Skip to content

rdap 1.1.0: API server, outbound proxies/bind, Docker and systemd packaging - #1

Merged
showfom merged 27 commits into
mainfrom
feat/serve
Oct 29, 2026
Merged

rdap 1.1.0: API server, outbound proxies/bind, Docker and systemd packaging#1
showfom merged 27 commits into
mainfrom
feat/serve

Conversation

@showfom

@showfom showfom commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

rdap 1.1.0: a local HTTP API server (rdap --serve), outbound proxy / source-address control, and Docker/systemd packaging.

API server (rdap --serve, default 127.0.0.1:8888)

  • RFC 7482 paths (/domain, /ip incl. CIDR, /autnum, /nameserver, /entity, /help, searches), so any RDAP client can use it as an upstream; upstream JSON is passed through unchanged
  • JSON endpoints: /query, /abuse, /batch, /healthz, /version
  • RFC 7480 error objects with correct status codes for every "nothing found" case (404 / no RDAP server / empty search / abuse null)
  • Optional bearer token (RDAP_API_TOKEN), size-bounded result cache with request coalescing, gzip/br, optional CORS, graceful shutdown

Outbound networking (CLI and server)

  • --proxy (socks5/socks5h/http/https, rotated, failover with cooldown), --bind (rotated source IPs, v4/v6 aware), --interface
  • Per-host concurrency limit, response size limit; --update uses the same routes

Hardening (from security review)

  • SSRF guard for registrar referrals and redirects (public upstreams can only point to public https URLs)
  • Header-read timeout and connection caps (slowloris), DNS-rebinding Host check without a token, batch cancellation and global concurrency, credential redaction everywhere, sanitized upstream error text

Fixes in existing code

  • --timeout and -k had no effect; failed queries were sent twice; Verisign 404s surfaced as decode errors; panic on non-ASCII input; unvalidated IP queries in upstream paths; bootstrap registries were refetched on every query (now cached 24 h, coalesced, stale fallback)

Packaging

  • Docker: ghcr.io/xtomcom/rdap (amd64 + arm64, native runners, published by release.yml), runtime gcr.io/distroless/cc-debian13:nonroot (38 MB); compose.yaml + .env.example
  • deb/rpm: built from glibc 2.17 binaries, hardened rdap.service (not enabled on install), depend on ca-certificates, gzip rpm payload for CentOS 7
  • README rewritten; dependencies updated

Test plan

  • cargo fmt --check, cargo clippy -D warnings (with and without server feature), cargo test (69 tests), cargo audit clean
  • End-to-end against local mock upstreams with oha (up to 236k rps cached, no errors, no deadlocks with --max-per-host 1)
  • SOCKS5 proxy via Docker (auth, failover, no credential leaks at RUST_LOG=trace)
  • deb/rpm install on Ubuntu 16.04/22.04, Debian 10/12, CentOS 7, Rocky 9
  • Real systemd (privileged containers): Debian 13 deb and Rocky 9 rpm, DynamicUser sandbox, token from env file, restart on crash, conffile handling
  • Docker image: read-only fs, cap_drop ALL, --update into volume, graceful stop
  • CI: Windows / macOS builds (not verifiable locally)
  • Release workflow: GHCR multi-arch publish (runs on tag) — after first publish, set the rdap package visibility to public

🤖 Generated with Claude Code

showfom and others added 27 commits October 29, 2026 08:46
@showfom @claude
refactor: shared HTTP pool, cached bootstrap, query validation; scaff…
72af5d9 
…old --serve

Groundwork for the 1.1.0 API server:
- net::HttpPool: single choke point for outbound HTTP (proxies, source
  address, limits to follow); --timeout and --insecure now take effect
- Cache bootstrap registries in memory for 24h instead of refetching
  from IANA on every query
- RdapRequest::from_user_query shared by CLI and server: detection, IP
  shorthand expansion and strict validation; fix panic on non-ASCII
  input in AS prefix handling; never put unvalidated text in ip/ paths
- RdapQueryResult is Serialize
- CLI flags for --serve and outbound networking, `server` cargo feature
- Bump version to 1.1.0

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
feat(net): proxy rotation, source binding, failover and limits
e238feb 
Complete the outbound HTTP pool: one route per proxy (socks5, socks5h,
http, https) or per --bind address, rotated round-robin per request.
--bind applies to the proxy connection when proxying; --interface
applies to every route and is checked at startup.

Connection-level failures (connect, proxy handshake, timeout before a
response) fail over to the next route, up to min(routes, 3) attempts,
and put the route in a short cooldown; nothing is retried once a status
was received. Mixed IPv4/IPv6 bind addresses only use routes of the
destination's family. Per-host concurrency limit held until the body
is read; body size limit enforced via Content-Length and while
streaming. Proxy URLs are validated at startup and credentials are
redacted from logs and errors.

New RdapError variants: Proxy, BodyTooLarge, Config. Timeouts now map
to RdapError::Timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(client): don't repeat the whole query after it fails
17cd073 
query_with_referral fell through to a second query_servers() call on
every non-CIDR-retry error, doubling upstream requests for each 404 or
failure. Return the original error instead.

Also expose RdapClient::route_labels() and list https:// in --proxy help.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
feat(packaging): add Docker image, compose file and systemd service
fea8794 
- Dockerfile: multi-stage build on rust:1.99-slim-trixie, runtime on
  debian:trixie-slim (same glibc), BuildKit cache mounts, non-root UID
  10001 with writable HOME, runs `rdap --serve --listen 0.0.0.0` on 8888
- compose.yaml: reference setup on 127.0.0.1:8888 with read-only root fs,
  no capabilities, no-new-privileges and a $HOME volume
- packaging/rdap.service: hardened DynamicUser unit, HOME in StateDirectory
- packaging/rdap.env: installed as /etc/default/rdap (conffile, mode 600)
- deb/rpm ship the unit but never enable or start it on install

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
docs: document API server, network options, Docker and systemd
27ad30b 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
feat(server): implement rdap --serve HTTP API
b955057 
- RDAP-compatible endpoints (RFC 7482): /domain, /ip (incl. CIDR),
  /autnum, /nameserver, /entity, /domains, /nameservers, /entities, /help;
  `rdap -s http://127.0.0.1:8888/ <query>` works against the server
- JSON API: /query/{q}?type=&source=, /abuse/{q} (null when absent),
  POST /batch (ordered, deduplicated, bounded concurrency, 64 KiB limit),
  /healthz, /version
- RFC 7480 error objects with correct status for every failure: upstream
  404, no bootstrap server, 2xx error objects, empty searches, wrong
  object class, 429, timeouts (504), other upstream failures (502)
- Optional bearer token (constant-time compare, /healthz exempt),
  gzip/br, opt-in CORS, nosniff, request timeout, graceful shutdown
- moka result cache with coalesced misses; 404s cached for at most 60 s,
  errors never cached
- -s/--server with --serve sends every query to one upstream

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom
Merge branch 'worktree-agent-a7721e6f7db1d9e12' into feat/serve
1d081cf
@showfom @claude
fix(docker): set RDAP_LISTEN via ENV so it can be overridden; fix cac…
8df5854 
…he docs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom
Merge branch 'worktree-agent-a709bf5953dc7b4b6' into feat/serve
bf45e3a
@showfom @claude
fix: lossless RDAP passthrough, 404s with broken bodies, typed bootst…
94df959 
…rap errors

- Keep upstream JSON in RdapQueryResult (registry_json / registrar_json)
  and serve it as-is, so fields the models don't cover (RFC 9537
  redaction, vendor extensions, nested unknowns) aren't dropped
- A body read error after a non-2xx status no longer hides the status:
  Verisign 404s (closed TLS without close_notify) came back as
  "error decoding response body" (502 / CLI error) instead of not found
- New RdapError::NoServer / NeedsServer instead of matching Bootstrap
  message text in the server
- Referral same-server check compares port too
- Print credential-redacted outbound routes on server startup

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix: review findings in failover, caching and query validation
948b5d0 
- Fail over only on connect-level errors; a slow upstream (response-head
  timeout) is no longer retried on every route, which tripled waits and
  upstream load. Connect timeout is now below the request timeout so a
  stuck proxy handshake still fails over.
- Results whose registrar referral failed are cached for at most 60 s
  instead of the full TTL
- Canonicalize AS numbers (digits only: rejects +5, AS0013335 -> AS13335)
- Validate CIDR prefix length (<= 32 / 128, digits only)
- build_url parses Ip / Autnum queries, so library callers can't inject
  ../ path segments either
- Docs: cache key, failover semantics, no disk cache

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(security): guard registrar referrals and redirects against SSRF
1769213 
URLs taken from upstream data could make the server fetch internal
addresses: a registry's related link or a 302 to http://127.0.0.1:...,
169.254.169.254, RFC 1918 etc. was followed and could be returned.

- A public upstream may only refer/redirect to public https URLs (no
  downgrade, no loopback/private/link-local/CGNAT/ULA IP literals, no
  localhost); referral hostnames must also resolve to public addresses
  (skipped through proxies to avoid local DNS leaks)
- A local/private upstream (self-hosted -s, tests) is unrestricted
- At most 3 redirects, each checked; no Referer header

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(security): never show upstream URL credentials in API responses o…
177addc 
…r logs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(security): sanitize upstream error text, trust HTTP status, hide …
e512963 
…proxy details, reject dot-only queries

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(security): size-bounded result cache, 4 MiB upstream bodies when …
9a13d9c 
…serving, abort abandoned batches, global batch concurrency, trim token

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(security): header-read timeout and connection limits for the API …
8cb730f 
…server

axum::serve has no header timeout or connection cap, so anyone reaching
the port (token or not: auth runs after headers) could hold connections
open with partial or no headers until fds ran out.

- Own accept loop (hyper-util, HTTP/1 only so there is no untimed
  protocol sniffing): 10 s header-read timeout, also for idle keep-alive
- At most 1024 connections (capped to half of RLIMIT_NOFILE) and 64 per
  client IP; accept errors back off instead of spinning
- Graceful shutdown waits up to 30 s for in-flight requests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix: coalesce bootstrap fetches, back off and fall back to stale copy…
aebd964 
… on failure; route nameserver queries via DNS bootstrap

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(security): reject non-loopback Host without a token on loopback (…
405d90a 
…DNS rebinding); CORS allows Authorization

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(cli): empty RDAP_* env vars count as unset, skip empty list items…
7b3d049 
…, boolish RDAP_CORS

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(cli): rdap --update downloads through the configured proxies / so…
7822706 
…urce address

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(net): keep the real status for oversized error bodies; prune idle…
fc07b76 
… per-host limiters

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
chore(security): systemd/compose resource limits and metadata IP deny…
209c47a 
…, read-only workflow default permissions, docs for the hardening

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
ci: publish amd64 + arm64 images to ghcr.io/xtomcom/rdap; simpler com…
3e6863d 
…pose with .env

- release.yml: build each architecture natively (ubuntu-latest /
  ubuntu-24.04-arm), push by digest, merge into one manifest tagged
  X.Y.Z, X.Y, X and latest (pre-releases only get their own tag). Runs
  for tag pushes and for the auto-release workflow_dispatch.
- Image version label comes from the tag instead of a hardcoded value
- compose.yaml uses the published image, settings move to .env
  (.env.example); .env is git- and docker-ignored

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
fix(packaging): default RDAP_ARGS to empty to silence systemd's unset…
cd1161d 
…-variable warning

Verified under real systemd (Debian 13 deb, Rocky 9 rpm): disabled on
install, DynamicUser + sandbox (no caps, NoNewPrivs, seccomp, IP deny,
memory/task limits), token from /etc/default/rdap, restart on SIGKILL,
conffile kept on reinstall/remove and dropped on purge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
feat(docker): distroless cc-debian13:nonroot runtime; rewrite README
bbd065f 
- Runtime image is gcr.io/distroless/cc-debian13:nonroot (glibc +
  libgcc_s, no shell or package manager), built on rust:slim-trixie so
  glibc matches; runs as nonroot (65532), HOME=/home/nonroot. 93 MB ->
  38 MB. compose volume moves to /home/nonroot.
- README: drop emoji and marketing tone, tighten (1337 -> 826 lines),
  re-capture examples from real output, document search endpoints and
  other missing behaviour, add a Docker deployment guide (compose, .env,
  exposing behind a reverse proxy, proxies/bind, health checks, builds)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
chore(deps): update dependencies to latest compatible versions
dbf3ce0 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom @claude
docs: rewrite README; publish image to Docker Hub as xtomcom/rdap
7e89bf4 
- README: professional wording, xtom.com / AS8888 examples taken from real
  output, accurate platform list
- release: push the multi-arch image to docker.io/xtomcom/rdap alongside
  ghcr.io/xtomcom/rdap (needs the DOCKERHUB_TOKEN secret)
- compose: default to xtomcom/rdap:latest, GHCR as a commented alternative

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@showfom
showfom merged commit ef12e73 into main Oct 29, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@showfom