Skip to content

Latest commit

 

History

History
352 lines (278 loc) · 14.1 KB

raspberry-pi.md

File metadata and controls

352 lines (278 loc) · 14.1 KB

Raspberry Pi Deployment

Run EggPool on a Raspberry Pi for always-on LAN access.

Quick Start

# Install (verified standalone raw binary, no Python >=3.11 required)
bash -o pipefail -c 'curl -fsSL https://raw.githubusercontent.com/eggstack/eggpool/main/scripts/install.sh | bash' && export PATH="$HOME/.local/bin:$PATH"

# Explicit wheel alternative (requires Python >=3.11 for package management)
# pipx install eggpool

# Set up providers interactively
eggpool onboard

# Start on boot (writes systemd unit, enables, starts)
sudo env "PATH=$PATH" "$(command -v eggpool)" deploy systemd --install

# Verify
sudo systemctl status eggpool
curl http://localhost:11300/v1/healthz

A current-native curl install on Linux/aarch64 does not consult pipx or uv and does not require Python >=3.11; that floor applies only to explicit package-manager and historical compatibility paths.

See deployment.md for full details on both personal and production deployment paths.

Requirements

  • Raspberry Pi 4 (4GB+ RAM) or Pi 5
  • Raspberry Pi OS (Debian-based) or Ubuntu Server
  • 32GB+ microSD card (or USB SSD)
  • Ethernet recommended over WiFi

Pi Setup

sudo apt update && sudo apt upgrade -y
sudo apt install -y curl
# No uv/pipx/Python setup is needed for the default raw-binary install.
# Only install a package manager if you explicitly prefer a wheel install:
# curl -LsSf https://astral.sh/uv/install.sh | sh
# export PATH="$HOME/.local/bin:$PATH"

Then follow the Quick Start above, or deployment.md for the full flow.

Pi-Specific Config

For a complete copyable low-wear configuration, use config.sbc.example.toml. It intentionally uses a loopback bind, one database worker, disables optional diagnostics, and keeps correctness-critical request state durable while allowing buffered analytics to lag. Startup and check-config require a server API key before accepting a non-loopback bind. If LAN access is needed, configure the key before changing [server].host to 0.0.0.0 or another non-loopback address.

If you are adapting an existing configuration, increase timeouts for slower SD card storage:

[database]
busy_timeout_ms = 10000

[upstream]
max_connections = 16
max_keepalive = 4

[models]
refresh_interval_s = 7200

Catalog discovery is separate from catalog rewriting: unchanged semantic model/provider rows are not rewritten on each refresh. Successful refresh freshness uses compact per-account state, and steady successful pings are sampled internally while failures remain immediately durable.

Recommended Performance Profile

The default config is tuned for Pi-class devices. The supported native runtime default (threads = 1) uses bounded task concurrency for high throughput. If you need to explicitly set the recommended profile:

[server]
threads = 1          # supported single-loop default

[database]
worker_threads = 1   # lean default; use 2 for a separate stats connection

[metrics]
write_mode = "low_wear"
flush_interval_s = 120

[routing.trace]
mode = "off"
sample_rate = 0.0
include_score_components = false

For minimum-footprint mode on very constrained devices:

[server]
threads = 1

[database]
worker_threads = 1

[routing.trace]
mode = "off"
sample_rate = 0.0
include_score_components = false

Process Model

EggPool's native Rust runtime is one eggpool serve process. The installed wheel does not launch a Python interpreter or a separate web-framework worker. Keep the default threads = 1 on constrained SBC hardware. It is retained for configuration compatibility and diagnostics; Tokio's current-thread runtime does not use it to select an I/O worker count:

[server]
threads = 1

The PID file path is resolved by rust/src/operations/paths.rs in this precedence: $EGGPOOL_PID_FILE → $XDG_RUNTIME_DIR/eggpool.pid → ~/.local/state/eggpool/eggpool.pid → /tmp/eggpool-<UID>.pid, and is owned by the supervisor. If eggpool serve ever exits non-zero with a message about an existing instance, that is the duplicate-instance guard catching a live PID or a successful /v1/healthz probe: check pgrep -f eggpool before retrying.

Reduce SD Card Wear

Log to tmpfs by adding to /etc/fstab:

tmpfs /var/log/eggpool tmpfs defaults,noatime,nosuid,mode=0750,size=50M,uid=eggpool,gid=eggpool 0 0

Temperature Monitoring

vcgencmd measure_temp          # current temp
watch -n 5 vcgencmd measure_temp  # continuous

Thermal throttling starts at 80°C: use a heatsink or fan if sustained loads are expected.

Optional Target-Class Characterization

For a release candidate built on the Pi (or copied byte-for-byte from a qualified Linux/aarch64 build), run the existing guarded qualification first (ordinary Q008 contract, runtime-q008.v1), then repeat the bounded benchmark three times from fresh temporary roots with the benchmark-only fixture:

uv run python scripts/qualification_sbc.py \
  --binary rust/target/release/eggpool \
  --candidate-origin on-device-release-build \
  --config-fixture tests/tooling/fixtures/qualification/sbc-benchmark.toml \
  --benchmark-samples 30 \
  --output artifacts/qualification/236-sbc-benchmark-run-1.json

The ordinary fixture (sbc.toml) keeps its aggressive lifecycle cadence for Q008 coverage; the benchmark fixture (sbc-benchmark.toml) mirrors the low-wear steady-state profile so timing batches are not contaminated. The translated Responses-client stream requires downstream response.completed evidence plus proof that the fixture observed the Anthropic Messages upstream path. Benchmark mode emits the extended runtime-q008.v2 contract with cadence facts; default mode stays on runtime-q008.v1.

Use q005-qualified-aarch64-copy when the candidate was copied, and verify its SHA-256 before running. The provider stays on loopback; Ethernet is recommended for ordinary client use but is not part of the timing result. The report is descriptive and sanitized, not a hardware-CI gate or SLA. Never report a hosted ARM VM or cloud ARM instance as SBC evidence.

Optional Finite-Tail Diagnostic

To localize a finite-request tail, append the diagnostic-only flag to the benchmark command (10–200 samples, default off; requires benchmark mode):

uv run python scripts/qualification_sbc.py \
  --binary rust/target/release/eggpool \
  --candidate-origin on-device-release-build \
  --config-fixture tests/tooling/fixtures/qualification/sbc-benchmark.toml \
  --benchmark-samples 30 \
  --diagnose-finite-tail 60 \
  --output artifacts/qualification/237-sbc-finite-tail-run-1.json

It runs 60 sequential native finite requests with monotonic provider-boundary phase timing (client start, provider receive/finish, first byte, done) plus a 30-request direct-provider control, all scalar-only with no p99. If the slowest request is pre-provider dominated, repeat once with the temporary root on a RAM-backed filesystem (for example TMPDIR=/dev/shm/eggpool-qual) keeping all EggPool settings identical; record the storage class. On Pi 5 this localized the tail to the durable publication / SQLite / storage path: pre-provider dominated in all three runs with a stable direct control, and the tmpfs run removed the tail entirely. See Plan 237 for the decision matrix and the sanitized aggregate artifact.

Publication/storage diagnostic

Plan 238 keeps the diagnostic tooling separate from the normal benchmark corpus. With the benchmark fixture, run the physical-SBC-gated mode directly for a fresh-root MMC pass:

uv run python scripts/qualification_sbc.py \
  --binary rust/target/release/eggpool \
  --config-fixture tests/tooling/fixtures/qualification/sbc-benchmark.toml \
  --diagnose-publication-storage 60 \
  --output artifacts/qualification/238-sbc-publication-storage-mmc-run-1.json

For the comparison, pass a directory on a mounted tmpfs with --diagnostic-database-dir DIR. Only the SQLite database/WAL/SHM move there; logs, config, runtime files, and backups remain in the qualification root. Repeat three fresh-root runs per class, retain only sanitized aggregate scalars, and record unavailable hardware/tmpfs dimensions as not measured. This mode is diagnostic-only and does not justify a production durability or publication change by itself.

On-prem small-team envelope (M012, Pi 5 managed)

On-prem milestone M012 qualified one managed Pi 5 as a solo/small-team envelope (closure plans/closure/on-prem-user-access/012-status.md). Target frozen at measurement: Raspberry Pi 5 Model B Rev 1.0, 4 cores, 8 GB RAM, Ubuntu 24.04.4, kernel 6.8.0-1065-raspi, ext4 on MMC (mmcblk0p2), release 0.8.2 built on-device (SHA-256 8609119a…1538, 34.9 MB, schema 62, WAL/NORMAL, one database worker, threads = 1). Inference and catalog use a synthetic administrator personal token; the shared server key is rejected for those surfaces (managed fencing), and runtime-status CLI plus the dashboard shell require a browser session (expected; full runtime projection remains available there per M011).

Managed functional plus three fresh-root benchmark runs (30 samples, concurrency 4, 32-request batch) all pass with secret-free aggregates in artifacts/qualification/on-prem-m012-pi5-functional.json and on-prem-m012-pi5-benchmark-run-{1,2,3}.json: 25 functional cells, 145 requests per benchmark run with zero pending/active reservations, backup, recovery, restart, and shutdown green. Sequential p50 is about 5 ms (finite/streaming) and 13–15 ms concurrent; streaming TTFT p95 is 4–19 ms. MMC tails reach seconds (concurrent p95 up to about 14 s, maxima similar) with no OOM, swap, drops, or WAL divergence; RSS stays about 20 MB baseline to 20.8 MB peak. Argon2id logins (19 MiB, t=2, p=1, two slots) measure about 51 ms p50 one-way idle and 71–73 ms two-way, unchanged under gentle inference load. Provisioning reaches 49 named users (one admin plus 48 members) at about 51 ms p50 each with all spot logins passing; five personal tokens issue cleanly. Bounded concurrency 1/2/4/8 completes (1–2 at p50 5–6 ms with second-scale tails; 8 saturates at p50 about 4.1 s).

Deploy accordingly: solo personal use is practical in one process with no extra services; up to 2–4 concurrent requests work with degraded tails; 8 concurrent exceeds this Pi/MMC class. Forty-nine named users are inexpensive metadata, not 49 simultaneous generations: no 50-concurrent claim is made. Le Potato and a separate ordinary-Linux reference host were not measured here (docs promise Pi 4/Pi 5 only; paired same-Pi controls ground the solo deltas); 16-way concurrency was not attempted because 8-way already saturates.

The ordinary-server team baseline is maintained separately from this Pi microSD solo envelope; do not use the Pi limits as the team-host sizing source. M002 now has a measured synthetic reference on one durable-NVMe ordinary Linux host. Its host-specific observations and limits are in the closure record and deployment guidance.

Verify from LAN

  1. Find Pi IP: hostname -I
  2. Test: curl http://<pi-ip>:11300/v1/healthz
  3. Dashboard: http://<pi-ip>:11300/
  4. Point OpenCode at http://<pi-ip>:11300

Browser password sign-in is not offered over remote plaintext HTTP. From a solo operator workstation, create a loopback tunnel with ssh -L 11300:127.0.0.1:11300 operator@<pi-ip>, then open http://127.0.0.1:11300/login. This adds no service to the Pi. Team browser administration should use an operator-managed HTTPS proxy or VPN; the LAN inference endpoint remains separate from browser password transport.

Headless Pi + Desktop Clients

On the Pi, advertise one stable URL once (live-reloadable; never the bind address):

[integrations]
advertise_base_url = "http://<pi-lan-ip>:11300/v1"

Then export a secret-free profile without touching a desktop filesystem:

eggpool configremote codex
eggpool configremote opencode --format token

Copy the printed bootstrap block (or the epc1 token) to each desktop that can reach the Pi. The bootstrap verifies SHA-256 before executing anything, so desktops need no EggPool checkout, Rust toolchain, or Python: only the configremote block for their shell (--shell posix for macOS/Linux, --shell powershell for Windows). Each desktop still provisions its own authorized EggPool key (EGGPOOL_API_KEY) in the environment that launches the client. Desktops fetch current models from authenticated GET /api/integrations/v1/profile during install; the token itself carries no credential. The Windows helper does not add Windows proxy support for the server; the Pi remains the only server in this topology. See Agent Configuration and Desktop Bootstrap.

Troubleshooting

Symptom Check
Won't start journalctl -u eggpool --since "5 min ago" or tail -50 /var/log/eggpool/eggpool.log
Slow perf CPU temp, use Ethernet, increase busy_timeout_ms
DB locked pgrep -f eggpool: ensure only one instance
SD full df -h /var/lib/eggpool, check retention config

The startup log includes a structured operational profile line (Milestone A6) with effective config: workers, threads, database connections, WAL mode, routing trace settings, and background task counts. Use it to confirm the recommended profile is active. Background task cadence diagnostics (last_tick_drift_s, configured_interval_s) are visible via eggpool runtime-status --json under each task's snapshot fields.

For a short before/after resource check, use the same database/config shape and a fixed stabilization interval on each build, then record:

eggpool runtime-status --json
ps -o pid,rss,vsz,pcpu,etime,command -p <worker-pid>
lsof -nP -a -p <worker-pid> -iTCP -sTCP=ESTABLISHED

Compare at least three runs and keep upstream latency separate from the local local_pre_upstream and dispatch_overhead fields. These checks are descriptive and non-gating; measurements from macOS or an unconstrained workstation must not be reported as Raspberry Pi results.