Lucebox builds plug-and-play hardware and the open inference stack that powers it. Local-first, OpenAI/Anthropic-compatible, 3 to 5x faster than competing boxes at the same price.
Lucebox is a 9.56L aluminum box that runs frontier open models locally at speeds people thought needed cloud GPUs. Inside: an AMD Radeon™ AI PRO R9700 (32GB GDDR6) paired with an AMD Ryzen AI MAX+ 395 APU (128GB unified LPDDR5X), 2TB NVMe, Corsair 1000W 80+ Gold. Outside: a single power cable and an OpenAI/Anthropic-compatible endpoint reachable from Claude Code, Codex, OpenCode, Hermes, OpenClaw, Open WebUI, and Ollama in roughly one minute from unboxing.
lucebox is the open inference engine with custom high performance optimizations. It runs on any RTX 30/40/50, on Strix Halo, Radeon 7900 XTX and Radeon AI PRO R9700 not just on our hardware.
- Privacy. Prompts and weights never leave the device. Default-fit for legal, medical, finance, and any team where the data is the moat.
- Cost. $5,499 once, then zero per token. Replaces $200 to $2,000 per month in cloud API spend for sustained agent workloads.
- Throughput. Up to 207 tok/s on Qwen3.5-27B and 134 tok/s at 128K context, matching or beating cloud latency on a desk.
- Open. Apache 2.0 inference stack, GGUF models, no vendor lock-in.
| Repository | Description | Stars | Forks |
|---|---|---|---|
| lucebox | Fast LLM speculative inference server for consumer hardware. OpenAI/Anthropic compatible HTTP server. |  |
 |
| Component | What it does | Speedup |
|---|---|---|
| DFlash | Speculative decode with draft model + tree verification (DDTree) | 3 to 5x on 27B |
| PFlash | Block-sparse speculative prefill, register-resident FA-2 kernels | ~5.6x on long context, 5.4x at 128K |
| KVFlash | Auto-sized active pool of KV cache on the GPU and paging cold 64-token chunks dynamically | 256k context |
| Spark | Dynamic Expert Pinning, only hot experts directly inside the GPU VRAM | 1.5x tok/s |
| Megakernel | Fused 24-layer persistent CUDA kernel for small drafts | ~2x on 0.8B (413 tok/s) |
git clone --recurse-submodules https://github.com/Luce-Org/lucebox-hub
cd lucebox-hub
cmake -B server/build -S server -DCMAKE_CUDA_ARCHITECTURES=86
cmake --build server/build --target dflash_server -j
./server/build/dflash_server \
model.gguf \
--draft draft.gguf \
--port 8000Then point any OpenAI-compatible client at http://localhost:8000/v1.
| Model | Hardware | Throughput | Method |
|---|---|---|---|
| Qwen3.5-27B AWQ | RTX 3090 | 207 tok/s | DFlash + DDTree |
| Qwen3.6-27B Q4_K_M | RTX 3090 | 134 tok/s @ 128K | PFlash sliding target_feat |
| Laguna-XS.2 33B | RTX 3090 | 5.4x @ 128K | PFlash |
| Qwen3.5-0.8B | RTX 3090 | 413 tok/s | bf16 Megakernel |
| gfx1151 iGPU (Strix Halo) | Ryzen AI MAX+ 395 | 26.85 tok/s | HIP, 2.23x vs llama.cpp HIP |
- NVIDIA: RTX 3090, RTX 4090, RTX 5090, DGX Spark (CUDA 12+)
- AMD: Ryzen AI MAX+ 395 Strix Halo (HIP / ROCm 6+), RX 7900 XTX (HIP)
- OS (Inference engine): Linux, Windows
- OS (Lucebox appliance): Linux, pre-tuned
The plug-and-play Lucebox ships pre-tuned with the full stack loaded. $6,499, three year warranty.
- Product Site
- Blog
- DFlash on 27B
- PFlash Speculative Prefill
- Megakernel Decode
- Laguna-XS.2 @128K
- Gemma vs DeepSeek
- Client Harnesses
- AMD Strix Halo Notes
- eGPU Myth
- Issue Tracker
Website • GitHub • X • Discord
Apache 2.0. Built in Italy.