v1: runnable extraction, sources, Q&A, batch and web demo

.env.example

@@ -1,19 +1,28 @@
-# document-extractor example configuration. Copy to .env and edit.
+# document-extractor configuration. Copy to .env and edit.
 # Placeholder values only. Never commit real keys.
 
-# Choose a backend: "api" (bring your own key) or "hf" (open-source on Hugging Face)
-MS_BACKEND=api
+# Choose a backend: "api" (any OpenAI-compatible endpoint) or "hf" (open models via transformers)
+MS_BACKEND=hf
 
-# --- (a) Bring your own API key: any OpenAI-compatible endpoint ---
-OPENAI_BASE_URL=https://your-provider.example/v1
-OPENAI_API_KEY=your-api-key
-LLM_MODEL=your-chat-model
-EMBEDDING_MODEL=your-embedding-model
+# --- (a) api: bring your own key, any OpenAI-compatible endpoint ---
+MS_API_BASE=https://your-provider.example/v1
+MS_API_KEY=your-api-key
+MS_MODEL=your-chat-model
+# Optional: a vision-capable model on the same endpoint, used for OCR of scanned pages
+MS_VISION_MODEL=
+# Optional: an embedding model on the same endpoint for Q&A retrieval (BM25 is used if empty)
+MS_EMBED_MODEL=
 
-# --- (b) Open-source models from Hugging Face ---
-HF_MODE=local                  # local | inference
-HF_TOKEN=your-hf-token         # only needed for gated models or HF Inference
-HF_LLM_MODEL=Qwen/Qwen2.5-7B-Instruct
-HF_EMBEDDING_MODEL=BAAI/bge-m3
-HF_RERANKER_MODEL=BAAI/bge-reranker-v2-m3
-HF_OCR_MODEL=microsoft/Florence-2-large
+# --- (b) hf: open-source models from Hugging Face, run locally with transformers ---
+# Qwen2.5-7B-Instruct needs a GPU or ~32 GB RAM. On a CPU-only machine use the 1.5B model.
+MS_HF_MODEL=Qwen/Qwen2.5-7B-Instruct
+# MS_HF_MODEL=Qwen/Qwen2.5-1.5B-Instruct
+MS_HF_EMBED_MODEL=BAAI/bge-m3
+MS_HF_OCR_MODEL=florence-community/Florence-2-large
+# HF_TOKEN=your-hf-token   # only needed for gated models
+
+# --- shared ---
+# Q&A retrieval: auto (embeddings when available, else BM25) | embed | bm25
+MS_RETRIEVER=auto
+# Where the web demo keeps saved results
+MS_CACHE_DIR=.cache/results

.github/workflows/ci.yml

@@ -0,0 +1,39 @@
+name: CI
+
+on:
+  push:
+    branches: [main, v1]
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v5
+      - uses: actions/setup-python@v6
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+      - name: Install (no model backends; tests use a mocked LLM)
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[dev]"
+      - name: Samples are reproducible
+        run: |
+          python scripts/make_samples.py
+          git diff --exit-code -- samples/
+      - name: Tests
+        run: pytest -q
+      - name: CLI smoke test
+        run: |
+          document-extractor --version
+          # api backend without a vision model: a scanned PDF must fail with a clear message (exit 2)
+          MS_BACKEND=api MS_API_BASE=http://127.0.0.1:9/v1 MS_MODEL=none \
+            document-extractor extract samples/invoice-acme-scanned.pdf --schema invoice || test $? -eq 2

README.md

@@ -5,7 +5,7 @@
 <p align="center"><em>Turn PDF invoices and contracts into structured fields, each linked to its source.</em></p>
 
 <p align="center">
-  <img alt="Status: in development" src="https://img.shields.io/badge/status-in%20development-8a3a24?style=flat-square">
+  <img alt="Status: v1" src="https://img.shields.io/badge/status-v1-8a3a24?style=flat-square">
   <img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-1c1a17?style=flat-square">
   <img alt="Python 3.11+" src="https://img.shields.io/badge/python-3.11%2B-1c1a17?style=flat-square">
   <img alt="Backends: API or Hugging Face" src="https://img.shields.io/badge/backends-API%20%7C%20Hugging%20Face-1c1a17?style=flat-square">
@@ -15,24 +15,30 @@
 
 ## Features
 
-Planned for v1. Progress is tracked in the Roadmap below.
+All five features below work in v1. See [Limitations](#limitations) for what they do not cover yet.
 
 - **Field extraction to JSON**: define the fields you want (invoice number, totals, parties, dates) and get typed JSON back.
 - **Source citation per value**: every value carries the page and bounding box it came from, so it can be checked in one click.
 - **Q&A over the document (RAG)**: ask a question and get an answer grounded in cited passages.
-- **Batch mode**: point it at a folder and get one JSON file and one CSV summary per run.
-- **Web demo**: drop a PDF, see fields highlighted on the page.
+- **Batch mode**: point it at a folder and get one CSV or JSON file per run.
+- **Web demo**: pick a sample or upload a PDF, see fields highlighted on the page.
+
+<p align="center">
+  <img src="assets/demo.gif" alt="Web demo: a synthetic invoice is selected, the extracted fields appear with their source boxes on the page, and a question is answered with a cited passage" width="100%">
+</p>
+<p align="center"><sub>Web demo on a synthetic invoice (<a href="assets/demo.mp4">MP4</a>). Results in the recording were computed earlier by Qwen2.5-1.5B-Instruct on CPU and loaded from the demo's result cache.</sub></p>
 
 ## Architecture
 
 ```mermaid
 %%{init: {"theme":"base","flowchart":{"curve":"basis","wrappingWidth":260},"themeVariables":{"fontFamily":"Georgia, Times New Roman, serif","fontSize":"14px","primaryColor":"#f3efe6","primaryTextColor":"#1c1a17","primaryBorderColor":"#1c1a17","lineColor":"#1c1a17","clusterBkg":"#faf7f0","clusterBorder":"#8a3a24","titleColor":"#8a3a24","edgeLabelBackground":"#f3efe6"}}}%%
 flowchart TB
   P["PDF / scan"]:::ink --> L["Layout and OCR<br/>pages, blocks, boxes"]:::ink
+  L --> X["Field extractor<br/>schema to JSON"]:::ink
   L --> C["Chunk and embed"]:::ink
-  C --> X["Field extractor<br/>schema to JSON"]:::ink
   C --> Q["Q&A (RAG)<br/>retrieve, answer, cite"]:::ink
-  X --> O["JSON + source links<br/>page, box, snippet"]:::ink
+  X --> G["Grounding<br/>match each value to page words"]:::ink
+  G --> O["JSON + source links<br/>page, box, snippet"]:::ink
   Q --> O
   subgraph BK["Backend · pick one"]
     direction LR
@@ -54,8 +60,10 @@ Pick one with `MS_BACKEND` in `.env`. The rest of the pipeline does not change.
 
 | | (a) Bring your own API key | (b) Open-source on Hugging Face |
 |---|---|---|
-| **What** | Any OpenAI-compatible endpoint | Open models, run locally or via Hugging Face Inference |
-| **Set** | `MS_BACKEND=api`, `OPENAI_BASE_URL`, `OPENAI_API_KEY` | `MS_BACKEND=hf`, `HF_MODE=local` or `inference` |
+| **What** | Any OpenAI-compatible endpoint | Open models, run locally with `transformers` |
+| **Set** | `MS_BACKEND=api`, `MS_API_BASE`, `MS_API_KEY`, `MS_MODEL` | `MS_BACKEND=hf`, optionally `MS_HF_MODEL` |
+| **Scanned pages** | `MS_VISION_MODEL` (a vision model on the same endpoint) | Florence-2 |
+| **Q&A retrieval** | `MS_EMBED_MODEL` if set, otherwise BM25 | bge-m3 |
 | **Good for** | Fastest start, strongest hosted models | Data stays on your machines, no per-call cost |
 
 **Default open-source models**
@@ -64,31 +72,96 @@ Pick one with `MS_BACKEND` in `.env`. The rest of the pipeline does not change.
 |---|---|---|
 | [`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) | field extraction and answers | Apache-2.0 |
 | [`BAAI/bge-m3`](https://huggingface.co/BAAI/bge-m3) | embeddings for retrieval | MIT |
-| [`BAAI/bge-reranker-v2-m3`](https://huggingface.co/BAAI/bge-reranker-v2-m3) | reranking retrieved passages | Apache-2.0 |
-| [`microsoft/Florence-2-large`](https://huggingface.co/microsoft/Florence-2-large) | OCR for scanned pages | MIT |
+| [`florence-community/Florence-2-large`](https://huggingface.co/florence-community/Florence-2-large) | OCR for scanned pages (`transformers`-native copy of [`microsoft/Florence-2-large`](https://huggingface.co/microsoft/Florence-2-large)) | MIT |
+
+On a machine without a GPU, set `MS_HF_MODEL=Qwen/Qwen2.5-1.5B-Instruct`. The 7B model needs a GPU or a lot of memory.
+`MS_RETRIEVER=bm25` skips the bge-m3 download and uses keyword retrieval instead.
 
 ## Quickstart
 
-> **Coming soon.** The commands below show the planned interface.
+Python 3.11 or newer.
 
 ```bash
 git clone https://github.com/ModelShifts/document-extractor
 cd document-extractor
-cp .env.example .env   # choose a backend
-pip install -e .
-document-extractor extract invoice.pdf --schema schemas/invoice.yaml
-document-extractor ask contract.pdf "When does the notice period start?"
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[hf,demo]"      # open models; use ".[api,demo]" for the API backend
+cp .env.example .env              # choose a backend and models
 ```
 
+On a CPU-only machine, install the CPU build of PyTorch first (`pip install torch --index-url https://download.pytorch.org/whl/cpu`) and set `MS_HF_MODEL=Qwen/Qwen2.5-1.5B-Instruct` in `.env`. Models download from Hugging Face on first use.
+
+```bash
+# fields as JSON, each with {page, bbox, text}
+document-extractor extract samples/invoice-northwind-classic.pdf --schema schemas/invoice.yaml
+
+# a question, answered with cited passages
+document-extractor ask samples/contract-master-services.pdf "When does the notice period start?"
+
+# a folder to one CSV (or .json)
+document-extractor batch samples --glob "invoice-*.pdf" --schema invoice --out results.csv
+
+# the web demo at http://127.0.0.1:7860 (python app.py does the same)
+document-extractor serve
+```
+
+### Using the API backend
+
+Any OpenAI-compatible endpoint works. In `.env`:
+
+```bash
+MS_BACKEND=api
+MS_API_BASE=https://your-provider.example/v1
+MS_API_KEY=your-api-key
+MS_MODEL=your-chat-model
+```
+
+To try it without a paid key, run an open model behind a local OpenAI-compatible server, for example [llama-cpp-python](https://github.com/abetlen/llama-cpp-python):
+
+```bash
+pip install "llama-cpp-python[server]" huggingface_hub \
+  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu   # prebuilt CPU wheels, no compiler needed
+hf download Qwen/Qwen2.5-1.5B-Instruct-GGUF qwen2.5-1.5b-instruct-q4_k_m.gguf --local-dir models
+python -m llama_cpp.server --model models/qwen2.5-1.5b-instruct-q4_k_m.gguf --n_ctx 8192 --port 8001
+# then: MS_API_BASE=http://127.0.0.1:8001/v1  MS_API_KEY=local  MS_MODEL=qwen2.5-1.5b-instruct-q4_k_m
+```
+
+### Schemas
+
+A schema is a YAML file that lists fields and their types (`string`, `date`, `number`, `integer`). See [`schemas/invoice.yaml`](schemas/invoice.yaml) and [`schemas/contract.yaml`](schemas/contract.yaml). Dates come back as `YYYY-MM-DD` and amounts as numbers. The model is asked to copy each value as printed, and the value is then matched back to the words on the page to find its box.
+
+### Sample documents and tests
+
+`samples/` holds seven synthetic PDFs (four invoice layouts, two contracts, one image-only scan) with a ground-truth file for each. All names and figures in them are fictional.
+
+```bash
+pip install -e ".[dev]"
+python scripts/make_samples.py   # regenerate samples/
+pytest                           # uses a mocked model, no downloads
+python scripts/evaluate.py       # compare the configured backend with the ground truth
+```
+
+## Limitations
+
+- The samples are synthetic and have simple layouts. The project has not been measured on real-world documents.
+- On a CPU, extraction with the open models takes tens of seconds per document, and loading Qwen, bge-m3 and Florence-2 together needs several GB of memory. The 7B default needs a GPU.
+- Florence-2 reads a scanned page in one pass, so small or dense text can be misread. On the API backend, OCR through `MS_VISION_MODEL` is implemented but has only been tested with a mocked model.
+- Source boxes come from matching the value against the page text. If a value is printed more than once, the copy closest to a matching label is used, which is not always the one the model read.
+- Numeric dates such as `03.02.2026` are read day-first.
+- The web demo saves results under `.cache/results` and reuses them for the same file, schema and model.
+
 ## Roadmap
 
 - [x] Repository scaffold and README
-- [ ] PDF parsing with page and box coordinates
-- [ ] Schema-driven field extraction
-- [ ] Source citation per value
-- [ ] Q&A over the document (RAG)
-- [ ] Batch mode
-- [ ] Web demo
+- [x] PDF parsing with page and box coordinates
+- [x] Schema-driven field extraction
+- [x] Source citation per value
+- [x] Q&A over the document (RAG)
+- [x] Batch mode
+- [x] Web demo
+- [ ] Reranking retrieved passages with `BAAI/bge-reranker-v2-m3`
+- [ ] Scanned-page OCR tested with a real vision model on the API backend
+- [ ] Hosted demo
 
 ## License
 

app.py

@@ -0,0 +1,11 @@
+"""Entry point for the web demo, also usable as a Hugging Face Space app file.
+
+    python app.py            # same as: document-extractor serve
+"""
+
+import os
+
+from document_extractor.webapp import launch
+
+if __name__ == "__main__":
+    launch(host=os.environ.get("GRADIO_SERVER_NAME", "127.0.0.1"), port=int(os.environ.get("GRADIO_SERVER_PORT", 7860)))

pyproject.toml

@@ -4,17 +4,29 @@ build-backend = "hatchling.build"
 
 [project]
 name = "document-extractor"
-version = "0.0.1"
+version = "0.1.0"
 description = "Turn PDF invoices and contracts into structured fields, each linked to its source."
 readme = "README.md"
 license = "MIT"
 requires-python = ">=3.11"
 authors = [{ name = "Modelshifts", email = "info@modelshifts.com" }]
-dependencies = ["pydantic>=2", "httpx>=0.27"]
+dependencies = [
+  "pydantic>=2",
+  "pdfplumber>=0.11",
+  "pypdfium2>=4.30",
+  "pillow>=10",
+  "pyyaml>=6",
+  "rank-bm25>=0.2.2",
+  "python-dotenv>=1",
+  "numpy>=1.26",
+]
 
 [project.optional-dependencies]
 api = ["openai>=1.40"]
-hf = ["transformers>=4.45", "sentence-transformers>=3", "torch>=2.3"]
+hf = ["transformers>=4.56", "sentence-transformers>=3", "torch>=2.3", "accelerate>=0.33"]
+demo = ["gradio>=5"]
+samples = ["reportlab>=4"]
+dev = ["pytest>=8", "reportlab>=4", "openai>=1.40"]
 
 [project.urls]
 Homepage = "https://modelshifts.com/work"
@@ -25,3 +37,9 @@ document-extractor = "document_extractor.cli:main"
 
 [tool.hatch.build.targets.wheel]
 packages = ["src/document_extractor"]
+
+[tool.hatch.build.targets.wheel.force-include]
+"schemas" = "document_extractor/schemas"
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

samples/contract-master-services.truth.json

@@ -0,0 +1,14 @@
+{
+  "file": "contract-master-services.pdf",
+  "schema": "contract",
+  "fields": {
+    "contract_title": "Master Services Agreement",
+    "party_a": "Northwind Parts Pte. Ltd.",
+    "party_b": "Example Logistics Pte. Ltd.",
+    "effective_date": "2026-07-01",
+    "term_months": 24,
+    "notice_period_days": 60,
+    "payment_terms_days": 30,
+    "governing_law": "Singapore"
+  }
+}

samples/contract-supply.truth.json

@@ -0,0 +1,14 @@
+{
+  "file": "contract-supply.pdf",
+  "schema": "contract",
+  "fields": {
+    "contract_title": "Supply Agreement",
+    "party_a": "Sample Robotics Ltd.",
+    "party_b": "Demo Freight GmbH",
+    "effective_date": "2026-09-15",
+    "term_months": 12,
+    "notice_period_days": 90,
+    "payment_terms_days": 45,
+    "governing_law": "England and Wales"
+  }
+}

samples/invoice-acme-scanned.truth.json

@@ -0,0 +1,15 @@
+{
+  "file": "invoice-acme-scanned.pdf",
+  "schema": "invoice",
+  "fields": {
+    "invoice_number": "AES-5120",
+    "invoice_date": "2026-06-08",
+    "due_date": "2026-07-08",
+    "seller_name": "Acme Example Services Pte. Ltd.",
+    "buyer_name": "Northwind Parts Pte. Ltd.",
+    "currency": "SGD",
+    "subtotal": 5219.37,
+    "tax": 469.74,
+    "total": 5689.11
+  }
+}

samples/invoice-demo-freight-compact.truth.json

@@ -0,0 +1,15 @@
+{
+  "file": "invoice-demo-freight-compact.pdf",
+  "schema": "invoice",
+  "fields": {
+    "invoice_number": "DF-88213",
+    "invoice_date": "2026-02-03",
+    "due_date": "2026-03-05",
+    "seller_name": "Demo Freight GmbH",
+    "buyer_name": "Northwind Parts Pte. Ltd.",
+    "currency": "EUR",
+    "subtotal": 3151.06,
+    "tax": 598.7,
+    "total": 3749.76
+  }
+}

samples/invoice-example-logistics-banner.truth.json

@@ -0,0 +1,15 @@
+{
+  "file": "invoice-example-logistics-banner.pdf",
+  "schema": "invoice",
+  "fields": {
+    "invoice_number": "EL/INV/7731",
+    "invoice_date": "2026-05-02",
+    "due_date": "2026-06-01",
+    "seller_name": "Example Logistics Inc.",
+    "buyer_name": "Sample Robotics Ltd.",
+    "currency": "USD",
+    "subtotal": 12594.58,
+    "tax": 881.62,
+    "total": 13476.2
+  }
+}