Skip to content

perf(preview): focus Hugging Face paper extraction on main - #25

Merged
phatpham9 merged 1 commit into
mainfrom
perf/hf-preview-cpu
Nov 13, 2026
Merged

phatpham9 merged 1 commit into
mainfrom
perf/hf-preview-cpu

Conversation

@phatpham9

@phatpham9 phatpham9 commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Summary

Cloudflare logs showed 10 503 exceededCpu events on Hugging Face paper previews (10–35ms CPU vs the Workers Free 10ms limit). This reduces the HF paper extraction cost without changing its output:

  • Parse only <head>+<main> for Hugging Face paper pages before DOM extraction. This keeps the title/site/date metadata and the paper/link/citation blocks while dropping header/footer/hydration chrome.
  • Combine the 16 noise-removal selectors into one comma-separated selector, so node-html-parser traverses the tree once instead of once per tag.
  • Add a regression test proving HF extraction stays inside <main> and does not pick up surrounding chrome.

Verification

  • npm run typecheck
  • npx vitest run --exclude '.claude/**' — 18 files, 146 tests
  • Benchmark on the 10 HF URLs from the production CPU-exceeded events:
    • Median extraction time dropped from 3.2–5.3ms to 2.5–4.9ms (~20% lower).
    • Block counts, titles, bylines, publication dates, and site names are unchanged on all 10 pages.
    • The worst page (2611.08175) moved from 5.33ms to 4.87ms median.
  • PR preview pr-25-7fad10e against production D1:
    • 2611.02857 → 200, 10 blocks, 0.88s
    • 2611.05036 → 200, 12 blocks, 0.72s
    • 2611.06741 → 200, 20 blocks, 0.97s
    • Titles, bylines, publication dates, and reading times unchanged.
  • npm run db:migrate:local
  • wrangler deploy --dry-run

Notes

  • This is a safe reduction, not a guarantee against cold-start/concurrency CPU spikes; the same HF URLs succeeded individually before this change.
  • If production still logs exceededCpu for HF URLs, the next lever is the paper Markdown-content endpoint or a stored-summary fallback; both trade away the HF citation/link extraction.
  • No client changes or cache-key bump; preview output is unchanged.

🤖 Generated with Claude Code

Hugging Face paper pages are ~150-220 KB of HTML with large header,
footer, and hydration chrome. Parse only <head>+<main> before
extraction, preserving title/byline/date metadata and all paper,
link, and citation blocks while cutting DOM work.

Combine the 16 noise-removal selectors into one comma-separated
selector so node-html-parser traverses the tree once instead of once
per tag. Local benchmarks on the ten HF URLs that hit the Workers Free
exceededCpu limit show ~20% lower extraction time with identical
block counts; a regression test covers the main-only boundary.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🔎 Cloudflare preview: https://1cd975a8-feedreader.phatpham9.workers.dev

Uploaded from 7fad10e6ae34307cff5dfb4a8c965e86d312811e. This is a Worker version preview — it shares the production D1 database, so it reads/writes real data; it does not receive cron-triggered refreshes.

@phatpham9
phatpham9 merged commit 45a4ca9 into main Nov 13, 2026
3 of 5 checks passed
@phatpham9
phatpham9 deleted the perf/hf-preview-cpu branch November 13, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant