Repository navigation
feat(preview): add Workers AI Markdown fallback - #21
Merged
Merged
Conversation
PDFs, Office documents, and HTML pages the built-in extractor cannot read now fall back to Workers AI's toMarkdown conversion. The converted Markdown is mapped into the same validated PreviewBlock shape, with unsafe links and images dropped; core stays platform-free behind an injected fallback, and FEEDREADER_AI_PREVIEW_FALLBACK=false disables it. Bump the preview cache key to v10 so previously cached empty previews get a fallback attempt. Bound non-HTML downloads at 5 MB, skip oversized documents using Content-Length, and race the AI call against 10 seconds. Strip PDF filename/page-marker headings and trailing JSON-LD. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
🔎 Cloudflare preview: https://ece1c824-feedreader.phatpham9.workers.dev Uploaded from |
When the AI fallback is enabled but the document exceeds the 5 MB cap (declared by Content-Length or discovered while reading), return too_large instead of unsupported_content_type so the client shows the size message. No fallback, unsupported formats, and failed conversions still report unsupported_content_type. Co-Authored-By: Claude Code <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
toMarkdownfallback for previews the built-in extractor cannot read: non-HTML documents (PDF/Office/CSV/XML), HTML pages with no readable blocks, and bodies truncated before content.fetchArticlePreviewaccepts an injected fallback,platforms/cloudflare/src/previewFallback.tsownsenv.AI, andcore/previewMarkdown.tsmaps the converted Markdown into the same validatedPreviewBlockshape.no_content,too_large, andunsupported_content_type.blocked,timeout,rate_limited,upstream_error, andempty_bodykeep their existing paths.Content-Length; oversized documents now reporttoo_large), 10-second AI timeout, existing 10-second fetch timeout, 120 blocks / 40k chars in the mapper. Kill switch:FEEDREADER_AI_PREVIEW_FALLBACK=false.preview:v10so previously cached empty previews get a fallback attempt.Verification
npm run typechecknpx vitest run --exclude '.claude/**'— 14 files, 119 testsnpm run db:migrate:localwrangler deploy --dry-run— 359.01 KiB / 95.58 KiB gzip200, 19 blocks, 1.8–3.0 s, 14,865 tokens502 too_largein ~0.2 s, no 5 MB download (Content-Length guard)200via AI in ~0.3 spr-21-6fbbc54, shared production D1):200, 86 blocks (4.3 s first extraction; cache hit afterwards)502 too_largein ~0.9 s502 blocked(upstream_status: 403; fallback correctly skipped)200, 100 blocks,cache-control: private, no-storeprivate, no-storeclient header still behave as before.Notes
🤖 Generated with Claude Code