feat(seo): real bot content for the five doc pages

[MarkusNeusinger]

Summary

• The bot renderings of /libraries, /stats, /about, /legal and /palette were 29–59-word title+description stubs — a crawler following llms.txt's own link descriptions ("the fifteen supported plotting libraries", …) found none of the promised content (AI-access audit 2026-08-19; confirmed by the machine-files verifier in the post-deploy sweep).
• Each page now derives its body from its single source of truth, so none can drift: /libraries renders the full registry from core/constants.py (grouped by language, with versions, descriptions, doc links); /stats shows the live catalogue counts via the same cached /stats data the startup prewarm fills (registry-derived fallback without a DB); /about carries the pipeline story mirroring AboutPage.tsx with registry-derived counts; /legal carries the operator/privacy/hosting facts mirroring LegalPage.tsx; /palette lists all 8 categorical hexes plus the amber/neutral/muted anchors straight from core/palette.py.

Plan

One of the three "for later" items from the AI-access verification sweep, done on request. Independent of #10490 (different seo.py hunks) and of the nginx charset PR.

Test plan

• ruff check + ruff format --check clean; mypy api core clean; 1,751 unit + 67 integration tests pass (new/updated tests assert the registry list per language, the palette hexes from the module, operator/privacy facts, the pipeline copy, and both stats bodies — live counts with DB, registry fallback without)
• After merge + deploy: curl -A Googlebot https://anyplot.ai/libraries lists all 15 libraries; /stats shows the live counts; /palette carries #009E73

[Copilot]

Pull request overview

This PR upgrades the bot-rendered (crawler) HTML for five documentation-style pages (/libraries, /stats, /about, /legal, /palette) so their bodies contain real, indexable content derived from the same canonical sources the app uses (registry, stats cache, and palette module), reducing drift between human and crawler experiences.

Changes:

• Build crawler bodies for /libraries, /about, /legal, and /palette from canonical in-repo sources (registry and palette module) and inject them into the bot HTML template.
• Update /stats bot rendering to reuse the same cached stats computation as the JSON /stats endpoint (with a registry-derived fallback when no DB is available).
• Expand unit tests to validate the new bot page bodies and add an [Unreleased] changelog entry describing the SEO/crawler-facing improvement.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
api/routers/seo.py	Adds registry-/palette-/stats-backed crawler bodies for the five pages; /stats now uses the cached stats machinery when DB is present.
tests/unit/api/test_routers.py	Adds/updates unit tests asserting the crawler HTML contains the intended real content for the five pages (plus DB/no-DB behavior for stats).
CHANGELOG.md	Documents the crawler-content upgrade under [Unreleased].

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (1)

api/routers/seo.py:1119

• Inside seo_stats, the nested _fetch coroutine is missing a return type annotation, which breaks the file’s otherwise-consistent typing pattern for these cache factories (and makes it harder for type checkers to infer the cached value type). Add an explicit return type (and a local import if needed).

    if db is not None:

        async def _fetch():
            return await _compute_stats(db)

[codecov]

Codecov Report

❌ Patch coverage is 92.00000% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
api/routers/seo.py	92.00%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (2)

CHANGELOG.md:145

• This changelog entry says the five bot pages are each derived from a "single source of truth" so they "cannot drift", but /about and /legal are hard-coded HTML fragments in api/routers/seo.py (mirroring TSX, not derived). Please reword this bullet to avoid claiming drift-proof derivation for the about/legal copy.

  content (AI-access audit 2026-08-19). They now serve the full library registry (derived from
  `core/constants.py`), the live catalogue counts (reusing the cached `/stats` data), the
  pipeline story, the operator/privacy/transparency facts, and the actual palette hex values
  (straight from `core/palette.py`) — each derived from its single source of truth so the pages
  cannot drift (#10491).

api/routers/seo.py:1064

• _PALETTE_BOT_BODY is built at import time and uses zip(..., strict=True). A future palette/slot-name length mismatch would raise at import and prevent the API from starting, even though this is just crawler HTML. Consider making this non-fatal (and let tests enforce the invariant) by dropping strict=True or handling mismatch explicitly.

    + "".join(
        f"<li>{name} <code>{hex_value}</code></li>"
        for name, hex_value in zip(_PALETTE_SLOT_NAMES, palette.IMPRINT, strict=True)
    )