[MarkusNeusinger]

Summary

• /insights/related (the related-plots block on every spec and implementation page) cached its response per spec, library, mode and limit. Those thousands of keys competed for the shared 1,000-entry TTLCache with /plots/filter, /og/* and seo:*, so nearly every call reloaded all 325 specs and 4,863 impls (SpecRepository.get_all). Production over the LinkedIn weekend of 2026-10-18/19: p50 755 ms, p99 1.6 s over 3,134 calls, while every other endpoint had a p50 of 8–120 ms.
• The catalogue is now cached once as a plain-value index (_RelatedEntry: tags per mode and library plus the best preview, no ORM objects) under the single key insights:related_index. Each request ranks against that index in memory with the unchanged algorithm (same Jaccard, stable sort, seeded shuffle and jitter).
• Invalidation is unchanged: the index is cleared with the rest of the cache after a DB sync, and the 24 h TTL still applies. docs/reference/api.md gets a caching note and the correct limit range (1–24, which the docs gave as 1–12).

Measured on the live catalogue (read-only):

	Before	After
DB load per cache miss	get_all ≈ 1.1 s (local, almost every call)	once per cache lifetime, index build 16 ms
Ranking per call	2.0 ms	0.64 ms
Output		byte-identical for 1,266 combinations (325 specs in spec mode, 60 specs × 15 libraries in full mode, 40 specs in full mode with no library at limit 12, one unknown spec)

Plan

N/A. This is the Todoist finding from the traffic check on 2026-10-19. Second opinion: Fable and Gemini 4 Argon (results are summarized in a comment).

Test plan

• uv run pytest tests/unit tests/integration (CI extras test and agents): 12,227 passed.
• New tests:
  • Tag sets and preview choice of _related_entry. These replace the tests of the removed _collect_impl_tags.
  • A router test with the real cache: four different URLs (spec mode, limit, full mode with library, unknown spec) cause exactly one get_all.
• ruff check, ruff format --check, mypy api core, and tools.changelog check pass.
• Old and new code on the live catalogue (read-only): 0 differences in 1,266 cases.
• After deploy: confirm the /insights/related p50 in the request logs drops from about 750 ms to the level of the other endpoints.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M

[Copilot]

🟢 Approval recommended

The focused refactor preserves ranking and cache invalidation behavior, with relevant regression tests and no identified blocking issues.

No open issues 🎉

What changed in this PR

Replaces per-request related-plots caching with one shared catalogue index, reducing database reads while preserving recommendation behavior.

Changes:

• Caches plain-value catalogue entries and ranks results in memory.
• Adds tests for tag selection, preview selection, and cache reuse.
• Documents caching behavior and corrects the supported limit range.

File	Description
tests/​unit/​api/​test_routers.py	Verifies different requests share one catalogue load.
tests/​unit/​api/​test_insights_helpers.py	Tests index tag sets and preview selection.
docs/​reference/​api.md	Documents caching and the 1–24 limit range.
changelog.d/​related-shared-index.md	Records the performance improvement.
api/​routers/​insights.py	Introduces the shared index and in-memory ranking.

_{Review effort: Balanced}

[MarkusNeusinger]

Second opinion: Fable and Gemini 4 Argon

Both returned sound with changes. Both confirmed:

• The ranking is equivalent: same stable sort, same seeded shuffle, same tie order within one get_all() snapshot.
• A ranking request cannot race clear_cache(), because _build_related is synchronous and holds its own list reference.
• The index key stays hot in the LRU, because every related call touches it. Under a crawler burst the worst case is one reload, not one reload per call.

Adopted in f0448d13e:

• Stale-while-revalidate on the index key (refresh_after=settings.cache_refresh_after plus a get_db_context refresh factory), as specs_list, specs_map, dashboard and potd already do. Staleness is now capped at about one hour even if a post-sync flush is missed, and a TTL expiry no longer costs a live request the full DB load. (Both reviewers.)
• Impl tags are unioned per library. The unique key is (spec, language, library), and the dict comprehension kept only the last impl. No such pair exists in today's catalogue. (Fable, with a test.)
• eq=False on the frozen dataclass, so the dict field never meets a synthesized __hash__. (Both reviewers.)
• The router test also asserts that clear_cache() forces a reload, which pins the invalidation contract. (Fable.)

Not adopted:

• Startup prewarm of the index (Gemini). The API runs one instance that restarts about once a day, so the saving is a single slow request per day.
• Clearing the index in clear_spec_cache (Gemini). The old per-response keys were not cleared there either, and the post-sync flush is clear_cache().
• A library_id tie-break for equal-score previews (Gemini). It would change output that is byte-identical today.
• Reusing the index for /plots/filter (both reviewers advise against doing it in this PR). It needs different fields.

Rerun on the live catalogue after the changes: still 0 differences in 1,266 cases, ranking 0.73 ms per call.

[codecov]

Codecov Report

❌ Patch coverage is 94.73684% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
api/routers/insights.py	94.73%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!