Skip to content

Site-rot check: hidden pages and the sitemap (owner-ratified rule, Oct 22) - #182

Merged
fosterstack-admin merged 6 commits into
mainfrom
www/noindex-sitemap-rule
Oct 22, 2026
Merged

fosterstack-admin merged 6 commits into
mainfrom
www/noindex-sitemap-rule

Conversation

@fosterstack-admin

Copy link
Copy Markdown
Contributor

Builds the owner's ratified rule (ops/docs/ratify/2026-10-22-www-noindex-and-sitemap.md), in bin/check-site-rot.py, run by the existing Public-repo hygiene workflow. No workflow edit, no new workflow.

Rule, word for word: "No page that's hidden from search may be listed in the sitemap. On the live site, no page except the 404 page may be hidden from search."

  • Hidden = a robots or googlebot meta tag with noindex or none, or an X-Robots-Tag noindex/none rule in _headers whose path pattern matches the page.
  • Sentence 1 always applies: a hidden page listed in sitemap.xml fails ("hidden: ..."). A hidden page is no longer required to be in the sitemap.
  • Sentence 2 applies with --live, which the script turns on by itself when GitHub's own GITHUB_BASE_REF is main (pull request based on main) or GITHUB_REF is refs/heads/main. 404.html is exempt. A draft stack based on another branch stays testable.
  • Tests first (bin/test-check-site-rot.sh): 17 new cases (sentence 1, sentence 2, each way of hiding, the 404 exemption, the three environment cases); all 80+ self-tests pass; the current site still passes (48 pages, 0 problems).
  • Checked on the other branches (not changed here): the preview www/v030-revamp fails both sentences (27 hidden pages, all listed); the release-day branch (RELEASE DAY (do not merge before v0.3.0 is live): remove noindex and draft wording #181) passes both (56 pages, 0 problems).

…the 404 page may be hidden (owner's rule, Oct 22)
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Deploying www with  Cloudflare Pages  Cloudflare Pages

Latest commit: d5ea07e
Status: ✅  Deploy successful!
Preview URL: https://94121ea7.www-4rw.pages.dev
Branch Preview URL: https://www-noindex-sitemap-rule.www-4rw.pages.dev

View logs

@fosterstack-admin
fosterstack-admin merged commit b84e7c7 into main Oct 22, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant