Sitemaps: Don't return a 404 status for valid sitemap requests when the main query has no posts

[i-am-chitti]

Ticket

Trac ticket: https://core.trac.wordpress.org/ticket/65945

Summary

• On a site with zero published posts, /wp-sitemap.xml renders correct XML but responds with HTTP 404, so search engines discard it.
• WP::handle_404() 404s the main query when it has no posts and no exception applies. Sitemap requests aren't in that list — they were only shielded by falling through to is_home = true, which r62664 (Core-51542, Core-51543) correctly removed.
• Same class of issue as Core-39157, fixed for feeds in 4.7.3.

Changes

1. WP::handle_404() — exempt sitemaps alongside the other machine-readable endpoints. Placed in the first branch rather than next to is_feed(), since provider routes map paged from the URL and so never reach the elseif ( ! is_paged() ) branch.
2. WP_Sitemaps::render_sitemaps() — 404 explicitly for an unregistered provider, which previously relied on handle_404() doing it.
3. WP_Sitemaps::render_sitemaps() — 404 explicitly for an unrecognized stylesheet type.
4. WP_Sitemaps::render_sitemaps() — 404 explicitly for a route whose query vars do not survive sanitizing.
5. WP_Sitemaps::render_sitemaps() — send nocache_headers() with every sitemap 404.

Changes 2 through 4 are consequences of change 1 rather than cleanup. Once handle_404() stops deciding the status for these requests, anything it now exempts and render_sitemaps() declines to serve is left as a 200:

• /wp-sitemap-bogus-1.xml matches the provider rewrite rule and would fall through to the theme.
• /does-not-exist/?sitemap-stylesheet=1 reaches WP_Sitemaps_Stylesheet::render_stylesheet(), which sends an XML content type, echoes nothing for a type other than sitemap or index, and exits — a 200 with an empty body.
• /does-not-exist/?sitemap=<> and /does-not-exist/?sitemap-stylesheet[]=sitemap are exempted on the raw query var — WP_Query::$is_sitemap is ! empty() and the stylesheet check is bare truthiness — but render_sitemaps() acts on the sanitized value, which is empty in both cases, so it bailed without a status. The early bail now tests the same raw vars handle_404() does, so the two agree on what a sitemap route is.

All of these are crawlable soft 404s on arbitrary URLs, and all returned 404 on trunk.

Change 5 restores something handle_404() was providing. Its 404 branch calls nocache_headers(); the branch that declines to 404 sends only status_header( 200 ). A sitemap 404 is frequently transient — /wp-sitemap-posts-post-9.xml is a 404 only until the site has nine pages of posts — and intermediaries commonly cache 404s, so a bare 404 can outlive the condition that produced it. The three lines are collected into a private send_404() helper and called from each bail.

The early bail is also moved above the sanitizing, since render_sitemaps() runs on template_redirect for every front-end request and on nearly all of them both query vars are empty. The sanitizing itself is untouched, byte for byte as trunk has it.

Note that this leaves three pre-existing expects string, mixed given reports on the relocated sanitize_text_field() calls, since get_query_var() returns mixed. Tightening that is worth doing, but sanitize_text_field() is much looser than these values allow — the rewrite rules constrain a provider name to [a-z]+, a subtype to [a-z\d_-]+, and a stylesheet type to sitemap or index — so the right fix is narrower validation, not a cast. That belongs with the 7.2 work below rather than a point release.

The empty-URL-list branch keeps its existing behavior, so Core-61293 is unaffected. The production change is confined to WP_Sitemaps::render_sitemaps(), one new private helper, and the three-line handle_404() exemption.

Known behavior, not addressed here

Because sitemap and sitemap-stylesheet are public query vars, they can be appended to any path, so /no-such-page/?sitemap=index now returns the sitemap index with a 200 where trunk returned the same body with a 404. Requiring that the request matched a sitemap rewrite rule would fix that, but it would also break sitemaps on sites without pretty permalinks, where no rewrite rules apply and /?sitemap=index is the only access path — a case the ticket explicitly relies on. Narrowing this belongs in a major release, not a point release.

Deferred to 7.2: converting these bails to wp_die()

An earlier revision of this PR replaced the set_404() bails with wp_die( $message, 404 ), giving each failure a descriptive message instead of the theme's generic 404. That remains desirable — the current responses tell an operator nothing about why a sitemap URL failed, and "no such provider" versus "provider registered but empty" is exactly the distinction someone debugging a missing sitemap needs. The intent is to revisit it for 7.2.

It was reverted here because it is not needed to fix Core-65945 and does not belong in a minor release:

• New strings. The wp_die() form adds three translatable strings. 7.1.1 is a minor release, so they would ship untranslated in every locale.
• wp_die() ends the request inside template_redirect. Later callbacks on that hook, template_include, and the theme's 404.php never run. Sites currently serving a themed 404 for these URLs would start getting core's error page, and page-cache and 404-logging plugins hooked after priority 10 would stop seeing the requests.
• The response isn't always HTML. wp_die() dispatches through _xml_wp_die_handler() whenever wp_is_xml_request() is true — i.e. whenever the client sends Accept: text/xml or application/rss+xml, both plausible for a sitemap fetcher. That handler emits an <error><message> document and applies htmlspecialchars() on top of the escaping already applied, so a provider or subtype containing & renders double-escaped.

None of that is disqualifying for a major release, where the behavior change can be evaluated on its own merits and the strings land with a full translation cycle. It is disqualifying for a point-release bug fix.

Test

tests/phpunit/tests/wp/handle404.php (12 tests): index, provider, paged, taxonomy, user and stylesheet routes with no posts, plus guards that unknown URLs and unregistered providers still 404.

tests/phpunit/tests/sitemaps/sitemaps.php: the two pre-existing should_return_404 tests are unchanged from trunk. Cases are added for every route handle_404() now defers on — empty URL list for a paged subtype, unregistered provider, unrecognized stylesheet type, and three data-provider cases for query vars that do not survive sanitizing. Each asserts both halves of the interaction: is_404() false after go_to(), proving the exemption applied, then true after render_sitemaps().

• --group sitemaps — 150 tests, OK
• --group wp --group query --group feed --group canonical --group rewrite — 1393 tests, OK

Screenshots

The 404 responses, unchanged from trunk — the theme's own 404 template still renders:

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Claude Opus 5
Used for: Root-cause investigation, tracing history, and drafting the fix and tests.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props iamchitti, westonruter, adamsilverstein.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[github-actions]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
  it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

[adamsilverstein]

Please note the RC for 7.1.1 is September 10th if we want it included in that version. Otherwise it is fine to punt to 7.1.2.

[westonruter]

The tests seem like they need to be updated to listen for calls to wp_die() instead of looking at is_404().

[westonruter]

Sorry for the back-and-forth on this. I think for 7.1.1 we'll go back to just the theme's 404 template as it is a smaller change with less risk, and then in 7.2 we can improve the error handling with wider testing. I'm going to have Claude share some review feedback that led me to this.

[westonruter]

🤖 Comment by Claude Opus 5

A medium-effort review of the current diff. The root-cause analysis holds up, and change 1 — exempting sitemap requests in WP::handle_404() — looks correct and well placed: provider routes do map paged from the URL, so the elseif ( ! is_paged() ) branch would indeed never be reached.

The main suggestion is that change 2 be split, so 7.1.1 carries only what the fix requires.

The explicit 404 for an unregistered provider is load-bearing

This is worth stating plainly, because it reads like cleanup and is not. WP_Query::$is_sitemap is set from the bare presence of the query var (class-wp-query.php:828), so it is true for any value. /wp-sitemap-bogus-1.xml matches the provider rewrite rule and sets sitemap=bogus, so it is exempted by the new handle_404() branch — which would turn a request that currently 404s into a 200 if if ( ! $provider ) { return; } were left as-is. That branch does have to gain an explicit status.

But wp_die() is not required to do it, and carries costs

The blocking concern is new translatable strings: three are added here, and 7.1.1 is a minor release, so they would ship untranslated for every locale. Beyond that, wp_die() changes the request lifecycle — see the inline notes.

A smaller form for 7.1.1 would keep the idiom already used twice in this same method:

if ( ! $provider ) {
	$wp_query->set_404();
	status_header( 404 );
	return;
}

That leaves the sitemaps-disabled and empty-URL-list branches exactly as trunk has them, and holds the wp_die() conversion for 7.2, where it can be evaluated on its own merits rather than as a rider on a point-release bug fix.

Checked and cleared

• wp_die( $msg, 404 ) does set a 404 — the int $title is converted to response inside wp_die() itself.
• redirect_canonical() does not start redirecting sitemap URLs now that is_404() is false; the explicit sitemap / sitemap-stylesheet guard at canonical.php:709 already covers that.
• Dropping global $wp_query and its @global tag is correct as the diff stands, since no other use remains — though it would need restoring under the suggestion above.
• A concern that the new wp_die() might break SEO plugins routing through the shared sitemap query var was investigated and does not apply. Yoast (inc/sitemaps/class-sitemaps.php:102) renders on pre_get_posts at priority 1, and Rank Math (includes/modules/sitemap/class-router.php:35) on parse_query at priority 1; both exit before template_redirect fires at all.

Minor

The PR description states that "The sitemaps-disabled branch still uses set_404()", which no longer matches the diff.

[Copilot]

🟢 Approval recommended

The changes are narrowly scoped, align with the described behavior, and are backed by targeted tests covering both the exemption and the relocated 404 decisions.

Pull request overview

Fixes sitemap endpoints returning HTTP 404 when the main query is empty (e.g., sites with zero published posts), by exempting sitemap requests from WP::handle_404() and ensuring sitemap-specific 404s are issued from the sitemaps component itself.

Changes:

• Update WP::handle_404() to never set a 404 for sitemap or sitemap-stylesheet requests.
• Update WP_Sitemaps::render_sitemaps() to explicitly 404 for invalid/unservable sitemap routes (unregistered provider, unrecognized stylesheet type, and values that don’t survive sanitizing), centralizing sitemap 404 behavior.
• Add/extend PHPUnit coverage for sitemap + stylesheet routes to verify the handle_404() exemption and that render_sitemaps() still produces correct 404 outcomes.

File summaries

File	Description
tests/phpunit/tests/wp/handle404.php	New tests asserting sitemap/stylesheet requests aren’t 404ed by WP::handle_404() when the main query is empty.
tests/phpunit/tests/sitemaps/sitemaps.php	Adds tests ensuring render_sitemaps() still sets 404 for invalid sitemap/stylesheet routes now exempted from handle_404().
src/wp-includes/sitemaps/class-wp-sitemaps.php	Adds a centralized send_404() helper and explicit 404 bails in render_sitemaps() for invalid sitemap routes.
src/wp-includes/class-wp.php	Exempts sitemap and sitemap-stylesheet requests from WP::handle_404()’s default 404 behavior.

Review details

Suppressed comments (1)

tests/phpunit/tests/wp/handle404.php:51

• The data provider key label is missing a space after the comma, which reads like a typo and is inconsistent with the other labels in the same provider.

			'taxonomies provider,paged' => array( '/?sitemap=taxonomies&sitemap-subtype=category&paged=3' ),

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[westonruter]

Note, when testing this, it raised a question for me:

Something else I'm seeing: When there are no URLs, the /wp-sitemap.xml response will have a link to /wp-sitemap-posts-post-1.xml even when there are no URLs. When navigating to that URL, I am then served a 404 from the theme unexpectedly. I would have expected an XML response which tells the crawler that they successfully accessed the sitemap URL, but there are no URLs currently to index. Right? Is it valid to have a <urlset> with no children?

🤖 Claude Opus 5 analysis:

Three separate answers, all verified.

1. No — an empty <urlset> is not schema-valid. From the actual XSD at sitemaps.org/schemas/sitemap/0.9/sitemap.xsd:

<xsd:sequence>
  <xsd:any namespace="##other" minOccurs="0" maxOccurs="unbounded" processContents="strict"/>
  <xsd:element name="url" type="tUrl" maxOccurs="unbounded"/>
</xsd:sequence>

minOccurs is omitted on url, so it defaults to 1. At least one <url> is required. (Note the contrast with the xsd:any right above it, which sets minOccurs="0" explicitly — so the omission on url is deliberate, not an oversight.)

2. Your instinct is nonetheless the consensus — and it's already an accepted ticket. Core-61293, "Display an empty sitemap instead of 404," milestone 7.2, has-patch, reported by Sybre Waaijer of The SEO Framework. Their argument is exactly yours, with the schema caveat folded in: an empty sitemap makes Google Search Console say "Sitemap can be read, but has errors" rather than "Sitemap could not be read" — errors because the empty urlset is invalid, but "readable with errors" is a far more actionable signal for a site owner than a hard 404. So the proposal knowingly trades schema validity for diagnosability.

That's why this PR deliberately leaves the empty-URL-list branch alone, and why the description says so — it's 61293's call to make, in 7.2, where the milestone already is.

3. The specific case you describe doesn't come from core alone. I checked: with zero posts of every type, WP_Sitemaps_Posts::get_max_num_pages( 'post' ) returns max( 0, 0 ) === 0, so get_sitemap_entries()'s loop never runs and the index contains no posts-post entry at all. Emptying every post type in a test and dumping the index gives exactly one entry:

show_on_front=posts
INDEX: array ( 0 => 'http://example.org/?sitemap=posts&sitemap-subtype=page&paged=1' )

That one is listed because of the $min_num_pages = 1 special case at class-wp-sitemaps-posts.php:211, and it is not empty — line 108 prepends home_url( '/' ) to page 1 when show_on_front is posts. So it returns 200 with one URL.

For the index to advertise a sitemap that then 404s, something has to make get_max_num_pages() and get_url_list() disagree — which is precisely Cybr's scenario in 61293: a plugin filtering entries out for indexability while the page count still counts the underlying posts. So if you're seeing a link to wp-sitemap-posts-post-1.xml on a site with no posts, I'd look for a plugin on wp_sitemaps_posts_pre_max_num_pages or wp_sitemaps_posts_query_args — or a stale index, since these responses are cacheable. Worth confirming which site you saw it on; if it reproduces on a clean install I'd want to see it, because that would be a distinct bug from 61293.

[github-actions]

A commit was made that fixes the Trac ticket referenced in the description of this pull request.

SVN changeset: 63570
GitHub commit: 285d4d1

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.