KSES: Reimplement with Tag Processor

[dmsnell]

Trac ticket: Core-65984

Merge after #13273, which accounts for three of the failing tests.

• self-closing non-HTML elements
• [~] remove opening tag when required attributes are missing, and closing tag
  • while this would be a nice enhancement it’s going to be left out of this work to preserve existing behaviors. with the HTML Processor powering wp_kses(), it’s possible to simply wait until an opened element is closed based on depth, and skip that closing element if it exists.
• replace C0 controls with their escapes, rather than stripping them away
• replace C0 controls in attribute values?
• [~] handle incomplete parsing, including closing all open elements
  • plenty of existing code in Core calls wp_kses() with intentionally-incomplete input, for example, a wrapper opening tag with part of the content, separately from the closer. closing open elements does a good job of isolating content, but legacy behaviors depend too much on the more procedural use of wp_kses() so isolation cannot be reasonably added without mangling websites.
• if SVG or MATH are not allowed, the entire element should disappear
• remove default pre_kses filters but then call pre_kses

Notes

• Branch tip with HTML Processor bdbae3a

[github-actions]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
  it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props dmsnell, jonsurrell.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[sirreal]

I'm supportive of this direction overall. I'll leave some feedback on different points.

I started this review at 0b1535e but it moved while I was reviewing, so some of this review may be stale by the time I post.

---

General things to debug:

echo wp_kses( '<template>foo</template>', [ 'template' => true ] );
// Prints: </template>

echo wp_kses( '<div style>x</div>', [ 'div' => ['style' => true] ] );
// Prints: <div style="1">x</div>

echo wp_kses( '<div style="col&#x01;or:red">x</div>', [ 'div' => [ 'style' => true ] ] );
// Before (trunk): <div>x</div>
// After         : <div style="color:red">x</div>

---

This is an interesting one, the block delimiter is preserved (improvement) but for some reason wp:image becomes wp:core/image:

echo wp_kses( '<!-- wp:image {"alt":"<script>x</script>"} -->', [] );
// Before: &lt;!-- wp:image {&quot;alt&quot;:&quot;x"} --&gt;
// After:  <!-- wp:core/image {"alt":""} -->

---

Incomplete tokens are interesting.

KSES would leave something like a trailing <b this is lost as HTML text (it would escape <). Browsers do not do that and this PR does not do that, the <b… is dropped.

That means that partial tokens like the unclosed <b above but also "atomic elements" are stripped entirely:

echo wp_kses( '<p>keep me</p><b this is lost', ['p'=>[],'b'=>[]] ) . "\n";
echo wp_kses( '<p>keep me</p><title>this is lost', ['p'=>[],'title'=>[]] ) . "\n";
echo wp_kses( '<p>keep me</p><style>*{content:"this is lost";}', ['p'=>[],'title'=>[]] ) . "\n";
echo wp_kses( '<p>keep me</p><style>*{content:"this is lost";}', ['p'=>[],'style'=>[]] ) . "\n";
echo wp_kses( '<p>keep me', ['p'=>[]] ) . "\n";

Before/after diff of output:

diff --git 1/tmp/before.txt 2/tmp/after.txt
index 5122acd7a7..d87549cc15 100644
--- 1/tmp/before.txt
+++ 2/tmp/after.txt
@@ -1,5 +1,5 @@
-<p>keep me</p>&lt;b this is lost
+<p>keep me</p>
-<p>keep me</p><title>this is lost
+<p>keep me</p>
-<p>keep me</p>*{content:"this is lost";}
+<p>keep me</p>
-<p>keep me</p><style>*{content:"this is lost";}
+<p>keep me</p>
 <p>keep me

This is worth considering carefully. The HTML snippets KSES sees don't necessarily align with coherent boundaries. In particular echo wp_kses_post( 'a<b', [] ); is a<b on trunk (which seems reasonable) but on this branch its a.

We could align with the previous behavior, assume that the incomplete tokens are HTML text, and escape them as such. There may be negative implications to that, it's worth some careful consideration.

---

Foreign content is difficult, is this a case of integration points bailing because they're risky?

echo wp_kses( '<math><mtext>x</mtext></math>', ['math'=>[],'mtext'=>[]] );
// (no output)

---

</br> produces warnings.

echo wp_kses( '</br>', [ 'br' => true ] );
// Warning: foreach() argument must be of type array|object, true given in /var/www/html/wp-includes/kses.php on line 1499
// Warning: foreach() argument must be of type array|object, null given in /var/www/html/wp-includes/kses.php on line 1536
// Prints: <br>

First warning is the allowed tags. I used true for the allowed tags value.

This change produces a warning where before it did not:

wordpress-develop/src/wp-includes/kses.php

Line 1499 in 0b1535e

foreach ( $element_attributes as $name => $spec ) {

Maybe true is intentionally supported:

wordpress-develop/src/wp-includes/kses.php

Lines 1533 to 1537 in 049287a

	// Are any attributes allowed at all for this element?
	$element_low = strtolower( $element );
	if ( empty( $allowed_html[ $element_low ] ) || true === $allowed_html[ $element_low ] ) {
	return "<$element$xhtml_slash>";
	}

Either way, it's filterable and important to to follow whatever KSES did previously. I think if the value is true it should use an empty array (no attributes allowed).

Second warning is here:

wordpress-develop/src/wp-includes/kses.php

Line 1536 in 0b1535e

foreach ( $attribute_names as $name ) {

As it is now, this needs a null guard. I had proposed a fix at the tag processor level in #9657 that we could consider.