Harden XML parsing via commons-secure-xml - #766
Conversation
Create XmlStringLookup's document builder and XPath factories through org.apache.commons:commons-secure-xml. The secure factories enable FEATURE_SECURE_PROCESSING and install a non-removable entity-resolver floor on every parser they produce: external DTD and entity lookups are resolved to empty content instead of being fetched, and internal entity expansion is bounded, regardless of the JAXP implementation on the classpath. Changes: - Add the commons-secure-xml dependency (1.0.0-SNAPSHOT until its first release). - Route factory creation through SecureDocumentBuilderFactory and SecureXPathFactory in XmlStringLookup when the instance's feature map enables secure processing. The documented opt-outs keep their meaning: feature maps without secure processing, and the standard javax.xml.accessExternalDTD system property (which the secure factory's resolver floor would otherwise ignore), keep using the plain JAXP factories, so external entity resolution can be restored where it is wanted. - Adapt the secure-path tests to the secure contract: a parser may either reject a document with an external reference or parse it with the reference resolved to empty content; the tests now assert that the external content does not leak into the result instead of expecting one fixed failure mode. - Run the CI and CodeQL builds with -Puse-apache-snapshots (inherited from the org.apache:apache parent POM) so the commons-secure-xml SNAPSHOT resolves; CodeQL's autobuild receives the profile through MAVEN_ARGS. Assisted-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MHgnMnGWHQoH2zD2jFdoMT
68e8cc3 to
f758b18
Compare
Bump org.apache.commons:commons-secure-xml from 1.0.0-SNAPSHOT to 1.0.0 and add the temporary staging repository https://repository.apache.org/content/repositories/orgapachecommons-1962/ after Central, so the vote gets downstream CI results. Drop the -Puse-apache-snapshots profile from the CI workflows, which the release version no longer needs. Remove the staging repository once 1.0.0 is released. Assisted-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0167e29ScPEdfzJnEFm95imK
| // The secure factory installs a non-removable resolver floor that ignores the JAXP access properties, | ||
| // so the documented opt-outs keep a plain factory: a feature map without secure processing, or the | ||
| // standard javax.xml.accessExternalDTD system property re-allowing external access. | ||
| final boolean secure = Boolean.TRUE.equals(xmlFactoryFeatures.get(XMLConstants.FEATURE_SECURE_PROCESSING)) |
There was a problem hiding this comment.
This looks like an attempt by the agent to keep the *ExternalEntityOn tests working. Those tests rely on the specifics of the hardening to allow fetching external entities. On the built-in JDK parser:
- Setting FSP to
false, - or setting
javax.xml.accessExternalDTDto "all",
is enough to reenable external entity fetching.
Commons Secure XML use a resolver to block the fetch instead, so the technique above will fail, since the resolver will block before javax.xml.accessExternalDTD is even evaluated.
I will of course clean up these (they don't make sense), but we might need a way to allow users to specify external fetches. My proposal is to use a resolver that uses the nested PathFence to determine if the fetch is allowed or not. WDYT?
There was a problem hiding this comment.
So the test should configure the objects for its own needs and leave the main code to factory class changes only, right?
There was a problem hiding this comment.
Yes, although right now there is not way to have a test with “external entities on”.
In one of the tests I see a property XmlStringLookup.secure being set, but I don't see that property being used anywhere in the code: was there every such a property?
There was a problem hiding this comment.
Interesting, I see it referred to from Javadoc comments at
and
but not from code, which tell me the comments must be left over from an earlier implementation or is was an incomplete implementation.
There was a problem hiding this comment.
Should we implement XmlStringLookup.secure then? Or a resolver that uses PathFence? Or maybe we should use both?
There was a problem hiding this comment.
Well, right now, you do get secure processing enabled by default in master, and unlike what the Javadoc says, it doesn't appear you can turn it off with a sys prop, BUT, you can set custom features (StringLookupFactory.xmlStringLookup(Map<String, Boolean>, Path...) which ends up calling setFeature()) so you can shoot yourself in the foot to your heart's desire.
The simple path would be:
- Update the Javadoc to reflect the code: no sys prop, unless you can find it used ;)
- Redo the main side of this PR to use Commons Secure XML cleanly and without leaving cruft behind
For example, I think the statics DEFAULT_* can go away
Path fences are mixing concerns for this PR. We want to integrate Secure XML and then see if we need improvements in Secure XML and/or call sites, later if possible.
| try { | ||
| final String result = lookup.get(); | ||
| assertFalse(result != null && result.contains(external), () -> "external content leaked: " + result); | ||
| } catch (final IllegalArgumentException e) { |
There was a problem hiding this comment.
@ppkarwasz Is this expected? If so, use assertThrows().
| try (InputStream inputStream = Files.newInputStream(getPath(documentPath))) { | ||
| final Document doc = dbFactory.newDocumentBuilder().parse(inputStream); | ||
| final XPathFactory xpFactory = XPathFactory.newInstance(); | ||
| final XPathFactory xpFactory = Boolean.TRUE.equals(xPathFactoryFeatures.get(XMLConstants.FEATURE_SECURE_PROCESSING)) |
2 participants
Warning
This PR was submitted automatically to smoke-test
Apache Commons Secure XML
and has not yet been verified by a human.
It will stay a draft until a committer reviews it and marks it ready.
Creates
XmlStringLookup's document builder and XPath factories throughorg.apache.commons:commons-secure-xml(1.0.0-SNAPSHOT until its first release) on the default path. The secure factories enable XML secure processing and install a non-removable entity-resolver floor: external DTD and entity lookups are resolved to empty content instead of being fetched, and internal entity expansion is bounded. The documentedXmlStringLookup.secure=falsesystem property keeps its meaning: that path still uses the plain JAXP factories, so external entity resolution can be restored where it is wanted.The secure-path tests are adapted to the secure contract: a parser may either reject a document with an external reference or parse it with the reference resolved to empty content, so they now assert that the external content does not leak into the result instead of expecting one fixed failure mode. CI and CodeQL run with
-Puse-apache-snapshotsso the SNAPSHOT dependency resolves.🤖 Generated with Claude Code