fix: parse XML bodies whose elements are named after HTML void elements - #22
Open
quirky4 wants to merge 1 commit into
Open
fix: parse XML bodies whose elements are named after HTML void elements#22quirky4 wants to merge 1 commit into
quirky4 wants to merge 1 commit into
Conversation
…ts (DEF-51266) The XML request-body decoder ran with AutoClose = xml.HTMLAutoClose, so it self-closed every element on Go's HTML void list. param, link, input and col are ordinary container elements in XML-RPC, Atom and SOAP vocabularies, and once the decoder had invented their end tag, their real one aborted decoding with "unexpected end element". ProcessRequest returned before populating the collection, leaving REQUEST_XML empty and REQBODY_ERROR set, so every rule keyed on XML variables matched nothing: XML-RPC brute-force, pingback and multicall-amplification detection never fired, and the captcha and RBL blocking downstream of it never triggered. Nothing depended on AutoClose for leniency. readXML reads attribute values off StartElement and text off CharData and never inspects EndElement, so self-closing an element changed no member it collects; the only effect it had here was turning a document's own end tag into a stray one, which Strict false does not forgive because it invents missing end tags rather than tolerating extra ones. What the flexibility for malformed payloads actually rests on is untouched: an element the body never terminates is closed by an ancestor's end tag under Strict false, and a body cut short is tolerated by the unexpected EOF branch. Across 738 malformed, truncated and HTML-shaped payloads parsed both ways, none lost an attribute or text node and 35 fewer failed to parse. AutoClose entered in corazawaf#622, which added flexible XML body processing for malformed payloads and shipped no test exercising it. The defect is still present upstream at HEAD and is reported, with the same wp.getUsersBlogs body used in the regression test here, at corazawaf#1441, where the cause has not been identified and reporters instead disable CRS rules 200000 and 200002. corazawaf#1452, in this fork since f18e237, added the unexpected EOF tolerance but does not cover unexpected end element. The XML:// XPath selectors these rules would rather key on remain unsupported, so this restores the //@* and /* keys only: corazawaf#1322. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes DEF-51266.
Problem
The XML request-body decoder ran with
dec.AutoClose = xml.HTMLAutoClose, whichself-closes every element on Go's HTML void list — including
param,link,inputandcol. Those are ordinary container elements in XML-RPC, Atom andSOAP, so once the decoder invented their end tag, the document's real end tag
arrived as a stray one and decoding aborted:
XML syntax error on line 5: unexpected end element
ProcessRequestreturned that error before reachingcol.Set, soREQUEST_XMLstayed empty and
REQBODY_ERRORwas set. Every rule keyed on XML variablesmatched nothing: XML-RPC brute-force, pingback and multicall-amplification
detection (77317950/51, 77350179/81, 77317980, 77231011) fired ~0 times on the
Coraza fleet against millions on Apache/LiteSpeed, and the XMLRpcBruteforce
captcha and www-brute RBL blocking downstream never triggered. In practice this
means essentially every WordPress
xmlrpc.phpcall was uninspected.Fix
Remove the one line.
Strict = falseandEntity = xml.HTMLEntityare unchanged.Does this reduce the leniency PR corazawaf#622 added?
No — it strictly increases it. This is the main thing worth reviewing, since the
line was added deliberately to be lenient with malformed payloads.
readXMLcollects attribute values offStartElementand text offCharData.It never inspects
EndElementand never builds a tree, so self-closing anelement cannot change any member it collects.
AutoClose's only observableeffect here was on the error path — and
Strict = falsedoes not forgive astray end tag, because it invents missing end tags rather than tolerating
extra ones.
The flexibility for malformed payloads rests on two mechanisms this PR does not
touch:
Strict = falselets an unterminated element be closed by an ancestor'send tag, and the
unexpected EOFbranch tolerates a body cut short.Measured over 738 payloads (hand-written malformed/HTML-shaped bodies plus
deterministic mutations — truncations, byte deletions, injected
<br>), parsedboth with and without
AutoClose:AutoCloseTesting
TestXMLHTMLVoidElementNames— XML-RPCwp.getUsersBlogsandsystem.multicall, Atom<link>, plus two rows pinning that unterminatedvoid elements still decode (the invariant the fix relies on).
xmlrpc_multicallrow added to the existingTestXMLOrdinaryDocumentsAcceptedtable, which goes through
ProcessRequest— the bug was thatProcessRequestreturned before populating, which areadXML-level testcannot catch.
AutoClosefails all four regression cases, thecollection-level one reporting
rejected a 16794 byte document holding 0 members— the ticket's symptom verbatim.go vetandgofmtclean.Out of scope
The
XML://methodName/text()XPath selectors the rules would rather key on arestill unsupported (only
//@*and/*keys exist) — this is the engine halfonly; the rule-side rewrite is a separate WPT item, upstream
#1322.
Upstream
corazawaf/corazaHEAD; no open upstream PR touchesinternal/bodyprocessors/xml.go.bug with the same
wp.getUsersBlogspayload, but the root cause was neveridentified there —
AutoCloseis mentioned nowhere upstream (no issue, PR ordiscussion). Reporters work around it by disabling CRS rules 200000/200002.
add "flexible XML body processing for malformed payloads", with no test
exercising it.
f18e237, addedunexpected EOFtolerance but does not coverunexpected end element.An upstream PR and a root-cause comment on corazawaf#1441 are follow-ups.