Problem in brief
HtmlUnit sets the Fetch Metadata and Client Hint request headers as fixed defaults in WebClient.addDefaultHeaders(...) so requests end up with wrong header values, or with headers a real browser would never send.
1. Sec-Fetch-Site is always same-origin
Should be none for a user-initiated top-level navigation, and cross-site / same-site for cross-origin requests.
2. Sec-Fetch-Mode is always navigate
Should also reflect the request mode like cors and no-cors.
3. Sec-Fetch-User is sent on every request
The spec says to only send it on navigation requests that had user activation, and to omit it otherwise. HtmlUnit hardcodes ?1 on all requests, so images, CSS, XHR, etc. get a Sec-Fetch-User header that a real browser would never send.
4. Sec-Fetch-Dest is always document
Should match the request destination.
5. Headers are also sent over plain HTTP
Specs say these headers only go to potentially trustworthy origins. HtmlUnit adds them regardless of scheme.
Specs: https://www.w3.org/TR/fetch-metadata/
Problem in brief
HtmlUnit sets the Fetch Metadata and Client Hint request headers as fixed defaults in
WebClient.addDefaultHeaders(...)so requests end up with wrong header values, or with headers a real browser would never send.1.
Sec-Fetch-Siteis alwayssame-originShould be
nonefor a user-initiated top-level navigation, andcross-site/same-sitefor cross-origin requests.2.
Sec-Fetch-Modeis alwaysnavigateShould also reflect the request mode like
corsandno-cors.3.
Sec-Fetch-Useris sent on every requestThe spec says to only send it on navigation requests that had user activation, and to omit it otherwise. HtmlUnit hardcodes
?1on all requests, so images, CSS, XHR, etc. get aSec-Fetch-Userheader that a real browser would never send.4.
Sec-Fetch-Destis alwaysdocumentShould match the request destination.
5. Headers are also sent over plain HTTP
Specs say these headers only go to potentially trustworthy origins. HtmlUnit adds them regardless of scheme.
Specs: https://www.w3.org/TR/fetch-metadata/