Add JA4 TLS client fingerprinting (core, BSD-scoped) and fix GREASE filtering#20
Add JA4 TLS client fingerprinting (core, BSD-scoped) and fix GREASE filtering#20kriszyp wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request implements core JA4 TLS client fingerprinting and blocking support across the Rust and TypeScript layers, including configuration options, event emissions, comprehensive tests, and documentation. The review feedback highlights two important spec-compliance and robustness improvements in src/sni.rs: first, correcting the ALPN character mapping logic to use the hex representation of the ALPN value when non-alphanumeric characters are encountered; second, checking if sig_algs is not empty rather than relying on has_sig_algs to prevent a potential trailing underscore in the hash input.
| let alpn_chars: String = match &alpn_first { | ||
| None => "00".to_string(), | ||
| Some(proto) if proto.is_empty() => "00".to_string(), | ||
| Some(proto) => { | ||
| let to_alnum = |b: u8| -> char { | ||
| let c = b as char; | ||
| if c.is_ascii_alphanumeric() { c.to_ascii_lowercase() } else { '9' } | ||
| }; | ||
| format!("{}{}", to_alnum(proto[0]), to_alnum(proto[proto.len() - 1])) | ||
| } | ||
| }; |
There was a problem hiding this comment.
The current ALPN character mapping logic replaces non-alphanumeric characters with '9'. However, according to the JA4 specification, if the first or last byte of the first ALPN is not an ASCII alphanumeric character, the first and last characters of the hex representation of the entire first ALPN value must be used instead. This ensures compliance with the JA4 spec for custom or non-standard ALPN protocols.
| let alpn_chars: String = match &alpn_first { | |
| None => "00".to_string(), | |
| Some(proto) if proto.is_empty() => "00".to_string(), | |
| Some(proto) => { | |
| let to_alnum = |b: u8| -> char { | |
| let c = b as char; | |
| if c.is_ascii_alphanumeric() { c.to_ascii_lowercase() } else { '9' } | |
| }; | |
| format!("{}{}", to_alnum(proto[0]), to_alnum(proto[proto.len() - 1])) | |
| } | |
| }; | |
| let alpn_chars: String = match &alpn_first { | |
| None => "00".to_string(), | |
| Some(proto) if proto.is_empty() => "00".to_string(), | |
| Some(proto) => { | |
| let first = proto[0]; | |
| let last = proto[proto.len() - 1]; | |
| let is_alnum = |b: u8| b.is_ascii_alphanumeric(); | |
| if is_alnum(first) && is_alnum(last) { | |
| format!("{}{}", (first as char).to_ascii_lowercase(), (last as char).to_ascii_lowercase()) | |
| } else { | |
| let hex = bytes_to_hex(proto); | |
| let hex_bytes = hex.as_bytes(); | |
| format!("{}{}", hex_bytes[0] as char, hex_bytes[hex_bytes.len() - 1] as char) | |
| } | |
| } | |
| }; |
| let hash_input = if has_sig_algs { | ||
| let sig_str = sig_algs.iter().map(|v| format!("{v:04x}")).collect::<Vec<_>>().join(","); | ||
| format!("{ext_str}_{sig_str}") | ||
| } else { | ||
| ext_str | ||
| }; |
There was a problem hiding this comment.
According to the JA4 specification, if there are no signature algorithms in the ClientHello packet, the string ends without an underscore and is hashed. Checking has_sig_algs can result in a trailing underscore if the signature algorithms extension is present but empty (e.g., due to truncation or malformed data). Checking !sig_algs.is_empty() is more robust and fully compliant with the spec.
| let hash_input = if has_sig_algs { | |
| let sig_str = sig_algs.iter().map(|v| format!("{v:04x}")).collect::<Vec<_>>().join(","); | |
| format!("{ext_str}_{sig_str}") | |
| } else { | |
| ext_str | |
| }; | |
| let hash_input = if !sig_algs.is_empty() { | |
| let sig_str = sig_algs.iter().map(|v| format!("{v:04x}")).collect::<Vec<_>>().join(","); | |
| format!("{ext_str}_{sig_str}") | |
| } else { | |
| ext_str | |
| }; |
Fixes the is_grease bitmask (was matching only 0x0A0A; now correctly covers all 16 GREASE values per RFC 8701: both bytes equal, low nibble 0xA). Adds JA4 (core TLS only — BSD-licensed; JA4+ variants intentionally omitted, FoxIO proprietary license). Computed on every peeked ClientHello alongside JA3: - Parses supported_versions (ext 43), ALPN (ext 16), and signature_algorithms (ext 13) from the extension list in a single pass. - Part A: t<ver><sni><cc><ec><alpn> (10 chars, version prefers supported_versions). - Part B: SHA-256/12 of sorted 4-hex cipher list (GREASE excluded). - Part C: SHA-256/12 of sorted extension IDs (GREASE/SNI/ALPN removed) + optional _<unsorted sigalgs> suffix when ext 13 is present. Surfaces ja4 in PeekInfo, blocked events (ja3 and ja4 fields added), and the ja4Blocklist protection config option (JsProtectionConfig, ProtectionConfig, parse_protection_config, ts/types.ts, proxy.ts). Values are normalized to lowercase on ingest; comparison is case-insensitive. Adds 10 Rust unit tests (is_grease full 16-value coverage, JA4 format, version detection, SNI indicator, counts, ALPN chars, known synthetic vector, GREASE exclusion, SNI hash exclusion) and a Node integration test that discovers the live JA4 of Node's TLS stack via a blocked event and asserts ja4_blocked. Depends on sha2 = "0.10" (keeps md-5 for JA3). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… shift Review findings: the d/i indicator was derived from extract_sni's validated result, so a present-but-rejected SNI (e.g. IPv6 literal) produced 'i' where the FoxIO spec requires 'd'. Also documents that the is_grease fix changes previously-collected JA3 values. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds two related edge-proxy capabilities (issue #9): 1. PROXY protocol v2 (binary, TLV-capable) header emission alongside the existing v1 text encoder, selectable per route via sourceAddressHeader: 'proxyProtocolV2'. v1 stays the UDS default since Harper core's own UDS reader currently parses v1 only. 2. forwardFingerprint ('ja3' | 'ja4' | 'none') forwards the ClientHello fingerprint symphony already computes to the upstream so backends can act on it. Carrier follows the mode: a PROXY v2 TLV (0xE0=JA3 / 0xE1=JA4, in HAProxy's private range) under proxyProtocolV2 — which works in passthrough since it prefixes the raw TLS bytes — otherwise an injected X-JA3/X-JA4 HTTP header for terminated L7 upstreams. Fingerprint-agnostic: JA4 rides the same path as JA3 (built on the JA4 branch, #2/#20). proxy_conn.rs threads the fingerprint + the connection's local_addr (the v2 destination) into apply_source_header; inject_x_forwarded_for generalized to inject_request_headers for XFF + fingerprint headers in one request rewrite. Drive-by: toJsRoute/resolveConnection dropped the http2 field entirely — it was declared in types + parsed in Rust but never forwarded by the JS wrapper; fixed alongside forwardFingerprint. Also unstuck a stale server.spec version assertion (0.4.0 → 0.5.0). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes #12, closes #2.
What this does
Adds core JA4 TLS client fingerprinting (FoxIO spec) beside JA3 in the MSG_PEEK ClientHello parser, and accepts JA4 entries in the protection blocklist.
PeekInfogainsja4(empty string on parse failure, likeja3). The parser now captures ALPN (ext 16), signature_algorithms (ext 13), and supported_versions (ext 43) payloads.ja4Blocklist?: string[]inProtectionConfig, checked beside the JA3 check; entries normalized to lowercase on both sides.blockedevents now carryja3/ja4, so fingerprints can be collected from symphony's own logs for blocklisting.is_greasefix (root cause, in this path)The GREASE predicate matched only
0x0A0Ainstead of all 16 RFC 8701 values. Fixed (lo == hi && (lo & 0x0F) == 0x0A) — this makes the redsni::tests::test_is_greasegreen and is required for correct JA4.is_greasefix — whichever merges second rebases trivially (same predicate).0x0A0AGREASE values, so JA3 for GREASE-rotating clients (Chrome) was nonstandard and unstable per-connection. Post-fix values are spec-correct; JA3 blocklist entries collected from earlier symphony versions should be re-collected (README upgrade note included).JA4 spec conformance
Part A (
t+ version-from-supported_versions + SNI d/i keyed on extension presence per spec + GREASE-excluded counts, with SNI/ALPN included in the extension count per spec + ALPN first/last chars with non-alphanumeric →9), Part B (sha256/12 of sorted GREASE-excluded ciphers), Part C (sorted extensions minus SNI/ALPN + unsorted sigalgs, suffix omitted when ext 13 absent). Verified in review against the FoxIO spec field-by-field; the unit vector is synthetic (a captured-Chrome reference fixture would be a nice follow-up).Review
Cross-model reviewed (Gemini diff-only leg + Opus domain pass). The domain pass specifically hunted for panic paths in the extended parser — this code runs pre-handshake on raw untrusted bytes in a napi cdylib where a panic aborts the Node process — and confirmed every slice is length-guarded (no remote-DoS path). Review fixes landed in
b5add11.Testing
cargo test: 16/16 (including the previously-redtest_is_grease, now covering all 16 GREASE values + near-misses).npm test: 33/34 — sole failure is the pre-existing hardcoded'0.4.0'version assertion, fixed in Fix CI: check out core submodule, drop retired macos-13 runner #18.ja4Blocklist.— Drafted and driven by KrAIs (Claude Opus 4.8 / Fable 5) on Kris's behalf; adversarially reviewed as described above.
🤖 Generated with Claude Code