Surrogate-safe JSON serialization for payloads, responses, and RPC bodies

[feruzm]

Follow-up to #52. Path-attributed diagnostics showed the surrogate failure surfacing in a second place: pipe(): error on /search-api/similar. Root cause: System.Text.Json's writer also throws on lone-surrogate strings (verified with both default and relaxed-encoder ToJsonString), so any JsonNode tree containing one failed at serialization — upstream payloads built from client bodies, response bodies echoing upstream data, and Hive RPC request bodies carrying client-derived names.

All three now serialize with JsJson.Stringify, which tolerates lone surrogates by construction (re-emitting them as escapes, like well-formed JSON.stringify) and byte-matches what the Node service sent. The Solana raw-text reconstruction deliberately keeps ToJsonString: it depends on raw u64 preservation and those RPC responses contain no user-controlled strings.

Verified: 43 tests green; end-to-end smoke of the exact failing shape (/search-api/similar with a lone-surrogate title) now proxies 200 with the payload delivered upstream and zero exceptions.

Summary by CodeRabbit

• Bug Fixes
  • Improved JSON request and response handling for usernames and other values containing unusual Unicode characters.
  • Prevented serialization errors when processing lone-surrogate characters.
  • Maintained consistent JSON output across API calls and upstream responses.

[greptile-apps]

Greptile Summary

This PR makes JSON serialization match the Node service for surrogate-safe proxying. The main changes are:

• Hive RPC request bodies now serialize through JsJson.Stringify.
• API JSON responses now use the same surrogate-safe serializer.
• Upstream payload and response forwarding now avoids the strict System.Text.Json writer.
• Project guidance now documents JavaScript number and lone-surrogate behavior.

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the changed code.

Important Files Changed

Filename	Overview
dotnet/EcencyApi/Infrastructure/HiveRpcClient.cs	Hive RPC bodies now use JsJson.Stringify so client-derived surrogate escapes can be sent without writer failures.
dotnet/EcencyApi/Infrastructure/HttpContextExtensions.cs	SendJson now writes non-null JSON nodes with JsJson.Stringify while preserving status, content type, and null output.
dotnet/EcencyApi/Infrastructure/Upstream.cs	Upstream request and response serialization now uses JsJson.Stringify for Node-compatible JSON output.
CLAUDE.md	The repository notes now document the expected JavaScript number and lone-surrogate JSON behavior.

_{Reviews (2): Last reviewed commit: "docs: codify JS-double number semantics ..." | Re-trigger Greptile}

[coderabbitai]

Warning

Review limit reached

@feruzm, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 39 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 420c2318-6064-4ac9-90cb-d56bf17f4896

📥 Commits

Reviewing files that changed from the base of the PR and between e39385f and b9b3106.

📒 Files selected for processing (1)

• CLAUDE.md

📝 Walkthrough

Walkthrough

Infrastructure JSON serialization now uses JsJson.Stringify for RPC requests, API request bodies, HTTP JSON responses, and Express-like upstream object bodies. Null handling and response content types remain unchanged, while obsolete serializer options are removed.

Changes

JSON serialization updates

Layer / File(s)	Summary
Outbound request serialization dotnet/EcencyApi/Infrastructure/HiveRpcClient.cs, dotnet/EcencyApi/Infrastructure/Upstream.cs	RPC and API request payloads now use JsJson.Stringify, retaining the {} fallback for null API payloads.
Transport response and upstream serialization dotnet/EcencyApi/Infrastructure/HttpContextExtensions.cs, dotnet/EcencyApi/Infrastructure/Upstream.cs	HTTP JSON responses and Express-like upstream object bodies use JsJson.Stringify; the previous serializer options are removed. Response content type behavior remains unchanged.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

I’m a rabbit with JSON in my burrow tonight,
Lone surrogates hop through without a fright.
Requests and responses now stringify with care,
Old serializer settings vanish in air.
Squeak—JavaScript parity is everywhere!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: switching payload, response, and RPC JSON serialization to surrogate-safe handling.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/surrogate-safe-serialization

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[feruzm]

@greptile-apps Repository convention to remember for future reviews of this repo — this class of finding has now been raised (in both directions) on several PRs, and it is a designed invariant, not a defect:

Number precision on proxied JSON is intentionally JavaScript-double semantics. This service is a behavior-preserving replacement for a Node/Express proxy: every request body and upstream response historically went through JSON.parse → JSON.stringify, so integers above 2^53 round (verified: 18446744073709551615 → 18446744073709552000 in both the old Node service and JsJson.Stringify, byte-identical). Raw-literal preservation on these paths would be a parity break, not an improvement. The single place raw text is load-bearing — the Solana lamports u64 scan in PrivateApi.Chain — deliberately uses ToJsonString and is documented.

Likewise, lone-surrogate \u escapes are valid in this codebase's JSON handling (JS accepts them; strict System.Text.Json APIs throw), which is why string extraction uses JsVal.TryGetStringLenient and serialization uses JsJson.Stringify.

Both invariants are now codified in CLAUDE.md ("Critical invariants", items 7–8) alongside the existing ones (e.g. the {} body on upstream GETs, item 4). Please treat findings that contradict those invariants as false positives unless accompanied by evidence the golden-vector tests (EcencyApi.Tests, generated from real dhive/V8 output) are wrong.

[greptile-apps]

Understood and saved. View rule

Going forward, findings on this repo that contradict either invariant will be treated as false positives:

• JS-double number semantics on proxied paths — rounding above 2^53 is correct parity with the old Node service; only PrivateApi.Chain (Solana lamports) is load-bearing and uses ToJsonString deliberately.
• Lone-surrogate \u escapes — JsVal.TryGetStringLenient / JsJson.Stringify are the correct APIs here; strict System.Text.Json paths are intentionally avoided on proxied strings.

Both are gated by the golden-vector tests in EcencyApi.Tests, so that's the right bar for evidence if a future finding genuinely needs re-evaluation.