Skip to content

fix(e13-s5): accept markdown-fenced JSON from real LLMs in --ai parse - #63

Merged
imneov merged 1 commit into
mainfrom
fix/e13-ai-markdown-fence
Jul 8, 2026
Merged

fix(e13-s5): accept markdown-fenced JSON from real LLMs in --ai parse#63
imneov merged 1 commit into
mainfrom
fix/e13-ai-markdown-fence

Conversation

@imneov

@imneov imneov commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What

Live-testing the --ai path against a real Anthropic model (claude-sonnet-4-5, via a relay) surfaced a bug that mocked tests could not: the model returns perfectly correct content but wraps the JSON in a ```json``` markdown fence, despite the system prompt asking for no markdown. ParseCoreJSON's strict decode then failed on the leading fence, so --ai refused every real response — the feature was effectively unusable against live models.

Fix

internal/aiparse now strips a single outer markdown code fence before the strict JSON decode. Strictness is not weakened: after stripping it still DisallowUnknownFields, rejects nested objects / arrays / nulls / trailing data / empty objects, and only peels one well-formed outer fence (multiple/incomplete/odd-language fences fall through to refusal).

Evidence

  • Unit tests: fenced live-response fixture parses; bare ``` fence parses; fenced_unknown_field still refuses (strictness not bypassed); all prior non-conforming cases still refuse.
  • Live end-to-end (real relay, synthetic secret-free blob) after the fix:
    base_url=https://api.anthropic.com model=claude-3-5-sonnet-20241022 api_key={{CLAUDECM_SECRET_1}} small_fast_model=claude-3-5-haiku-20241022 — parsed successfully; placeholder preserved for local re-injection.
  • Fresh-context adversarial review: APPROVE (strictness preserved, stripping robust, request/secret/credential path untouched).

Response-parse side only; no change to the outbound redaction, secret handling, or credential path.

🤖 Generated with Claude Code

@imneov
imneov merged commit c6fc3d0 into main Jul 8, 2026
8 of 9 checks passed
@imneov
imneov deleted the fix/e13-ai-markdown-fence branch July 8, 2026 04:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant