Skip to content

fix(json): only convert unambiguous date/datetime/UUID strings in additional data#664

Open
56badkarma wants to merge 1 commit into
microsoft:mainfrom
56badkarma:fix/json-additional-data-string-coercion
Open

fix(json): only convert unambiguous date/datetime/UUID strings in additional data#664
56badkarma wants to merge 1 commit into
microsoft:mainfrom
56badkarma:fix/json-additional-data-string-coercion

Conversation

@56badkarma

Copy link
Copy Markdown

Overview

JsonParseNode.try_get_anything runs every untyped string (everything landing in additional_data) through five lenient parsers in sequence: datetime_from_iso_format_compat, UUID, parse_timedelta_string, date.fromisoformat, time_from_iso_format_compat. On Python 3.11+ the lenient parsers accept many ordinary text values:

'19-2026'       -> datetime.time(19, 0, tzinfo=timezone(timedelta(days=-1, seconds=12840)))  # invoice number
'100-012863299' -> datetime.time(10, 0, ...)                                                 # invoice number
'PT'            -> datetime.timedelta(0)                                                     # country code
'10:30'         -> datetime.timedelta(seconds=37800)
'11.0'          -> datetime.time(11, 0)                                                      # version string
'd41d8cd98f00b204e9800998ecf8427e' -> UUID(...)                                              # 32-hex string

This is not just a display problem: in our production SharePoint AP-invoicing app, invoice numbers like 19-2026 were coerced to time objects, later stringified on update, and persisted back — permanently corrupting stored data (details with a captured wire payload in microsoftgraph/msgraph-sdk-python#1340).

This PR replaces the speculative parsing with two anchored patterns, mirroring the .NET JsonParseNode.TryGetAnything, which only attempts strict DateTime/DateTimeOffset/Guid parses and otherwise returns the string:

  • full ISO date or datetime (^\d{4}-\d{2}-\d{2}([Tt ]\d{2}:\d{2}|$)) → datetime_from_iso_format_compat
  • canonical hyphenated UUID → UUID

Behavior is unchanged for real payload values: full ISO datetimes still become datetime, date-only strings still become datetime (same as today, via the same compat parser), canonical UUIDs still become UUID. The timedelta/time guessing is removed entirely — Graph does not send bare durations or times as untyped strings, and their grammars overlap with too much real-world text. The __is_four_digit_number/isdigit() special cases (added for earlier symptom reports like microsoftgraph/msgraph-sdk-python#653) become unnecessary since all-digit strings cannot match either anchored pattern, and are removed.

Related Issue

Fixes #487
Addresses microsoftgraph/msgraph-sdk-python#1340 and microsoftgraph/msgraph-sdk-python#653

Notes

  • Strings that previously coerced to timedelta/time/unhyphenated-UUID will now stay strings in additional_data. That is the intent of the fix, but callers who relied on those accidental conversions would see a change; typed model fields are unaffected (they go through declared field deserializers and never reach try_get_anything).
  • Verified against a live Microsoft Graph SharePoint tenant: text columns round-trip unchanged, Created/Modified datetimes still parse.

Testing Instructions

  • cd packages/serialization/json && pytest tests/
  • New parametrized test test_get_anything_does_not_coerce_text_values covers each reported corruption pattern (19-2026, PT, 10:30, 23.085, 11.0, 32-hex, date fragments)
  • New tests pin the preserved conversions: date-only → datetime, canonical UUID → UUID; existing tests cover full ISO datetimes

🤖 Generated with Claude Code

https://claude.ai/code/session_019Uui4g7E7UmzKRGHMW5zbH

…itional data

try_get_anything ran every untyped string through five lenient parsers
(datetime, UUID, timedelta, date, time). On Python 3.11+ this mangled
ordinary text values landing in additional_data: "19-2026" became
time(19, 0) at offset -20:26, "PT" became timedelta(0), "11.0" became
time(11, 0), and 32-char hex strings became UUIDs. Round-tripping such
records then persisted the mangled strings, corrupting user data.

Only convert strings matching an anchored full ISO date/datetime shape
or the canonical hyphenated UUID form, mirroring the .NET JsonParseNode
which attempts strict DateTime/DateTimeOffset/Guid parses only. Date-only
strings still parse through datetime_from_iso_format_compat, preserving
the existing datetime result for real payload values. The previous
four-digit and isdigit special cases are no longer needed since all-digit
strings cannot match either pattern.

Fixes microsoft#487

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Uui4g7E7UmzKRGHMW5zbH
@sonarqubecloud

Copy link
Copy Markdown

@56badkarma

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR tightens JsonParseNode.try_get_anything so that untyped strings routed into additional_data are only coerced when they are unambiguously ISO date/datetime literals or canonical hyphenated UUIDs, preventing accidental corruption of ordinary text values (e.g., invoice numbers, country codes) on round-trip updates.

Changes:

  • Replaces speculative “try a bunch of parsers” logic with two anchored regex gates (ISO date/datetime prefix and canonical UUID) before attempting conversion.
  • Removes the implicit timedelta/time guessing behavior for untyped strings in additional_data.
  • Adds unit tests covering previously-corrupted text patterns and pins the intended conversions (date-only → datetime, canonical UUID → UUID).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
packages/serialization/json/kiota_serialization_json/json_parse_node.py Restricts try_get_anything string coercions to unambiguous ISO date/datetime and canonical UUID shapes.
packages/serialization/json/tests/unit/test_json_parse_node.py Adds parameterized coverage to ensure common text values are not coerced, and validates the remaining intended conversions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

try_get_anything - converts a 32 hex string to UUID

2 participants