fix: parse hyphenated strings correctly inside container literals#678
Open
Yadidiah-k wants to merge 1 commit into
Open
fix: parse hyphenated strings correctly inside container literals#678Yadidiah-k wants to merge 1 commit into
Yadidiah-k wants to merge 1 commit into
Conversation
When a hyphenated string like 'foo-bar' appears inside a tuple, list, set, or dict (e.g. '(foo-bar,baz)'), ast.parse turns the hyphen into a BinOp(Sub) node. The AST walker only replaced bare Name nodes, leaving the BinOp unresolved. ast.literal_eval then failed on the BinOp and the entire container fell back to a plain string instead of a tuple/list/dict. Add _BinOpSubToStr to recursively convert a Name-Sub-Name chain back to a hyphenated string, and call it in the walker alongside _Replacement. This handles multi-segment names like 'a-b-c' too. Fixes google#561
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #561
What was wrong
When a hyphenated string like
foo-barappears inside a container literal — tuple, list, set, or dict — Python'sast.parseturns the hyphen into aBinOp(Sub)node (foo - bar). The AST walker in_LiteralEvalonly replaced bareNamenodes with strings; it leftBinOpnodes untouched. Whenast.literal_evalthen encountered the unresolvedBinOp, it raisedValueError, and the entire container fell back to a plain string.Fix
Added
_BinOpSubToStr, a helper that recursively converts aName-Sub-Namechain back to a hyphenated string (handles multi-segment names likea-b-ctoo). The walker now calls it alongside_Replacementfor both list children and single-value fields. Works for tuples, lists, and dicts.Numeric subtraction like
123-456is unaffected —Constantnodes are notNamenodes and_BinOpSubToStrreturnsNonefor them, so no replacement happens.Tests added
testDefaultParseValueHyphenatedStringsInTupletestDefaultParseValueHyphenatedStringsInListtestDefaultParseValueHyphenatedStringsInDictAll 23 parser tests pass.