Support inline # comments on TypedDict and dataclass fields#364
Support inline # comments on TypedDict and dataclass fields#364hemanth1999k wants to merge 1 commit into
Conversation
…crosoft#242) When no Annotated[..., Doc(...)] is present, fall back to the inline # comment on the field's source line. Uses inspect + ast to extract comments at class-definition time; fails silently so dynamic/REPL- defined types are unaffected. Doc annotation still takes priority.
@microsoft-github-policy-service agree |
|
@microsoft-github-policy-service agree |
robgruen
left a comment
There was a problem hiding this comment.
I really like the idea of this PR. Having inline comments would be fantastic. However, I noted a few different schemas that could break the implementation as provided. Instead of doing a find string, use the Python tokenizer to get each raw AST token and then grab the contents of the comment token.
Here's a suggestion from copilot (requires importing io & tokenize):
def _extract_inline_field_comments(py_type: type) -> dict[str, str]:
"""Return a mapping of field name -> trailing '#' comment for a TypedDict/dataclass."""
try:
source_lines, _ = inspect.getsourcelines(py_type)
except (OSError, TypeError):
# Source unavailable (e.g. dynamically created types) -> no comments.
return {}
source = textwrap.dedent("".join(source_lines))
# Only real COMMENT tokens count. A '#' inside "..." is a STRING token, so it's ignored.
comment_by_line: dict[int, str] = {}
try:
for token in tokenize.generate_tokens(io.StringIO(source).readline):
if token.type == tokenize.COMMENT:
comment_by_line[token.start[0]] = token.string[1:].strip()
except (tokenize.TokenError, IndentationError):
pass # keep whatever we gathered before an incomplete-source error
if not comment_by_line:
return {}
try:
tree = ast.parse(source)
except SyntaxError:
return {}
comments: dict[str, str] = {}
for node in ast.walk(tree):
if isinstance(node, ast.ClassDef):
for stmt in node.body:
if (
isinstance(stmt, ast.AnnAssign)
and isinstance(stmt.target, ast.Name)
and stmt.end_lineno is not None
):
comment = comment_by_line.get(stmt.end_lineno)
if comment:
comments[stmt.target.id] = comment
return comments| for stmt in node.body: | ||
| if isinstance(stmt, ast.AnnAssign) and isinstance(stmt.target, ast.Name): | ||
| line = source_lines[stmt.end_lineno - 1] | ||
| if "#" in line: |
There was a problem hiding this comment.
This will cause problems....see these schemas that break:
# BREAK 1 — hex colors, NO comment intended
class Theme(TypedDict):
background: Literal["#FFFFFF", "#000000"]
accent: Literal["#FF0000"]
# BREAK 2 — real comment gets corrupted (string '#' comes first)
class Swatch(TypedDict):
color: Literal["#00FF00"] # the accent color as a hex triplet
# BREAK 3 — dataclass default
@dataclass
class GitConfig:
comment_marker: str = "value # not a comment"
# BREAK 4 — URL fragment in a default leaks out
@dataclass
class ApiEndpoint:
url: str = "https://api.example.com/v1/items#latest"
# BREAK 5 — multi-line annotation whose last line has '#' in a string
class ButtonSpec(TypedDict):
variant: Literal[
"primary",
"hash-#-danger"] # pick one variant
Fixes #242.
Currently the only way to document a field is the verbose
Annotated[int, Doc("...")]syntax. This adds support for native Python inline comments:which produces the same TypeScript as before:
Uses
inspect.getsourcelines+astto pull# commentsoff field definitions at schema-generation time.Annotated/Docstill takes priority when both are present. Fails silently for dynamically-defined types (REPL,exec, etc.) so nothing breaks.Adds three snapshot tests covering TypedDict inline comments, Doc-takes-priority, and dataclass inline comments.