Validation rejects {{ $item }} as a syntax error; the runtime rewrites it

[jeremymanning]

The runtime accepts {{ $item }}; validation calls it a syntax error

UnifiedTemplateResolver._preprocess_dollar_variables (unified_template_resolver.py:545) rewrites {{ $variable }} to {{ variable }} before rendering, because "Jinja2 doesn't support variables starting with $". It does the same inside {% %} blocks.

TemplateValidator._validate_syntax parses the raw text, so it reports:

syntax_error

for a template the runtime handles correctly.

Reproduction

TemplateValidator().validate_pipeline_templates(
    {"id": "p", "steps": [{"id": "s", "for_each": "{{ rows }}",
                           "parameters": {"t": "{{ $item }}"}}]},
    {"rows": [1]},
)
# -> is_valid=False, ['syntax_error']

Scope

32 uses across the catalogue — $item (18), $iteration (10), $index (2), $is_first (2) — in files including:

• examples/control_flow_for_loop.yaml
• examples/iterative_fact_checker.yaml
• examples/control_flow_advanced.yaml
• examples/enhanced_until_conditions_demo.yaml
• examples/original_research_report_pipeline.yaml

All of these currently fail validation, and this is at least one of the reasons.

Same class as #465, #469 and #472

The compiler/runtime accepts a spelling the validator does not. Each time, the validator's private notion of the language disagreed with the runtime's:

issue	disagreement
#465	inferred dependencies rejected rather than used
#469	bare loop names unknown to the template validator
#472	one binding set for every loop construct; source validated in loop scope
this	$-prefixed spelling rejected as a syntax error

Fix

Validation should apply the same preprocessing the runtime applies, from a shared implementation rather than a second regex — a reimplementation would be the fourth mechanism problem #466 removed for dependencies.

Once {{ $item }} is normalised to {{ item }} before parsing, the DOLLAR_LOOP_VARIABLES raw-text scan in _validate_variables becomes redundant: the names appear in the AST like any other. That scan exists only because $item cannot be parsed.

Tests

• {{ $item }} inside a for_each validates and renders to the item
• {{ $item }} outside a loop is still rejected, as {{ item }} is
• $-names in {% if %} / {% for %} blocks are handled
• the preprocessing has one implementation, asserted by identity
• a pipeline using $item compiles and runs, producing the expected artifact
• the affected catalogue examples move into the validating baseline

[jeremymanning]

Another instance of this, found while writing the loop documentation: the named-loop syntax is affected too, and it is the only way to reach an outer loop's variables from a nested loop.

This runs. Executed hermetically, it writes 0.txt and 1.txt containing named=<A> bare=<A> and named=<B> bare=<B>:

steps:
  - id: outer
    for_each: "['A','B']"
    loop_name: outer_loop
    steps:
      - id: w
        tool: filesystem
        action: write
        parameters:
          path: "out/{{ index }}.txt"
          content: "named=<{{ $outer_loop.item }}> bare=<{{ item }}>"

TemplateValidator on the same pipeline:

valid: False [('syntax_error', "Template syntax error: unexpected char '$' at 3")]

So the documented feature for nested loops cannot pass validation. That raises the cost of leaving this open: it is not only the {{ $item }} shorthand (which has a working bare equivalent) but a construct with no validating spelling.

Note also that the $ set is not uniform: $item, $index, $is_first, $is_last, $iteration, $loop_id, $loop_name, $loop_state are rewritten, while $position, $length, $remaining, $has_next, $has_prev raise unexpected char '$' at run time as well. So the fix is not "accept $ everywhere" — validation and runtime need to agree on one list, from one implementation.