create_parallel_queue reports success while silently skipping a step that did not render

[jeremymanning]

Observed while probing loop bindings by execution.

Inside a create_parallel_queue, a step whose template references an unbound bare name does not write its file, and the pipeline still reports success:

- id: loop
  action: create_parallel_queue
  create_parallel_queue:
    "on": "['A']"
    action_loop:
      - action: filesystem
        parameters:
          action: write
          path: "out/probe.txt"
          content: "{{ nosuch_name }}"

The write is refused internally with still contains unresolved template references, no file appears, and the run's result reports success. Nothing surfaces to the caller.

The $ spelling fails differently and just as quietly: {{ $nosuch }} is stripped to nothing and the literal text is written to the file, so the artifact exists and its contents are wrong.

Two more findings from the same probe, same construct:

1. A template in on: breaks queue splitting. "on": "['x', '{{ 1 + 1 }}']" yields a single queue item whose value is the list ['x', '2'], rather than two items. Without a template it splits correctly.
2. Inner actions cannot use tool:. tool: filesystem inside the nested action_loop raises Could not import tool filesystem: No module named 'orchestrator.tools.filesystem_tools'; action: filesystem is required. So the step syntax inside this construct differs from the step syntax everywhere else, undocumented.

The silent-success case is the serious one: it is the failure mode the resilience work was meant to remove, and no test covers this construct. examples/ contains no create_parallel_queue pipeline that validates, so nothing exercises it.

[jeremymanning]

Root cause

Traced. The error is raised correctly and thrown away two frames below the code that decides success.

Reproduction (hermetic, filesystem tool only):

- id: loop
  action: create_parallel_queue
  create_parallel_queue:
    "on": "['A']"
    action_loop:
      - action: filesystem
        parameters: {action: write, path: "out/out.txt", content: "{{ nosuch_name }}"}

Direct action execution failed for filesystem: Parameter 'content' of tool 'filesystem'
still contains unresolved template references: '{{ nosuch_name }}'.
RUN COMPLETED, no exception
artifacts: []

UnresolvedTemplateError is raised by _reject_unresolved_templates exactly as designed. Then:

1. parallel_queue_handler.py:617 — _execute_subtask_action catches every exception and converts it into a return value: {"status": "failed", "error": ..., "result": None}.
2. parallel_queue_handler.py:411 — the outer execute_subtask awaits that call. No exception arrives, so it takes the success path: subtask.complete(result), mark_item_completed, task.completed_items.add(...), and returns {"status": "completed"}. A failed action is recorded as a completed subtask.
3. parallel_queue_handler.py:694 — _aggregate_results counts by status. Every subtask says completed, so failed_items is 0 and failure_rate is 0.
4. parallel_queue_handler.py:211 — execute_parallel_queue sees no exception, calls task.complete(final_result) and sets ParallelQueueStatus.COMPLETED.

So the per-item failure record at step 1 is real and correct; it is simply never read by anything that decides success. The status field it sets is overwritten by the caller one frame up.

Direction

Two changes, in this order:

1. The inner catch must not swallow. Re-raising routes into the outer execute_subtask's existing except branch, which already does the right thing: subtask.fail(e), mark_item_failed, task.failed_items.add(...), and a status: "failed" record. That path is correct and currently unreachable for this class of error.
2. The queue step must fail when items failed. execute_parallel_queue marks COMPLETED unconditionally when nothing propagates. It should consult the declared failure policy — on_error / continue_on_error already exist (engine/pipeline_spec.py:139 should_continue_on_error, tests/test_failure_policy.py) and this handler does not consult them at all. Absent an explicit policy, a failed item must fail the step.

That is what makes the stated invariant hold: a successful pipeline means every required step either succeeded or was explicitly handled by a declared policy.

Note on the existing coverage

tests/test_parallel_queue_phase2.py is in the non-blocking legacy job, so nothing here is gated. Any fix should move the queue execution tests into the blocking suite, otherwise the same regression can return silently — which is how this survived.