Stream compound command output progressively (#43)

[konard]

Summary

Fix compound commands so .stream() emits each virtual or nested process chunk as it arrives instead of misclassifying the whole command or waiting for nested work to finish.

Closes #43.

Root cause

The JavaScript runner deliberately skipped its enhanced shell parser whenever streaming was active. A command such as:

$`echo before; long-running-build`.stream()

therefore fell through to the simple virtual-command parser, which could treat everything after echo as arguments. The earlier draft worked around that with substring checks and forced compound streams into a system shell. That would make common cases stream, but it also:

• treats quoted metacharacters as operators;
• removes command-stream built-ins and registered virtual commands from sequences;
• changes cross-platform behavior to whatever shell happens to be installed;
• does not connect parent cancellation to a real process nested in internal orchestration.

Solution

• Run streamed compound commands through the existing parser/orchestrator.
• Keep built-ins and custom virtual commands in-process without finishing the parent after each subcommand.
• Forward output from nested real processes to the parent immediately.
• Propagate kill() and async-iterator cancellation to the active nested process.
• Stop a sequence after cancellation and preserve the cancellation exit code.
• Detect unsupported shell features quote-aware and fall back to the system shell only when required.
• Handle a lone background & without allowing the tokenizer to stall.
• Release virtual sleep timers, polling intervals, and abort listeners on both completion and cancellation.

Reproduction and regression coverage

The minimal regression registers a virtual command that blocks until the test releases it:

const runner = $({ mirror: false })`echo before; issue-43-blocked`;

Before the fix, the first chunk was the misparsed literal command text. After the fix, before\n is available while the second command is still blocked.

js/tests/issue-43-stream-output.test.mjs also verifies:

• live forwarding from a gated real child process nested in a sequence;
• quoted ;, *, [, and ? remain ordinary virtual-command arguments;
• breaking a stream kills the active nested child and returns the signal exit code;
• a background operator safely falls back to the real shell;
• the runnable compound-stream example exits without leaked virtual-sleep resources.

Execution models and documentation

The new Real-time Streaming section in js/BEST-PRACTICES.md compares the main approaches used by Bun Shell, zx, Execa, and Node:

1. library parser/interpreter for portable built-ins and virtual commands;
2. explicit system shell for trusted commands needing shell-only syntax;
3. direct process spawning/programmatic pipelines for application-controlled composition.

It documents when to choose each model and includes bad examples for completion-buffered output, unnecessary sh -c, untrusted raw syntax, and producer-side buffering. js/examples/streaming-compound-commands.mjs is a runnable progressive-output example.

Validation

• bun test js/tests/ --timeout 10000: 790 passed, 5 skipped, 0 failed
• bun run lint: passed (one pre-existing warning in commands/$.cd.mjs)
• bun run format:check
• bun run check:duplication
• bun scripts/validate-changeset.mjs
• Node module-load and terminal-artifact tests
• cargo fmt --all -- --check
• cargo clippy --all-targets --all-features
• cargo test --all-features --verbose
• cargo test --doc --all-features --verbose
• all Rust release-script unit tests
• cargo build --release --verbose
• cargo package --allow-dirty

This is intentionally JavaScript-only and carries parity-exempt: Rust's StreamingRunner already launches the complete compound command in one system shell and streams that child, while the bug was in JavaScript's virtual-command orchestration lifecycle.

Release

Adds a patch changeset for command-stream.

[konard]

Resolve conflicts, and double check that is the best possible way to address all these issues. If there is a better way fix it. If that is not documented, make sure we document well enough all the best practices, and show bad examples to avoid.

If there is multiple different solutions we should check competitors best practices and provide all solutions they have.

[konard]

🤖 AI Work Session Started

Starting automated work session at 2026-07-28T20:24:46.636Z

The PR has been converted to draft mode while work is in progress.

This comment marks the beginning of an AI work session. Please wait for the session to finish, and provide your feedback.

[konard]

Working session summary

PR #106 is ready for review, conflict-free, mergeable, and green on commit d1fb9a3.

Implemented:

• Progressive compound-command streaming while preserving virtual commands.
• Real-time nested-process output and cancellation propagation.
• Quote-aware shell fallback and safe background-operator handling.
• Virtual sleep resource cleanup.
• Six focused regressions, runnable example, patch changeset, and best-practices documentation comparing Bun Shell, zx, Execa, and Node.

Validation:

• Local: 790 passed, 5 skipped, 0 failed.
• Lint, formatting, duplication, changeset, Node, Rust tests/build/package all passed.
• Fresh CI passed across Ubuntu, macOS, Windows, and Node 20/22/24.
• One unrelated Windows PTY handshake timeout was preserved in ci-logs/; its same-SHA rerun passed.
• Worktree is clean and origin/main is merged.

---

This summary was automatically extracted from the AI working session output.

[konard]

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

• Model: GPT-5.6 Sol
• Provider: OpenAI
• Public pricing estimate: $22.313423

📊 Context and tokens usage:

• 519.5K / 200K (260%) input tokens, 68.7K / 128K (54%) output tokens

Total: (519.5K + 14.0M cached) input tokens, 68.7K output tokens, $22.313423 cost

🤖 Models used:

• Tool: OpenAI Codex
• Requested: gpt-5.6-sol
• Thinking level: xhigh (~31999 tokens)
• Model: GPT 5.6 Sol (gpt-5.6-sol)

📎 Log file uploaded as Gist (5090KB)

• View complete solution draft log

---

Now working session is ended, feel free to review and add any feedback on the solution draft.

[konard]

🎉 Auto-merged

This pull request has been automatically merged by hive-mind.

• All CI checks have passed

---

Auto-merged by hive-mind with --auto-merge flag