Skip to content

fix: import clgraph without PyYAML installed (0.0.7) - #76

Merged
mingjerli merged 1 commit into
mainfrom
fix/optional-orchestrator-imports
Aug 2, 2026
Merged

fix: import clgraph without PyYAML installed (0.0.7)#76
mingjerli merged 1 commit into
mainfrom
fix/optional-orchestrator-imports

Conversation

@mingjerli

Copy link
Copy Markdown
Owner

The bug

A clean pip install clgraph followed by import clgraph fails outright:

File ".../clgraph/orchestrators/kestra.py", line 10, in <module>
    import yaml
ModuleNotFoundError: No module named 'yaml'

clgraph/orchestrators/kestra.py imported yaml at module scope, and clgraph/orchestrators/__init__.py imports every backend eagerly, so the failure propagates all the way up to import clgraph.

PyYAML is not a declared dependency — not in dependencies, not in any extra. It only ever arrived transitively in development environments (via airflow and friends), which is why the full suite and every CI job stayed green: all of them install .[dev].

Bisected across published releases:

version bare import clgraph
0.0.3 works
0.0.5 fails
0.0.6 fails (inherited)

So the currently published release is unusable out of the box. Found while verifying the 0.0.6 artifact on PyPI; it predates that release and is unrelated to its contents.

The fix

PyYAML is imported at point of use through a _require_yaml() helper. KestraOrchestrator now imports and constructs without it — only to_flow(), to_flow_with_triggers() and to_flow_dict() need it, and they raise an ImportError that names the package and the install command:

KestraOrchestrator requires PyYAML, which is not installed.
Install it with `pip install 'clgraph[kestra]'` or `pip install pyyaml`.

I checked every orchestrator, not just the reported one: Airflow, Dagster, Prefect and Mage emit code as text and import nothing beyond stdlib and .base, so Kestra was the only one affected.

Adding pyyaml to core dependencies would also have worked, but it would put a YAML parser in every install to serve one optional backend — the lazy import matches how the other four already avoid depending on their target systems.

Also included

  • clgraph[kestra] extra, so the error message's install hint is real.
  • bare-install CI job — builds the wheel, installs it into a clean venv with no extras, imports it, builds a pipeline, and asserts Kestra raises the actionable error. Every existing job installs .[dev], so nothing in the pipeline could have caught this class of bug. Runs on 3.10 and 3.13.
  • Regression tests (tests/test_optional_orchestrator_deps.py) that run in a subprocess with yaml made unimportable via a meta_path blocker — the dev environment genuinely has PyYAML, so the failure only reproduces when it is absent. Includes a test_yaml_blocker_actually_blocks guard so the suite cannot pass vacuously if the blocker ever stops working.

Test plan

  • 8 new tests; verified RED first (7 failed reproducing the exact bug, with the blocker-guard passing).
  • Full suite: 1613 passed, 40 skipped, 2 xfailed.
  • ruff check / ruff format --check clean.
  • Verified end to end in a real bare venv with zero pyyaml installed: import clgraph works, a pipeline builds (4 columns), and Kestra raises the actionable ImportError.
  • Kestra output unchanged when PyYAML is present (to_flow and to_flow_dict exercised).
  • ty reports 52 errors, the same 52 as main — no new ones.

`clgraph/orchestrators/kestra.py` imported yaml at module scope, and
`clgraph/orchestrators/__init__.py` imports every backend eagerly, so a bare
`pip install clgraph` followed by `import clgraph` raised:

    ModuleNotFoundError: No module named 'yaml'

PyYAML was never a declared dependency. It only ever arrived transitively in
development environments, which is why the full suite and every CI job stayed
green - all of them install ".[dev]". Bisected across published releases:
0.0.3 imports fine, 0.0.5 and 0.0.6 do not.

PyYAML is now imported at point of use via `_require_yaml()`. KestraOrchestrator
imports and constructs without it; only to_flow(), to_flow_with_triggers() and
to_flow_dict() need it, and they raise an ImportError naming the package and
the install command. Airflow, Dagster, Prefect and Mage emit code as text and
were never affected.

Also adds:
- a `clgraph[kestra]` extra, so the error message's install hint is real
- a `bare-install` CI job that installs the built wheel into a clean
  environment with no extras and imports it - nothing in the pipeline would
  have caught this otherwise
- regression tests that run in a subprocess with yaml made unimportable, since
  the dev environment has PyYAML and the failure only reproduces without it

Bumps to 0.0.7.
@mingjerli
mingjerli merged commit 4bf946c into main Aug 2, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant