Summary
At clgraph 0.0.5, Pipeline.from_dbt_models() silently breaks column lineage for any dbt model that references two or more {{ ref('...') }} targets (the standard dbt mart pattern).
Repro (minimal, 3 models)
models/staging/stg_a.sql: SELECT id, x FROM raw.a
models/staging/stg_c.sql: SELECT id, k FROM raw.c
models/marts/mart_b.sql: SELECT a.x, c.k, a.id AS y FROM {{ ref('stg_a') }} a JOIN {{ ref('stg_c') }} c ON a.id = c.id
p = Pipeline.from_dbt_models(project_dir, schema_map={"staging": "staging", "marts": "marts"}, dialect="bigquery")
Observed
- Table graph contains literal unresolved names
"{{ ref('stg_a') }}" / "{{ ref('stg_c') }}" alongside the real staging.stg_a/staging.stg_c tables.
marts.mart_b's columns trace to disconnected placeholder nodes (mart_b:unknown.x, …) instead of the real upstream columns — lineage is silently broken, no error or warning.
Expected
All ref() targets resolve to their schema-mapped table names; the mart's columns carry edges from both staging tables' columns.
Root-cause hypothesis
The templated-reference resolution drops every ref() after the first in a query (Jinja-function handling in the template resolver), leaving later refs as raw text through parsing.
Workaround (verified)
Pre-resolve ref() calls to literal target-table names on the output of wrap_dbt_models() before constructing Pipeline directly — clgraph-studio does this in both its demo build script and its live dbt analyze route (mingjerli/clgraph-studio, backend/dbt_io.py::build_dbt_pipeline).
Found while building clgraph-studio's dbt demo; confirmed independently by a second reviewer with the repro above.
Summary
At clgraph 0.0.5,
Pipeline.from_dbt_models()silently breaks column lineage for any dbt model that references two or more{{ ref('...') }}targets (the standard dbt mart pattern).Repro (minimal, 3 models)
Observed
"{{ ref('stg_a') }}"/"{{ ref('stg_c') }}"alongside the realstaging.stg_a/staging.stg_ctables.marts.mart_b's columns trace to disconnected placeholder nodes (mart_b:unknown.x, …) instead of the real upstream columns — lineage is silently broken, no error or warning.Expected
All
ref()targets resolve to their schema-mapped table names; the mart's columns carry edges from both staging tables' columns.Root-cause hypothesis
The templated-reference resolution drops every
ref()after the first in a query (Jinja-function handling in the template resolver), leaving later refs as raw text through parsing.Workaround (verified)
Pre-resolve
ref()calls to literal target-table names on the output ofwrap_dbt_models()before constructingPipelinedirectly — clgraph-studio does this in both its demo build script and its live dbt analyze route (mingjerli/clgraph-studio,backend/dbt_io.py::build_dbt_pipeline).Found while building clgraph-studio's dbt demo; confirmed independently by a second reviewer with the repro above.