Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,15 @@
- **Behavior change:** `dbt_sqlserver_use_default_schema_concat` now defaults to `True`: a custom schema is concatenated onto `target.schema` (dbt-core's standard `generate_schema_name` behavior) instead of being used directly. This also matches the behavior dbt-core v2 (Fusion) will ship unconditionally. The `False` (legacy, no prefix) behavior is deprecated and will be removed in a future release; set the flag explicitly to keep it during the deprecation window, or override `sqlserver__generate_schema_name` in your project for a permanent solution. [#800](https://github.com/dbt-msft/dbt-sqlserver/issues/800)
- Add an experimental `adbc` backend (`backend: adbc`), an alternative to `pyodbc`/`mssql-python` built on [ADBC](https://arrow.apache.org/adbc/) that talks to SQL Server via `go-mssqldb` instead of an ODBC/DB-API bridge. Install with `dbt-sqlserver[adbc]` plus the separate `dbc` CLI-installed driver binary; supports SQL Server (user/password) authentication only for now. See [docs/adbc_backend.md](docs/adbc_backend.md). [#771](https://github.com/dbt-msft/dbt-sqlserver/issues/771)
- Add a model-level `denies` config that re-applies object-level `DENY` permissions after each build, diffed against `sys.database_permissions`, so an object DENY survives dbt's drop-and-recreate. A SQL Server object DENY is the only way to carve an exception out of a schema-level GRANT, but it is stored against `object_id` and was silently discarded on every rebuild (every run for a view), leaving a fail-open posture. Shaped like `grants` (`{privilege: [principals]}`); covers `table`, `view`, `incremental` and `snapshot`; emits `DENY`/`REVOKE` only for what changed; warns-and-skips an absent principal; and is a no-op on other adapters. Mirrors the existing `masks` re-application. See the README for details.
- Add the `sqlserver__openquery` macro for safely executing pass-through queries against linked servers, including remote-SQL quote escaping, carriage-return stripping and the SQL Server 8 KB query-length validation. Add a SQL Server best-practices guide covering when to use `OPENQUERY` instead of distributed four-part-name joins.

#### Bugfixes

- Fix a `view` model silently skipping a rebuild when text was removed from the *start* of its body (e.g. deleting a leading comment or CTE). The skip test compared the stored definition against the model with `endswith()`, so any edit whose new body was a tail of the old one looked unchanged: `dbt run` reported `PASS` but the change never reached the database, and `--full-refresh` did not fix it. The header (`CREATE [OR ALTER] VIEW <name> AS`) is now split off at its separating ` AS ` and the body compared exactly. The comparison also no longer lowercases or strips whitespace, both of which made genuinely different bodies (a string literal differing only in case, or any literal containing spaces) compare equal; where the definition cannot be parsed with certainty the view is rebuilt rather than skipped.
- Fix snapshots failing on their second and later runs with `Invalid object name '..._dbt_tmp'`, and contract-enforced models silently losing their in-transaction `pre_hook` writes. `get_column_schema_from_query` reads a query's column shape by executing it, then returned without fetching the rows or closing the cursor. Closing a cursor whose result set the server is still producing makes the driver cancel the request, and SQL Server answers that cancel by rolling back the open transaction, since every connection runs `SET XACT_ABORT ON` (#718). Nothing is raised for any of it, so the snapshot lost the staging table it had just built and failed against it a statement later. The probe now drains and closes its cursor, as does the row-count probe in `expand_column_types`. Only queries opening with a CTE were affected - anything else is wrapped as `select * from (...) where 1 = 0` by `sqlserver__get_empty_subquery_sql` and returns no rows - which is why snapshot staging queries (`with snapshot_query as ...`, both `check` and `timestamp` strategies) and CTE-headed contract models were the ones that broke.
- Fix models failing with `Incorrect syntax near '\'` when the schema name needs delimiters, such as a domain-qualified `domain\user`. The clustered columnstore index name embeds the schema and was emitted as a bare identifier, so the generated DDL did not parse. [#409](https://github.com/dbt-msft/dbt-sqlserver/issues/409)
- Fix identifiers built inside string literals not being quoted, which broke schema names containing a `.` or a `"`. `OBJECT_ID('schema.table')` returns `NULL` rather than erroring for such a name, so the failures were silent: the drop-before-create guards in `create_table_as` treated an existing table as absent (then hit `Msg 2714`), and the mask introspection in `apply_masks` found no columns, so configured masks were never applied. `sp_rename` was affected too, failing the table rename-swap with `No item by the name of ...`. All now pass quoted, qualified names. [#785](https://github.com/dbt-msft/dbt-sqlserver/issues/785)
- Fix the `sqlserver__openquery` macro to quote linked-server names through `adapter.quote()`, keeping its generated identifier style consistent with the rest of the adapter.

#### Under the hood

Expand Down
22 changes: 16 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

[dbt](https://www.getdbt.com) adapter for Microsoft SQL Server and Azure SQL services.

The adapter supports dbt-core 1.11 or newer and follows the same versioning scheme.
E.g. version 1.11.x of the adapter is compatible with dbt-core 1.11.x.
The adapter supports dbt-core 1.12 or newer and follows the same versioning scheme.
E.g. version 1.12.x of the adapter is compatible with dbt-core 1.12.x.

## Supported Python versions

Expand All @@ -15,6 +15,7 @@ The adapter is tested against:
| 3.11 | Officially supported |
| 3.12 | Officially supported |
| 3.13 | Officially supported |
| 3.14 | Officially supported |

## Supported SQL Server versions

Expand Down Expand Up @@ -178,19 +179,28 @@ Safe expansions are further gated by `column_type_expansion_max_rows` (default 1

### `dbt_sqlserver_use_dbt_transactions`

_(default: `false`)_ When enabled, makes dbt's transaction hooks real at the SQL Server level by emitting `BEGIN TRANSACTION` / `COMMIT TRANSACTION` through the adapter's `add_begin_query` and `add_commit_query` methods.
_(default: `true`)_ Makes dbt's transaction hooks real at the SQL Server level by emitting `BEGIN TRANSACTION` / `COMMIT TRANSACTION` through the adapter's `add_begin_query` and `add_commit_query` methods.

The default is `false`, preserving existing behavior where `begin`/`commit` hooks are logical no-ops and the ODBC driver auto-commits each statement. When `dbt_sqlserver_use_dbt_transactions: true`, the adapter emits real T-SQL transaction statements, and rollback uses `IF @@TRANCOUNT > 0 ROLLBACK TRANSACTION`.
The default is `true`, so dbt-managed transaction hooks emit real T-SQL transaction statements and rollback uses `IF @@TRANCOUNT > 0 ROLLBACK TRANSACTION`. Set it to `false` to opt back into the deprecated legacy behavior where `begin`/`commit` hooks are logical no-ops and the driver auto-commits each statement.

The driver connection remains in autocommit mode (`autocommit=true`) in both modes.

This mode is opt-in and should be tested carefully with project-specific materializations and hooks.
This is now the default and should be tested carefully with project-specific materializations and hooks. Projects that depend on autocommit-only behavior should set the flag to `false` during migration.

```yaml
# dbt_project.yml
flags:
dbt_sqlserver_enable_safe_type_expansion: true
dbt_sqlserver_use_dbt_transactions: true # <-- opt-in; default is false
dbt_sqlserver_use_dbt_transactions: true # default
```

### `dbt_sqlserver_use_native_string_types`

*(default: `true`)* Controls the SQL Server-native mappings used for dbt string types. With the default enabled, `STRING` maps to `VARCHAR(MAX)`, `NCHAR` maps to `NCHAR(1)`, and `NVARCHAR` maps to `NVARCHAR(4000)`. Set it to `false` to opt back into the deprecated legacy mappings: `STRING` and `NVARCHAR` map to `VARCHAR(8000)`, while `NCHAR` maps to `CHAR(1)`.

```yaml
flags:
dbt_sqlserver_use_native_string_types: false # deprecated legacy behavior
```

### `xact_abort`
Expand Down
2 changes: 1 addition & 1 deletion dbt/adapters/sqlserver/__version__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
version = "1.12.0rc2"
version = "1.12.0rc3"
5 changes: 2 additions & 3 deletions docs/adbc_backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,8 @@ pyodbc and mssql-python. ADBC (Arrow Database Connectivity) uses the
[Apache Arrow](https://arrow.apache.org/) columnar format natively, avoiding
the overhead of row-based ODBC/DB-API bridges.

> **Experimental.** The ADBC backend is new and opt-in. It passes the full
> dbt-sqlserver functional test suite (320 passed, 0 failed, as of v1.12.0-rc1)
> and is covered by a dedicated CI job against the latest SQL Server. Please
> **Experimental.** The ADBC backend is new and opt-in. It is covered by a
> dedicated CI job against the latest SQL Server. Please
> report issues at
> [dbt-msft/dbt-sqlserver#771](https://github.com/dbt-msft/dbt-sqlserver/issues/771).

Expand Down
2 changes: 1 addition & 1 deletion docs/sqlserver-best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ select * from {{ sqlserver__openquery(
The adapter ships
[`sqlserver__openquery`](../dbt/include/sqlserver/macros/utils/openquery.sql) for
this: it doubles single quotes in the remote SQL, strips carriage returns,
brackets the server name, and fails compilation with a specific message if the
quotes the server name through `adapter.quote()`, and fails compilation with a specific message if the
escaped query exceeds the 8 KB limit or either argument is empty. Writing
`OPENQUERY(...)` by hand means escaping every literal yourself — the date
predicate above would need `''2026-01-01''`.
Expand Down
2 changes: 1 addition & 1 deletion tests/functional/adapter/mssql/test_index_macros.py
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@
# name (#409), the dot and the quote reach identifiers built inside string
# literals. Raw string: what is written here is exactly what Jinja parses.
backslash_schema_model = r"""
{{ config(materialized='table', schema=target.schema ~ '_dom\\usr.x\"q') }}
{{ config(materialized='table', schema='dom\\usr.x\"q') }}
select 1 as id
"""

Expand Down
8 changes: 5 additions & 3 deletions tests/functional/adapter/mssql/test_masks.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ def select_as_unprivileged(project, table_name, columns):
# A schema whose name needs delimiters: a dot made the bare OBJECT_ID() lookup
# in the mask macros return NULL, so masks were silently never applied (#785).
dotted_schema_model_sql = """
{{ config(materialized="table", schema=target.schema ~ '.x') }}
{{ config(materialized="table", schema='dotted.x') }}
select cast('Smith' as varchar(50)) as surname
"""

Expand Down Expand Up @@ -119,7 +119,7 @@ def models(self):
@pytest.fixture(scope="class", autouse=True)
def drop_dotted_schema(self, project):
yield
schema = f"{project.test_schema}.x"
schema = f"{project.test_schema}_dotted.x"
with get_connection(project.adapter):
project.adapter.execute(f'DROP TABLE IF EXISTS "{schema}".dotted_schema_model')
project.adapter.execute(f'DROP SCHEMA IF EXISTS "{schema}"')
Expand All @@ -142,7 +142,9 @@ def test_masks_applied_in_a_schema_needing_delimiters(self, project):
"""A dot in the schema made OBJECT_ID() return NULL, so the mask
introspection found nothing and masks were silently skipped (#785)."""
run_dbt(["run"])
masks = masked_columns(project, "dotted_schema_model", schema=f"{project.test_schema}.x")
masks = masked_columns(
project, "dotted_schema_model", schema=f"{project.test_schema}_dotted.x"
)
assert masks.get("surname") == "default()"

def test_unprivileged_user_sees_masked_values(self, project):
Expand Down