v1.6.9

What's Changed

• Backport: Pin Databricks SDK, Pandas by @benc-db in #612

Full Changelog: v1.6.8...v1.6.9

v1.7.9

What's Changed

• Fix for U2M flow on windows (sharding long passwords) (thanks @thijs-nijhuis-shell!) (597)
• Fix regression in incremental behavior, and align more with dbt-core expectations (604)
• Don't fail for unknown types when listing schema (600)

Full Changelog: v1.7.8...v1.7.9

v1.8.0b1

This beta brings expanded support for materialized views and streaming tables. See this discussion for full details.

What's Changed

• Materialized View Config Level Up by @benc-db in #536
• Streaming Table Support by @benc-db in #569
• Upgrade databricks-sql-connector to 3.1.0 by @benc-db in #593

v1.7.8

What's Changed

Fixes

• Fixed the behavior of the incremental schema change ignore option to properly handle the scenario when columns are dropped (thanks @case-k-git!) (580)
• Fixed export of saved queries (thanks @peterallenwebb!) (588)
• Properly match against null for merging matching rows (590)

Full Changelog: v1.7.4...v1.7.8

v1.7.7

1.7.6 was accidentally pushed with beta Materialized View code, hence the skip to 1.7.7. This release rolls back databricks-sql-connector to version 2.9.3, but otherwise includes dbt-databricks 1.7.4 fixes, as version 3.0.x has retry behavior that was insufficient for connecting to cold SQL Warehouses. I will work on addressing that separately so that we can go back to using the 3.x branch of the sql connector.

v1.7.5

Update: This version has also been pulled. Unfortunately the change was not sufficient to fix the issue for affected customers.

What's Changed

• Pin to databricks-sdk 0.17.0 to fix SQL Warehouse connection issues by @benc-db in #571

Full Changelog: v1.7.4...v1.7.5

v1.7.4

UPDATE
This release has been pulled due to discovered connection issues with (non-serverless) SQL Warehouses. 1.7.5 with a fix will be released shortly.

For anyone waiting for materialized views / streaming tables updates, those will be coming in 1.7.6b1 (and possibly 1.7.6b2), but we had so many fixes piling up that we wanted to get this release out faster. In particular, pandas 2.2.0 release is causing havoc, so we are pinning to < 2.2.0.

What's Changed

Fixes

• Added python model specific connection handling to prevent using invalid sessions (547)
• Allow schema to be specified in testing (thanks @case-k-git!) (538)
• Fix dbt incremental_strategy behavior by fixing schema table existing check (thanks @case-k-git!) (530)
• Fixed bug that was causing streaming tables to be dropped and recreated instead of refreshed. (552)
• Fixed Hive performance regression by streamlining materialization type acquisition (557)
• Fix: Python models authentication could be overridden by a .netrc file in the user's home directory (338)
• Fix: MV/ST REST api authentication could be overriden by a .netrc file in the user's home directory (555)
• Show details in connection errors (562)
• Updated connection debugging logging and setting connection last used time on session open.(565)

Under the Hood

• Adding retries around API calls in python model submission (549)
• Upgrade to databricks-sql-connector 3.0.0 (554)
• Pinning pandas to < 2.2.0 to keep from breaking multiple tests (564)

New Contributors

• @case-k-git made their first contribution in #537
• @lennartkats-db made their first contribution in #562

Full Changelog: v1.7.3...v1.7.4

v1.6.8

What's Changed

Fixes

• Backport of fix for where we were invoking create schema or not exists when the schema already exists (leading to permission issue) (529)

Under the Hood

• Update dependency to dbt-spark 1.6.2 (thanks @ChenyuLInx!)

Full Changelog: v1.6.7...v1.6.8

v1.7.3

What's Changed

The big change in this release is that we fixed the issue where every single dbt action initiated a new connection to Databricks. We will now reuse a connection if there is a thread-local connection that matches the compute the user has selected.

This change will be most apparent if your dbt operations are very short lived, such as tests against a small table, as there is now less time spent in connection negotiation; for longer operations, the time spent in computing and transmitting the result set is more significant than the time spent on connecting.

If for some unforeseen reason this change negatively impacts performance:

a.) You can turn it off by setting the DBT_DATABRICKS_LONG_SESSIONS environment variable to false.
b.) Please file an issue so we can investigate.

Fixes

• Allow session re-use by @rcypher-databricks in #517
• Fixing list_schemas so that we stop trying to create schemas that exist by @benc-db in #529

Under the Hood

• Refactor macro tests so that we can move macros by @benc-db in #524
• Updating Python Functional Tests by @benc-db in #526
• Refactoring to align with dbt-core organization: Part I by @benc-db in #525

Full Changelog: v1.7.2...v1.7.3

1.7.2

The big news is that the ability to choose separate compute by model is now available. Until I get updated docs out, please look here for usage notes: #333 (comment)

What's Changed

• Updates to warehouse-per-model logging and tests by @rcypher-databricks in #511
• Persist docs on incremental only if changed by @benc-db in #513
• Enable access control for job runs by @benc-db (thanks @srggrs!) in #518
• View column comments can now be persisted, hive docs generate now works by @benc-db in #519

Full Changelog: v1.7.2b2...v1.7.2