diff --git a/website/docs/guides/databricks-qs.md b/website/docs/guides/databricks-qs.md index b969786b384..bb248e09320 100644 --- a/website/docs/guides/databricks-qs.md +++ b/website/docs/guides/databricks-qs.md @@ -169,16 +169,63 @@ If you get a session error and don’t get redirected to this page, you can go b There are two ways to connect dbt Cloud to Databricks. The first option is Partner Connect, which provides a streamlined setup to create your dbt Cloud account from within your new Databricks trial account. The second option is to create your dbt Cloud account separately and build the Databricks connection yourself (connect manually). If you want to get started quickly, dbt Labs recommends using Partner Connect. If you want to customize your setup from the very beginning and gain familiarity with the dbt Cloud setup flow, dbt Labs recommends connecting manually. -If you want to use Partner Connect, refer to [Connect to dbt Cloud using Partner Connect](https://docs.databricks.com/partners/prep/dbt-cloud.html#connect-to-dbt-cloud-using-partner-connect) in the Databricks docs for instructions. +## Set up the integration from Partner Connect -If you want to connect manually, refer to [Connect to dbt Cloud manually](https://docs.databricks.com/partners/prep/dbt-cloud.html#connect-to-dbt-cloud-manually) in the Databricks docs for instructions. +:::note + Partner Connect is intended for trial partner accounts. If your organization already has a dbt Cloud account, connect manually. Refer to [Connect to dbt Cloud manually](https://docs.databricks.com/partners/prep/dbt-cloud.html#connect-to-dbt-cloud-manually) in the Databricks docs for instructions. +::: + +To connect dbt Cloud to Databricks using Partner Connect, do the following: + +1. In the sidebar of your Databricks account, click **Partner Connect**. + +2. Click the **dbt tile**. + +3. Select a catalog from the drop-down list, and then click **Next**. The drop-down list displays catalogs you have read and write access to. If your workspace isn't `-enabled`, the legacy Hive metastore (`hive_metastore`) is used. + +5. If there are SQL warehouses in your workspace, select a SQL warehouse from the drop-down list. If your SQL warehouse is stopped, click **Start**. + +6. If there are no SQL warehouses in your workspace: + + 1. Click **Create warehouse**. A new tab opens in your browser that displays the **New SQL Warehouse** page in the Databricks SQL UI. + 2. Follow the steps in [Create a SQL warehouse](https://docs.databricks.com/en/sql/admin/create-sql-warehouse.html#create-a-sql-warehouse) in the Databricks docs. + 3. Return to the Partner Connect tab in your browser, and then close the **dbt tile**. + 4. Re-open the **dbt tile**. + 5. Select the SQL warehouse you just created from the drop-down list. + +7. Select a schema from the drop-down list, and then click **Add**. The drop-down list displays schemas you have read and write access to. You can repeat this step to add multiple schemas. -## Set up a dbt Cloud managed repository -If you used Partner Connect, you can skip to [initializing your dbt project](#initialize-your-dbt-project-and-start-developing) as the Partner Connect provides you with a managed repository. Otherwise, you will need to create your repository connection. + Partner Connect creates the following resources in your workspace: + + - A Databricks service principal named **DBT_CLOUD_USER**. + - A Databricks personal access token that is associated with the **DBT_CLOUD_USER** service principal. + + Partner Connect also grants the following privileges to the **DBT_CLOUD_USER** service principal: + + - (Unity Catalog) **USE CATALOG**: Required to interact with objects within the selected catalog. + - (Unity Catalog) **USE SCHEMA**: Required to interact with objects within the selected schema. + - (Unity Catalog) **CREATE SCHEMA**: Grants the ability to create schemas in the selected catalog. + - (Hive metastore) **USAGE**: Required to grant the **SELECT** and **READ_METADATA** privileges for the schemas you selected. + - **SELECT**: Grants the ability to read the schemas you selected. + - (Hive metastore) **READ_METADATA**: Grants the ability to read metadata for the schemas you selected. + - **CAN_USE**: Grants permissions to use the SQL warehouse you selected. + +8. Click **Next**. + + The **Email** box displays the email address for your Databricks account. dbt Labs uses this email address to prompt you to create a trial dbt Cloud account. + +9. Click **Connect to dbt Cloud**. + + A new tab opens in your web browser, which displays the getdbt.com website. + +10. Complete the on-screen instructions on the getdbt.com website to create your trial dbt Cloud account. + +## Set up a dbt Cloud managed repository ## Initialize your dbt project​ and start developing + Now that you have a repository configured, you can initialize your project and start development in dbt Cloud: 1. Click **Start developing in the IDE**. It might take a few minutes for your project to spin up for the first time as it establishes your git connection, clones your repo, and tests the connection to the warehouse. diff --git a/website/docs/reference/resource-configs/schema.md b/website/docs/reference/resource-configs/schema.md index 311dc01e767..5a1a61d3943 100644 --- a/website/docs/reference/resource-configs/schema.md +++ b/website/docs/reference/resource-configs/schema.md @@ -149,14 +149,16 @@ seeds: ### Tests -Customize the name of the schema in which tests [configured to store failures](/reference/resource-configs/store_failures) will save their results: +Customize the name of the schema in which tests [configured to store failures](/reference/resource-configs/store_failures) will save their results. +The resulting schema is `{{ profile.schema }}_{{ tests.schema }}`, with a default suffix of `dbt_test__audit`. +To use the same profile schema, set `+schema: null`. ```yml tests: +store_failures: true - +schema: the_island_of_misfit_tests + +schema: _sad_test_failures # Will write tables to my_database.my_schema__sad_test_failures ``` diff --git a/website/docs/reference/resource-configs/store_failures.md b/website/docs/reference/resource-configs/store_failures.md index 633895b4fe2..26b5b372701 100644 --- a/website/docs/reference/resource-configs/store_failures.md +++ b/website/docs/reference/resource-configs/store_failures.md @@ -10,7 +10,7 @@ Optionally set a test to always or never store its failures in the database. - If specified as `true` or `false`, the `store_failures` config will take precedence over the presence or absence of the `--store-failures` flag. - If the `store_failures` config is `none` or omitted, the resource will use the value of the `--store-failures` flag. -- When true, `store_failures` saves all the record(s) that failed the test only if [limit](/reference/resource-configs/limit) is not set or if there are fewer records than the limit. `store_failures` are saved in a new table with the name of the test. +- When true, `store_failures` saves all records (up to [limit](/reference/resource-configs/limit)) that failed the test. Failures are saved in a new table with the name of the test. By default, `store_failures` uses the schema `{{ profile.schema }}_dbt_test__audit`, but you can [configure](/reference/resource-configs/schema#tests) the schema suffix to a different value. - A test's results will always **replace** previous failures for the same test, even if that test results in no failures. - By default, `store_failures` uses a schema named `dbt_test__audit`, but, you can [configure](/reference/resource-configs/schema#tests) the schema to a different value. Ensure you have the authorization to create or access schemas for your work. For more details, refer to the [FAQ](#faqs).