Merge branch 'current' into amychen1776-patch-1

website/blog/2023-11-14-specify-prod-environment.md

@@ -14,8 +14,8 @@ is_featured: false
 
 ---
 
-:::note You can now use a Staging environment!
-This blog post was written before Staging environments. You can now use dbt Cloud can to support the patterns discussed here. Read more about [Staging environments](/docs/deploy/deploy-environments#staging-environment).
+:::note You can now specify a Staging environment too!
+This blog post was written before dbt Cloud added full support for Staging environments. Now that they exist, you should mark your CI environment as Staging as well. Read more about [Staging environments](/docs/deploy/deploy-environments#staging-environment).
 :::
 
 :::tip The Bottom Line:

website/docs/docs/build/custom-aliases.md

@@ -157,3 +157,8 @@ If these models should indeed have the same database identifier, you can work ar
 
 By default, dbt will create versioned models with the alias `<model_name>_v<v>`, where `<v>` is that version's unique identifier. You can customize this behavior just like for non-versioned models by configuring a custom `alias` or re-implementing the `generate_alias_name` macro.
 
+## Related docs
+
+- [Customize dbt models database, schema, and alias](/guides/customize-schema-alias?step=1) to learn how to customize dbt models database, schema, and alias
+- [Custom schema](/docs/build/custom-schemas) to learn how to customize dbt schema 
+- [Custom database](/docs/build/custom-databases) to learn how to customize dbt database

website/docs/docs/build/custom-databases.md

@@ -98,3 +98,9 @@ See docs on macro `dispatch`: ["Managing different global overrides across packa
 ### BigQuery
 
 When dbt opens a BigQuery connection, it will do so using the `project_id` defined in your active `profiles.yml` target. This `project_id` will be billed for the queries that are executed in the dbt run, even if some models are configured to be built in other projects.
+
+## Related docs
+
+- [Customize dbt models database, schema, and alias](/guides/customize-schema-alias?step=1) to learn how to customize dbt models database, schema, and alias
+- [Custom schema](/docs/build/custom-schemas) to learn how to customize dbt model schema
+- [Custom aliases](/docs/build/custom-aliases) to learn how to customize dbt model alias name

website/docs/docs/build/custom-schemas.md

@@ -207,3 +207,9 @@ In the `generate_schema_name` macro examples shown in the [built-in alternative
 If your schema names are being generated incorrectly, double-check your target name in the relevant environment.
 
 For more information, consult the [managing environments in dbt Core](/docs/core/dbt-core-environments) guide.
+
+## Related docs
+
+- [Customize dbt models database, schema, and alias](/guides/customize-schema-alias?step=1) to learn how to customize dbt models database, schema, and alias
+- [Custom database](/docs/build/custom-databases) to learn how to customize dbt model database
+- [Custom aliases](/docs/build/custom-aliases) to learn how to customize dbt model alias name

website/docs/docs/build/metricflow-time-spine.md

@@ -7,7 +7,6 @@ tags: [Metrics, Semantic Layer]
 ---
 <VersionBlock firstVersion="1.9">
 
-<!-- this whole section is for 1.9 and higher + Release Tracks -->
 
 It's common in analytics engineering to have a date dimension or "time spine" table as a base table for different types of time-based joins and aggregations. The structure of this table is typically a base column of daily or hourly dates, with additional columns for other time grains, like fiscal quarters, defined based on the base column. You can join other tables to the time spine on the base column to calculate metrics like revenue at a point in time, or to aggregate to a specific time grain.
 
@@ -23,7 +22,7 @@ To see the generated SQL for the metric and dimension types that use time spine
 
 ## Configuring time spine in YAML
 
- Time spine models are normal dbt models with extra configurations that tell dbt and MetricFlow how to use specific columns by defining their properties. Add the [`models` key](/reference/model-properties) for the time spine in your `models/` directory. If your project already includes a calendar table or date dimension, you can configure that table as a time spine. Otherwise, review the [example time-spine tables](#example-time-spine-tables) to create one.
+ Time spine models are normal dbt models with extra configurations that tell dbt and MetricFlow how to use specific columns by defining their properties. Add the [`models` key](/reference/model-properties) for the time spine in your `models/` directory. If your project already includes a calendar table or date dimension, you can configure that table as a time spine. Otherwise, review the [example time-spine tables](#example-time-spine-tables) to create one. If the relevant model file (`util/_models.yml`) doesn't exist, create it and add the configuration mentioned in the [next section](#creating-a-time-spine-table).
 
  Some things to note when configuring time spine models:
 
@@ -34,9 +33,9 @@ To see the generated SQL for the metric and dimension types that use time spine
 - If you're looking to specify the grain of a time dimension so that MetricFlow can transform the underlying column to the required granularity, refer to the [Time granularity documentation](/docs/build/dimensions?dimension=time_gran)
 
 :::tip
-If you previously used a model called `metricflow_time_spine`, you no longer need to create this specific model. You can now configure MetricFlow to use any date dimension or time spine table already in your project by updating the `model` setting in the Semantic Layer.
-
-If you don’t have a date dimension table, you can still create one by using the code snippet in the [next section](#creating-a-time-spine-table) to build your time spine model.
+- If you previously used a `metricflow_time_spine.sql` model, you can delete it after configuring the `time_spine` property in YAML. The Semantic Layer automatically recognizes the new configuration. No additional `.yml` files are needed. 
+- You can also configure MetricFlow to use any date dimension or time spine table already in your project by updating the `model` setting in the Semantic Layer.
+- If you don’t have a date dimension table, you can still create one by using the code snippet in the [next section](#creating-a-time-spine-table) to build your time spine model.
 :::
 
 ### Creating a time spine table  
@@ -112,9 +111,37 @@ models:
 
 For an example project, refer to our [Jaffle shop](https://github.com/dbt-labs/jaffle-sl-template/blob/main/models/marts/_models.yml) example.
 
+### Migrating from SQL to YAML
+If your project already includes a time spine (`metricflow_time_spine.sql`), you can migrate its configuration to YAML to address any deprecation warnings you may get.
+
+1. Add the following configuration to a new or existing YAML file using the [`models` key](/reference/model-properties) for the time spine in your `models/` directory. Name the YAML file whatever you want (for example, `util/_models.yml`):
+
+  <File name="models/_models.yml">
+
+  ```yaml
+  models:
+    - name: all_days
+      description: A time spine with one row per day, ranging from 2020-01-01 to 2039-12-31.
+      time_spine:
+        standard_granularity_column: date_day  # Column for the standard grain of your table
+      columns:
+        - name: date_day
+          granularity: day  # Set the granularity of the column
+  ```
+  </File>
+
+2. After adding the YAML configuration, delete the existing `metricflow_time_spine.sql` file from your project to avoid any issues.
+
+3. Test the configuration to ensure compatibility with your production jobs.
+
+Note that if you're migrating from a `metricflow_time_spine.sql` file:
+
+- Replace its functionality by adding the `time_spine` property to YAML as shown in the previous example.
+- Once configured, MetricFlow will recognize the YAML settings, and then the SQL model file can be safely removed.
+
 ### Considerations when choosing which granularities to create{#granularity-considerations}
 
-- MetricFlow will use the time spine with the largest compatible granularity for a given query to ensure the most efficient query possible. For example, if you have a time spine at a monthly grain, and query a dimension at a monthly grain, MetricFlow will use the monthly time spine. If you only have a daily time spine, MetricFlow will use the daily time spine and date_trunc to month.
+- MetricFlow will use the time spine with the largest compatible granularity for a given query to ensure the most efficient query possible. For example, if you have a time spine at a monthly grain, and query a dimension at a monthly grain, MetricFlow will use the monthly time spine. If you only have a daily time spine, MetricFlow will use the daily time spine and `date_trunc` to month.
 - You can add a time spine for each granularity you intend to use if query efficiency is more important to you than configuration time, or storage constraints. For most engines, the query performance difference should be minimal and transforming your time spine to a coarser grain at query time shouldn't add significant overhead to your queries.
 - We recommend having a time spine at the finest grain used in any of your dimensions to avoid unexpected errors. For example, if you have dimensions at an hourly grain, you should have a time spine at an hourly grain.
 

website/docs/docs/build/sources.md

@@ -14,7 +14,7 @@ search_weight: "heavy"
 
 ## Using sources
 Sources make it possible to name and describe the data loaded into your warehouse by your Extract and Load tools. By declaring these tables as sources in dbt, you can then
-- select from source tables in your models using the `{{ source() }}` function, helping define the lineage of your data
+- select from source tables in your models using the [`{{ source() }}` function,](/reference/dbt-jinja-functions/source) helping define the lineage of your data
 - test your assumptions about your source data
 - calculate the freshness of your source data
 

website/docs/docs/build/unit-tests.md

@@ -10,9 +10,6 @@ keywords:
 
 <VersionCallout version="1.8" />
 
-
-
-
 Historically, dbt's test coverage was confined to [“data” tests](/docs/build/data-tests), assessing the quality of input data or resulting datasets' structure. However, these tests could only be executed _after_ building a model. 
 
 Starting in dbt Core v1.8, we have introduced an additional type of test to dbt - unit tests. In software programming, unit tests validate small portions of your functional code, and they work much the same way here. Unit tests allow you to validate your SQL modeling logic on a small set of static inputs _before_ you materialize your full model in production. Unit tests enable test-driven development, benefiting developer efficiency and code reliability. 
@@ -219,10 +216,19 @@ dbt test --select test_is_valid_email_address
 
 Your model is now ready for production! Adding this unit test helped catch an issue with the SQL logic _before_ you materialized `dim_customers` in your warehouse and will better ensure the reliability of this model in the future. 
 
-
 ## Unit testing incremental models
 
-When configuring your unit test, you can override the output of macros, vars, or environment variables. This enables you to unit test your incremental models in "full refresh" and "incremental" modes. 
+When configuring your unit test, you can override the output of macros, vars, or environment variables. This enables you to unit test your incremental models in "full refresh" and "incremental" modes.
+
+:::note
+Incremental models need to exist in the database first before running unit tests or doing a `dbt build`. Use the [`--empty` flag](/reference/commands/build#the---empty-flag) to build an empty version of the models to save warehouse spend. You can also optionally select only your incremental models using the [`--select` flag](/reference/node-selection/syntax#shorthand).
+
+  ```shell
+  dbt run --select "config.materialized:incremental" --empty
+  ```
+
+  After running the command, you can then perform a regular `dbt build` for that model and then run your unit test.
+:::
 
 When testing an incremental model, the expected output is the __result of the materialization__ (what will be merged/inserted), not the resulting model itself (what the final table will look like after the merge/insert).
 

website/docs/docs/cloud/git/git-configuration-in-dbt-cloud.md

@@ -43,4 +43,9 @@ Whether you use a Git integration that natively connects with dbt Cloud or prefe
     link="/docs/cloud/git/connect-azure-devops"
     icon="dbt-bit"/>
 
+<Card
+    title="Availability of CI features by Git provider"
+    body="Learn which Git providers have native support for Continuous Integration workflows"
+    link="/docs/deploy/continuous-integration#git-providers-who-support-ci"
+    icon="dbt-bit"/>
 </div>

website/docs/docs/cloud/git/import-a-project-by-git-url.md

@@ -8,6 +8,10 @@ In dbt Cloud, you can import a git repository from any valid git URL that points
 ## Git protocols
 You must use the `git@...` or `ssh:..`. version of your git URL, not the `https://...` version. dbt Cloud uses the SSH protocol to clone repositories, so dbt Cloud will be unable to clone repos supplied with the HTTP protocol.
 
+import GitProvidersCI from '/snippets/_git-providers-supporting-ci.md';
+
+<GitProvidersCI />
+
 ## Managing deploy keys
 
 After importing a project by Git URL, dbt Cloud will generate a Deploy Key for your repository. To find the deploy key in dbt Cloud:

website/docs/docs/cloud/manage-access/audit-log.md

@@ -102,9 +102,11 @@ The audit log supports various events for different objects in dbt Cloud. You wi
 | Invite Added                 | invite.Added              | User invitation added and sent to the user      |
 | Invite Redeemed              | invite.Redeemed           | User redeemed invitation                        |
 | User Added to Account        | account.UserAdded         | New user added to the account                   |
-| User Added to Group          | user_group_user.Added     | An existing user is added to a group            |
-| User Removed from Account    | account.UserRemoved       | User removed from the account
-| User Removed from Group      | user_group_user.Removed   | An existing user is removed from a group        |
+| User Added to Group          | user_group_user.Added     | An existing user was added to a group            |
+| User Removed from Account    | account.UserRemoved       | User removed from the account                  |
+| User Removed from Group      | user_group_user.Removed   | An existing user was removed from a group        |
+| User License Created         | user_license.added        | A new user license was consumed                  |
+| User License Removed         | user_license.removed      | A user license was removed from the seat count   |
 | Verification Email Confirmed | user.jit.email.Confirmed  | Email verification confirmed by user            |
 | Verification Email Sent      | user.jit.email.Sent       | Email verification sent to user created via JIT |
 

website/docs/docs/cloud/manage-access/set-up-sso-okta.md

@@ -75,16 +75,16 @@ so pick a slug that uniquely identifies your company.
 * **Single sign on URL**: `https://YOUR_AUTH0_URI/login/callback?connection=<login slug>`
 * **Audience URI (SP Entity ID)**: `urn:auth0:<YOUR_AUTH0_ENTITYID>:{login slug}`
 * **Relay State**: `<login slug>`
+* **Name ID format**: `Unspecified`
+* **Application username**: `Custom` / `user.getInternalProperty("id")`
+* **Update Application username on**: `Create and update`
 
 <Lightbox
     collapsed={false}
     src="/img/docs/dbt-cloud/dbt-cloud-enterprise/okta/okta-3-saml-settings-top.png"
     title="Configure the app's SAML Settings"
 />
 
-<!-- TODO : Will users need to change the Name ID format and Application
-username on this screen? -->
-
 Use the **Attribute Statements** and **Group Attribute Statements** forms to
 map your organization's Okta User and Group Attributes to the format that
 dbt Cloud expects.

website/docs/docs/cloud/manage-access/set-up-sso-saml-2.0.md

@@ -59,7 +59,9 @@ Additionally, you may configure the IdP attributes passed from your identity pro
 | email | Unspecified | user.email | The user's email address |
 | first_name | Unspecified | user.first_name | The user's first name |
 | last_name | Unspecified | user.last_name | The user's last name |
-| NameID (if applicable) | Unspecified | user.email | The user's email address |
+| NameID | Unspecified | ID | The user's unchanging ID |
+
+`NameID` values can be persistent (`urn:oasis:names:tc:SAML:2.0:nameid-format:persistent`) rather than unspecified if your IdP supports these values.  Using an email address for `NameID` will work, but dbt Cloud creates an entirely new user if that email address changes.  Configuring a value that will not change, even if the user's email address does, is a best practice.
 
 dbt Cloud's [role-based access control](/docs/cloud/manage-access/about-user-access#role-based-access-control) relies
 on group mappings from the IdP to assign dbt Cloud users to dbt Cloud groups. To
@@ -144,6 +146,9 @@ Login slugs must be unique across all dbt Cloud accounts, so pick a slug that un
    * **Single sign on URL**: `https://YOUR_AUTH0_URI/login/callback?connection=<login slug>`
    * **Audience URI (SP Entity ID)**: `urn:auth0:<YOUR_AUTH0_ENTITYID>:<login slug>`
    * **Relay State**: `<login slug>`
+   * **Name ID format**: `Unspecified`
+   * **Application username**: `Custom` / `user.getInternalProperty("id")`
+   * **Update Application username on**: `Create and update`
 
   <Lightbox collapsed={false} src="/img/docs/dbt-cloud/dbt-cloud-enterprise/okta/okta-3-saml-settings-top.png" title="Configure the app's SAML Settings"/>
 
@@ -245,7 +250,7 @@ Login slugs must be unique across all dbt Cloud accounts, so pick a slug that un
    * **Audience URI (SP Entity ID)**: `urn:auth0:<YOUR_AUTH0_ENTITYID>:<login slug>`
    - **Start URL**: `<login slug>`
 5. Select the **Signed response** checkbox.
-6. The default **Name ID** is the primary email. Multi-value input is not supported.
+6. The default **Name ID** is the primary email. Multi-value input is not supported.  If your user profile has a unique, stable value that will persist across email address changes, it's best to use that; otherwise, email will work.
 7. Use the **Attribute mapping** page to map your organization's Google Directory Attributes to the format that
 dbt Cloud expects.
 8. Click **Add another mapping** to map additional attributes.
@@ -329,20 +334,22 @@ Follow these steps to set up single sign-on (SSO) with dbt Cloud:
 From the Set up Single Sign-On with SAML page:
 
 1. Click **Edit** in the User Attributes & Claims section.
-2. Leave the claim under "Required claim" as is.
-3. Delete all claims under "Additional claims."
-4. Click **Add new claim** and add these three new claims:
+2. Click **Unique User Identifier (Name ID)** under **Required claim.**
+3. Set **Name identifier format** to **Unspecified**.
+4. Set **Source attribute** to **user.objectid**.
+5. Delete all claims under **Additional claims.**
+6. Click **Add new claim** and add the following new claims:
 
    | Name | Source attribute |
    | ----- | ----- |
    | **email** | user.mail |
    | **first_name** | user.givenname |
    | **last_name** | user.surname |
 
-5. Click **Add a group claim** from User Attributes and Claims.
-6. If you'll assign users directly to the enterprise application, select **Security Groups**. If not, select **Groups assigned to the application**.
-7. Set **Source attribute** to **Group ID**.
-8. Under **Advanced options**, check **Customize the name of the group claim** and specify **Name** to **groups**.
+7. Click **Add a group claim** from **User Attributes and Claims.**
+8. If you assign users directly to the enterprise application, select **Security Groups**. If not, select **Groups assigned to the application**.
+9. Set **Source attribute** to **Group ID**.
+10. Under **Advanced options**, check **Customize the name of the group claim** and specify **Name** to **groups**.
 
 **Note:** Keep in mind that the Group ID in Entra ID maps to that group's GUID. It should be specified in lowercase for the mappings to work as expected. The Source Attribute field alternatively can be set to a different value of your preference.
 
@@ -386,7 +393,7 @@ We recommend using the following values:
 
 | name | name format | value |
 | ---- | ----------- | ----- |
-| NameID | Unspecified | Email |
+| NameID | Unspecified | OneLogin ID |
 | email | Unspecified | Email |
 | first_name | Unspecified | First Name |
 | last_name | Unspecified | Last Name |