Merge branch 'current' into v1-9-beta

website/docs/best-practices/how-we-build-our-metrics/semantic-layer-9-conclusion.md

@@ -28,6 +28,7 @@ pagination_next: null
 
 - 🗺️ Use these best practices to map out your team's plan to **incrementally adopt the Semantic Layer**.
 - 🤗 Get involved in the community and ask questions, **help craft best practices**, and share your progress in building a dbt Semantic Layer.
+- [Validate semantic nodes in CI](/docs/deploy/ci-jobs#semantic-validations-in-ci) to ensure code changes made to dbt models don't break these metrics.
 
 The dbt Semantic Layer is the biggest paradigm shift thus far in the young practice of analytics engineering. It's ready to provide value right away, but is most impactful if you move your project towards increasing normalization, and allow MetricFlow to do the denormalization for you with maximum dimensionality.
 

website/docs/docs/build/saved-queries.md

@@ -230,5 +230,5 @@ To include all saved queries in the dbt build run, use the [`--resource-type` fl
 </detailsToggle>
 
 ## Related docs
-
+- [Validate semantic nodes in a CI job](/docs/deploy/ci-jobs#semantic-validations-in-ci)
 - Configure [caching](/docs/use-dbt-semantic-layer/sl-cache)

website/docs/docs/cloud/configure-cloud-cli.md

@@ -110,17 +110,34 @@ As a tip, most command-line tools have a `--help` flag to show available command
 - `dbt run --help`: Lists the flags available for the `run` command
 :::
 
-### SQLFluff integration
-The dbt Cloud CLI supports [SQLFluff](https://sqlfluff.com/), a modular and configuration SQL linter, which warns you of complex functions, syntax, formatting, and compilation errors.
+### Lint SQL files 
 
-To get started, run `dbt sqlfluff -h` to see the list of supported commands and flags, such as `dbt sqlfluff lint` to lint SQL files.
+From the dbt Cloud CLI, you can invoke [SQLFluff](https://sqlfluff.com/) which is a modular and configurable SQL linter that warns you of complex functions, syntax, formatting, and compilation errors. Many of the same flags that you can pass to SQLFluff are available from the dbt Cloud CLI.
+
+The available SQLFluff commands are: 
+
+- `lint` — Lint SQL files by passing a list of files or from standard input (stdin).
+- `fix` — Fix SQL files.
+- `format` — Autoformat SQL files.
+
+
+To lint SQL files, run the command as follows:  
+
+```shell
+dbt sqlfluff lint [PATHS]... [flags]
+```
+
+When no path is set, dbt lints all SQL files in the current project. To lint a specific SQL file or a directory, set `PATHS` to the path of the SQL file(s) or directory of files. To lint multiple files or directories, pass multiple `PATHS` flags.  
+
+To show detailed information on all the dbt supported commands and flags, run the `dbt sqlfluff -h` command. 
 
 #### Considerations
-Keep the following points in mind when using SQLFluff with the dbt Cloud:
 
-- When you run `dbt sqlfluff`, it picks up changes to your local .sqlfluff config.
-- To use SQLFluff in continuous integration/continuous development, you need to have a `dbt_cloud.yml` file in your project and run commands from a valid dbt project.
-- SQLFluff commands in the dbt Cloud CLI do not return exit codes yet.
+When running `dbt sqlfluff` from the dbt Cloud CLI, the following are important behaviors to consider:
+
+- dbt reads the `.sqlfluff` file, if it exists, for any custom configurations you might have.
+- For continuous integration/continuous development (CI/CD) workflows, your project must have a `dbt_cloud.yml` file and you have successfully run commands from within this dbt project.
+- An SQLFluff command will return an exit code of 0 if it ran without any file violations. This dbt behavior differs from SQLFluff behavior, where a linting violation returns a non-zero exit code. dbt Labs plans on addressing this in a later release.
 
 ## FAQs
 <Expandable alt_header="How to create a .dbt directory and move your file">

website/docs/docs/dbt-versions/release-notes.md

@@ -18,6 +18,10 @@ Release notes are grouped by month for both multi-tenant and virtual private clo
 
 [^*] The official release date for this new format of release notes is May 15th, 2024. Historical release notes for prior dates may not reflect all available features released earlier this year or their tenancy availability. 
 
+## July 2024
+- **New**: The ability to lint your SQL files from the dbt Cloud CLI is now available. To learn more, refer to [Lint SQL files](/docs/cloud/configure-cloud-cli#lint-sql-files).
+- **New**: Introduced Semantic validations in CI pipelines. Automatically test your semantic nodes (metrics, semantic models, and saved queries) during code reviews by adding warehouse validation checks in your CI job using the `dbt sl validate` command. You can also validate modified semantic nodes to guarantee code changes made to dbt models don't break these metrics. Refer to [Semantic validations in CI](/docs/deploy/ci-jobs#semantic-validations-in-ci) to learn about the additional commands and use cases.
+
 ## June 2024
 - **New:** Introduced new granularity support for cumulative metrics in MetricFlow. Granularity options for cumulative metrics are slightly different than granularity for other metric types. For other metrics, we use the `date_trunc` function to implement granularity. However, because cumulative metrics are non-additive (values can't be added up), we can't use the `date_trunc` function to change their time grain granularity. 
 

website/docs/docs/deploy/ci-jobs.md

@@ -10,7 +10,6 @@ You can set up [continuous integration](/docs/deploy/continuous-integration) (CI
 
 dbt Labs recommends that you create your CI job in a dedicated dbt Cloud [deployment environment](/docs/deploy/deploy-environments#create-a-deployment-environment) that's connected to a staging database. Having a separate environment dedicated for CI will provide better isolation between your temporary CI schema builds and your production data builds. Additionally, sometimes teams need their CI jobs to be triggered when a PR is made to a branch other than main. If your team maintains a staging branch as part of your release process, having a separate environment will allow you to set a [custom branch](/faqs/environments/custom-branch-settings) and, accordingly, the CI job in that dedicated environment will be triggered only when PRs are made to the specified custom branch. To learn more, refer to [Get started with CI tests](/guides/set-up-ci).
 
-
 ### Prerequisites
 - You have a dbt Cloud account. 
 - For the [Concurrent CI checks](/docs/deploy/continuous-integration#concurrent-ci-checks) and [Smart cancellation of stale builds](/docs/deploy/continuous-integration#smart-cancellation) features, your dbt Cloud account must be on the [Team or Enterprise plan](https://www.getdbt.com/pricing/).
@@ -77,6 +76,92 @@ If you're not using dbt Cloud’s native Git integration with [GitHub](/docs/cl
       - `non_native_pull_request_id` (for example, BitBucket)
    - Provide the `git_sha` or `git_branch` to target the correct commit or branch to run the job against. 
 
+## Semantic validations in CI  <Lifecycle status="team,enterprise" />
+
+Automatically test your semantic nodes (metrics, semantic models, and saved queries) during code reviews by adding warehouse validation checks in your CI job, guaranteeing that any code changes made to dbt models don't break these metrics. 
+
+To do this, add the command `dbt sl validate --select state:modified+` in the CI job. This ensures the validation of modified semantic nodes and their downstream dependencies.
+
+- Testing semantic nodes in a CI job supports deferral and selection of semantic nodes.
+- It allows you to catch issues early in the development process and deliver high-quality data to your end users.
+- Semantic validation executes an explain query in the data warehouse for semantic nodes to ensure the generated SQL will execute.
+- For semantic nodes and models that aren't downstream of modified models, dbt Cloud defers to the production models
+
+To learn how to set this up, refer to the following steps:
+
+1. Navigate to the **Job setting** page and click **Edit**.
+2. Add the `dbt sl validate --select state:modified+` command under **Commands** in the **Execution settings** section. The command uses state selection and deferral to run validation on any semantic nodes downstream of model changes. To reduce job times, we recommend only running CI on modified semantic models.
+3. Click **Save** to save your changes.
+
+There are additional commands and use cases described in the [next section](#use-cases), such as validating all semantic nodes, validating specific semantic nodes, and so on.
+
+<Lightbox src="/img/docs/dbt-cloud/deployment/ci-dbt-sl-validate-downstream.jpg" width="90%" title="Validate semantic nodes downstream of model changes in your CI job." />
+
+### Use cases
+
+Use or combine different selectors or commands to validate semantic nodes in your CI job. Semantic validations in CI supports the following use cases:
+
+<Expandable alt_header="Semantic nodes downstream of model changes (recommended)" > 
+
+To validate semantic nodes that are downstream of a model change, add the two commands in your job **Execution settings** section:
+
+```bash
+dbt build --select state:modified+
+dbt sl validate --select state:modified+
+```
+
+- The first command builds the modified models.
+- The second command validates the semantic nodes downstream of the modified models.
+
+Before running semantic validations, dbt Cloud must build the modified models. This process ensures that downstream semantic nodes are validated using the CI schema through the dbt Semantic Layer API. 
+
+For semantic nodes and models that aren't downstream of modified models, dbt Cloud defers to the production models.
+
+<Lightbox src="/img/docs/dbt-cloud/deployment/ci-dbt-sl-validate-downstream.jpg" width="90%" title="Validate semantic nodes downstream of model changes in your CI job." />
+
+</Expandable>
+
+<Expandable alt_header="Semantic nodes that are modified or affected by downstream modified nodes.">
+
+To only validate modified semantic nodes, use the following command (with [state selection](/reference/node-selection/syntax#stateful-selection)):
+
+```bash
+dbt sl validate --select state:modified+
+```
+
+<Lightbox src="/img/docs/dbt-cloud/deployment/ci-dbt-sl-validate-modified.jpg" width="90%" title="Use state selection to validate modified metric definition models in your CI job." />
+
+This will only validate semantic nodes. It will use the defer state set configured in your orchestration job, deferring to your production models.
+
+</Expandable>
+
+<Expandable alt_header="Select specific semantic nodes">
+
+Use the selector syntax to select the _specific_ semantic node(s) you want to validate:
+
+```bash
+dbt sl validate --select metric:revenue
+```
+
+<Lightbox src="/img/docs/dbt-cloud/deployment/ci-dbt-sl-validate-select.jpg" width="90%" title="Use state selection to validate modified metric definition models in your CI job." />
+
+In this example, the CI job will validate the selected `metric:revenue` semantic node. To select multiple semantic nodes, use the selector syntax: `dbt sl validate --select metric:revenue metric:customers`.
+
+If you don't specify a selector, dbt Cloud will validate all semantic nodes in your project.
+
+</Expandable>
+
+<Expandable alt_header="Select all semantic nodes">
+
+To validate _all_ semantic nodes in your project, add the following command to defer to your production schema when generating the warehouse validation queries:
+
+   ```bash
+   dbt sl validate
+   ```
+
+<Lightbox src="/img/docs/dbt-cloud/deployment/ci-dbt-sl-validate-all.jpg" width="90%" title="Validate all semantic nodes in your CI job by adding the command: 'dbt sl validate' in your job execution settings." />
+
+</Expandable>
 
 ## Troubleshooting
 

website/docs/docs/deploy/continuous-integration.md

@@ -16,9 +16,9 @@ Using CI helps:
 
 ## How CI works
 
-When you [set up CI jobs](/docs/deploy/ci-jobs#set-up-ci-jobs), dbt Cloud listens for notification from your Git provider indicating that a new PR has been opened or updated with new commits. When dbt Cloud receives one of these notifications, it enqueues a new run of the CI job. 
+When you [set up CI jobs](/docs/deploy/ci-jobs#set-up-ci-jobs), dbt Cloud listens for notification from your Git provider indicating that a new PR has been opened or updated with new commits. When dbt Cloud receives one of these notifications, it enqueues a new run of the CI job.
 
-dbt Cloud builds and tests the models affected by the code change in a temporary schema, unique to the PR. This process ensures that the code builds without error and that it matches the expectations as defined by the project's dbt tests. The unique schema name follows the naming convention `dbt_cloud_pr_<job_id>_<pr_id>` (for example, `dbt_cloud_pr_1862_1704`) and can be found in the run details for the given run, as shown in the following image:
+dbt Cloud builds and tests models, semantic models, metrics, and saved queries affected by the code change in a temporary schema, unique to the PR. This process ensures that the code builds without error and that it matches the expectations as defined by the project's dbt tests. The unique schema name follows the naming convention `dbt_cloud_pr_<job_id>_<pr_id>` (for example, `dbt_cloud_pr_1862_1704`) and can be found in the run details for the given run, as shown in the following image:
 
 <Lightbox src="/img/docs/dbt-cloud/using-dbt-cloud/using_ci_dbt_cloud.png" width="90%"title="Viewing the temporary schema name for a run triggered by a PR"/>
 

website/docs/docs/deploy/deploy-environments.md

@@ -39,7 +39,9 @@ In dbt Cloud, each project can have one designated deployment environment, which
 
 ### Semantic Layer
 
-For Semantic Layer-eligible customers, the next section of environment settings is the Semantic Layer configurations. [The Semantic Layer setup guide](/docs/use-dbt-semantic-layer/setup-sl) has the most up-to-date setup instructions!
+For customers using the dbt Semantic Layer, the next section of environment settings is the Semantic Layer configurations. [The Semantic Layer setup guide](/docs/use-dbt-semantic-layer/setup-sl) has the most up-to-date setup instructions.
+
+You can also leverage the dbt Job scheduler to [validate your semantic nodes in a CI job](/docs/deploy/ci-jobs#semantic-validations-in-ci) to ensure code changes made to dbt models don't break these metrics.
 
 ## Staging environment
 

website/docs/docs/deploy/job-commands.md

@@ -28,7 +28,6 @@ Every job invocation automatically includes the [`dbt deps`](/reference/commands
 
 **Job outcome** — During a job run, the built-in commands are "chained" together.  This means if one of the run steps in the chain fails, then the next commands aren't executed, and the entire job fails with an "Error" job status.
 
-
 <Lightbox src="/img/docs/dbt-cloud/using-dbt-cloud/fail-dbtdeps.jpg" width="85%" title="A failed job that had an error during the dbt deps run step."/>
 
 ### Checkbox commands
@@ -49,9 +48,8 @@ You can add or remove as many dbt commands as necessary for every job. However,
 Use [selectors](/reference/node-selection/syntax) as a powerful way to select and execute portions of your project in a job run. For example, to run tests for one_specific_model, use the selector: `dbt test --select one_specific_model`. The job will still run if a selector doesn't match any models. 
 
 :::
-
-
-**Job outcome**  &mdash;  During a job run, the commands are "chained" together and executed as run steps. If one of the run steps in the chain fails, then the subsequent steps aren't executed, and the job will fail.
+
+**Job outcome** &mdash; During a job run, the commands are "chained" together and executed as run steps. If one of the run steps in the chain fails, then the subsequent steps aren't executed, and the job will fail.
 
 In the following example image, the first four run steps are successful. However, if the fifth run step (`dbt run --select state:modified+ --full-refresh --fail-fast`) fails, then the next run steps aren't executed, and the entire job fails. The failed job returns a non-zero [exit code](/reference/exit-codes) and "Error" job status:
 

website/docs/docs/use-dbt-semantic-layer/exports.md

@@ -203,5 +203,6 @@ To include all saved queries in the dbt build run, use the [`--resource-type` fl
 </detailsToggle>
 
 ## Related docs
+- [Validate semantic nodes in a CI job](/docs/deploy/ci-jobs#semantic-validations-in-ci)
 - Configure [caching](/docs/use-dbt-semantic-layer/sl-cache)
 - [dbt Semantic Layer FAQs](/docs/use-dbt-semantic-layer/sl-faqs)

website/docs/docs/use-dbt-semantic-layer/setup-sl.md

@@ -43,10 +43,16 @@ import SlSetUp from '/snippets/_new-sl-setup.md';
 8. You’re done 🎉! The semantic layer should is now enabled for your project. 
 -->
 
+## Next steps
+
+- Now that you've set up the dbt Semantic Layer, start querying your metrics with the [available integrations](/docs/cloud-integrations/avail-sl-integrations).
+- [Optimize querying performance](/docs/use-dbt-semantic-layer/sl-cache) using declarative caching.
+- [Validate semantic nodes in CI](/docs/deploy/ci-jobs#semantic-validations-in-ci) to ensure code changes made to dbt models don't break these metrics.
+- If you haven't already, learn how to [build you metrics and semantic models](/docs/build/build-metrics-intro) in your development tool of choice.
+
 ## Related docs
 
 - [Build your metrics](/docs/build/build-metrics-intro)
-- [Available integrations](/docs/cloud-integrations/avail-sl-integrations)
 - [Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview)
 - [Get started with the dbt Semantic Layer](/guides/sl-snowflake-qs)
 - [dbt Semantic Layer FAQs](/docs/use-dbt-semantic-layer/sl-faqs)

website/docs/docs/use-dbt-semantic-layer/sl-cache.md

@@ -132,6 +132,8 @@ If an upstream model has data in it that was created after the cache was created
 
 You can manually invalidate the cache through the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) using the `InvalidateCacheResult` field.
 
+
 ## Related docs
+- [Validate semantic nodes in CI](/docs/deploy/ci-jobs#semantic-validations-in-ci)
 - [Saved queries](/docs/build/saved-queries)
 - [dbt Semantic Layer FAQs](/docs/use-dbt-semantic-layer/sl-faqs)

website/docs/docs/use-dbt-semantic-layer/sl-faqs.md

@@ -226,6 +226,15 @@ Yes, we approach this by specifying a [dimension](/docs/build/dimensions) that a
 Yes, while [entities](/docs/build/entities) must be defined under “entities,” they can be queried like dimensions in downstream tools. Additionally, if the entity isn't used to perform joins across your semantic models, you may optionally define it as a dimension.
 </Expandable>
 
+<Expandable alt_header="Can I test my semantic models and metrics?">
+
+Yes! You can validate your semantic nodes (semantic models, metrics, saved queries) in a few ways:
+
+- [Query and validate you metrics](/docs/build/metricflow-commands) in your development tool before submitting your code changes.
+- [Validate semantic nodes in CI](/docs/deploy/ci-jobs#semantic-validations-in-ci) to ensure code changes made to dbt models don't break these metrics.
+
+</Expandable>
+
 ## Available integrations
 
 <Expandable alt_header="What integrations are supported today?">