diff --git a/website/docs/best-practices/how-we-mesh/mesh-5-faqs.md b/website/docs/best-practices/how-we-mesh/mesh-5-faqs.md index 3443da614a9..1ae49928ae5 100644 --- a/website/docs/best-practices/how-we-mesh/mesh-5-faqs.md +++ b/website/docs/best-practices/how-we-mesh/mesh-5-faqs.md @@ -264,12 +264,16 @@ Historical limitations required customers to "duplicate" projects so that one ac +import SLMeshFAQs from '/snippets/_sl-dbt-mesh-faq.md'; + The [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) and dbt Mesh are complementary mechanisms enabled by dbt Cloud that work together to enhance the management, usability, and governance of data in large-scale data environments. The Semantic Layer in dbt Cloud allows teams to centrally define business metrics and dimensions. It ensures consistent and reliable metric definitions across various analytics tools and platforms. dbt Mesh enables organizations to split their data architecture into multiple domain-specific projects, while retaining the ability to reference “public” models across projects. It is also possible to reference a “public” model from another project for the purpose of defining semantic models and metrics. Your organization can have multiple dbt projects feed into a unified semantic layer, ensuring that metrics and dimensions are consistently defined and understood across these domains. + + diff --git a/website/docs/docs/build/metricflow-commands.md b/website/docs/docs/build/metricflow-commands.md index b94e657f712..405f9b08ca4 100644 --- a/website/docs/docs/build/metricflow-commands.md +++ b/website/docs/docs/build/metricflow-commands.md @@ -74,7 +74,7 @@ You can use the `dbt sl` prefix before the command name to execute them in the d - [`list dimensions`](#list) — Lists unique dimensions for metrics. - [`list dimension-values`](#list-dimension-values) — List dimensions with metrics. - [`list entities`](#list-entities) — Lists all unique entities. -- [`list saved queries`](#list-saved-queries) — Lists available saved queries. Use the `--show-exports` flag to display each export listed under a saved query. +- [`list saved-queries`](#list-saved-queries) — Lists available saved queries. Use the `--show-exports` flag to display each export listed under a saved query. - [`query`](#query) — Query metrics, saved queries, and dimensions you want to see in the command line interface. Refer to [query examples](#query-examples) to help you get started. - [`export`](#export) — Runs exports for a singular saved query for testing and generating exports in your development environment. You can also use the `--select` flag to specify particular exports from a saved query. - [`export-all`](#export-all) — Runs exports for multiple saved queries at once, saving time and effort. diff --git a/website/docs/docs/build/validation.md b/website/docs/docs/build/validation.md index 02ce48729a4..6ca5a680895 100644 --- a/website/docs/docs/build/validation.md +++ b/website/docs/docs/build/validation.md @@ -12,13 +12,13 @@ These validations ensure that configuration files follow the expected schema, th The code that handles validation [can be found here](https://github.com/dbt-labs/dbt-semantic-interfaces/tree/main/dbt_semantic_interfaces/validations) for those who want to dive deeper into this topic. - ## Validations command -You can run validations from dbt Cloud or the command line with the following [MetricFlow commands](/docs/build/metricflow-commands): +You can run validations against the defined semantic model configurations from the command line with the following [MetricFlow commands](/docs/build/metricflow-commands): + +Note, in dbt Cloud you don't need to validate the Semantic Layer config separately. Running a dbt command (such as `dbt parse`, `dbt build`, `dbt compile`, or `dbt run`) automatically checks it. ```bash -dbt sl validate-configs # dbt Cloud users mf validate-configs # dbt Core users ``` @@ -32,9 +32,9 @@ In this validation step, we ensure your config files follow the defined schema f * Dimensions * Metrics -## Semantic +## Semantic syntax -This validation step occurs after we've built your semantic graph. The Semantic Layer, powered by MetricFlow, runs a suite of tests to ensure that your semantic graph doesn't violate any constraints. For example, we check to see if measure names are unique, or if metrics referenced in materialization exist. The current semantic rules we check for are: +This syntactic validation step occurs after we've built your semantic graph. The Semantic Layer, powered by MetricFlow, runs a suite of tests to ensure that your semantic graph doesn't violate any constraints. For example, we check to see if measure names are unique, or if metrics referenced in materialization exist. The current semantic rules we check for are: 1. Check those semantic models with measures have a valid time dimension 2. Check that there is only one primary identifier defined in each semantic model @@ -45,9 +45,10 @@ This validation step occurs after we've built your semantic graph. The Semantic ## Data platform -This type of validation Checks to see if the semantic definitions in your semantic graph exist in the underlying physical table. To test this, we run queries against your data platform to ensure the generated SQL for semantic models, dimensions, and metrics will execute. We run the following checks +This type of validation checks to see if the semantic definitions in your semantic graph exist in the underlying physical table. To test this, we run queries against your data platform to ensure the generated SQL for semantic models, dimensions, and metrics will execute. We run the following checks: -* Check that measures and dimensions exist -* Check that underlying tables for data sources exist -* Check that the generated SQL for metrics will execute +* Measures and dimensions exist +* Underlying tables for data sources exist +* Generated SQL for metrics will execute +You can run semantic validations (against your semantic layer) in a CI job to guarantee any code changes made to dbt models don't break these metrics. For more information, refer to [semantic validation in CI](/docs/deploy/ci-jobs#semantic-validations-in-ci). diff --git a/website/docs/docs/cloud/about-cloud-develop-defer.md b/website/docs/docs/cloud/about-cloud-develop-defer.md index 034fd9d2a4e..4e2f70b7b82 100644 --- a/website/docs/docs/cloud/about-cloud-develop-defer.md +++ b/website/docs/docs/cloud/about-cloud-develop-defer.md @@ -11,6 +11,8 @@ pagination_next: "docs/cloud/cloud-cli-installation" Both the dbt Cloud IDE and the dbt Cloud CLI enable users to natively defer to production metadata directly in their development workflows. + + By default, dbt follows these rules: - dbt uses the production locations of parent models to resolve `{{ ref() }}` functions, based on metadata from the production environment. diff --git a/website/docs/docs/use-dbt-semantic-layer/sl-faqs.md b/website/docs/docs/use-dbt-semantic-layer/sl-faqs.md index 9a122aa644a..40b84ada40a 100644 --- a/website/docs/docs/use-dbt-semantic-layer/sl-faqs.md +++ b/website/docs/docs/use-dbt-semantic-layer/sl-faqs.md @@ -279,6 +279,14 @@ We are currently working on introducing more fine-grained access controls, inclu ## Implementation + + +import SLMeshFAQs from '/snippets/_sl-dbt-mesh-faq.md'; + + + + + We recommend to build your semantic layer on top of the [marts layer](/best-practices/how-we-structure/4-marts), which represents the clean and transformed data from your dbt models. diff --git a/website/docs/guides/mesh-qs.md b/website/docs/guides/mesh-qs.md index f2e02437819..0d13d043059 100644 --- a/website/docs/guides/mesh-qs.md +++ b/website/docs/guides/mesh-qs.md @@ -636,6 +636,7 @@ Here are some additional resources to help you continue your journey: - [How we build our dbt mesh projects](https://docs.getdbt.com/best-practices/how-we-mesh/mesh-1-intro) - [dbt Mesh FAQs](https://docs.getdbt.com/best-practices/how-we-mesh/mesh-5-faqs) +- [Implement dbt Mesh with the Semantic Layer](/docs/use-dbt-semantic-layer/sl-faqs#how-can-i-implement-dbt-mesh-with-the-dbt-semantic-layer) - [Cross-project references](/docs/collaborate/govern/project-dependencies#how-to-write-cross-project-ref) - [dbt Explorer](/docs/collaborate/explore-projects) diff --git a/website/docs/guides/sl-snowflake-qs.md b/website/docs/guides/sl-snowflake-qs.md index 367e41083bb..782073fdc38 100644 --- a/website/docs/guides/sl-snowflake-qs.md +++ b/website/docs/guides/sl-snowflake-qs.md @@ -1056,7 +1056,7 @@ You've learned how to: - Build, test, and manage dbt Cloud projects, focusing on metrics and semantic layers. - Run production jobs and query metrics with our available integrations. -For next steps, you can start defining your own metrics and learn additional configuration options such as [exports](/docs/use-dbt-semantic-layer/exports), [fill null values](/docs/build/advanced-topics), and more. +For next steps, you can start defining your own metrics and learn additional configuration options such as [exports](/docs/use-dbt-semantic-layer/exports), [fill null values](/docs/build/advanced-topics), [implementing dbt Mesh with the Semantic Layer](/docs/use-dbt-semantic-layer/sl-faqs#how-can-i-implement-dbt-mesh-with-the-dbt-semantic-layer), and more. Here are some additional resources to help you continue your journey: diff --git a/website/docs/reference/node-selection/defer.md b/website/docs/reference/node-selection/defer.md index 27cc3dd78c5..1653bf7e04c 100644 --- a/website/docs/reference/node-selection/defer.md +++ b/website/docs/reference/node-selection/defer.md @@ -4,6 +4,8 @@ title: "Defer" Defer is a powerful feature that makes it possible to run a subset of models or tests in a [sandbox environment](/docs/environments-in-dbt) without having to first build their upstream parents. This can save time and computational resources when you want to test a small number of models in a large project. + + Defer requires that a manifest from a previous dbt invocation be passed to the `--state` flag or env var. Together with the `state:` selection method, these features enable "Slim CI". Read more about [state](/reference/node-selection/syntax#about-node-selection). An alternative command that accomplishes similar functionality for different use cases is `dbt clone` - see the docs for [clone](/reference/commands/clone#when-to-use-dbt-clone-instead-of-deferral) for more information. diff --git a/website/snippets/_sl-dbt-mesh-faq.md b/website/snippets/_sl-dbt-mesh-faq.md new file mode 100644 index 00000000000..1a517f1aa21 --- /dev/null +++ b/website/snippets/_sl-dbt-mesh-faq.md @@ -0,0 +1,30 @@ +When using the dbt Semantic Layer in a [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) setting, we recommend the following: + +- You have one standalone project that contains your semantic models and metrics. +- Then as you build your Semantic Layer, you can [cross-reference dbt models](/docs/collaborate/govern/project-dependencies) across your various projects or packages to create your semantic models using the [two-argument `ref` function](/reference/dbt-jinja-functions/ref#ref-project-specific-models)( `ref('project_name', 'model_name')`). +- Your dbt Semantic Layer project serves as a global source of truth across the rest of your projects. + +#### Usage example +For example, let's say you have a public model (`fct_orders`) that lives in the `jaffle_finance` project. As you build your semantic model, use the following syntax to ref the model: + + + +```yaml +semantic_models: + - name: customer_orders + defaults: + agg_time_dimension: first_ordered_at + description: | + Customer grain mart that aggregates customer orders. + model: ref('jaffle_finance', 'fct_orders') # ref('project_name', 'model_name') + entities: + ...rest of configuration... + dimensions: + ...rest of configuration... + measures: + ...rest of configuration... +``` + + +Notice that in the `model` parameter, we're using the `ref` function with two arguments to reference the public model `fct_orders` defined in the `jaffle_finance` project. +
diff --git a/website/snippets/cloud-feature-parity.md b/website/snippets/cloud-feature-parity.md index b6c49aa0faa..3f7f447050c 100644 --- a/website/snippets/cloud-feature-parity.md +++ b/website/snippets/cloud-feature-parity.md @@ -1,7 +1,7 @@ The following table outlines which dbt Cloud features are supported on the different SaaS options available today. For more information about feature availability, please [contact us](https://www.getdbt.com/contact/). | Feature | AWS Multi-tenant | AWS single tenant |Azure multi-tenant ([Preview](/docs/dbt-versions/product-lifecycles#dbt-cloud)) | Azure single tenant | -|-------------------------------|------------------|-----------------------|---------------------|------------------- -| +|-------------------------------|------------------|-----------------------|---------------------|---------------------| | Audit logs | ✅ | ✅ | ✅ | ✅ | | Continuous integration jobs | ✅ | ✅ | ✅ | ✅ | | dbt Cloud CLI | ✅ | ✅ | ✅ | ✅ | diff --git a/website/static/img/docs/reference/defer-diagram.png b/website/static/img/docs/reference/defer-diagram.png new file mode 100644 index 00000000000..abd571c112c Binary files /dev/null and b/website/static/img/docs/reference/defer-diagram.png differ