diff --git a/License.md b/License.md index 8dada3edaf5..6a141c4f6ee 100644 --- a/License.md +++ b/License.md @@ -186,7 +186,7 @@ same "printed page" as the copyright notice for easier identification within third-party archives. - Copyright {yyyy} {name of copyright owner} + Copyright \{yyyy\} \{name of copyright owner\} Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. diff --git a/contributing/adding-page-components.md b/contributing/adding-page-components.md index a07d0ff02e4..68294e7d149 100644 --- a/contributing/adding-page-components.md +++ b/contributing/adding-page-components.md @@ -4,12 +4,12 @@ You can use the following components to provide code snippets for each supported Identify code by labeling with the warehouse names: -```js +```code
- ```sql + ``` select * from `dbt-tutorial.jaffle_shop.customers` ``` @@ -17,7 +17,7 @@ Identify code by labeling with the warehouse names:
- ```sql + ``` select * from default.jaffle_shop_customers ``` @@ -45,11 +45,11 @@ Identify code and code files by labeling with the component they are describing: - ```sql + ``` - {{ config( + \{\{ config( - ) }} + ) \}\} select ... @@ -60,9 +60,9 @@ Identify code and code files by labeling with the component they are describing: - ```yml + ``` models: - [](/reference/resource-configs/resource-path): + [resource-path](/reference/resource-configs/resource-path): ``` @@ -75,9 +75,9 @@ Identify code and code files by labeling with the component they are describing: - ```yml + ``` sources: - [](/reference/resource-configs/resource-path): + [resource-path](/reference/resource-configs/resource-path): ``` diff --git a/contributing/content-style-guide.md b/contributing/content-style-guide.md index 9e573ba0170..022efa127a5 100644 --- a/contributing/content-style-guide.md +++ b/contributing/content-style-guide.md @@ -359,10 +359,10 @@ There won't be many instances where you need to display 4 or 5 cards on the docs Otherwise, the text will appear squished and provide users with a bad experience. -- ``: creates 2 column cards -- ``: creates 3 columns cards -- ``: creates 4 columns cards (use sparingly) -- ``: creates 5 columns cards (use sparingly) +- `
`: creates 2 column cards +- `
`: creates 3 columns cards +- `
`: creates 4 columns cards (use sparingly) +- `
`: creates 5 columns cards (use sparingly) - You can't create cards with 6 or more columns as that would provide users a poor experience. Refer to [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features) and [Quickstarts](/docs/guides) as examples. @@ -456,7 +456,7 @@ In general, when the spelling of English words varies by locale — use the standardize | standardise license | licence color | colour -
+ Avoid regional idiomatic phrases as well. For example, a common saying amongst English speakers in India is "do the needful," but this phrase may be unrecognizable to English speakers from other regions. @@ -533,7 +533,7 @@ terminal | shell username | login dbt Cloud CLI | CLI, dbt CLI dbt Core | CLI, dbt CLI -
+ Note, let's make sure we're talking to our readers and keep them close to the content and documentation (second person). diff --git a/contributing/lightbox.md b/contributing/lightbox.md index baccbe769f7..5f35b4d9639 100644 --- a/contributing/lightbox.md +++ b/contributing/lightbox.md @@ -14,14 +14,15 @@ You can use the Lightbox component to add an image or screenshot to your page. I ## Example usage -
-src="/img/hamburger-icon.jpg"
-lt="Alt text"
-title="This text is visible"
-collapsed={true}
-width="600px"
-alignment="left"
-/> - +```mdx + +``` diff --git a/contributing/single-sourcing-content.md b/contributing/single-sourcing-content.md index 7c345a6631a..2b8a82dfb81 100644 --- a/contributing/single-sourcing-content.md +++ b/contributing/single-sourcing-content.md @@ -107,7 +107,7 @@ Below is how we can implement the same versioning using the new **VersionBlock** You see this block when the selected version is >= 0.21: -```markdown +```code @@ -116,11 +116,12 @@ $ dbt run --select [...] --defer --state path/to/artifacts $ dbt test --select [...] --defer --state path/to/artifacts \``` -``` -You see this version block when the selected version is <= 0.20 +``` -```markdown +You see this version block when the selected version is `<= 0.20` + +```code ```shell diff --git a/website/blog/2021-09-11-sql-dateadd.md b/website/blog/2021-09-11-sql-dateadd.md index 23e5a29dcb9..7ffd9f0367f 100644 --- a/website/blog/2021-09-11-sql-dateadd.md +++ b/website/blog/2021-09-11-sql-dateadd.md @@ -6,7 +6,7 @@ canonical_url: https://docs.getdbt.com/sql-reference/dateadd authors: david_krevitt -tags: [SQL magic] +tags: [sql magic] hide_table_of_contents: false date: 2021-11-15 diff --git a/website/blog/2021-09-11-union-all.md b/website/blog/2021-09-11-union-all.md index 036e46a0abd..6795d37803a 100644 --- a/website/blog/2021-09-11-union-all.md +++ b/website/blog/2021-09-11-union-all.md @@ -5,7 +5,7 @@ slug: sql-union-all authors: david_krevitt -tags: [SQL magic] +tags: [sql magic] hide_table_of_contents: false date: 2021-11-15 diff --git a/website/blog/2021-11-22-exact-commands-we-run-in-production.md b/website/blog/2021-11-22-exact-commands-we-run-in-production.md index 350afa45052..63692b20948 100644 --- a/website/blog/2021-11-22-exact-commands-we-run-in-production.md +++ b/website/blog/2021-11-22-exact-commands-we-run-in-production.md @@ -42,7 +42,7 @@ Some projects can afford to have older data in the warehouse, others can’t. ### 3) Simplify your regularly scheduled runs -The more complex your run commands are, the harder it gets to maintain this in the long run. Feel free to rely on dbt’s DAG (more info on why we <3 DAGs in my colleagues Christine + Randy’s [modular data modeling technique](https://getdbt.com/analytics-engineering/modular-data-modeling-technique/) post). +The more complex your run commands are, the harder it gets to maintain this in the long run. Feel free to rely on dbt’s DAG (more info on why we <3 DAGs in my colleagues Christine + Randy’s [modular data modeling technique](https://getdbt.com/analytics-engineering/modular-data-modeling-technique/) post). Given those principles, we’ll now take a look at the most common run commands for production jobs, and why we think they could work for your organization! Do note that yours may vary slightly (depending on your team’s specific needs), but as long as you stick to the principles mentioned above, your project should be in good shape! diff --git a/website/blog/2021-11-29-the-missing-role-of-design-in-analytics.md b/website/blog/2021-11-29-the-missing-role-of-design-in-analytics.md index 94338cbff5c..180d1751c51 100644 --- a/website/blog/2021-11-29-the-missing-role-of-design-in-analytics.md +++ b/website/blog/2021-11-29-the-missing-role-of-design-in-analytics.md @@ -41,40 +41,28 @@ One solution I proposed back in 2019 is hiring a [data product manager](https:// - - + + - - + + - - + + - - + + - - + + - - + +
Don’t - Do - Don’tDo
Think only within the constraints of your current tooling. - First define the ideal user experience, irrespective of tooling. - Think only within the constraints of your current tooling.First define the ideal user experience, irrespective of tooling.
Think users can simply build their own solution given a self-service interface. - Recognize that hard ongoing problems require a design-focused analyst. - Think users can simply build their own solution given a self-service interface.Recognize that hard ongoing problems require a design-focused analyst.
Stop at the data visualization. - Think about how to group visualizations, the interactions, and purpose-built exploratory flows. - Stop at the data visualization.Think about how to group visualizations, the interactions, and purpose-built exploratory flows.
Ignore look and feel. - Think about the overall aesthetics of your output. - Ignore look and feel.Think about the overall aesthetics of your output.
Just respond to tickets and user requests. - Truly try to understand the problem and design the appropriate solution. - Just respond to tickets and user requests.Truly try to understand the problem and design the appropriate solution.
@@ -104,58 +92,40 @@ Here are a few examples: - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + +
Traditional Dashboards - Analytical Applications - Traditional DashboardsAnalytical Applications
Built for generic use cases - Purpose-built for specific use cases - Built for generic use casesPurpose-built for specific use cases
Standard dashboard interactions - Interactive based on the desired workflow - Standard dashboard interactionsInteractive based on the desired workflow
Fixed, static layout - Dynamic layout determined by logic - Fixed, static layoutDynamic layout determined by logic
Each element is a tile - Elements can be grouped and purposefully-arranged - Each element is a tileElements can be grouped and purposefully-arranged
Filters are global - Users have preferences and their own defaults - Filters are globalUsers have preferences and their own defaults
Minimal software development lifecycle - Strong SDLC to promote user trust - Minimal software development lifecycleStrong SDLC to promote user trust
Look and feel are ignored - Custom look and feel to match company products - Look and feel are ignoredCustom look and feel to match company products
Low bar for performance - High bar for performance - Low bar for performanceHigh bar for performance
diff --git a/website/blog/2021-12-05-how-to-build-a-mature-dbt-project-from-scratch.md b/website/blog/2021-12-05-how-to-build-a-mature-dbt-project-from-scratch.md index 2375d31d448..7e5c49b2dc7 100644 --- a/website/blog/2021-12-05-how-to-build-a-mature-dbt-project-from-scratch.md +++ b/website/blog/2021-12-05-how-to-build-a-mature-dbt-project-from-scratch.md @@ -85,7 +85,7 @@ The most important thing we’re introducing when your project is an infant is t * Configure your first [sources](/docs/build/sources) -* Introduce modularity with [{{ ref() }}](/reference/dbt-jinja-functions/ref) and [{{ source() }}](/reference/dbt-jinja-functions/source) +* Introduce modularity with [\{\{ ref() \}\}](/reference/dbt-jinja-functions/ref) and [\{\{ source() \}\}](/reference/dbt-jinja-functions/source) * [Document](/docs/build/documentation) and [test](/docs/build/data-tests) your first models @@ -99,9 +99,9 @@ Specifically, now is when it's useful to introduce **_modularity_** to our proje We’re going to: -* Break out reused code into separate models and use [{{ ](/reference/dbt-jinja-functions/ref)[ref](/reference/dbt-jinja-functions/ref)[() }}](/reference/dbt-jinja-functions/ref) to build dependencies +* Break out reused code into separate models and use [\{\{ ref() }}](/reference/dbt-jinja-functions/ref) to build dependencies -* Use the[ {{ source() }}](/reference/commands/source) macro to declare our raw data dependencies +* Use the[ \{\{ source() \}\}](/reference/commands/source) macro to declare our raw data dependencies * Dip our toes into testing and documenting our models diff --git a/website/blog/2022-06-30-coalesce-sql.md b/website/blog/2022-06-30-coalesce-sql.md index dc85daf8a53..926adf17558 100644 --- a/website/blog/2022-06-30-coalesce-sql.md +++ b/website/blog/2022-06-30-coalesce-sql.md @@ -5,7 +5,7 @@ slug: coalesce-sql-love-letter authors: [kira_furuichi] -tags: [SQL Magic] +tags: [sql magic] hide_table_of_contents: false date: 2022-05-08 diff --git a/website/blog/2022-06-30-extract-sql-function.md b/website/blog/2022-06-30-extract-sql-function.md index b81a7254a76..73c88f09b8e 100644 --- a/website/blog/2022-06-30-extract-sql-function.md +++ b/website/blog/2022-06-30-extract-sql-function.md @@ -5,7 +5,7 @@ slug: extract-sql-love-letter authors: [kira_furuichi] -tags: [SQL Magic] +tags: [sql magic] hide_table_of_contents: false date: 2022-05-15 diff --git a/website/blog/2022-06-30-lower-sql-function.md b/website/blog/2022-06-30-lower-sql-function.md index 3f7cff44ccb..55c0808a071 100644 --- a/website/blog/2022-06-30-lower-sql-function.md +++ b/website/blog/2022-06-30-lower-sql-function.md @@ -6,7 +6,7 @@ canonical_url: https://docs.getdbt.com/sql-reference/lower authors: [kira_furuichi] -tags: [SQL Magic] +tags: [sql magic] hide_table_of_contents: false date: 2022-05-11 diff --git a/website/blog/2022-07-05-date-trunc-sql-love-letter.md b/website/blog/2022-07-05-date-trunc-sql-love-letter.md index 916f5b505a3..f6dc9a36102 100644 --- a/website/blog/2022-07-05-date-trunc-sql-love-letter.md +++ b/website/blog/2022-07-05-date-trunc-sql-love-letter.md @@ -57,7 +57,7 @@ date_trunc(, ) ``` > **A note on BigQuery:** -> BigQuery’s DATE_TRUNC function supports the truncation of date types, whereas Snowflake, Redshift, and Databricks’ can be a date or timestamp data type. BigQuery also supports DATETIME_TRUNC and TIMESTAMP_TRUNC functions to support truncation of more granular date/time types. +> BigQuery’s DATE_TRUNC function supports the truncation of date types, whereas Snowflake, Redshift, and Databricks’ `date/time field` can be a date or timestamp data type. BigQuery also supports DATETIME_TRUNC and TIMESTAMP_TRUNC functions to support truncation of more granular date/time types. ## A dbt macro to remember diff --git a/website/blog/2022-07-26-configuring-grants.md b/website/blog/2022-07-26-configuring-grants.md index 5888f6ca782..43d59eda9dc 100644 --- a/website/blog/2022-07-26-configuring-grants.md +++ b/website/blog/2022-07-26-configuring-grants.md @@ -204,6 +204,6 @@ This is just a sample of the issues we've seen: - [Post hooks that call macros get parsed with execute = False #2370](https://github.com/dbt-labs/dbt-core/issues/2370) - [get_relation returns none in hook context #2938](https://github.com/dbt-labs/dbt-core/issues/2938) - [this.is_view and this.is_table not working in BigQuery inside a hook #3529](https://github.com/dbt-labs/dbt-core/issues/3529) -- [custom table schema path of {{ this }} parsed in correctly in post-hook macro #3985](https://github.com/dbt-labs/dbt-core/issues/3985) +- [custom table schema path of \{\{ this \}\} parsed in correctly in post-hook macro #3985](https://github.com/dbt-labs/dbt-core/issues/3985) - [Post-hook doesn't resolve custom schema #4023](https://github.com/dbt-labs/dbt-core/issues/4023) - [[CT-80] [Bug] post-hook macro generates SQL with incorrect source table #4606](https://github.com/dbt-labs/dbt-core/issues/4606) diff --git a/website/blog/2023-12-15-serverless-free-tier-data-stack-with-dlt-and-dbt-core.md b/website/blog/2023-12-15-serverless-free-tier-data-stack-with-dlt-and-dbt-core.md index aed00e1c650..87aa82710a3 100644 --- a/website/blog/2023-12-15-serverless-free-tier-data-stack-with-dlt-and-dbt-core.md +++ b/website/blog/2023-12-15-serverless-free-tier-data-stack-with-dlt-and-dbt-core.md @@ -63,7 +63,7 @@ When I'm happy with the results, I use the [ready-made dlt Sheets source connect For transforming I use my favorite solution, dbt Core. For running and orchestrating dbt on Cloud Functions, I am using dlt’s dbt Core runner. The benefit of the runner in this context is that I can re-use the same credential setup, instead of creating a separate profiles.yml file. -This is the package I created: +This is the package I created: [https://github.com/euanjohnston-dev/idealista_dbt_pipeline](https://github.com/euanjohnston-dev/idealista_dbt_pipeline) ### Production-readying the pipeline diff --git a/website/blog/categories.yml b/website/blog/categories.yml index 45acf246dff..9e23635151b 100644 --- a/website/blog/categories.yml +++ b/website/blog/categories.yml @@ -15,9 +15,9 @@ display_title: dbt tutorials description: Best practices in the usage of our favorite data transformation tool. is_featured: true -- name: SQL magic +- name: sql magic display_title: SQL magic description: Stories of dbt developers making SQL sing across warehouses. is_featured: true - name: dbt Cloud - description: Using dbt Cloud to build for scale \ No newline at end of file + description: Using dbt Cloud to build for scale diff --git a/website/docs/best-practices/how-we-mesh/mesh-5-faqs.md b/website/docs/best-practices/how-we-mesh/mesh-5-faqs.md index 3fd7300287e..1ae49928ae5 100644 --- a/website/docs/best-practices/how-we-mesh/mesh-5-faqs.md +++ b/website/docs/best-practices/how-we-mesh/mesh-5-faqs.md @@ -9,7 +9,7 @@ dbt Mesh is a new architecture enabled by dbt Cloud. It allows you to better man ## Overview of Mesh - + Here are some benefits of implementing dbt Mesh: @@ -20,15 +20,15 @@ Here are some benefits of implementing dbt Mesh: Most importantly, all this can be accomplished without the central data team losing the ability to see lineage across the entire organization, or compromising on governance mechanisms. - + - + dbt [model contracts](/docs/collaborate/govern/model-contracts) serve as a governance tool enabling the definition and enforcement of data structure standards in your dbt models. They allow you to specify and uphold data model guarantees, including column data types, allowing for the stability of dependent models. Should a model fail to adhere to its established contracts, it will not build successfully. - + - + dbt [model versions](https://docs.getdbt.com/docs/collaborate/govern/model-versions) are iterations of your dbt models made over time. In many cases, you might knowingly choose to change a model’s structure in a way that “breaks” the previous model contract, and may break downstream queries depending on that model’s structure. When you do so, creating a new version of the model is useful to signify this change. @@ -38,9 +38,9 @@ You can use model versions to: - Bump the latest version, to be used as the canonical "source of truth." - Offer a migration window off the "old" version. - + - + A [model access modifier](/docs/collaborate/govern/model-access) in dbt determines if a model is accessible as an input to other dbt models and projects. It specifies where a model can be referenced using [the `ref` function](/reference/dbt-jinja-functions/ref). There are three types of access modifiers: @@ -48,23 +48,23 @@ A [model access modifier](/docs/collaborate/govern/model-access) in dbt determin * **Protected:** Models with a protected access modifier can be referenced by any other model within the same dbt project or when the project is installed as a package. This is the default setting for all models, ensuring backward compatibility, especially when groups are assigned to an existing set of models. * **Public:** A public model can be referenced across different groups, packages, or projects. This is suitable for stable and mature models that serve as interfaces for other teams or projects. - + - + A [model group](/docs/collaborate/govern/model-access#groups) in dbt is a concept used to organize models under a common category or ownership. This categorization can be based on various criteria, such as the team responsible for the models or the specific data source they model. - + - + This is a new way of working, and the intentionality required to build, and then maintain, cross-project interfaces and dependencies may feel like a slowdown versus what some developers are used to. The intentional friction introduced promotes thoughtful changes, fostering a mindset that values stability and systematic adjustments over rapid transformations. Orchestration across multiple projects is also likely to be slightly more challenging for many organizations, although we’re currently developing new functionality that will make this process simpler. - + - + dbt Mesh allows you to better _operationalize_ data mesh by enabling decentralized, domain-specific data ownership and collaboration. @@ -72,52 +72,52 @@ In data mesh, each business domain is responsible for its data as a product. Thi dbt Mesh also enhances the interoperability and reusability of data across different domains, a key aspect of the data mesh philosophy. By allowing cross-project references and shared governance through model contracts and access controls, dbt Mesh ensures that while data ownership is decentralized, there is still a governed structure to the overall data architecture. - + ## How dbt Mesh works - + import CycleDetection from '/snippets/_mesh-cycle-detection.md'; - + - + While it’s not currently possible to share sources across projects, it would be possible to have a shared foundational project, with staging models on top of those sources, exposed as “public” models to other teams/projects. - + - + This would be a breaking change for downstream consumers of that model. If the maintainers of the upstream project wish to remove the model (or “downgrade” its access modifier, effectively the same thing), they should mark that model for deprecation (using [deprecation_date](/reference/resource-properties/deprecation_date)), which will deliver a warning to all downstream consumers of that model. In the future, we plan for dbt Cloud to also be able to proactively flag this scenario in [continuous integration](/docs/deploy/continuous-integration) for the maintainers of the upstream public model. - + - + No, unless upstream projects are installed as [packages](/docs/build/packages) (source code). In that case, the models in project installed as a project become “your” models, and you can select or run them. There are cases in which this can be desirable; see docs on [project dependencies](/docs/collaborate/govern/project-dependencies). - + - + Yes, as long as they’re in the same data platform (BigQuery, Databricks, Redshift, Snowflake, etc.) and you have configured permissions and sharing in that data platform provider to allow this. - + - + Yes, because the cross-project collaboration is done using the `{{ ref() }}` macro, you can use those models from other teams in [singular tests](/docs/build/data-tests#singular-data-tests). - + - + Each team defines their connection to the data warehouse, and the default schema names for dbt to use when materializing datasets. @@ -128,88 +128,88 @@ By default, each project belonging to a team will create: Depending on each team’s needs, this can be customized with model-level [schema configurations](/docs/build/custom-schemas), including the ability to define different rules by environment. - + - + No, contracts can only be applied at the [model level](/docs/collaborate/govern/model-contracts). It is a recommended best practice to [define staging models](/best-practices/how-we-structure/2-staging) on top of sources, and it is possible to define contracts on top of those staging models. - + - + No. A contract applies to an entire model, including all columns in the model’s output. This is the same set of columns that a consumer would see when viewing the model’s details in Explorer, or when querying the model in the data platform. - If you wish to contract only a subset of columns, you can create a separate model (materialized as a view) selecting only that subset. - If you wish to limit which rows or columns a downstream consumer can see when they query the model’s data, depending on who they are, some data platforms offer advanced capabilities around dynamic row-level access and column-level data masking. - + - + No, a [group](/docs/collaborate/govern/model-access#groups) can only be assigned to a single owner. However, the assigned owner can be a _team_, rather than an individual. - + - + Not directly, but contracts are [assigned to models](/docs/collaborate/govern/model-contracts) and models can be assigned to individual owners. You can use meta fields for this purpose. - + - + This is not currently possible, but something we hope to enable in the near future. If you’re interested in this functionality, please reach out to your dbt Labs account team. - + - + dbt Cloud will soon offer the capability to trigger jobs on the completion of another job, including a job in a different project. This offers one mechanism for executing a pipeline from start to finish across projects. - + - + Yes. In addition to being viewable natively through [dbt Explorer](https://www.getdbt.com/product/dbt-explorer), it is possible to view cross-project lineage connect using partner integrations with data cataloging tools. For a list of available dbt Cloud integrations, refer to the [Integrations page](https://www.getdbt.com/product/integrations). - + - + Tests and model contracts in dbt help eliminate the need to restate data in the first place. With these tools, you can incorporate checks at the source and output layers of your dbt projects to assess data quality in the most critical places. When there are changes in transformation logic (for example, the definition of a particular column is changed), restating the data is as easy as merging the updated code and running a dbt Cloud job. If a data quality issue does slip through, you also have the option of simply rolling back the git commit, and then re-running the dbt Cloud job with the old code. - + - + Yes, all of this metadata is accessible via the [dbt Cloud Admin API](/docs/dbt-cloud-apis/admin-cloud-api). This metadata can be fed into a monitoring tool, or used to create reports and dashboards. We also expose some of this information in dbt Cloud itself in [jobs](/docs/deploy/jobs), [environments](/docs/environments-in-dbt) and in [dbt Explorer](https://www.getdbt.com/product/dbt-explorer). - + - + You can reference models in other accounts within the same data platform by leveraging the data-sharing capabilities of that platform, as long as the database identifier of the public model is consistent across the producer and consumer. For example, [Snowflake cross-account data shares](https://docs.snowflake.com/en/user-guide/data-sharing-intro), [Databricks Unity Catalog across workspaces](https://docs.databricks.com/en/data-governance/unity-catalog/index.html), or multiple BigQuery projects. - + ## Permissions and access - + The existence of projects that have at least one public model will be visible to everyone in the organization with [read-only access](/docs/cloud/manage-access/seats-and-users). Private or protected models require a user to have read-only access to the specific project to see its existence. - + - + There’s model-level access within dbt, role-based access for users and groups in dbt Cloud, and access to the underlying data within the data platform. @@ -230,39 +230,39 @@ Within dbt Explorer, `public` models are discoverable for every user in the dbt Because dbt does not implicitly coordinate data warehouse `grants` with model-level `access`, it is possible for there to be a mismatch between them. For example, a `public` model’s metadata is viewable to all dbt Cloud users, anyone can write a `ref` to that model, but when they actually run or preview, they realize they do not have access to the underlying data in the data warehouse. **This is intentional.** In this way, your organization can retain least-privileged access to underlying data, while providing visibility and discoverability for the wider organization. Armed with the knowledge of which other “data products” (public models) exist — their descriptions, their ownership, which columns they contain — an analyst on another team can prepare a well-informed request for access to the underlying data. - + - + Not currently! But this is something we may evaluate in the future. - + - + Yes! As long as a user has permissions (at least read-only access) on all projects in a dbt Cloud account, they can navigate across the entirety of the organization’s DAG in dbt Explorer, and see models at all levels of detail. - + - + By default, cross-project references resolve to the “Production” deployment environment of the upstream project. If your organization has genuinely different data in production versus non-production environments, this poses an issue. For this reason, we rolled out canonical type of deployment environment: “[Staging](/docs/deploy/deploy-environments#staging-environment).” If a project defines both a “Production” environment and a “Staging” environment, then cross-project references from development and “Staging” environments will resolve to “Staging,” whereas only references coming from “Production” environments will resolve to “Production.” In this way, you are guaranteed separation of data environments, without needing to duplicate project configurations. - + - + The short answer is "no." Cross-project references require that each project `name` be unique in your dbt Cloud account. Historical limitations required customers to "duplicate" projects so that one actual dbt project (codebase) would map to more than one dbt Cloud project. To that end, we are working to remove the historical limitations that required customers to "duplicate" projects in dbt Cloud — Staging environments for data isolation, environment-level permissions, and environment-level data warehouse connections (coming soon). Once those pieces are in place, it should no longer be necessary to define separate dbt Cloud projects to isolate data environments or permissions. - + ## Compatibility with other features - + import SLMeshFAQs from '/snippets/_sl-dbt-mesh-faq.md'; @@ -274,31 +274,31 @@ dbt Mesh enables organizations to split their data architecture into multiple do - + - + **[dbt Explorer](/docs/collaborate/explore-projects)** is a tool within dbt Cloud that serves as a knowledge base and lineage visualization platform. It provides a comprehensive view of your dbt assets, including models, tests, sources, and their interdependencies. Used in conjunction with dbt Mesh, dbt Explorer becomes a powerful tool for visualizing and understanding the relationships and dependencies between models across multiple dbt projects. - + - + The [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) allows users to develop and run dbt commands from their preferred development environments, like VS Code, Sublime Text, or terminal interfaces. This flexibility is particularly beneficial in a dbt Mesh setup, where managing multiple projects can be complex. Developers can work in their preferred tools while leveraging the centralized capabilities of dbt Cloud. - + ## Availability - + Yes, your account must be on [at least dbt v1.6](/docs/dbt-versions/upgrade-dbt-version-in-cloud) to take advantage of [cross-project dependencies](/docs/collaborate/govern/project-dependencies), one of the most crucial underlying capabilities required to implement a dbt Mesh. - + - + While dbt Core defines several of the foundational elements for dbt Mesh, dbt Cloud offers an enhanced experience that leverages these elements for scaled collaboration across multiple teams, facilitated by multi-project discovery in dbt Explorer that’s tailored to each user’s access. @@ -308,33 +308,33 @@ To reference models defined in another project, users can also leverage [package The major distinction comes with dbt Cloud's metadata service, which is unique to the dbt Cloud platform and allows for the resolution of references to only the public models in a project. This service enables users to take dependencies on upstream projects, and reference just their `public` models, *without* needing to load the full complexity of those upstream projects into their local development environment. - + - + Yes, a [dbt Cloud Enterprise](https://www.getdbt.com/pricing) plan is required to set up multiple projects and reference models across them. - + ## Tips on implementing dbt Mesh - + Refer to our developer guide on [How we structure our dbt Mesh projects](https://docs.getdbt.com/best-practices/how-we-mesh/mesh-1-intro). You may also be interested in watching the recording of this talk from Coalesce 2023: [Unlocking model governance and multi-project deployments with dbt-meshify](https://www.youtube.com/watch?v=FAsY0Qx8EyU). You can also learn how to implement dbt Mesh by following our [Quickstart dbt Mesh](/guides/mesh-qs) guide. - + - + `dbt-meshify` is a [CLI tool](https://github.com/dbt-labs/dbt-meshify) that automates the creation of model governance and cross-project lineage features introduced in dbt-core v1.5 and v1.6. This package will leverage your dbt project metadata to create and/or edit the files in your project to properly configure the models in your project with these features. - + - + Let’s say your organization has fewer than 500 models and fewer than a dozen regular contributors to dbt. You're operating at a scale well served by the monolith (a single project), and the larger pattern of dbt Mesh probably won't provide any immediate benefits. It’s never too early to think about how you’re organizing models _within_ that project. Use model `groups` to define clear ownership boundaries and `private` access to restrict purpose-built models from becoming load-bearing blocks in an unrelated section of the DAG. Your future selves will thank you for defining these interfaces, especially if you reach a scale where it makes sense to “graduate” the interfaces between `groups` into boundaries between projects. - + diff --git a/website/docs/best-practices/materializations/materializations-guide-4-incremental-models.md b/website/docs/best-practices/materializations/materializations-guide-4-incremental-models.md index 10776eef1fc..e9c975b2f66 100644 --- a/website/docs/best-practices/materializations/materializations-guide-4-incremental-models.md +++ b/website/docs/best-practices/materializations/materializations-guide-4-incremental-models.md @@ -79,7 +79,7 @@ So we’ve found a way to isolate the new rows we need to process. How then do w ![Diagram visualizing how incremental models work](/img/best-practices/materializations/incremental-diagram.png) :::info -😌 Incremental models can be confusing at first, **take your time reviewing** this visual and the previous steps until you have a **clear mental model.** Be patient with yourself. This materialization will become second nature soon, but it’s tough at first. If you’re feeling confused the [dbt Community is here for you on the Forum and Slack](community/join). +😌 Incremental models can be confusing at first, **take your time reviewing** this visual and the previous steps until you have a **clear mental model.** Be patient with yourself. This materialization will become second nature soon, but it’s tough at first. If you’re feeling confused the [dbt Community is here for you on the Forum and Slack](https://www.getdbt.com/community/join-the-community). ::: Thankfully dbt has some additional configuration and special syntax just for incremental models. diff --git a/website/docs/community/contributing/contributing-online-community.md b/website/docs/community/contributing/contributing-online-community.md index 400a4dadc32..546c165110a 100644 --- a/website/docs/community/contributing/contributing-online-community.md +++ b/website/docs/community/contributing/contributing-online-community.md @@ -21,7 +21,7 @@ For new channels that you'd like to create and maintain, post a message in the # #### Get started -- Read the guide to [Maintaining a Slack Channel](community/resources/maintaining-a-channel) +- Read the guide to [Maintaining a Slack Channel](/community/resources/maintaining-a-channel) ### Participating on the Community Forum diff --git a/website/docs/community/resources/jobs-terms-and-conditions.md b/website/docs/community/resources/jobs-terms-and-conditions.md index f2f2134f847..222c3147a77 100644 --- a/website/docs/community/resources/jobs-terms-and-conditions.md +++ b/website/docs/community/resources/jobs-terms-and-conditions.md @@ -4,7 +4,7 @@ id: "jobs-terms-and-conditions" description: "Before posting a job in the dbt Community or submitting an application, review these terms and conditions." --- -I agree to abide by the [dbt Community Code of Conduct](community/resources/code-of-conduct) and all laws applicable to me in my use of the dbt Community's #jobs channels. I further agree: +I agree to abide by the [dbt Community Code of Conduct](/community/resources/code-of-conduct) and all laws applicable to me in my use of the dbt Community's #jobs channels. I further agree: - dbt Labs is not responsible for not does it warrant or guarantee the validity, accuracy, completeness, legality, or reliability of any functionality of any #jobs channel, any posting's content, or any application and/or solicitation of any kind of employment. - dbt Labs does not review and approve job-related content. diff --git a/website/docs/community/resources/maintaining-a-channel.md b/website/docs/community/resources/maintaining-a-channel.md index 869d3daa980..f9671140188 100644 --- a/website/docs/community/resources/maintaining-a-channel.md +++ b/website/docs/community/resources/maintaining-a-channel.md @@ -24,7 +24,7 @@ A maintainer can be a dbt Labs employee but does not have to be. *Slack channel - Fill out the fields and click **Submit** to submit your request. Someone will get in touch from there. ![request-slack-chnl](https://github.com/siljamardla/docs.getdbt.com/assets/89008547/b14abc52-4164-40a8-b48a-e8061fb4b51a) -3. If you are a vendor, review the [Vendor Expectations](community/resources/community-rules-of-the-road#vendor-expectations). +3. If you are a vendor, review the [Vendor Expectations](/community/resources/community-rules-of-the-road#vendor-expectations). 4. Add the Topic and Description to the channel. @Mention your name in the channel Description, identifying yourself as the maintainer. Ex: *Maintainer: First Last (pronouns).* If you are a vendor, make sure your Handle contains your affiliation. diff --git a/website/docs/docs/build/about-metricflow.md b/website/docs/docs/build/about-metricflow.md index 5c4160ea2db..5c555e62e4a 100644 --- a/website/docs/docs/build/about-metricflow.md +++ b/website/docs/docs/build/about-metricflow.md @@ -245,7 +245,7 @@ metrics: ## FAQs - + Not at all! While a cleaned and well-modeled data set can be extraordinarily powerful and is the ideal input, you can use any dataset from raw to fully denormalized datasets. @@ -253,34 +253,34 @@ It's recommended that you apply quality data consistency, such as filtering bad If you have not invested in data consistency, that is okay. The Semantic Layer can take SQL queries or expressions to define consistent datasets. - + - + MetricFlow is built to do denormalization efficiently. There are better tools to take raw datasets and accomplish the various tasks required to build data consistency and organized data models. On the other end, by putting in denormalized data you are potentially creating redundancy which is technically challenging to manage, and you are reducing the potential granularity that MetricFlow can use to aggregate metrics. - + - + One principle of MetricFlow is to reduce the duplication of logic sometimes referred to as Don't Repeat Yourself(DRY). Many metrics are constructed from reused measures and in some cases constructed from measures from different semantic models. This allows for metrics to be built breadth-first (metrics that can stand alone) instead of depth-first (where you have multiple metrics acting as functions of each other). Additionally, not all metrics are constructed off of measures. As an example, a conversion metric is likely defined as the presence or absence of an event record after some other event record. - - + + The dbt Semantic Layer, powered by MetricFlow, builds joins based on the types of keys and parameters that are passed to entities. To better understand how joins are constructed see our documentation on join types. Rather than capturing arbitrary join logic, MetricFlow captures the types of each identifier and then helps the user to navigate to appropriate joins. This allows us to avoid the construction of fan out and chasm joins as well as generate legible SQL. - + - + If it helps you to think of entities as join keys, that is very reasonable. Entities in MetricFlow have applications beyond joining two tables, such as acting as a dimension. - + - + Yes, but because a dimension is considered an attribute of the primary or unique ent of the table, they are only usable by the metrics that are defined in that table. They cannot be joined to metrics from other tables. This is common in event logs. - + ## Related docs - [Joins](/docs/build/join-logic) diff --git a/website/docs/docs/build/custom-aliases.md b/website/docs/docs/build/custom-aliases.md index 4f22de63e3f..ae1d93e66a7 100644 --- a/website/docs/docs/build/custom-aliases.md +++ b/website/docs/docs/build/custom-aliases.md @@ -18,10 +18,10 @@ The `alias` config can be used to change the name of a model's identifier in the | Model | Config | Relation Type | Database Identifier | | ----- | ------ | --------------| ------------------- | -| ga_sessions.sql | {{ config(materialization='view') }} | | "analytics"."ga_sessions" | -| ga_sessions.sql | {{ config(materialization='view', alias='sessions') }} | | "analytics"."sessions" | -| ga_sessions.sql | {{ config(materialization='ephemeral') }} | | "\__dbt\__cte\__ga_sessions" | -| ga_sessions.sql | {{ config(materialization='ephemeral', alias='sessions') }} | | "\__dbt\__cte\__sessions" | +| ga_sessions.sql | \{\{ config(materialization='view') \}\} | | "analytics"."ga_sessions" | +| ga_sessions.sql | \{\{ config(materialization='view', alias='sessions') \}\} | | "analytics"."sessions" | +| ga_sessions.sql | \{\{ config(materialization='ephemeral') \}\} | | "\__dbt\__cte\__ga_sessions" | +| ga_sessions.sql | \{\{ config(materialization='ephemeral', alias='sessions') \}\} | | "\__dbt\__cte\__sessions" | To configure an alias for a model, supply a value for the model's `alias` configuration parameter. For example: diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index 158b8427136..98627aa0194 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -310,7 +310,7 @@ To suppress warnings about the rename, add `TestsConfigDeprecation` to the `sile ## FAQs - + diff --git a/website/docs/docs/build/entities.md b/website/docs/docs/build/entities.md index 020e0cc4878..7b0ddfffabf 100644 --- a/website/docs/docs/build/entities.md +++ b/website/docs/docs/build/entities.md @@ -56,7 +56,7 @@ entities: ### Combine columns with a key -If a table doesn't have any key (like a primary key), use _surrogate combination_ to form a key that will help you identify a record by combining two columns. This applies to any [entity type](/docs//build/entities#entity-types). For example, you can combine `date_key` and `brand_code` from the `raw_brand_target_weekly` table to form a _surrogate key_. The following example creates a surrogate key by joining `date_key` and `brand_code` using a pipe (`|`) as a separator. +If a table doesn't have any key (like a primary key), use _surrogate combination_ to form a key that will help you identify a record by combining two columns. This applies to any [entity type](/docs/build/entities#entity-types). For example, you can combine `date_key` and `brand_code` from the `raw_brand_target_weekly` table to form a _surrogate key_. The following example creates a surrogate key by joining `date_key` and `brand_code` using a pipe (`|`) as a separator. ```yaml entities: diff --git a/website/docs/docs/build/incremental-models-overview.md b/website/docs/docs/build/incremental-models-overview.md index 419c2a9eb02..16c950eb331 100644 --- a/website/docs/docs/build/incremental-models-overview.md +++ b/website/docs/docs/build/incremental-models-overview.md @@ -12,7 +12,7 @@ Incremental models in dbt is a [materialization](/docs/build/materializations) s This page will provide you with a brief overview of incremental models, their importance in data transformations, and the core concepts of incremental materializations in dbt. - A visual representation of how incremental models work. Source: Materialization best practices guide. /> + ## Understand incremental models diff --git a/website/docs/docs/build/incremental-models.md b/website/docs/docs/build/incremental-models.md index df95504ceab..2f8bbc46c3a 100644 --- a/website/docs/docs/build/incremental-models.md +++ b/website/docs/docs/build/incremental-models.md @@ -45,7 +45,7 @@ Note that the SQL in your model needs to be valid whether `is_incremental()` eva To tell dbt which rows it should transform on an incremental run, wrap valid SQL that filters for these rows in the `is_incremental()` macro. -Often, you'll want to filter for "new" rows, as in, rows that have been created since the last time dbt ran this model. The best way to find the timestamp of the most recent run of this model is by checking the most recent timestamp in your target table. dbt makes it easy to query your target table by using the "[{{ this }}](/reference/dbt-jinja-functions/this)" variable. +Often, you'll want to filter for "new" rows, as in, rows that have been created since the last time dbt ran this model. The best way to find the timestamp of the most recent run of this model is by checking the most recent timestamp in your target table. dbt makes it easy to query your target table by using the "[\{\{ this \}\}](/reference/dbt-jinja-functions/this)" variable. Also common is wanting to capture both new and updated records. For updated records, you'll need to [define a unique key](#defining-a-unique-key-optional) to ensure you don't bring in modified records as duplicates. Your `is_incremental()` code will check for rows created *or modified* since the last time dbt ran this model. diff --git a/website/docs/docs/build/metricflow-commands.md b/website/docs/docs/build/metricflow-commands.md index c8ba951733c..b94e657f712 100644 --- a/website/docs/docs/build/metricflow-commands.md +++ b/website/docs/docs/build/metricflow-commands.md @@ -605,7 +605,7 @@ dbt sl export-all ## FAQs - + To add a dimension filter to a where filter, you have to indicate that the filter item is part of your model and use a template wrapper: `{{Dimension('primary_entity__dimension_name')}}`. @@ -642,17 +642,17 @@ Keep in mind that modifying your shell configuration files can have an impact on - + - + The default `limit` for query issues from the dbt Cloud CLI is 100 rows. We set this default to prevent returning unnecessarily large data sets as the dbt Cloud CLI is typically used to query the dbt Semantic Layer during the development process, not for production reporting or to access large data sets. For most workflows, you only need to return a subset of the data. However, you can change this limit if needed by setting the `--limit` option in your query. For example, to return 1000 rows, you can run `dbt sl list metrics --limit 1000`. - + - + To query multiple metrics, group bys, or where statements in your command, follow this guidance: @@ -663,9 +663,9 @@ To query multiple metrics, group bys, or where statements in your command, follo - To query multiple where statements, use the `--where` syntax and wrap the statement in quotes: - Multiple where statement example: `dbt sl query --metrics accounts_active,users_active --group-by metric_time__week,accounts__plan_tier --where "metric_time__week >= '2024-02-01' and accounts__plan_tier = 'coco'"` - + - + When you query metrics, use `--order-by` to specify metrics or groupings to order by. The `order_by` option applies to metrics, dimensions, and group bys. @@ -674,4 +674,4 @@ Add the `-` prefix to sort your query in descending (DESC) order. Leave blank fo - For example, to query a metric and sort `metric_time` in descending order, run `dbt sl query --metrics order_total --group-by metric_time --order-by -metric_time`. Note that the `-` prefix in `-metric_time` sorts the query in descending order. - To query a metric and sort `metric_time` in ascending order and `revenue` in descending order, run `dbt sl query --metrics order_total --order-by metric_time,-revenue`. Note that `metric_time` without a prefix is sorted in ascending order and `-revenue` with a `-` prefix sorts the query in descending order. - + diff --git a/website/docs/docs/build/saved-queries.md b/website/docs/docs/build/saved-queries.md index 4ef63a637be..e9beffca15f 100644 --- a/website/docs/docs/build/saved-queries.md +++ b/website/docs/docs/build/saved-queries.md @@ -218,16 +218,16 @@ For more information on how to run exports, refer to the [Exports](/docs/use-dbt ## FAQs - + Yes, this is possible. However, the difference would be the name, schema, and materialization strategy of the export. - + - + To include all saved queries in the dbt build run, use the [`--resource-type` flag](/reference/global-configs/resource-type) and run the command `dbt build --resource-type saved_query`. - + ## Related docs - [Validate semantic nodes in a CI job](/docs/deploy/ci-jobs#semantic-validations-in-ci) diff --git a/website/docs/docs/build/sources.md b/website/docs/docs/build/sources.md index ce97e34b746..1594dd72dba 100644 --- a/website/docs/docs/build/sources.md +++ b/website/docs/docs/build/sources.md @@ -177,7 +177,7 @@ To snapshot freshness information for your sources, use the `dbt source freshnes $ dbt source freshness ``` -Behind the scenes, dbt uses the freshness properties to construct a `select` query, shown below. You can find this query in the [query logs](/faqs/runs/checking-logs). +Behind the scenes, dbt uses the freshness properties to construct a `select` query, shown below. You can find this query in the [query logs](/faqs/Runs/checking-logs). ```sql select diff --git a/website/docs/docs/cloud/cloud-cli-installation.md b/website/docs/docs/cloud/cloud-cli-installation.md index a146b8c306f..8a058cbb90f 100644 --- a/website/docs/docs/cloud/cloud-cli-installation.md +++ b/website/docs/docs/cloud/cloud-cli-installation.md @@ -271,15 +271,15 @@ The Power User extension will handle installing the Cloud CLI on your behalf. ## FAQs - + The dbt Cloud CLI and dbt Core, an open-source project, are both command line tools that enable you to run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its features. - + - + For compatibility, both the dbt Cloud CLI and dbt Core are invoked by running `dbt`. This can create path conflicts if your operating system selects one over the other based on your $PATH environment variable (settings). @@ -291,13 +291,13 @@ If you have dbt Core installed locally, either: You can always uninstall the dbt Cloud CLI to return to using dbt Core. - + - + To create an alias for the dbt Cloud CLI:
-1. Open your shell's profile configuration file. Depending on your shell and system, this could be ~/.bashrc, ~/.bash_profile, ~/.zshrc, or another file.
+1. Open your shell's profile configuration file. Depending on your shell and system, this could be `~/.bashrc`, `~/.bash_profile`, `~/.zshrc`, or another file.
2. Add an alias that points to the dbt Cloud CLI binary. For example:alias dbt-cloud="path_to_dbt_cloud_cli_binary @@ -313,11 +313,11 @@ As an example, in bash you would run: source ~/.bashrc
This alias will allow you to use the dbt-cloud command to invoke the dbt Cloud CLI while having dbt Core installed natively. - + - + If you've ran a dbt command and receive a Session occupied error, you can reattach to your existing session with dbt reattach and then press Control-C and choose to cancel the invocation. - + diff --git a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md index f688dd239c2..4719095b87f 100644 --- a/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md +++ b/website/docs/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb.md @@ -100,11 +100,14 @@ Once the connection is saved, a public key will be generated and displayed for t
What is a Bastion server?
-
A bastion server in Amazon Web Services (AWS) is a host that allows dbt Cloud to open an SSH connection.

+
+ A bastion server in Amazon Web Services (AWS) is a host that allows dbt Cloud to open an SSH connection. + +

-dbt Cloud only sends queries and doesn't transmit large data volumes. This means the bastion server can run on an AWS instance of any size, like a t2.small instance or t2.micro.



+ dbt Cloud only sends queries and doesn't transmit large data volumes. This means the bastion server can run on an AWS instance of any size, like a t2.small instance or t2.micro.



-Make sure the location of the instance is the same Virtual Private Cloud (VPC) as the Redshift instance, and configure the security group for the bastion server to ensure that it's able to connect to the warehouse port. + Make sure the location of the instance is the same Virtual Private Cloud (VPC) as the Redshift instance, and configure the security group for the bastion server to ensure that it's able to connect to the warehouse port.
@@ -140,7 +143,7 @@ To grant users or roles database permissions (access rights and privileges), ref ## FAQs - + When setting up a database connection using an SSH tunnel, you need the following components: - A load balancer (like ELB or NLB) to manage traffic. @@ -163,4 +166,4 @@ To prevent premature disconnections, you can adjust the settings on the bastion - `ClientAliveInterval` — Configures when to check for client activity. For example, `ClientAliveInterval 30` checks every 30 seconds. The example adjustments ensure that inactive SSH clients are disconnected after about 300 seconds, reducing the chance of tunnel failures. - + diff --git a/website/docs/docs/cloud/connect-data-platform/connect-snowflake.md b/website/docs/docs/cloud/connect-data-platform/connect-snowflake.md index 68aa7c87a5c..d8dd8dfec11 100644 --- a/website/docs/docs/cloud/connect-data-platform/connect-snowflake.md +++ b/website/docs/docs/cloud/connect-data-platform/connect-snowflake.md @@ -116,7 +116,7 @@ This configuration may conflict with Snowflake OAuth when used with PrivateLink. If you're receiving a `Could not deserialize key data` or `JWT token` error, refer to the following causes and solutions: - + Possible cause and solution for the error "Could not deserialize key data" in dbt Cloud. - This could be because of mistakes like not copying correctly, missing dashes, or leaving out commented lines. @@ -124,9 +124,9 @@ Possible cause and solution for the error "Could not deserialize key data" in db **Solution**: - You can copy the key from its source and paste it into a text editor to verify it before using it in dbt Cloud. - + - + Possible cause and solution for the error "JWT token" in dbt Cloud. - This could be a transient issue between Snowflake and dbt Cloud. When connecting to Snowflake, dbt gets a JWT token valid for only 60 seconds. If there's no response from Snowflake within this time, you might see a `JWT token is invalid` error in dbt Cloud. @@ -136,4 +136,4 @@ Possible cause and solution for the error "JWT token" in dbt Cloud. - dbt needs to retry connections to Snowflake. - Confirm and enter Snowflake's public key correctly. Additionally, you can reach out to Snowflake for help or refer to this Snowflake doc for more info: [Key-Based Authentication Failed with JWT token is invalid Error](https://community.snowflake.com/s/article/Key-Based-Authentication-Failed-with-JWT-token-is-invalid-Error). - + diff --git a/website/docs/docs/cloud/dbt-cloud-ide/develop-in-the-cloud.md b/website/docs/docs/cloud/dbt-cloud-ide/develop-in-the-cloud.md index 8fb580ad054..438cb8c7981 100644 --- a/website/docs/docs/cloud/dbt-cloud-ide/develop-in-the-cloud.md +++ b/website/docs/docs/cloud/dbt-cloud-ide/develop-in-the-cloud.md @@ -143,17 +143,17 @@ Nice job, you're ready to start developing and building models 🎉! ## FAQs - + Not at all! You can use dbt Cloud when you sign up for the Free Developer plan, which comes with one developer seat. If you’d like to access more features or have more developer seats, you can upgrade your account to the Team or Enterprise plan.
Refer to dbt pricing plans for more details. - + - + As a proprietary product, dbt Cloud's source code isn't available for community contributions. If you want to build something in the dbt ecosystem, we encourage you to review [this article](/community/contributing/contributing-coding) about contributing to a dbt package, a plugin, dbt-core, or this documentation site. Participation in open-source is a great way to level yourself up as a developer, and give back to the community. - + - + You can develop dbt using the web-based IDE in dbt Cloud or on the command line interface using the dbt Cloud CLI or open-source dbt Core, all of which enable you to execute dbt commands. The key distinction between the dbt Cloud CLI and dbt Core is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its features: - dbt Cloud IDE: dbt Cloud is a web-based application that allows you to develop dbt projects with the IDE, includes a purpose-built scheduler, and provides an easier way to share your dbt documentation with your team. The IDE is a faster and more reliable way to deploy your dbt models and provides a real-time editing and execution environment for your dbt project. @@ -162,4 +162,4 @@ You can develop dbt using the web-based IDE in dbt Cloud or on the command line - dbt Core: dbt Core is an open-sourced software that’s freely available. You can build your dbt project in a code editor, and run dbt commands from the command line - + diff --git a/website/docs/docs/cloud/dbt-cloud-ide/lint-format.md b/website/docs/docs/cloud/dbt-cloud-ide/lint-format.md index 4b1446711e7..d14435a97e0 100644 --- a/website/docs/docs/cloud/dbt-cloud-ide/lint-format.md +++ b/website/docs/docs/cloud/dbt-cloud-ide/lint-format.md @@ -186,7 +186,7 @@ To format your Python code, dbt Cloud integrates with [Black](https://black.read ## FAQs - + SQLFluff and sqlfmt are both tools used for formatting SQL code, but some differences may make one preferable to the other depending on your use case.
@@ -200,32 +200,32 @@ You can use either SQLFluff or sqlfmt depending on your preference and what work - Use sqlfmt to only have your code well-formatted without analyzing it for errors and bugs. You can use sqlfmt out of the box, making it convenient to use right away without having to configure it. - + - + To ensure optimal code quality, consistent code, and styles — it's highly recommended you have one main `.sqlfluff` configuration file in the root folder of your project. Having multiple files can result in various different SQL styles in your project.

However, you can customize and include an additional child `.sqlfluff` configuration file within specific subfolders of your dbt project.

By nesting a `.sqlfluff` file in a subfolder, SQLFluff will apply the rules defined in that subfolder's configuration file to any files located within it. The rules specified in the parent `.sqlfluff` file will be used for all other files and folders outside of the subfolder. This hierarchical approach allows for tailored linting rules while maintaining consistency throughout your project. Refer to [SQLFluff documentation](https://docs.sqlfluff.com/en/stable/configuration.html#configuration-files) for more info. - + - + Currently, running SQLFluff commands from the terminal isn't supported. - + - + - Double-check that your SQLFluff version matches the one in dbt Cloud IDE (found in the Code Quality tab after a lint operation).

- If your lint operation passes despite clear rule violations, confirm you're not linting models with ephemeral models. Linting doesn't support ephemeral models in dbt v1.5 and lower. - + - + Currently, the dbt Cloud IDE can lint or fix files up to a certain size and complexity. If you attempt to lint or fix files that are too large, taking more than 60 seconds for the dbt Cloud backend to process, you will see an 'Unable to complete linting this file' error. To avoid this, break up your model into smaller models (files) so that they are less complex to lint or fix. Note that linting is simpler than fixing so there may be cases where a file can be linted but not fixed. - + ## Related docs diff --git a/website/docs/docs/cloud/git/import-a-project-by-git-url.md b/website/docs/docs/cloud/git/import-a-project-by-git-url.md index c058484eacd..90c54dbb1b1 100644 --- a/website/docs/docs/cloud/git/import-a-project-by-git-url.md +++ b/website/docs/docs/cloud/git/import-a-project-by-git-url.md @@ -96,7 +96,7 @@ dbt Cloud can work with dbt projects hosted on AWS CodeCommit, but there are som #### Step 4: Specify a custom branch in dbt Cloud -CodeCommit uses `master` as its default branch, and to initialize your project, you'll need to specify the `master` branch as a [custom branch](/faqs/environments/custom-branch-settings#development) in dbt Cloud. +CodeCommit uses `master` as its default branch, and to initialize your project, you'll need to specify the `master` branch as a [custom branch](/faqs/Environments/custom-branch-settings#development) in dbt Cloud. - Go to **Deploy** -> **Environments** -> **Development**. - Select **Settings** -> **Edit** and under **General Settings**, check the **Default to a custom branch** checkbox. diff --git a/website/docs/docs/cloud/git/setup-azure.md b/website/docs/docs/cloud/git/setup-azure.md index 47f8a3a11c0..6fdb2517f1a 100644 --- a/website/docs/docs/cloud/git/setup-azure.md +++ b/website/docs/docs/cloud/git/setup-azure.md @@ -200,7 +200,7 @@ To re-enable MFA for the user, select them again and click **Enable**. Note, you **Token (where applicable - API only):** - PublisherSecurity for access to all projects -- PublisherSecurity/ for per project access +- PublisherSecurity/<azure_devops_project_object_id> for per project access **UI/API/CLI:** API/CLI only @@ -233,7 +233,7 @@ az devops security permission update --organization https://dev.azure.com/ for per project access +- PublisherSecurity/<azure_devops_project_object_id> for per project access **UI/API/CLI:** API/CLI only @@ -268,7 +268,7 @@ az devops security permission update --organization https://dev.azure.com/ for per project access +- PublisherSecurity/<azure_devops_project_object_id> for per project access **UI/API/CLI:** API/CLI only @@ -305,8 +305,8 @@ az devops security permission update --organization https://dev.azure.com/ for per project access -- repoV2// for per repo access +- repoV2/<azure_devops_project_object_id> for per project access +- repoV2/<azure_devops_project_object_id>/<azure_devops_repository_object_id> for per repo access **UI/API/CLI:** UI, API, and CLI @@ -345,8 +345,8 @@ az devops security permission update --organization https://dev.azure.com/ for access to a single project at a time -- repoV2// for access to a single repo at a time +- repoV2/<azure_devops_project_object_id> for access to a single project at a time +- repoV2/<azure_devops_project_object_id>/<azure_devops_repository_object_id> for access to a single repo at a time **UI/API/CLI:** UI, API, and CLI diff --git a/website/docs/docs/cloud/manage-access/auth0-migration.md b/website/docs/docs/cloud/manage-access/auth0-migration.md index 0646263bfff..b7bab836810 100644 --- a/website/docs/docs/cloud/manage-access/auth0-migration.md +++ b/website/docs/docs/cloud/manage-access/auth0-migration.md @@ -33,7 +33,7 @@ Alternatively, you can start the process from the **Settings** page in the **Sin Once you have opted to begin the migration process, the following steps will vary depending on the configured identity provider. You can just skip to the section that's right for your environment. These steps only apply to customers going through the migration; new setups will use the existing [setup instructions](/docs/cloud/manage-access/sso-overview). -:::warning Login {slug} +:::warning Login \{slug\} Slugs should contain only letters, numbers, and dashes. Make sure to remove underscores (if they exist) from login slugs: * before migrating on the **Account Settings** page, or diff --git a/website/docs/docs/cloud/manage-access/set-up-sso-microsoft-entra-id.md b/website/docs/docs/cloud/manage-access/set-up-sso-microsoft-entra-id.md index 556a0740565..5dbbf219ccf 100644 --- a/website/docs/docs/cloud/manage-access/set-up-sso-microsoft-entra-id.md +++ b/website/docs/docs/cloud/manage-access/set-up-sso-microsoft-entra-id.md @@ -59,7 +59,7 @@ Depending on your Microsoft Entra ID settings, your App Registration page might -### Azure <-> dbt Cloud User and Group mapping +### Azure <-> dbt Cloud User and Group mapping The Azure users and groups you will create in the following steps are mapped to groups created in dbt Cloud based on the group name. Reference the docs on [enterprise permissions](enterprise-permissions) for additional information on how users, groups, and permission sets are configured in dbt Cloud. diff --git a/website/docs/docs/community-adapters.md b/website/docs/docs/community-adapters.md index e17a43d5f61..22dd2404765 100644 --- a/website/docs/docs/community-adapters.md +++ b/website/docs/docs/community-adapters.md @@ -5,7 +5,7 @@ id: "community-adapters" Community adapters are adapter plugins contributed and maintained by members of the community. We welcome and encourage [adapter plugins contributions](/docs/contribute-core-adapters#contribute-to-a-pre-existing-adapter) from the dbt community. Please be mindful that these [community maintainers](/docs/connect-adapters#maintainers) are intrepid volunteers who donate their time and effort — so be kind, understanding, and help out where you can! -| Data platforms (click to view setup guide) || +| Data platforms (click to view setup guide) ||| | ------------------------------------------ | -------------------------------- | ------------------------------------- | | [Clickhouse](/docs/core/connect-data-platform/clickhouse-setup) | [Databend Cloud](/docs/core/connect-data-platform/databend-setup) | [Doris & SelectDB](/docs/core/connect-data-platform/doris-setup) | | [DuckDB](/docs/core/connect-data-platform/duckdb-setup) | [Exasol Analytics](/docs/core/connect-data-platform/exasol-setup) | [Extrica](/docs/core/connect-data-platform/extrica-setup) | @@ -16,6 +16,3 @@ Community adapters are adapter plugins contributed and maintained by members of | [SingleStore](/docs/core/connect-data-platform/singlestore-setup)| [SQL Server & Azure SQL](/docs/core/connect-data-platform/mssql-setup) | [SQLite](/docs/core/connect-data-platform/sqlite-setup) | | [Starrocks](/docs/core/connect-data-platform/starrocks-setup) | [TiDB](/docs/core/connect-data-platform/tidb-setup)| [TimescaleDB](https://dbt-timescaledb.debruyn.dev/) | | [Upsolver](/docs/core/connect-data-platform/upsolver-setup) | [Vertica](/docs/core/connect-data-platform/vertica-setup) | [Yellowbrick](/docs/core/connect-data-platform/yellowbrick-setup) | - - - diff --git a/website/docs/docs/core/connect-data-platform/decodable-setup.md b/website/docs/docs/core/connect-data-platform/decodable-setup.md index 6c3cb487885..58b6ef19736 100644 --- a/website/docs/docs/core/connect-data-platform/decodable-setup.md +++ b/website/docs/docs/core/connect-data-platform/decodable-setup.md @@ -94,14 +94,16 @@ dbt run-operation delete_streams --args '{streams: [stream1, stream2], skip_erro ``` stop_pipelines(pipelines) -
  • pipelines: An optional list of pipeline names to deactivate. Defaults to none.
    • +
      +
    • pipelines: An optional list of pipeline names to deactivate. Defaults to none.
    Deactivate all pipelines for resources defined within the project. If the pipelines argument is provided, then only the specified pipelines are deactivated.

    delete_pipelines(pipelines) -
    • pipelines: An optional list of pipeline names to delete. Defaults to none.
      • +
        +
      • pipelines: An optional list of pipeline names to delete. Defaults to none.
      Delete all pipelines for resources defined within the project. If the pipelines argument is provided, then only the specified pipelines are deleted.

      @@ -109,8 +111,8 @@ Delete all pipelines for resources defined within the project. If the pipelines delete_streams(streams, skip_errors)
        -
      • streams: An optional list of stream names to delete. Defaults to none.
        • -
      • skip_errors: Specify whether to treat errors as warnings. When set to true, any stream deletion failures are reported as warnings. When set to false, the operation stops when a stream cannot be deleted. Defaults to true.
        • +
      • streams: An optional list of stream names to delete. Defaults to none.
        • +
      • skip_errors: Specify whether to treat errors as warnings. When set to true, any stream deletion failures are reported as warnings. When set to false, the operation stops when a stream cannot be deleted. Defaults to true.
      Delete all streams for resources defined within the project. If a pipeline is associated with a stream, then neither the pipeline nor stream are deleted. See the cleanup operation for a complete removal of stream/pipeline pairs.

      diff --git a/website/docs/docs/core/connect-data-platform/firebolt-setup.md b/website/docs/docs/core/connect-data-platform/firebolt-setup.md index 8fb91dea299..b1695c75b37 100644 --- a/website/docs/docs/core/connect-data-platform/firebolt-setup.md +++ b/website/docs/docs/core/connect-data-platform/firebolt-setup.md @@ -40,16 +40,15 @@ To connect to Firebolt from dbt, you'll need to add a [profile](https://docs.get outputs: : type: firebolt - user: "" - password: "" + client_id: "" + client_secret: "" database: "" engine_name: "" + account_name: "" schema: threads: 1 #optional fields - jar_path: host: "" - account_name: "" ``` @@ -57,30 +56,27 @@ To connect to Firebolt from dbt, you'll need to add a [profile](https://docs.get #### Description of Firebolt Profile Fields -To specify values as environment variables, use the format `{{ env_var('' }}`. For example, `{{ env_var('DATABASE_NAME' }}`. +To specify values as environment variables, use the format `{{ env_var('' }}`. For example, `{{ env_var('DATABASE_NAME' }}`. | Field | Description | |--------------------------|--------------------------------------------------------------------------------------------------------| | `type` | This must be included either in `profiles.yml` or in the `dbt_project.yml` file. Must be set to `firebolt`. | -| `user` | Required. A Firebolt username with adequate permissions to access the specified `engine_name`. | -| `password` | Required. The password associated with the specified `user`. | +| `client_id` | Required. Your [service account](https://docs.firebolt.io/godocs/Guides/managing-your-organization/service-accounts.html) id. | +| `client_secret` | Required. The secret associated with the specified `client_id`. | | `database` | Required. The name of the Firebolt database to connect to. | | `engine_name` | Required in version 0.21.10 and later. Optional in earlier versions. The name (not the URL) of the Firebolt engine to use in the specified `database`. This must be a general purpose read-write engine and the engine must be running. If omitted in earlier versions, the default engine for the specified `database` is used. | +| `account_name` | Required. Specifies the account name under which the specified `database` exists. | | `schema` | Recommended. A string to add as a prefix to the names of generated tables when using the [custom schemas workaround](https://docs.getdbt.com/reference/warehouse-profiles/firebolt-profile#supporting-concurrent-development). | -| `threads` | Required. Must be set to `1`. Multi-threading is not currently supported. | -| `jar_path` | Required only with versions earlier than 0.21.0. Ignored in 0.21.0 and later. The path to your JDBC driver on your local drive. | +| `threads` | Required. Set to higher number to improve performance. | | `host` | Optional. The host name of the connection. For all customers it is `api.app.firebolt.io`, which will be used if omitted. | -| `account_name` | Required if more than one account is associated with the specified `user1`. Specifies the account name (not the account ID) under which the specified `database` exists. If omitted, the default account is assumed. | - + #### Troubleshooting Connections If you encounter issues connecting to Firebolt from dbt, make sure the following criteria are met: -- The engine must be a general-purpose read-write engine, not an analytics engine. -- You must have adequate permissions to access the engine. +- You must have adequate permissions to access the engine and the database. +- Your service account must be attached to a user. - The engine must be running. -- If you're not using the default engine for the database, you must specify an engine name. -- If there is more than one account associated with your credentials, you must specify an account. ## Supporting Concurrent Development diff --git a/website/docs/docs/core/connect-data-platform/risingwave-setup.md b/website/docs/docs/core/connect-data-platform/risingwave-setup.md index 4e320d00f42..29ebc378c4e 100644 --- a/website/docs/docs/core/connect-data-platform/risingwave-setup.md +++ b/website/docs/docs/core/connect-data-platform/risingwave-setup.md @@ -77,7 +77,7 @@ The dbt models for managing data transformations in RisingWave are similar to ty |`materializedview`| To be deprecated. |It is available only for backward compatibility purposes (for v1.5.1 of the dbt-risingwave adapter plugin). If you are using v1.6.0 and later versions of the dbt-risingwave adapter plugin, use `materialized_view` instead.| |`materialized_view`| Yes| Creates a [materialized view](https://docs.risingwave.com/docs/dev/sql-create-mv/). This materialization corresponds the `incremental` one in dbt. To use this materialization, add `{{ config(materialized='materialized_view') }}` to your model SQL files.| | `incremental`|No|Please use `materialized_view` instead. Since RisingWave is designed to use materialized view to manage data transformation in an incremental way, you can just use the `materialized_view` materialization.| -|`source`| Yes| Creates a [source](https://docs.risingwave.com/docs/dev/sql-create-source/). To use this materialization, add {{ config(materialized='source') }} to your model SQL files. You need to provide your create source statement as a whole in this model. See [Example model files](https://docs.risingwave.com/docs/dev/use-dbt/#example-model-files) for details.| +|`source`| Yes| Creates a [source](https://docs.risingwave.com/docs/dev/sql-create-source/). To use this materialization, add \{\{ config(materialized='source') \}\} to your model SQL files. You need to provide your create source statement as a whole in this model. See [Example model files](https://docs.risingwave.com/docs/dev/use-dbt/#example-model-files) for details.| |`table_with_connector`| Yes| Creates a table with connector settings. In RisingWave, a table with connector settings is similar to a source. The difference is that a table object with connector settings persists raw streaming data in the source, while a source object does not. To use this materialization, add `{{ config(materialized='table_with_connector') }}` to your model SQL files. You need to provide your create table with connector statement as a whole in this model (see [Example model files](https://docs.risingwave.com/docs/dev/use-dbt/#example-model-files) for details). Because dbt tables have their own semantics, RisingWave use `table_with_connector` to distinguish itself from a dbt table.| |`sink`| Yes| Creates a [sink](https://docs.risingwave.com/docs/dev/sql-create-sink/). To use this materialization, add `{{ config(materialized='sink') }}` to your SQL files. You need to provide your create sink statement as a whole in this model. See [Example model files](https://docs.risingwave.com/docs/dev/use-dbt/#example-model-files) for details.| diff --git a/website/docs/docs/core/connect-data-platform/starrocks-setup.md b/website/docs/docs/core/connect-data-platform/starrocks-setup.md index ad0b119498b..198491c1ca5 100644 --- a/website/docs/docs/core/connect-data-platform/starrocks-setup.md +++ b/website/docs/docs/core/connect-data-platform/starrocks-setup.md @@ -83,7 +83,7 @@ my-starrocks-db: ## Supported features -| Starrocks <= 2.5 | Starrocks 2.5 ~ 3.1 | Starrocks >= 3.1 | Feature | +| Starrocks <= 2.5 | Starrocks 2.5 ~ 3.1 | Starrocks >= 3.1 | Feature | |:----------------:|:--------------------:|:-----------------:|:---------------------------------:| | ✅ | ✅ | ✅ | Table materialization | | ✅ | ✅ | ✅ | View materialization | diff --git a/website/docs/docs/core/connect-data-platform/teradata-setup.md b/website/docs/docs/core/connect-data-platform/teradata-setup.md index 98c3abba715..7067104fb94 100644 --- a/website/docs/docs/core/connect-data-platform/teradata-setup.md +++ b/website/docs/docs/core/connect-data-platform/teradata-setup.md @@ -198,12 +198,12 @@ For using cross-DB macros, teradata-utils as a macro namespace will not be used, #### examples for cross DB macros ##### replace - {{ dbt.replace("string_text_column", "old_chars", "new_chars") }} - {{ replace('abcgef', 'g', 'd') }} + \{\{ dbt.replace("string_text_column", "old_chars", "new_chars") \}\} + \{\{ replace('abcgef', 'g', 'd') \}\} ##### date_trunc - {{ dbt.date_trunc("date_part", "date") }} - {{ dbt.date_trunc("DD", "'2018-01-05 12:00:00'") }} + \{\{ dbt.date_trunc("date_part", "date") \}\} + \{\{ dbt.date_trunc("DD", "'2018-01-05 12:00:00'") \}\} ##### datediff `datediff` macro in teradata supports difference between dates. Differece between timestamps is not supported. diff --git a/website/docs/docs/core/pip-install.md b/website/docs/docs/core/pip-install.md index b052c50c1ea..6d94d92a64b 100644 --- a/website/docs/docs/core/pip-install.md +++ b/website/docs/docs/core/pip-install.md @@ -31,7 +31,7 @@ dbt-env\Scripts\activate # activate the environment for Windows To activate your dbt environment with every new shell window or session, you can create an alias for the source command in your $HOME/.bashrc, $HOME/.zshrc, or whichever config file your shell draws from. -For example, add the following to your rc file, replacing with the path to your virtual environment configuration. +For example, add the following to your rc file, replacing <PATH_TO_VIRTUAL_ENV_CONFIG> with the path to your virtual environment configuration. ```shell alias env_dbt='source /bin/activate' diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 672f34b2fec..ea0afbcfbad 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -56,7 +56,7 @@ The Semantic Layer JDBC API has built-in metadata calls which can provide a user Expand the following toggles for examples and metadata commands: - + You can use this query to fetch all defined metrics in your dbt project: @@ -65,9 +65,9 @@ select * from {{ semantic_layer.metrics() }} ``` - + - + You can use this query to fetch all dimensions for a metric. @@ -77,9 +77,9 @@ Note, metrics is a required argument that lists one or multiple metrics in it. select * from {{ semantic_layer.dimensions(metrics=['food_order_amount'])}} ``` - + - + You can use this query to fetch dimension values for one or multiple metrics and a single dimension. @@ -89,9 +89,9 @@ Note, metrics is a required argument that lists one or multiple metrics, and a s select * from {{ semantic_layer.dimension_values(metrics=['food_order_amount'], group_by=['customer__customer_name'])}} ``` - + - + You can use this query to fetch queryable granularities for a list of metrics. @@ -103,9 +103,9 @@ select * from {{ semantic_layer.queryable_granularities(metrics=['food_order_amount', 'order_gross_profit'])}} ``` - + - + You can use this query to fetch available metrics given dimensions. This command is essentially the opposite of getting dimensions given a list of metrics. @@ -117,9 +117,9 @@ select * from {{ }} ``` - + - + You can use this example query to fetch available granularities for all time dimensions (the similar queryable granularities API call only returns granularities for the primary time dimensions for metrics). @@ -133,9 +133,9 @@ select NAME, QUERYABLE_GRANULARITIES from {{ }} ``` - + - + It may be useful in your application to expose the names of the time dimensions that represent metric_time or the common thread across all metrics. @@ -147,9 +147,9 @@ select * from {{ }} ``` - + - + You can use this example query to list all available saved queries in your dbt project. @@ -165,7 +165,7 @@ select * from semantic_layer.saved_queries() | NAME | DESCRIPTION | LABEL | METRICS | GROUP_BY | WHERE_FILTER | ``` - +