diff --git a/website/docs/docs/cloud/about-cloud/regions-ip-addresses.md b/website/docs/docs/cloud/about-cloud/regions-ip-addresses.md index 82c1f277edb..23b6c8ca906 100644 --- a/website/docs/docs/cloud/about-cloud/regions-ip-addresses.md +++ b/website/docs/docs/cloud/about-cloud/regions-ip-addresses.md @@ -14,6 +14,7 @@ dbt Cloud is [hosted](/docs/cloud/about-cloud/architecture) in multiple regions |--------|----------|------------|--------------|----------------|-----------|-----------------| | North America [^1] | AWS us-east-1 (N. Virginia) | **Multi-tenant:** cloud.getdbt.com
**Cell based:** ACCOUNT_PREFIX.us1.dbt.com | 52.45.144.63
54.81.134.249
52.22.161.231
52.3.77.232
3.214.191.130
34.233.79.135 | ✅ | ✅ | ✅ | | EMEA [^1] | AWS eu-central-1 (Frankfurt) | emea.dbt.com | 3.123.45.39
3.126.140.248
3.72.153.148 | ❌ | ❌ | ✅ | +| EMEA [^1] | Azure
North Europe (Ireland) | **Cell based:** ACCOUNT_PREFIX.eu2.dbt.com (beta invite only) | 20.13.190.192/26 | ❌ | ❌ | ✅ | | APAC [^1] | AWS ap-southeast-2 (Sydney)| au.dbt.com | 52.65.89.235
3.106.40.33
13.239.155.206
| ❌ | ❌ | ✅ | | Virtual Private dbt or Single tenant | Customized | Customized | Ask [Support](/community/resources/getting-help#dbt-cloud-support) for your IPs | ❌ | ❌ | ✅ | diff --git a/website/docs/docs/collaborate/explore-multiple-projects.md b/website/docs/docs/collaborate/explore-multiple-projects.md index a37d67d058b..125d284a9a5 100644 --- a/website/docs/docs/collaborate/explore-multiple-projects.md +++ b/website/docs/docs/collaborate/explore-multiple-projects.md @@ -4,31 +4,46 @@ sidebar_label: "Explore multiple projects" description: "Learn about project-level lineage in dbt Explorer and its uses." --- -You can also view all the different projects and public models in the account, where the public models are defined, and how they are used to gain a better understanding about your cross-project resources. +View all the projects and public models in your account (where public models are defined) and gain a better understanding of your cross-project resources and how they're used. -The resource-level lineage graph for a given project displays the cross-project relationships in the DAG. The different icons indicate whether you’re looking at an upstream producer project (parent) or a downstream consumer project (child). +The resource-level lineage graph for a project displays the cross-project relationships in the DAG, with a **PRJ** icon indicating whether or not it's a project resource. That icon is located to the left side of the node name. -When you view an upstream (parent) project, its public models display a counter icon in the upper right corner indicating how many downstream (child) projects depend on them. Selecting a model reveals the lineage indicating the projects dependent on that model. These counts include all projects listing the upstream one as a dependency in its `dependencies.yml`, even without a direct `{{ ref() }}`. Selecting a project node from a public model opens its detailed lineage graph, which is subject to your [permission](/docs/cloud/manage-access/enterprise-permissions). +To view the project-level lineage graph, click the **View lineage** icon in the upper right corner from the main overview page: +- This view displays all the projects in your account and their relationships. +- Viewing an upstream (parent) project displays the downstream (child) projects that depend on it. +- Selecting a model reveals its dependent projects in the lineage. +- Click on an upstream (parent) project to view the other projects that reference it in the **Relationships** tab, showing the number of downstream (child) projects that depend on them. + - This includes all projects listing the upstream one as a dependency in its `dependencies.yml` file, even without a direct `{{ ref() }}`. +- Selecting a project node from a public model opens its detailed lineage graph if you have the [permissions](/docs/cloud/manage-access/enterprise-permissions) to do so. - + -When viewing a downstream (child) project that imports and refs public models from upstream (parent) projects, public models will show up in the lineage graph and display an icon on the graph edge that indicates what the relationship is to a model from another project. Hovering over this icon indicates the specific dbt Cloud project that produces that model. Double-clicking on a model from another project opens the resource-level lineage graph of the parent project, which is subject to your permissions. +When viewing a downstream (child) project that imports and refs public models from upstream (parent) projects: +- Public models will show up in the lineage graph and you can click on them to view the model details. +- Clicking on a model opens a side panel containing general information about the model, such as the specific dbt Cloud project that produces that model, description, package, and more. +- Double-clicking on a model from another project opens the resource-level lineage graph of the parent project, if you have the permissions to do so. - - + ## Explore the project-level lineage graph -For cross-project collaboration, you can interact with the DAG in all the same ways as described in [Explore your project's lineage](/docs/collaborate/explore-projects#project-lineage) but you can also interact with it at the project level and view the details. +For cross-project collaboration, you can interact with the DAG in all the same ways as described in [Explore your project's lineage](/docs/collaborate/explore-projects#project-lineage) but you can also interact with it at the project level and view the details. + +If you have permissions for a project in the account, you can view all public models used across the entire account. However, you can only view full public model details and private models if you have permissions for the specific project where those models are defined. -To get a list view of all the projects, select the account name at the top of the **Explore** page near the navigation bar. This view includes a public model list, project list, and a search bar for project searches. You can also view the project-level lineage graph by clicking the Lineage view icon in the page's upper right corner. +To view all the projects in your account (displayed as a lineage graph or list view): +- Navigate to the top left section of the **Explore** page, near the navigation bar. +- Hover over the project name and select the account name. This takes you to a account-level lineage graph page, where you can view all the projects in the account, including dependencies and relationships between different projects. +- Click the **List view** icon in the page's upper right corner to see a list view of all the projects in the account. +- The list view page displays a public model list, project list, and a search bar for project searches. +- Click the **Lineage view** icon in the page's upper right corner to view the account-level lineage graph. -If you have permissions for a project in the account, you can view all public models used across the entire account. However, you can only view full public model details and private models if you have permissions for a project where the models are defined. + -From the project-level lineage graph, you can: +From the account-level lineage graph, you can: -- Click the Lineage view icon (in the graph’s upper right corner) to view the cross-project lineage graph. -- Click the List view icon (in the graph’s upper right corner) to view the project list. +- Click the **Lineage view** icon (in the graph’s upper right corner) to view the cross-project lineage graph. +- Click the **List view** icon (in the graph’s upper right corner) to view the project list. - Select a project from the **Projects** tab to switch to that project’s main **Explore** page. - Select a model from the **Public Models** tab to view the [model’s details page](/docs/collaborate/explore-projects#view-resource-details). - Perform searches on your projects with the search bar. @@ -40,6 +55,6 @@ When you select a project node in the graph, a project details panel opens on th - View a list of its public models, if any. - View a list of other projects that uses the project, if any. - Click **Open Project Lineage** to switch to the project’s lineage graph. -- Click the Share icon to copy the project panel link to your clipboard so you can share the graph with someone. +- Click the **Share** icon to copy the project panel link to your clipboard so you can share the graph with someone. - + diff --git a/website/docs/docs/dbt-cloud-apis/authentication.md b/website/docs/docs/dbt-cloud-apis/authentication.md index 4d7c4d4c06a..8729cc0641d 100644 --- a/website/docs/docs/dbt-cloud-apis/authentication.md +++ b/website/docs/docs/dbt-cloud-apis/authentication.md @@ -33,7 +33,7 @@ pagination_prev: null You should use service tokens broadly for any production workflow where you need a service account. You should use PATs only for developmental workflows _or_ dbt Cloud client workflows that require user context. The following examples show you when to use a personal access token (PAT) or a service token: -* **Connecting a partner integration to dbt Cloud** — Some examples include Hightouch, Datafold, a custom app you’ve created, etc. These types of integrations should use a service token instead of a PAT because service tokens give you visibility, and you can scope them to only what the integration needs and ensure the least privilege. We highly recommend switching to a service token if you’re using a user API key for these integrations today. +* **Connecting a partner integration to dbt Cloud** — Some examples include the [dbt Semantic Layer Google Sheets integration](/docs/cloud-integrations/avail-sl-integrations), Hightouch, Datafold, a custom app you’ve created, etc. These types of integrations should use a service token instead of a PAT because service tokens give you visibility, and you can scope them to only what the integration needs and ensure the least privilege. We highly recommend switching to a service token if you’re using a user API key for these integrations today. * **Production Terraform** — Use a service token since this is a production workflow and is acting as a service account and not a user account. -* **Cloud CLI and Semantic Layer Sheets Integration** — Use a PAT since both the dbt Cloud CLI and Semantic Layer Google Sheets integrations work within the context of a user (the user is making the requests and has to operate within the context of their user account). -* **Testing a custom script and staging Terraform or Postman** — We recommend using a PAT as this is a developmental workflow and is scoped to the user making the changes. When you push this script or Terraform into production, use a service token instead. \ No newline at end of file +* **Cloud CLI** — Use a PAT since the dbt Cloud CLI works within the context of a user (the user is making the requests and has to operate within the context of their user account). +* **Testing a custom script and staging Terraform or Postman** — We recommend using a PAT as this is a developmental workflow and is scoped to the user making the changes. When you push this script or Terraform into production, use a service token instead. diff --git a/website/docs/docs/dbt-cloud-apis/service-tokens.md b/website/docs/docs/dbt-cloud-apis/service-tokens.md index 615e8a73536..26b0e58caa2 100644 --- a/website/docs/docs/dbt-cloud-apis/service-tokens.md +++ b/website/docs/docs/dbt-cloud-apis/service-tokens.md @@ -17,7 +17,7 @@ Service account tokens enable you to securely authenticate with the dbt Cloud AP You can use service account tokens for system-level integrations that do not run on behalf of any one user. Assign any permission sets available in dbt Cloud to your service account token, which can vary slightly depending on your plan: * Enterprise plans can apply any permission sets available to service tokens. -* Team plans can apply Account Admin, Member, Job Admin, Read-Only, and Metadata permissions set to service tokens. +* Team plans can apply Account Admin, Member, Job Admin, Read-Only, Metadata, and Semantic Layer permissions set to service tokens. You can assign as many permission sets as needed to one token. For more on permissions sets, see "[Enterprise Permissions](/docs/cloud/manage-access/enterprise-permissions)." diff --git a/website/docs/docs/dbt-versions/versionless-cloud.md b/website/docs/docs/dbt-versions/versionless-cloud.md new file mode 100644 index 00000000000..ae92e0d6a0a --- /dev/null +++ b/website/docs/docs/dbt-versions/versionless-cloud.md @@ -0,0 +1,67 @@ +--- +title: "Upgrade to \"Keep on latest version\" in dbt Cloud" +sidebar_label: "Upgrade to \"Keep on latest version\" " +description: "Learn how to go versionless in dbt Cloud. You never have to perform an upgrade again. Plus, you'll be able to access new features and enhancements as soon as they become available. " +--- + +dbt Cloud is going versionless. Soon, your environments and jobs will always run on the latest version of dbt. + +This will require you to make one final update to your current jobs and environments. When that's done, you'll never have to think about managing, coordinating, or upgrading dbt versions again. + +Move your environments and jobs to "Keep on latest version" to get all the functionality in the latest versions of dbt Core — and more! — along with access to the new features and fixes as soon as they’re released. + +## Tips for upgrading {#upgrade-tips} + +If you regularly develop your dbt project in dbt Cloud and this is your first time trying “Keep on latest version,” dbt Labs recommends that you start in development because it will be the fastest for investigation and iteration. [Override your dbt version in development](/docs/dbt-versions/upgrade-dbt-version-in-cloud#override-dbt-version). Then, launch the IDE or Cloud CLI and do your development work as usual. Everything should work as you expect. + +If you do see something unexpected or surprising, revert back to the previous version and record the differences you observed. [Contact dbt Cloud support](/docs/dbt-support#dbt-cloud-support) with your findings for a more detailed investigation. + +Next, we recommend that you try upgrading your project’s [deployment environment](/docs/dbt-versions/upgrade-dbt-version-in-cloud#environments). If your project has a [staging deployment environment](/docs/deploy/deploy-environments#staging-environment), upgrade and try working with it for a few days before you proceed with upgrading the production environment. + +If your organization has multiple dbt projects, we recommend starting your upgrade with projects that are smaller, newer, or more familiar for your team. That way, if you do encounter any issues, it'll be easier and faster to troubleshoot those before proceeding to upgrade larger or more complex projects. + +## Considerations + +The following is our guidance on some important considerations regarding dbt projects as part of the upgrade. + +To learn more about how dbt Labs deploys stable dbt upgrades in a safe manner to dbt Cloud, we recommend that you read our blog post [How we're making sure you can confidently "Keep on latest version" in dbt Cloud](https://docs.getdbt.com/blog/latest-dbt-stability) for details. + + + +If you're running dbt version 1.5 or older, please know that your version of dbt Core has reached [end-of-life (EOL)](/docs/dbt-versions/core#eol-version-support) and is no longer supported. We strongly recommend that you update to a newer version as soon as reasonably possible. In the coming months, we're planning to automatically migrate jobs and environments on these older, unsupported versions. + + + + + +The legacy dbt Semantic Layer was deprecated in the second half of 2023. We recommend that you refer to the [Legacy dbt Semantic Layer migration guide](/guides/sl-migration?step=1) for more information. + + + + + +When we talk about _latest version_, we’re referring to the underlying runtime for dbt, not the versions of packages you’re installing. Our continuous release for dbt includes testing against several popular dbt packages. This ensures that updates we make to dbt-core, adapters, or anywhere else are compatible with the code in those packages. + +If a new version of a dbt package includes a breaking change (for example, a change to one of the macros in `dbt_utils`), you don’t have to immediately use the new version. In your `packages` configuration (in `dependencies.yml` or `packages.yml`), you can still specify which versions or version ranges of packages you want dbt to install. If you're not already doing so, we strongly recommend [checking `package-lock.yml` into version control](/reference/commands/deps#predictable-package-installs) for predictable package installs in deployment environments and a clear change history whenever you install upgrades. + +If you upgrade to “Keep on latest version” and immediately see something that breaks, please [contact support](/docs/dbt-support#dbt-cloud-support) and, in the meantime, downgrade back to v1.7. + +If you’re already on “Keep on latest version” and you observe a breaking change (like something worked yesterday, but today it isn't working, or works in a surprising/different way), please [contact support](/docs/dbt-support#dbt-cloud-support) immediately. Depending on your contracted support agreement, the dbt Labs team will respond within our SLA time and we would seek to roll back the change and/or roll out a fix (just as we would for any other part of dbt Cloud). This is the same whether or not the root cause of the breaking change is in the project code or in the code of a package. + +If the package you’ve installed relies on _undocumented_ functionality of dbt, it doesn't have the same guarantees as functionality that we’ve documented and tested. However, we will still do our best to avoid breaking them. + + + + + +No. Going forward, “Keep on latest version” will be how all customers are going to access new functionality and ongoing support in dbt Cloud. We believe this is the best way for us to offer a reliable, stable, and secure runtime for dbt with continuous and consistent updates. + +In 2023 (and earlier), customers were expected to manage their own upgrades by selecting dbt Core versions, up to and including dbt Core v1.7, which was released in October 2023. (Way back in 2021, dbt Cloud customers would pick specific _patch releases_ of dbt Core, such as upgrading from `v0.21.0` to `v0.21.1`. We’ve come a long way since then!) + +In 2024, we've changed the way that new dbt functionality is made available for dbt Cloud customers: continuously. Behavior or breaking changes are gated behind opt-in flags. Users don't need to spend valuable time managing their own upgrades. This is called "Keep on latest version" and it's required for accessing any new functionality that we've put out in 2024+. + +We will absolutely continue to release new minor versions of dbt Core (OSS), including v1.9 which will be available later this year. When we do, it will be a subset of the functionality that's already available to dbt Cloud customers and always after the functionality has been available in dbt Cloud. + + + +If you have comments or concerns, we’re happy to help. If you’re an existing dbt Cloud customer, you may reach out to your account team or [contact support](/docs/dbt-support#dbt-cloud-support). \ No newline at end of file diff --git a/website/sidebars.js b/website/sidebars.js index 03e1ce7852c..07f552088d0 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -716,6 +716,7 @@ const sidebarSettings = { link: { type: "doc", id: "docs/dbt-versions/core" }, items: [ "docs/dbt-versions/core", + "docs/dbt-versions/versionless-cloud", "docs/dbt-versions/upgrade-dbt-version-in-cloud", "docs/dbt-versions/product-lifecycles", "docs/dbt-versions/experimental-features", diff --git a/website/static/img/docs/collaborate/dbt-explorer/account-level-lineage.gif b/website/static/img/docs/collaborate/dbt-explorer/account-level-lineage.gif new file mode 100644 index 00000000000..af6937f6d9a Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/account-level-lineage.gif differ diff --git a/website/static/img/docs/collaborate/dbt-explorer/cross-project-child.png b/website/static/img/docs/collaborate/dbt-explorer/cross-project-child.png new file mode 100644 index 00000000000..13461148465 Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/cross-project-child.png differ diff --git a/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-child.png b/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-child.png deleted file mode 100644 index aa2f0d06e00..00000000000 Binary files a/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-child.png and /dev/null differ diff --git a/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png b/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png index b667e9fa04f..55dfebf6bed 100644 Binary files a/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png and b/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png differ diff --git a/website/static/img/docs/collaborate/dbt-explorer/multi-project-overview.gif b/website/static/img/docs/collaborate/dbt-explorer/multi-project-overview.gif new file mode 100644 index 00000000000..5283fb19414 Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/multi-project-overview.gif differ