From c28fcefba58b69c437c9341e2130cfb30302cf8a Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 28 Feb 2024 16:27:29 -0500 Subject: [PATCH 01/89] Data_tests callout --- website/docs/docs/build/data-tests.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index 0806842a85c..b15dd58dafe 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -13,6 +13,29 @@ keywords: * [Data test configurations](/reference/data-test-configs) * [Test selection examples](/reference/node-selection/test-selection-examples) + + +:::important + +With the addition of unit tests in dbt v1.8, `tests` are now `data tests`. In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias but will be deprecated in the future in favor of `data_tests:`. Please update your configuration YML to prepare for these changes: + +```yml + +models: + +name: orders +columns: +name: order_id +data_tests: +unique +not_null + +``` + +::: + + + ## Overview Data tests are assertions you make about your models and other resources in your dbt project (e.g. sources, seeds and snapshots). When you run `dbt test`, dbt will tell you if each test in your project passes or fails. From beb6d8af1998aefdbe9a3e0ab22cdf8a08b1cb15 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 29 Feb 2024 14:52:24 +0000 Subject: [PATCH 02/89] add cache to sq --- website/docs/docs/build/saved-queries.md | 87 +++++++++++++++++++++--- 1 file changed, 76 insertions(+), 11 deletions(-) diff --git a/website/docs/docs/build/saved-queries.md b/website/docs/docs/build/saved-queries.md index 4083d2fb6b8..51efbb39da4 100644 --- a/website/docs/docs/build/saved-queries.md +++ b/website/docs/docs/build/saved-queries.md @@ -22,21 +22,33 @@ Saved queries are distinct from [exports](/docs/use-dbt-semantic-layer/exports), All metrics in a saved query need to use the same dimensions in the `group_by` or `where` clauses. The following is an example of a saved query: + + + ```yaml saved_queries: - - name: p0_booking - description: Booking-related metrics that are of the highest priority. + - name: test_saved_query + description: "{{ doc('saved_query_description') }}" + label: Test saved query + config: + cache: + enabled: True | False query_params: - metrics: - - bookings - - instant_bookings - group_by: - - TimeDimension('metric_time', 'day') - - Dimension('listing__capacity_latest') - where: - - "{{ Dimension('listing__capacity_latest') }} > 3" + metrics: + - simple_metric + group_by: + - "Dimension('user__ds')" + where: + - "{{ Dimension('user__ds', 'DAY') }} <= now()" + - "{{ Dimension('user__ds', 'DAY') }} >= '2023-01-01'" + exports: + - name: my_export + config: + alias: my_export_alias + export_as: table + schema: my_export_schema_name ``` @@ -48,6 +60,9 @@ To define a saved query, refer to the following parameters: |-------|---------|----------|----------------| | `name` | String | Required | Name of the saved query object. | | `description` | String | Required | A description of the saved query. | +| `label` | String | Required | The display name for your saved query. This value will be shown in downstream tools. | +| `config` | String | Required | A config section for any parameters specifying the saved query. | +| `config::cache` | String | Optional | A boolean to specify if a saved query should be used to populate the cache. Accepts `True` or `False`. Defaults to `False` | | `query_params` | Structure | Required | Contains the query parameters. | | `query_params::metrics` | List or String | Optional | A list of the metrics to be used in the query as specified in the command line interface. | | `query_params::group_by` | List or String | Optional | A list of the Entities and Dimensions to be used in the query, which include the `Dimension` or `TimeDimension`. | @@ -59,8 +74,58 @@ To define a saved query, refer to the following parameters: | `exports::config::schema` | String | Optional | The schema for creating the table or view. This option cannot be used for caching. | | `exports::config::alias` | String | Optional | The table alias to use to write the table or view. This option cannot be used for caching. | -All metrics in a saved query need to use the same dimensions in the `group_by` or `where` clauses. + + + + + + + +```yaml +saved_queries: + - name: test_saved_query + description: "{{ doc('saved_query_description') }}" + label: Test saved query + query_params: + metrics: + - simple_metric + group_by: + - "Dimension('user__ds')" + where: + - "{{ Dimension('user__ds', 'DAY') }} <= now()" + - "{{ Dimension('user__ds', 'DAY') }} >= '2023-01-01'" + exports: + - name: my_export + config: + alias: my_export_alias + export_as: table + schema: my_export_schema_name +``` + + +## Parameters + +To define a saved query, refer to the following parameters: + +| Parameter | Type | Required | Description | +|-------|---------|----------|----------------| +| `name` | String | Required | Name of the saved query object. | +| `description` | String | Required | A description of the saved query. | +| `label` | String | Required | The display name for your saved query. This value will be shown in downstream tools. | +| `query_params` | Structure | Required | Contains the query parameters. | +| `query_params::metrics` | List or String | Optional | A list of the metrics to be used in the query as specified in the command line interface. | +| `query_params::group_by` | List or String | Optional | A list of the Entities and Dimensions to be used in the query, which include the `Dimension` or `TimeDimension`. | +| `query_params::where` | List or String | Optional | A list of strings that may include the `Dimension` or `TimeDimension` objects. | +| `exports` | List or Structure | Optional | A list of exports to be specified within the exports structure. | +| `exports::name` | String | Required | Name of the export object. | +| `exports::config` | List or Structure | Required | A config section for any parameters specifying the export. | +| `exports::config::export_as` | String | Required | The type of export to run. Options include table or view currently and cache in the near future. | +| `exports::config::schema` | String | Optional | The schema for creating the table or view. This option cannot be used for caching. | +| `exports::config::alias` | String | Optional | The table alias to use to write the table or view. This option cannot be used for caching. | + + +All metrics in a saved query need to use the same dimensions in the `group_by` or `where` clauses. ## Related docs From 152396929ab64c8507a1575bc2fad979b3e3bc7f Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Thu, 29 Feb 2024 12:03:22 -0500 Subject: [PATCH 03/89] Updating text --- website/docs/docs/build/data-tests.md | 38 +++++++++++++++++---------- 1 file changed, 24 insertions(+), 14 deletions(-) diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index b15dd58dafe..f7665b4f359 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -17,20 +17,7 @@ keywords: :::important -With the addition of unit tests in dbt v1.8, `tests` are now `data tests`. In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias but will be deprecated in the future in favor of `data_tests:`. Please update your configuration YML to prepare for these changes: - -```yml - -models: - -name: orders -columns: -name: order_id -data_tests: -unique -not_null - -``` +With the addition of unit tests in dbt v1.8, `tests` are now `data tests`. In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias but will be deprecated in the future in favor of `data_tests:`. See [New syntax](#new-syntax) for more information. ::: @@ -274,6 +261,29 @@ Note that, if you select to store test failures: * Test result tables are created in a schema suffixed or named `dbt_test__audit`, by default. It is possible to change this value by setting a `schema` config. (For more details on schema naming, see [using custom schemas](/docs/build/custom-schemas).) - A test's results will always **replace** previous failures for the same test. + + +## New `tests:` syntax + +With the addition of unit tests in dbt v1.8, `tests` are now `data tests`. In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias but will be deprecated in the future in favor of `data_tests:`. + +As we progress towards this deprecation, the examples in our docs pages will be updated to reflect this new syntax, but we highly recommend you begin the migration process as soon as you upgrade to v1.8 to avoid interruptions or issues in the future. + +```yml + +models: + +name: orders +columns: +name: order_id +data_tests: +unique +not_null + + +``` + + ## FAQs From 292781db288dc5d645a9ffbc1580df2dae4a449e Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Thu, 29 Feb 2024 12:04:16 -0500 Subject: [PATCH 04/89] Update website/docs/docs/build/data-tests.md --- website/docs/docs/build/data-tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index f7665b4f359..545b9d8c04e 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -17,7 +17,7 @@ keywords: :::important -With the addition of unit tests in dbt v1.8, `tests` are now `data tests`. In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias but will be deprecated in the future in favor of `data_tests:`. See [New syntax](#new-syntax) for more information. +With the addition of unit tests in dbt v1.8, `tests` are now `data tests`. In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias but will be deprecated in the future in favor of `data_tests:`. See [New syntax](#new-`tests:`-syntax) for more information. ::: From f24dc17832e211751ae3eb4eb517b139c2aa5823 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Thu, 29 Feb 2024 16:09:15 -0500 Subject: [PATCH 05/89] Update website/docs/docs/build/data-tests.md --- website/docs/docs/build/data-tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index 545b9d8c04e..cf276e41f62 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -17,7 +17,7 @@ keywords: :::important -With the addition of unit tests in dbt v1.8, `tests` are now `data tests`. In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias but will be deprecated in the future in favor of `data_tests:`. See [New syntax](#new-`tests:`-syntax) for more information. +With the addition of unit tests in dbt v1.8, what was previously called "tests" are now "data tests". In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias for data tests but will be deprecated in the future in favor of `data_tests:`. See [New syntax](#new-`tests:`-syntax) for more information. ::: From 2652f2153a64ce55644c081617a04cdc64ed4e80 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Thu, 29 Feb 2024 16:11:17 -0500 Subject: [PATCH 06/89] Update website/docs/docs/build/data-tests.md --- website/docs/docs/build/data-tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index cf276e41f62..2bfcf604116 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -265,7 +265,7 @@ Note that, if you select to store test failures: ## New `tests:` syntax -With the addition of unit tests in dbt v1.8, `tests` are now `data tests`. In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias but will be deprecated in the future in favor of `data_tests:`. +Data tests were historically called "tests" in dbt as the only form of testing available. With the introduction of unit tests in v1.8, it was necessary to update our naming conventions and syntax. As of v1.8, `tests:` is still supported in your YML configuration file as an alias but will be deprecated in the future in favor of `data_tests:`. As we progress towards this deprecation, the examples in our docs pages will be updated to reflect this new syntax, but we highly recommend you begin the migration process as soon as you upgrade to v1.8 to avoid interruptions or issues in the future. From 26ec3b091607776943e9a25ad81d2b635ab9caa7 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Thu, 29 Feb 2024 16:21:49 -0500 Subject: [PATCH 07/89] Update website/docs/docs/build/data-tests.md --- website/docs/docs/build/data-tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index 2bfcf604116..260e1026092 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -17,7 +17,7 @@ keywords: :::important -With the addition of unit tests in dbt v1.8, what was previously called "tests" are now "data tests". In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias for data tests but will be deprecated in the future in favor of `data_tests:`. See [New syntax](#new-`tests:`-syntax) for more information. +With the addition of unit tests in dbt v1.8, what was previously called "tests" are now "data tests". In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias for data tests but will be deprecated in the future in favor of `data_tests:`. See [New syntax](#new-tests-syntax) for more information. ::: From 881ca9924e520d95e0fb47513376f233480d4bcb Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Thu, 29 Feb 2024 17:04:13 -0500 Subject: [PATCH 08/89] Update website/docs/docs/build/data-tests.md --- website/docs/docs/build/data-tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index 260e1026092..c3a654d5eb3 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -261,7 +261,7 @@ Note that, if you select to store test failures: * Test result tables are created in a schema suffixed or named `dbt_test__audit`, by default. It is possible to change this value by setting a `schema` config. (For more details on schema naming, see [using custom schemas](/docs/build/custom-schemas).) - A test's results will always **replace** previous failures for the same test. - + ## New `tests:` syntax From a7e503bf43c31579ee6d6f990d2f6a8ef94e8c35 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Thu, 29 Feb 2024 18:41:00 -0500 Subject: [PATCH 09/89] Update website/docs/docs/build/data-tests.md --- website/docs/docs/build/data-tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index c3a654d5eb3..65812c12a5f 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -17,7 +17,7 @@ keywords: :::important -With the addition of unit tests in dbt v1.8, what was previously called "tests" are now "data tests". In dbt v1.8, `tests:` is still supported in your YML configuration file as an alias for data tests but will be deprecated in the future in favor of `data_tests:`. See [New syntax](#new-tests-syntax) for more information. +In dbt v1.8, what was previously known as "tests" are now called "data tests" with the addition of [unit tests](/docs/build/unit-tests). The YAML key `tests:` is still supported as an alias for data tests but will be deprecated in the future in favor of `data_tests:`. Refer to [New syntax](#new-tests-syntax) for more information. ::: From 4c0d4df9b357fe3450b9413e7f2508c243bd2bb7 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Thu, 29 Feb 2024 19:10:42 -0500 Subject: [PATCH 10/89] Update data-tests.md --- website/docs/docs/build/data-tests.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index 65812c12a5f..eff8f9e2b70 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -272,13 +272,12 @@ As we progress towards this deprecation, the examples in our docs pages will be ```yml models: - -name: orders -columns: -name: order_id -data_tests: -unique -not_null + - name: orders + columns: + - name: order_id + data_tests: + - unique + - not_null ``` From 178b870b011b99c0f5f52fdf5f74114df4b28e05 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Thu, 29 Feb 2024 19:11:00 -0500 Subject: [PATCH 11/89] Update website/docs/docs/build/data-tests.md Co-authored-by: Grace Goheen <53586774+graciegoheen@users.noreply.github.com> --- website/docs/docs/build/data-tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/data-tests.md b/website/docs/docs/build/data-tests.md index eff8f9e2b70..0261ea2d8b2 100644 --- a/website/docs/docs/build/data-tests.md +++ b/website/docs/docs/build/data-tests.md @@ -263,7 +263,7 @@ Note that, if you select to store test failures: -## New `tests:` syntax +## New `data_tests:` syntax Data tests were historically called "tests" in dbt as the only form of testing available. With the introduction of unit tests in v1.8, it was necessary to update our naming conventions and syntax. As of v1.8, `tests:` is still supported in your YML configuration file as an alias but will be deprecated in the future in favor of `data_tests:`. From dd165e34605c6f10a17282b5a6de2f115cd23f79 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 11 Mar 2024 12:32:52 +0000 Subject: [PATCH 12/89] add body of first guide 1/3 --- website/docs/guides/core-to-cloud-1.md | 152 +++++++++++++++++++++++++ 1 file changed, 152 insertions(+) create mode 100644 website/docs/guides/core-to-cloud-1.md diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md new file mode 100644 index 00000000000..38fcb3bfacb --- /dev/null +++ b/website/docs/guides/core-to-cloud-1.md @@ -0,0 +1,152 @@ +--- +title: Switch from dbt Core to dbt Cloud +id: core-to-cloud-1 +description: "Create an adapter that connects dbt to you platform, and learn how to maintain and version that adapter." +hoverSnippet: "Learn how to build, test, document, and promote adapters as well as maintaining and versioning an adapter." +icon: 'guides' +hide_table_of_contents: true +tags: ['Migration','dbt Core','dbt Cloud'] +level: 'Intermediate' +recently_updated: true +--- + +## Switch to dbt Cloud + +Switching from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. + +This guide outlines the considerations needed to migrate your dbt project from Core to Cloud, highlighting collaboration enhancements and technical adjustments. + +- dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. dbt Cloud also supports a command line with dbt Cloud CLI, a Semantic Layer for consistent metrics, and domain ownership of data with multi-project “dbt Mesh” setups. Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features). +- dbt Core is an open-source tool that enables data teams to transform data following analytics engineering best practices using a command line interface. It must be self-hosted and maintained. + +## Prerequisites + +- You have dbt Core installed. +- You have an existing dbt Core project connected to a data platform and Git repository. +- You have a dbt Cloud account. **[Don't have one? Start your free trial today](https://www.getdbt.com/signup)**! + +## Account set up +*Time to complete: Approximately 30 mins to 1 hour* + +- [Create your dbt Cloud account](https://www.getdbt.com/signup). +- Learn about [access](/docs/cloud/manage-access/about-user-access) and [invite users](/docs/cloud/manage-access/about-user-access) to your dbt Cloud account and dbt Cloud project. +- Configure [Single Sign-On (SSO)](/docs/cloud/manage-access/sso-overview) or [Role-based access control (RBAC)](/docs/cloud/manage-access/about-user-access#role-based-access-control) for easy and secure access. + - This removes the need to save passwords and secret environment variables locally. +- In **Account settings**, switch on [partial parsing](/docs/deploy/deploy-environments#partial-parsing) to only reparse changed files, saving time. +- In **Account settings**, enable [Git repo caching](/docs/deploy/deploy-environments#git-repository-caching) to ensure job reliability, protection from third-party outages, and performance. + +## Environment variables + +- In dbt Cloud, [environment variables](/docs/build/environment-variables) are managed with a clear order of precedence, allowing users to define values at four levels (highest to lowest order of precedence): + + - The job level (job override) or in the IDE for an individual developer (personal override). _(Highest precedence)_ + - The environment level, which can be overridden by the job level or personal override. + - A project-wide default value, which can be overridden by the environment level, job level, or personal override. + - The optional default argument supplied to the `env_var` Jinja function in code. _(Lowest precedence)_ + + + +- dbt Cloud secures environment variables, offering additional measures for sensitive values, such as prefixing keys with `DBT_ENV_SECRET_` to obscure them in logs and the user interface (UI). + + + +- Ensure that all [environment variables](/docs/build/environment-variables) start with `DBT_` or `DBT_ENV_SECRET_`. If your dbt Core environment variables don’t follow this naming convention, perform a “find and replace” in your project to make sure all references to these environment variables contain the proper naming conventions. + + + +- To set these variables for an entire project or specific environments, navigate to **Deploy** > **Environments** > **Environment variables** tab. +- To set these variables at the job level, navigate to **Deploy** > **Jobs** > **Select your job** > **Settings** > **Advanced settings**. +- To set these variables at the personal override level, navigate to **Profile Settings** > **Credentials** > **Select your project** > **Environment variables**. + + + +## Data platform set up +_*Time to complete: Approximately 10-30 mins_ + +This section explains the considerations and methods to connect your data platform to dbt Cloud. + +- Set up your [data platform connections](/docs/cloud/connect-data-platform/about-connections) and [`environment variables`](#environment-variables) in dbt Cloud. + + You can verify your data platform connections by clicking the **Test connection** button in your deployment and development credentials settings. + +- Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enable secure authentication using your data platform’s SSO. + +## Git set up +_*Time to complete: Approximately 10 mins_ + +Your existing dbt project source code should live in a Git repository. In this step, you should connect your existing dbt project source code from Git to dbt Cloud. + +- Ensure your dbt project is in a Git repository +- Once you’ve set up a dbt Cloud account, you can [connect and configure Git](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud to connect your Git repository: + - Connect with one of the [native integrations](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud. + - [Import a git repository](/docs/cloud/git/import-a-project-by-git-url) from any valid git URL that points to a dbt project. +- Log into dbt Cloud using [OAuth connections](/docs/collaborate/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. + + Set up groups for dbt project access with those configured for repository access to streamline permissions. + +## Developer set up +_*Time to complete: Approximately 30 mins_ + +
+💡 What is a target in dbt Cloud? + +The concept of a `target` in dbt Core is the same as a [dbt Cloud environment](/docs/environments-in-dbt). + +The main difference between `target` in dbt Core and a dbt Cloud environment is that you can make these configurations through the dbt Cloud UI, as opposed to within your **`profiles.yml`** file. + +This difference streamlines the process of switching between development, staging, and production contexts, removing the need to manually edit a **`profiles.yml`** file. dbt Cloud environments also integrate with additional features such as job scheduling, version control, and more, making it easier to manage the full lifecycle of your dbt projects within a single platform You can [set up](/reference/dbt-jinja-functions/target) or [customize](/docs/build/custom-target-names) target names in dbt Cloud. +
+ +This section highlights what foundational configurations you’ll need to set up your dbt Cloud project. After setting up your dbt Cloud account and git integration: + +- Set up your [development](/docs/dbt-cloud-environments#create-a-development-environment) environment and [development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide). You’ll need this to access your dbt project and start developing. + - In your dbt Cloud environment and credentials, use the same dbt Core version you use locally. You can run `dbt --version` in the command line to find out which version of dbt Core you’re using. + - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. + - When using dbt Cloud, you can [connect to your data platform](/docs/cloud/connect-data-platform/about-connections) directly in the UI and don't need a `profiles.yml` file. Each environment is roughly analogous to an entry in your `profiles.yml` file. +- If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. +- Set up your development workspace with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) to edit and develop your dbt code in your tool of choice. +- Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. +- Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. + +## Orchestration set up +*Time to complete: Approximately 30 mins to 1 hour + +- To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one environment as the production environment. This would be the [deployment](/docs/deploy/deploy-environments) environment. +- In your environment settings, configure dbt Cloud with the same dbt Core version. + - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. +- [Configure your jobs](/docs/deploy/deploy-jobs) for automated or event-driven dbt jobs. You can use cron execution, manual, pull requests, or API triggers. + - Note that alongside [dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. +- You should set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. +- Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. +- Set up [notifications](/docs/deploy/job-notifications) by configuring email and Slack alerts to monitor your jobs. +- Use [monitoring tools](/docs/deploy/monitor-jobs) like run history, job retries, job chaining, dashboard status tiles, and more for a seamless orchestration experience. +- If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. + +### CI/CD set up +*Time to complete: Approximately 30 mins to 1 hour + +Building a custom solution to efficiently check code upon pull requests is complicated. With dbt Cloud, you can enable continuous integration / continuous deployment (CI/CD) and configure dbt Cloud to run your dbt projects in a temporary schema when new commits are pushed to open pull requests. This build-on-PR functionality is a great way to catch bugs before deploying to production, and an essential tool for data practitioners. + +- Set up an integration with a native git application (such as Azure DevOps, GitHub, GitLab) and a CI environment in dbt Cloud. +- Create [a CI/CD job](/docs/deploy/ci-jobs) to optimize workflows. +- Run your jobs in a production environment to fully implement CI/CD. Future pull requests will also leverage the last production runs to compare against. + +## Models configuration +*Time to complete: Approximately 30 mins to 1 hour + +In this section, you’ll be able to switch and configure your dbt models to dbt Cloud. You’ll want to make sure you set up your [development environment and credentials](#developer-set-up). + +- In your [development tool](/docs/cloud/about-develop-dbt) of choice, you can review your dbt project and ensure your project is set up correctly and you’re able to run commands. This means: + - Make sure your project compiles correctly. + - Run a few models in the dbt Cloud IDE or dbt Cloud CLI to ensure you’re experiencing accurate results in development. +- Once your first job has successfully run in your production environment, use [dbt Explorer](/docs/collaborate/explore-projects) to view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their [lineage](/terms/data-lineage) to gain a better understanding of its latest production state. + +## What’s next? + +- Link to the next guide (managing your migration or switch, etc.) +- Link to tips and faqs? + +## Related docs + +- Book [expert-led demos](https://www.getdbt.com/resources/dbt-cloud-demos-with-experts) and insights +- Work with the [dbt Labs’ Professional Services](https://www.getdbt.com/dbt-labs/services) team to support your data organization and migration. From aaef2ecbd3892ef189806af00874d79a2c4344a6 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 11 Mar 2024 14:12:05 +0000 Subject: [PATCH 13/89] tweak dev --- website/docs/guides/core-to-cloud-1.md | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 38fcb3bfacb..b1dcb0bafed 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -87,25 +87,28 @@ Your existing dbt project source code should live in a Git repository. In this s ## Developer set up _*Time to complete: Approximately 30 mins_ -
-💡 What is a target in dbt Cloud? +This section highlights what foundational configurations you’ll need to set up your dbt Cloud project. + +### dbt Cloud environments The concept of a `target` in dbt Core is the same as a [dbt Cloud environment](/docs/environments-in-dbt). The main difference between `target` in dbt Core and a dbt Cloud environment is that you can make these configurations through the dbt Cloud UI, as opposed to within your **`profiles.yml`** file. This difference streamlines the process of switching between development, staging, and production contexts, removing the need to manually edit a **`profiles.yml`** file. dbt Cloud environments also integrate with additional features such as job scheduling, version control, and more, making it easier to manage the full lifecycle of your dbt projects within a single platform You can [set up](/reference/dbt-jinja-functions/target) or [customize](/docs/build/custom-target-names) target names in dbt Cloud. -
-This section highlights what foundational configurations you’ll need to set up your dbt Cloud project. After setting up your dbt Cloud account and git integration: +### Initial set up steps +- **Set up development environment** — Set up your [development](/docs/dbt-cloud-environments#create-a-development-environment) environment and [development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide). You’ll need this to access your dbt project and start developing. +- **dbt Core version** — In your dbt Cloud environment and credentials, use the same dbt Core version you use locally. You can run `dbt --version` in the command line to find out which version of dbt Core you’re using. + - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. +- **Connect to your data platform** — When using dbt Cloud, you can [connect to your data platform](/docs/cloud/connect-data-platform/about-connections) directly in the UI and don't need a `profiles.yml` file. Each environment is roughly equivalent to an entry in your `profiles.yml` file. + +### Advanced configuration +- **Custom target names** — If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. +- **Development tools** — Set up your development workspace with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) to edit and develop your dbt code in your tool of choice. +- **API access** — Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. -- Set up your [development](/docs/dbt-cloud-environments#create-a-development-environment) environment and [development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide). You’ll need this to access your dbt project and start developing. - - In your dbt Cloud environment and credentials, use the same dbt Core version you use locally. You can run `dbt --version` in the command line to find out which version of dbt Core you’re using. - - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. - - When using dbt Cloud, you can [connect to your data platform](/docs/cloud/connect-data-platform/about-connections) directly in the UI and don't need a `profiles.yml` file. Each environment is roughly analogous to an entry in your `profiles.yml` file. -- If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. -- Set up your development workspace with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) to edit and develop your dbt code in your tool of choice. -- Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. +### dbt Cloud commands - Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. ## Orchestration set up From 16c0f12a98755877f5fe1fd5f8fa612baf541434 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 11 Mar 2024 14:23:50 +0000 Subject: [PATCH 14/89] tweak --- website/docs/guides/core-to-cloud-1.md | 27 +++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index b1dcb0bafed..dae77c1b752 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -112,21 +112,26 @@ This difference streamlines the process of switching between development, stagin - Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. ## Orchestration set up -*Time to complete: Approximately 30 mins to 1 hour +_*Time to complete: Approximately 30 mins to 1 hour_ -- To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one environment as the production environment. This would be the [deployment](/docs/deploy/deploy-environments) environment. -- In your environment settings, configure dbt Cloud with the same dbt Core version. +### dbt Cloud environments +To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one environment as the production environment. This would be the [deployment](/docs/deploy/deploy-environments) environment. You can set up multiple environments for different stages of your deployment pipeline, such as development, staging/QA, and production. + +### Initial set up steps +- **dbt Core version** — In your environment settings, configure dbt Cloud with the same dbt Core version. - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. -- [Configure your jobs](/docs/deploy/deploy-jobs) for automated or event-driven dbt jobs. You can use cron execution, manual, pull requests, or API triggers. +- **[Configure your jobs](/docs/deploy/deploy-jobs)** — You can create jobs for automated or event-driven dbt jobs. You can use cron execution, manual, pull requests, or API triggers. - Note that alongside [dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. -- You should set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. -- Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. -- Set up [notifications](/docs/deploy/job-notifications) by configuring email and Slack alerts to monitor your jobs. -- Use [monitoring tools](/docs/deploy/monitor-jobs) like run history, job retries, job chaining, dashboard status tiles, and more for a seamless orchestration experience. -- If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. + +### Additional configurations +- **Custom target names** — You should set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. +- **dbt commands** — Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. +- **Notifications** — Set up [notifications](/docs/deploy/job-notifications) by configuring email and Slack alerts to monitor your jobs. +- **Monitoring tools** — Use [monitoring tools](/docs/deploy/monitor-jobs) like run history, job retries, job chaining, dashboard status tiles, and more for a seamless orchestration experience. +- **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. ### CI/CD set up -*Time to complete: Approximately 30 mins to 1 hour +_*Time to complete: Approximately 30 mins to 1 hour_ Building a custom solution to efficiently check code upon pull requests is complicated. With dbt Cloud, you can enable continuous integration / continuous deployment (CI/CD) and configure dbt Cloud to run your dbt projects in a temporary schema when new commits are pushed to open pull requests. This build-on-PR functionality is a great way to catch bugs before deploying to production, and an essential tool for data practitioners. @@ -135,7 +140,7 @@ Building a custom solution to efficiently check code upon pull requests is compl - Run your jobs in a production environment to fully implement CI/CD. Future pull requests will also leverage the last production runs to compare against. ## Models configuration -*Time to complete: Approximately 30 mins to 1 hour +_*Time to complete: Approximately 30 mins to 1 hour_ In this section, you’ll be able to switch and configure your dbt models to dbt Cloud. You’ll want to make sure you set up your [development environment and credentials](#developer-set-up). From da06590f502da79bc7eb6c49a0d3b25b412223d0 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 11 Mar 2024 14:27:00 +0000 Subject: [PATCH 15/89] fix link --- website/docs/guides/core-to-cloud-1.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index dae77c1b752..7f5df53c9f1 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -80,7 +80,7 @@ Your existing dbt project source code should live in a Git repository. In this s - Once you’ve set up a dbt Cloud account, you can [connect and configure Git](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud to connect your Git repository: - Connect with one of the [native integrations](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud. - [Import a git repository](/docs/cloud/git/import-a-project-by-git-url) from any valid git URL that points to a dbt project. -- Log into dbt Cloud using [OAuth connections](/docs/collaborate/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. +- Log into dbt Cloud using [OAuth connections](/docs/cloud/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. Set up groups for dbt project access with those configured for repository access to streamline permissions. @@ -150,10 +150,13 @@ In this section, you’ll be able to switch and configure your dbt models to dbt - Once your first job has successfully run in your production environment, use [dbt Explorer](/docs/collaborate/explore-projects) to view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their [lineage](/terms/data-lineage) to gain a better understanding of its latest production state. ## What’s next? +Congrats on completing the first part of your migration to dbt Cloud! You should have learned how to set up your dbt Cloud account, connect your data platform, Git repository, and configure your development, orchestration, and CI/CD environments. You’ve also set up your models and are ready to run your first job in dbt Cloud. +For next steps, we will - Link to the next guide (managing your migration or switch, etc.) - Link to tips and faqs? + ## Related docs - Book [expert-led demos](https://www.getdbt.com/resources/dbt-cloud-demos-with-experts) and insights From 7d3bbfff46d55ef9faa9c66ac84ddc70ebed2c76 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 11 Mar 2024 16:36:39 +0000 Subject: [PATCH 16/89] add confetti and final touches --- website/docs/docs/core/about-core-setup.md | 6 +- .../docs/docs/core/installation-overview.md | 3 + website/docs/guides/core-to-cloud-1.md | 167 +++++++++++------- website/docs/guides/manual-install-qs.md | 7 +- website/package-lock.json | 15 ++ website/package.json | 1 + website/src/components/confetti/index.js | 58 ++++++ .../components/confetti/styles.modules.css | 0 website/src/css/custom.css | 1 + website/src/theme/MDXComponents/index.js | 1 + 10 files changed, 196 insertions(+), 63 deletions(-) create mode 100644 website/src/components/confetti/index.js create mode 100644 website/src/components/confetti/styles.modules.css diff --git a/website/docs/docs/core/about-core-setup.md b/website/docs/docs/core/about-core-setup.md index 8b170ba70d4..b4caf66ef1d 100644 --- a/website/docs/docs/core/about-core-setup.md +++ b/website/docs/docs/core/about-core-setup.md @@ -14,7 +14,9 @@ dbt Core is an [open-source](https://github.com/dbt-labs/dbt-core) tool that ena - [Connecting to a data platform](/docs/core/connect-data-platform/profiles.yml) - [How to run your dbt projects](/docs/running-a-dbt-project/run-your-dbt-projects) -To learn about developing dbt projects in dbt Cloud, refer to [Develop with dbt Cloud](/docs/cloud/about-develop-dbt). - - dbt Cloud provides a command line interface with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). Both dbt Core and the dbt Cloud CLI are command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). +You can also learn about [developing dbt projects in dbt Cloud](/docs/cloud/about-develop-dbt) — dbt Cloud provides a command line interface with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) that let you run dbt commands. The dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](https://docs.getdbt.com/guides). + +## Related docs +- [Switch from dbt Core to dbt Cloud](/guides/core-to-cloud-1?step=1) diff --git a/website/docs/docs/core/installation-overview.md b/website/docs/docs/core/installation-overview.md index 8c139012667..726950a8406 100644 --- a/website/docs/docs/core/installation-overview.md +++ b/website/docs/docs/core/installation-overview.md @@ -50,3 +50,6 @@ Most command-line tools, including dbt, have a `--help` flag that you can use to — `dbt run --help`: Lists the flags available for the `run` command ::: + +## Related docs +- [Switch from dbt Core to dbt Cloud](/guides/core-to-cloud-1?step=1) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 7f5df53c9f1..37619d692bc 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -14,9 +14,12 @@ recently_updated: true Switching from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. -This guide outlines the considerations needed to migrate your dbt project from Core to Cloud, highlighting collaboration enhancements and technical adjustments. +This guide outlines the considerations needed to switch your dbt project from dbt Core to dbt Cloud, highlighting collaboration enhancements and technical adjustments. + +- dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development, a [Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics, and domain ownership of data with multi-project “[dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro)” setups. + + Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features). -- dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. dbt Cloud also supports a command line with dbt Cloud CLI, a Semantic Layer for consistent metrics, and domain ownership of data with multi-project “dbt Mesh” setups. Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features). - dbt Core is an open-source tool that enables data teams to transform data following analytics engineering best practices using a command line interface. It must be self-hosted and maintained. ## Prerequisites @@ -28,66 +31,76 @@ This guide outlines the considerations needed to migrate your dbt project from C ## Account set up *Time to complete: Approximately 30 mins to 1 hour* -- [Create your dbt Cloud account](https://www.getdbt.com/signup). -- Learn about [access](/docs/cloud/manage-access/about-user-access) and [invite users](/docs/cloud/manage-access/about-user-access) to your dbt Cloud account and dbt Cloud project. -- Configure [Single Sign-On (SSO)](/docs/cloud/manage-access/sso-overview) or [Role-based access control (RBAC)](/docs/cloud/manage-access/about-user-access#role-based-access-control) for easy and secure access. - - This removes the need to save passwords and secret environment variables locally. -- In **Account settings**, switch on [partial parsing](/docs/deploy/deploy-environments#partial-parsing) to only reparse changed files, saving time. -- In **Account settings**, enable [Git repo caching](/docs/deploy/deploy-environments#git-repository-caching) to ensure job reliability, protection from third-party outages, and performance. +1. [Create your dbt Cloud account](https://www.getdbt.com/signup). -## Environment variables +2. Learn about [access](/docs/cloud/manage-access/about-user-access) and [invite users](/docs/cloud/manage-access/about-user-access) to your dbt Cloud account and dbt Cloud project. + +3. Configure [Single Sign-On (SSO)](/docs/cloud/manage-access/sso-overview) or [Role-based access control (RBAC)](/docs/cloud/manage-access/about-user-access#role-based-access-control) for easy and secure access. + - This removes the need to save passwords and secret environment variables locally. + +4. In **Account settings**, switch on [partial parsing](/docs/deploy/deploy-environments#partial-parsing) to only reparse changed files, saving time. -- In dbt Cloud, [environment variables](/docs/build/environment-variables) are managed with a clear order of precedence, allowing users to define values at four levels (highest to lowest order of precedence): +5. In **Account settings**, enable [Git repo caching](/docs/deploy/deploy-environments#git-repository-caching) to ensure job reliability, protection from third-party outages, and performance. - - The job level (job override) or in the IDE for an individual developer (personal override). _(Highest precedence)_ - - The environment level, which can be overridden by the job level or personal override. - - A project-wide default value, which can be overridden by the environment level, job level, or personal override. - - The optional default argument supplied to the `env_var` Jinja function in code. _(Lowest precedence)_ +## Environment variables + +1. In dbt Cloud, [environment variables](/docs/build/environment-variables) are managed with a clear order of precedence, allowing users to define values at four levels (highest to lowest order of precedence): + - The job level (job override) or in the IDE for an individual developer (personal override). _(Highest precedence)_ + - The environment level, which can be overridden by the job level or personal override. + - A project-wide default value, which can be overridden by the environment level, job level, or personal override. + - The optional default argument supplied to the `env_var` Jinja function in code. _(Lowest precedence)_ -- dbt Cloud secures environment variables, offering additional measures for sensitive values, such as prefixing keys with `DBT_ENV_SECRET_` to obscure them in logs and the user interface (UI). +2. dbt Cloud secures environment variables, offering additional measures for sensitive values, such as prefixing keys with `DBT_ENV_SECRET_` to obscure them in logs and the user interface (UI). -- Ensure that all [environment variables](/docs/build/environment-variables) start with `DBT_` or `DBT_ENV_SECRET_`. If your dbt Core environment variables don’t follow this naming convention, perform a “find and replace” in your project to make sure all references to these environment variables contain the proper naming conventions. +3. Ensure that all [environment variables](/docs/build/environment-variables) start with `DBT_` or `DBT_ENV_SECRET_`. If your dbt Core environment variables don’t follow this naming convention, perform a “find and replace” in your project to make sure all references to these environment variables contain the proper naming conventions. - +### How to set environment variables in dbt Cloud - To set these variables for an entire project or specific environments, navigate to **Deploy** > **Environments** > **Environment variables** tab. - To set these variables at the job level, navigate to **Deploy** > **Jobs** > **Select your job** > **Settings** > **Advanced settings**. - To set these variables at the personal override level, navigate to **Profile Settings** > **Credentials** > **Select your project** > **Environment variables**. - - ## Data platform set up _*Time to complete: Approximately 10-30 mins_ This section explains the considerations and methods to connect your data platform to dbt Cloud. -- Set up your [data platform connections](/docs/cloud/connect-data-platform/about-connections) and [`environment variables`](#environment-variables) in dbt Cloud. - - You can verify your data platform connections by clicking the **Test connection** button in your deployment and development credentials settings. +1. Set up your [data platform connections](/docs/cloud/connect-data-platform/about-connections) and [environment variables](/guides/core-to-cloud-1?step=4) in dbt Cloud. -- Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enable secure authentication using your data platform’s SSO. +2. You can verify your data platform connections by clicking the **Test connection** button in your deployment and development credentials settings. + +### Advanced configuration +3. Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enables secure authentication using your data platform’s SSO. ## Git set up _*Time to complete: Approximately 10 mins_ Your existing dbt project source code should live in a Git repository. In this step, you should connect your existing dbt project source code from Git to dbt Cloud. -- Ensure your dbt project is in a Git repository -- Once you’ve set up a dbt Cloud account, you can [connect and configure Git](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud to connect your Git repository: - - Connect with one of the [native integrations](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud. - - [Import a git repository](/docs/cloud/git/import-a-project-by-git-url) from any valid git URL that points to a dbt project. -- Log into dbt Cloud using [OAuth connections](/docs/cloud/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. +1. Ensure your dbt project is in a Git repository + +2. Once you’ve set up a dbt Cloud account, you can [connect and configure Git](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud to connect your Git repository: + - Connect with one of the [native integrations](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud (such as GitHub, GitLab, and Azure DevOps). + - [Import a git repository](/docs/cloud/git/import-a-project-by-git-url) from any valid git URL that points to a dbt project. + +### Advanced configuration +3. Log into dbt Cloud using [OAuth connections](/docs/cloud/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. Set up groups for dbt project access with those configured for repository access to streamline permissions. ## Developer set up _*Time to complete: Approximately 30 mins_ -This section highlights what foundational configurations you’ll need to set up your dbt Cloud project. +This section highlights the development configurations you’ll need for your dbt Cloud project. The following categories are covered in this section: + +- [dbt Cloud environments](/guides/core-to-cloud-1?step=7#dbt-cloud-environments) +- [Initial set up steps](/guides/core-to-cloud-1?step=7#initial-set-up-steps) +- [Advanced configuration](/guides/core-to-cloud-1?step=7#initial-set-up-steps) +- [dbt Cloud commands](/guides/core-to-cloud-1?step=7#dbt-cloud-commands) ### dbt Cloud environments @@ -98,66 +111,100 @@ The main difference between `target` in dbt Core and a dbt Cloud environment is This difference streamlines the process of switching between development, staging, and production contexts, removing the need to manually edit a **`profiles.yml`** file. dbt Cloud environments also integrate with additional features such as job scheduling, version control, and more, making it easier to manage the full lifecycle of your dbt projects within a single platform You can [set up](/reference/dbt-jinja-functions/target) or [customize](/docs/build/custom-target-names) target names in dbt Cloud. ### Initial set up steps -- **Set up development environment** — Set up your [development](/docs/dbt-cloud-environments#create-a-development-environment) environment and [development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide). You’ll need this to access your dbt project and start developing. -- **dbt Core version** — In your dbt Cloud environment and credentials, use the same dbt Core version you use locally. You can run `dbt --version` in the command line to find out which version of dbt Core you’re using. - - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. -- **Connect to your data platform** — When using dbt Cloud, you can [connect to your data platform](/docs/cloud/connect-data-platform/about-connections) directly in the UI and don't need a `profiles.yml` file. Each environment is roughly equivalent to an entry in your `profiles.yml` file. +1. **Set up development environment** — Set up your [development](/docs/dbt-cloud-environments#create-a-development-environment) environment and [development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide). You’ll need this to access your dbt project and start developing. + +2. **dbt Core version** — In your dbt Cloud environment and credentials, use the same dbt Core version you use locally. You can run `dbt --version` in the command line to find out which version of dbt Core you’re using. + - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. + +3. **Connect to your data platform** — When using dbt Cloud, you can [connect to your data platform](/docs/cloud/connect-data-platform/about-connections) directly in the UI and don't need a `profiles.yml` file. Each environment is roughly equivalent to an entry in your `profiles.yml` file. +4. **Development tools** — Set up your development workspace with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) to edit and develop your dbt code in your tool of choice. ### Advanced configuration -- **Custom target names** — If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. -- **Development tools** — Set up your development workspace with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) to edit and develop your dbt code in your tool of choice. -- **API access** — Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. +5. **Custom target names** — If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. + +6. **API access** — Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. ### dbt Cloud commands -- Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. +7. Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. ## Orchestration set up _*Time to complete: Approximately 30 mins to 1 hour_ +This section outlines the considerations and methods to set up your dbt Cloud environments and jobs for orchestration. The following categories are covered in this section: + +- [dbt Cloud environments](/guides/core-to-cloud-1?step=8#dbt-cloud-environments-1) +- [Initial set up steps](/guides/core-to-cloud-1?step=8#initial-set-up-steps-1) +- [Additional configurations](/guides/core-to-cloud-1?step=8#additional-configurations) +- [CI/CD set up](/guides/core-to-cloud-1?step=8#cicd-set-up) + ### dbt Cloud environments To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one environment as the production environment. This would be the [deployment](/docs/deploy/deploy-environments) environment. You can set up multiple environments for different stages of your deployment pipeline, such as development, staging/QA, and production. ### Initial set up steps -- **dbt Core version** — In your environment settings, configure dbt Cloud with the same dbt Core version. - - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. -- **[Configure your jobs](/docs/deploy/deploy-jobs)** — You can create jobs for automated or event-driven dbt jobs. You can use cron execution, manual, pull requests, or API triggers. - - Note that alongside [dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. +1. **dbt Core version** — In your environment settings, configure dbt Cloud with the same dbt Core version. + - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. + +2. **[Configure your jobs](/docs/deploy/deploy-jobs)** — You can create jobs for automated or event-driven dbt jobs. You can use cron execution, manual, pull requests, or API triggers. + - Note that alongside [dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. ### Additional configurations -- **Custom target names** — You should set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. -- **dbt commands** — Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. -- **Notifications** — Set up [notifications](/docs/deploy/job-notifications) by configuring email and Slack alerts to monitor your jobs. -- **Monitoring tools** — Use [monitoring tools](/docs/deploy/monitor-jobs) like run history, job retries, job chaining, dashboard status tiles, and more for a seamless orchestration experience. -- **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. +3. **Custom target names** — You should set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. + +4. **dbt commands** — Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. + +5. **Notifications** — Set up [notifications](/docs/deploy/job-notifications) by configuring email and Slack alerts to monitor your jobs. + +6. **Monitoring tools** — Use [monitoring tools](/docs/deploy/monitor-jobs) like run history, job retries, job chaining, dashboard status tiles, and more for a seamless orchestration experience. + +7. **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. ### CI/CD set up -_*Time to complete: Approximately 30 mins to 1 hour_ -Building a custom solution to efficiently check code upon pull requests is complicated. With dbt Cloud, you can enable continuous integration / continuous deployment (CI/CD) and configure dbt Cloud to run your dbt projects in a temporary schema when new commits are pushed to open pull requests. This build-on-PR functionality is a great way to catch bugs before deploying to production, and an essential tool for data practitioners. +Building a custom solution to efficiently check code upon pull requests is complicated. With dbt Cloud, you can enable [continuous integration / continuous deployment (CI/CD)](/docs/deploy/continuous-integration) and configure dbt Cloud to run your dbt projects in a temporary schema when new commits are pushed to open pull requests. + +This build-on-PR functionality is a great way to catch bugs before deploying to production, and an essential tool for data practitioners. -- Set up an integration with a native git application (such as Azure DevOps, GitHub, GitLab) and a CI environment in dbt Cloud. -- Create [a CI/CD job](/docs/deploy/ci-jobs) to optimize workflows. -- Run your jobs in a production environment to fully implement CI/CD. Future pull requests will also leverage the last production runs to compare against. +1. Set up an integration with a native git application (such as Azure DevOps, GitHub, GitLab) and a CI environment in dbt Cloud. +2. Create [a CI/CD job](/docs/deploy/ci-jobs) to optimize workflows. +3. Run your jobs in a production environment to fully implement CI/CD. Future pull requests will also leverage the last production runs to compare against. ## Models configuration _*Time to complete: Approximately 30 mins to 1 hour_ -In this section, you’ll be able to switch and configure your dbt models to dbt Cloud. You’ll want to make sure you set up your [development environment and credentials](#developer-set-up). +In this section, you’ll be able to validate whether your models run or compile correctly in your development tool of choice: [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). -- In your [development tool](/docs/cloud/about-develop-dbt) of choice, you can review your dbt project and ensure your project is set up correctly and you’re able to run commands. This means: - - Make sure your project compiles correctly. - - Run a few models in the dbt Cloud IDE or dbt Cloud CLI to ensure you’re experiencing accurate results in development. -- Once your first job has successfully run in your production environment, use [dbt Explorer](/docs/collaborate/explore-projects) to view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their [lineage](/terms/data-lineage) to gain a better understanding of its latest production state. +You’ll want to make sure you set up your [development environment and credentials](#developer-set-up). + +1. In your [development tool](/docs/cloud/about-develop-dbt) of choice, you can review your dbt project and ensure your project is set up correctly and you’re able to run commands. This means: + - Make sure your project compiles correctly. + - Run a few models in the dbt Cloud IDE or dbt Cloud CLI to ensure you’re experiencing accurate results in development. + +2. Once your first job has successfully run in your production environment, use [dbt Explorer](/docs/collaborate/explore-projects) to view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their [lineage](/terms/data-lineage) to gain a better understanding of its latest production state. ## What’s next? -Congrats on completing the first part of your migration to dbt Cloud! You should have learned how to set up your dbt Cloud account, connect your data platform, Git repository, and configure your development, orchestration, and CI/CD environments. You’ve also set up your models and are ready to run your first job in dbt Cloud. -For next steps, we will +import ConfettiTrigger from '/src/components/confetti/index.js'; + + + +Congrats on completing the first part of your switch to dbt Cloud 🎉! + +You should have learned: +- How to set up your dbt Cloud account +- Connect your data platform and Git repository +- Configure your development, orchestration, and CI/CD environments +- You’ve also set up your models and are ready to run your first job in dbt Cloud. + +For next steps, we'll soon share other guides on how to manage your switch and trips. Stay tuned! + + - -## Related docs +### Related docs - Book [expert-led demos](https://www.getdbt.com/resources/dbt-cloud-demos-with-experts) and insights - Work with the [dbt Labs’ Professional Services](https://www.getdbt.com/dbt-labs/services) team to support your data organization and migration. + + diff --git a/website/docs/guides/manual-install-qs.md b/website/docs/guides/manual-install-qs.md index fcd1e5e9599..2cc12d1db92 100644 --- a/website/docs/guides/manual-install-qs.md +++ b/website/docs/guides/manual-install-qs.md @@ -10,7 +10,9 @@ hide_table_of_contents: true --- ## Introduction -When you use dbt Core to work with dbt, you will be editing files locally using a code editor, and running projects using a command line interface (CLI). If you'd rather edit files and run projects using the web-based Integrated Development Environment (IDE), you should refer to the [dbt Cloud quickstarts](/guides). You can also develop and run dbt commands using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) — a dbt Cloud powered command line. +When you use dbt Core to work with dbt, you will be editing files locally using a code editor, and running projects using a command line interface (CLI). + +If you'd rather edit files and run projects using the web-based Integrated Development Environment (IDE), you should refer to the [dbt Cloud quickstarts](/guides). You can also develop and run dbt commands using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) — a dbt Cloud powered command line. ### Prerequisites @@ -23,6 +25,9 @@ When you use dbt Core to work with dbt, you will be editing files locally using After setting up BigQuery to work with dbt, you are ready to create a starter project with example models, before building your own models. +### Related docs +- [Switch from dbt Core to dbt Cloud](/guides/core-to-cloud-1?step=1) + ## Create a repository The following steps use [GitHub](https://github.com/) as the Git provider for this guide, but you can use any Git provider. You should have already [created a GitHub account](https://github.com/join). diff --git a/website/package-lock.json b/website/package-lock.json index 282056e5922..864954c3e8d 100644 --- a/website/package-lock.json +++ b/website/package-lock.json @@ -16,6 +16,7 @@ "@stoplight/elements": "^7.7.17", "@svgr/webpack": "^6.0.0", "axios": "^0.27.2", + "canvas-confetti": "^1.9.2", "classnames": "^2.3.1", "clsx": "^1.1.1", "color": "^3.1.2", @@ -8780,6 +8781,15 @@ } ] }, + "node_modules/canvas-confetti": { + "version": "1.9.2", + "resolved": "https://registry.npmjs.org/canvas-confetti/-/canvas-confetti-1.9.2.tgz", + "integrity": "sha512-6Xi7aHHzKwxZsem4mCKoqP6YwUG3HamaHHAlz1hTNQPCqXhARFpSXnkC9TWlahHY5CG6hSL5XexNjxK8irVErg==", + "funding": { + "type": "donate", + "url": "https://www.paypal.me/kirilvatev" + } + }, "node_modules/caseless": { "version": "0.12.0", "resolved": "https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz", @@ -32098,6 +32108,11 @@ "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001470.tgz", "integrity": "sha512-065uNwY6QtHCBOExzbV6m236DDhYCCtPmQUCoQtwkVqzud8v5QPidoMr6CoMkC2nfp6nksjttqWQRRh75LqUmA==" }, + "canvas-confetti": { + "version": "1.9.2", + "resolved": "https://registry.npmjs.org/canvas-confetti/-/canvas-confetti-1.9.2.tgz", + "integrity": "sha512-6Xi7aHHzKwxZsem4mCKoqP6YwUG3HamaHHAlz1hTNQPCqXhARFpSXnkC9TWlahHY5CG6hSL5XexNjxK8irVErg==" + }, "caseless": { "version": "0.12.0", "resolved": "https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz", diff --git a/website/package.json b/website/package.json index b0105102359..1c361319eb1 100644 --- a/website/package.json +++ b/website/package.json @@ -19,6 +19,7 @@ "@stoplight/elements": "^7.7.17", "@svgr/webpack": "^6.0.0", "axios": "^0.27.2", + "canvas-confetti": "^1.9.2", "classnames": "^2.3.1", "clsx": "^1.1.1", "color": "^3.1.2", diff --git a/website/src/components/confetti/index.js b/website/src/components/confetti/index.js new file mode 100644 index 00000000000..5ae2aa1b09d --- /dev/null +++ b/website/src/components/confetti/index.js @@ -0,0 +1,58 @@ +import React, { useRef, useEffect } from 'react'; +import confetti from 'canvas-confetti'; + +const ConfettiTrigger = ({ children }) => { + const triggerRef = useRef(null); // Use a ref to refer to the component's section + + useEffect(() => { + const triggerConfetti = () => { + if (event.target.closest('a, h2, h3')) { + // Do nothing if the click is on these elements + return; + } + // Configuration for the confetti + const confettiCount = 200; + const spread = 70; + const startVelocity = 30; + const scalar = 0.7; + const gravity = 1.5; + const decay = 0.9; + + // Launch confetti from multiple points + for (let i = 0; i < 5; i++) { + confetti({ + angle: 60, + spread: spread, + particleCount: confettiCount / 5, + origin: { y: 0.6, x: (i + 1) * 0.2 - 0.1 }, + startVelocity: startVelocity, + scalar: scalar, + gravity: gravity, + decay: decay, + zIndex: 9999, + }); + } + }; + + const triggerElement = triggerRef.current; + // Add event listener to the section + if (triggerElement) { + triggerElement.addEventListener('click', triggerConfetti); + } + + // Clean up the event listener on component unmount + return () => { + if (triggerElement) { + triggerElement.removeEventListener('click', triggerConfetti); + } + }; + }, []); + + return ( +
+ {children} {/* Render children passed to the component */} +
+ ); +}; + +export default ConfettiTrigger; diff --git a/website/src/components/confetti/styles.modules.css b/website/src/components/confetti/styles.modules.css new file mode 100644 index 00000000000..e69de29bb2d diff --git a/website/src/css/custom.css b/website/src/css/custom.css index 2f938605c51..38389a71b53 100644 --- a/website/src/css/custom.css +++ b/website/src/css/custom.css @@ -2090,6 +2090,7 @@ h2.anchor.clicked a.hash-link:before { display: none; } + @media (max-width: 996px) { .quickstart-container { flex-direction: column; diff --git a/website/src/theme/MDXComponents/index.js b/website/src/theme/MDXComponents/index.js index 345145c780f..aa66db52de6 100644 --- a/website/src/theme/MDXComponents/index.js +++ b/website/src/theme/MDXComponents/index.js @@ -45,6 +45,7 @@ import Icon from '@site/src/components/icon'; import Lifecycle from '@site/src/components/lifeCycle'; import detailsToggle from '@site/src/components/detailsToggle'; import expandable from '@site/src/components/expandable'; +import ConfettiTrigger from '@site/src/components/confetti/'; const MDXComponents = { head: MDXHead, From 581ddd89baeb4d2ab10d1cb7e8ac2d4ed03b9ce0 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 11 Mar 2024 16:39:24 +0000 Subject: [PATCH 17/89] adjust timing --- website/docs/guides/core-to-cloud-1.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 37619d692bc..2d1a2c19f2f 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -9,7 +9,6 @@ tags: ['Migration','dbt Core','dbt Cloud'] level: 'Intermediate' recently_updated: true --- - ## Switch to dbt Cloud Switching from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. @@ -22,7 +21,7 @@ This guide outlines the considerations needed to switch your dbt project from db - dbt Core is an open-source tool that enables data teams to transform data following analytics engineering best practices using a command line interface. It must be self-hosted and maintained. -## Prerequisites +### Prerequisites - You have dbt Core installed. - You have an existing dbt Core project connected to a data platform and Git repository. @@ -77,7 +76,7 @@ This section explains the considerations and methods to connect your data platfo 3. Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enables secure authentication using your data platform’s SSO. ## Git set up -_*Time to complete: Approximately 10 mins_ +_*Time to complete: Approximately 10 to 30 mins_ Your existing dbt project source code should live in a Git repository. In this step, you should connect your existing dbt project source code from Git to dbt Cloud. @@ -93,7 +92,7 @@ Your existing dbt project source code should live in a Git repository. In this s Set up groups for dbt project access with those configured for repository access to streamline permissions. ## Developer set up -_*Time to complete: Approximately 30 mins_ +_*Time to complete: Approximately 30 mins to 1 hour_ This section highlights the development configurations you’ll need for your dbt Cloud project. The following categories are covered in this section: @@ -169,7 +168,7 @@ This build-on-PR functionality is a great way to catch bugs before deploying to 3. Run your jobs in a production environment to fully implement CI/CD. Future pull requests will also leverage the last production runs to compare against. ## Models configuration -_*Time to complete: Approximately 30 mins to 1 hour_ +_*Time to complete: Approximately 30 mins_ In this section, you’ll be able to validate whether your models run or compile correctly in your development tool of choice: [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). From 284ef5d728d95d58edb98f0bfd32a386ecb532dd Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 11 Mar 2024 16:50:14 +0000 Subject: [PATCH 18/89] fix title --- website/docs/guides/core-to-cloud-1.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 2d1a2c19f2f..59f5dffd68b 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -1,15 +1,15 @@ --- -title: Switch from dbt Core to dbt Cloud +title: Switch to dbt Cloud - Getting started id: core-to-cloud-1 -description: "Create an adapter that connects dbt to you platform, and learn how to maintain and version that adapter." -hoverSnippet: "Learn how to build, test, document, and promote adapters as well as maintaining and versioning an adapter." +description: "Learn how to Switch from dbt Core to dbt Cloud and what you need to get started." +hoverSnippet: "Use this guide to learn how to switch from dbt Core to dbt Cloud." icon: 'guides' hide_table_of_contents: true tags: ['Migration','dbt Core','dbt Cloud'] level: 'Intermediate' recently_updated: true --- -## Switch to dbt Cloud +## Getting started Switching from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. From 97dbd09609359405470a84ba9f57e66450747904 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Mon, 11 Mar 2024 16:50:46 +0000 Subject: [PATCH 19/89] add --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 59f5dffd68b..b87eddb0a53 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -1,5 +1,5 @@ --- -title: Switch to dbt Cloud - Getting started +title: Switch from dbt Core to dbt Cloud - Getting started id: core-to-cloud-1 description: "Learn how to Switch from dbt Core to dbt Cloud and what you need to get started." hoverSnippet: "Use this guide to learn how to switch from dbt Core to dbt Cloud." From 813d1c6c7696c3476192002855644c7bf73b7d77 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 12 Mar 2024 11:08:00 +0000 Subject: [PATCH 20/89] tweak --- website/docs/guides/core-to-cloud-1.md | 37 ++++++++++++++++++++++---- 1 file changed, 32 insertions(+), 5 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index b87eddb0a53..7d00ba957a7 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -9,24 +9,42 @@ tags: ['Migration','dbt Core','dbt Cloud'] level: 'Intermediate' recently_updated: true --- -## Getting started +## Introduction Switching from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. -This guide outlines the considerations needed to switch your dbt project from dbt Core to dbt Cloud, highlighting collaboration enhancements and technical adjustments. - - dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development, a [Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics, and domain ownership of data with multi-project “[dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro)” setups. Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features). - dbt Core is an open-source tool that enables data teams to transform data following analytics engineering best practices using a command line interface. It must be self-hosted and maintained. -### Prerequisites +### What you'll learn + +This guide outlines what you need to do in order to switch from dbt Core to dbt Cloud and highlights the necessary technical changes: . + +- [Account set up](/guides/core-to-cloud-1?step=3#account-set-up): Learn how to create a dbt Cloud account, invite team members, and configure it for your team. +- [Environment variables](m/guides/core-to-cloud-1?step=4#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. +- [Data platform set up](/guides/core-to-cloud-1?step=5#data-platform-set-up): Find out about connecting your data platform to dbt Cloud. +- [Git set up](/guides/core-to-cloud-1?step=6#git-set-up): Learn to link your dbt project's Git repository with dbt Cloud. +- [Developer set up:](/guides/core-to-cloud-1?step=7#developer-set-up) Understand the setup needed for developing in dbt Cloud. +- [Orchestration set up](/guides/core-to-cloud-1?step=8#orchestration-set-up): Learn how to prepare your dbt Cloud environment and jobs for orchestration. +- [Models configuration](/guides/core-to-cloud-1?step=9#models-configuration): Get insights on validating and running your models in dbt Cloud, using either the IDE or CLI. +- [What's next?](/guides/core-to-cloud-1?step=10#whats-next): Summarizes key takeaways and introduces what to expect in the following guides. + +## Prerequisites - You have dbt Core installed. - You have an existing dbt Core project connected to a data platform and Git repository. - You have a dbt Cloud account. **[Don't have one? Start your free trial today](https://www.getdbt.com/signup)**! +### Related docs +- [Learn dbt Cloud](https://courses.getdbt.com/collections) +- [Develop with dbt Cloud](/docs/cloud/about-develop-dbt) +- [Deploy jobs](/docs/deploy/deployments) +- Book [expert-led demos](https://www.getdbt.com/resources/dbt-cloud-demos-with-experts) and insights +- Work with the [dbt Labs’ Professional Services](https://www.getdbt.com/dbt-labs/services) team to support your data organization and migration. + ## Account set up *Time to complete: Approximately 30 mins to 1 hour* @@ -68,7 +86,16 @@ _*Time to complete: Approximately 10-30 mins_ This section explains the considerations and methods to connect your data platform to dbt Cloud. -1. Set up your [data platform connections](/docs/cloud/connect-data-platform/about-connections) and [environment variables](/guides/core-to-cloud-1?step=4) in dbt Cloud. +1. Set up your [data platform connections](/docs/cloud/connect-data-platform/about-connections) and [environment variables](/guides/core-to-cloud-1?step=4) in dbt Cloud. dbt Cloud can connect with a variety of data platform providers including: + - [AlloyDB](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) + - [Amazon Redshift](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) + - [Apache Spark](/docs/cloud/connect-data-platform/connect-apache-spark) + - [Databricks](/docs/cloud/connect-data-platform/connect-databricks) + - [Google BigQuery](/docs/cloud/connect-data-platform/connect-bigquery) + - [Microsoft Fabric](/docs/cloud/connect-data-platform/connect-microsoft-fabric) + - [PostgreSQL](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) + - [Snowflake](/docs/cloud/connect-data-platform/connect-snowflake) + - [Starburst or Trino](/docs/cloud/connect-data-platform/connect-starburst-trino) 2. You can verify your data platform connections by clicking the **Test connection** button in your deployment and development credentials settings. From 9898ec4210a8968ae69fce0529ed956504b64ec4 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 12 Mar 2024 11:12:23 +0000 Subject: [PATCH 21/89] style guide --- website/docs/guides/core-to-cloud-1.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 7d00ba957a7..10c4fb48cfe 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -27,9 +27,9 @@ This guide outlines what you need to do in order to switch from dbt Core to dbt - [Environment variables](m/guides/core-to-cloud-1?step=4#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. - [Data platform set up](/guides/core-to-cloud-1?step=5#data-platform-set-up): Find out about connecting your data platform to dbt Cloud. - [Git set up](/guides/core-to-cloud-1?step=6#git-set-up): Learn to link your dbt project's Git repository with dbt Cloud. -- [Developer set up:](/guides/core-to-cloud-1?step=7#developer-set-up) Understand the setup needed for developing in dbt Cloud. +- [Developer set up:](/guides/core-to-cloud-1?step=7#developer-set-up) Understand the set up needed for developing in dbt Cloud. - [Orchestration set up](/guides/core-to-cloud-1?step=8#orchestration-set-up): Learn how to prepare your dbt Cloud environment and jobs for orchestration. -- [Models configuration](/guides/core-to-cloud-1?step=9#models-configuration): Get insights on validating and running your models in dbt Cloud, using either the IDE or CLI. +- [Models configuration](/guides/core-to-cloud-1?step=9#models-configuration): Get insights on validating and running your models in dbt Cloud, using either the dbt Cloud IDE or dbt Cloud CLI. - [What's next?](/guides/core-to-cloud-1?step=10#whats-next): Summarizes key takeaways and introduces what to expect in the following guides. ## Prerequisites From 6eee95ed567701a2e405af061fa2730b9a834eaf Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 12 Mar 2024 11:15:20 +0000 Subject: [PATCH 22/89] fix link --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 10c4fb48cfe..22af433b7f4 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -24,7 +24,7 @@ Switching from dbt Core to dbt Cloud streamlines analytics engineering workflows This guide outlines what you need to do in order to switch from dbt Core to dbt Cloud and highlights the necessary technical changes: . - [Account set up](/guides/core-to-cloud-1?step=3#account-set-up): Learn how to create a dbt Cloud account, invite team members, and configure it for your team. -- [Environment variables](m/guides/core-to-cloud-1?step=4#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. +- [Environment variables](/guides/core-to-cloud-1?step=4#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. - [Data platform set up](/guides/core-to-cloud-1?step=5#data-platform-set-up): Find out about connecting your data platform to dbt Cloud. - [Git set up](/guides/core-to-cloud-1?step=6#git-set-up): Learn to link your dbt project's Git repository with dbt Cloud. - [Developer set up:](/guides/core-to-cloud-1?step=7#developer-set-up) Understand the set up needed for developing in dbt Cloud. From b8eb6511d86c554accb3048d7f00340771acdb17 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 12 Mar 2024 16:03:30 +0000 Subject: [PATCH 23/89] matt's feedback --- website/docs/guides/core-to-cloud-1.md | 32 +++++++++++++------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 22af433b7f4..01b3e654b2b 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -23,12 +23,12 @@ Switching from dbt Core to dbt Cloud streamlines analytics engineering workflows This guide outlines what you need to do in order to switch from dbt Core to dbt Cloud and highlights the necessary technical changes: . -- [Account set up](/guides/core-to-cloud-1?step=3#account-set-up): Learn how to create a dbt Cloud account, invite team members, and configure it for your team. +- [Account setup](/guides/core-to-cloud-1?step=3#account-set-up): Learn how to create a dbt Cloud account, invite team members, and configure it for your team. - [Environment variables](/guides/core-to-cloud-1?step=4#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. -- [Data platform set up](/guides/core-to-cloud-1?step=5#data-platform-set-up): Find out about connecting your data platform to dbt Cloud. -- [Git set up](/guides/core-to-cloud-1?step=6#git-set-up): Learn to link your dbt project's Git repository with dbt Cloud. -- [Developer set up:](/guides/core-to-cloud-1?step=7#developer-set-up) Understand the set up needed for developing in dbt Cloud. -- [Orchestration set up](/guides/core-to-cloud-1?step=8#orchestration-set-up): Learn how to prepare your dbt Cloud environment and jobs for orchestration. +- [Data platform setup](/guides/core-to-cloud-1?step=5#data-platform-set-up): Find out about connecting your data platform to dbt Cloud. +- [Git setup](/guides/core-to-cloud-1?step=6#git-set-up): Learn to link your dbt project's Git repository with dbt Cloud. +- [Developer setup:](/guides/core-to-cloud-1?step=7#developer-set-up) Understand the setup needed for developing in dbt Cloud. +- [Orchestration setup](/guides/core-to-cloud-1?step=8#orchestration-set-up): Learn how to prepare your dbt Cloud environment and jobs for orchestration. - [Models configuration](/guides/core-to-cloud-1?step=9#models-configuration): Get insights on validating and running your models in dbt Cloud, using either the dbt Cloud IDE or dbt Cloud CLI. - [What's next?](/guides/core-to-cloud-1?step=10#whats-next): Summarizes key takeaways and introduces what to expect in the following guides. @@ -45,7 +45,7 @@ This guide outlines what you need to do in order to switch from dbt Core to dbt - Book [expert-led demos](https://www.getdbt.com/resources/dbt-cloud-demos-with-experts) and insights - Work with the [dbt Labs’ Professional Services](https://www.getdbt.com/dbt-labs/services) team to support your data organization and migration. -## Account set up +## Account setup *Time to complete: Approximately 30 mins to 1 hour* 1. [Create your dbt Cloud account](https://www.getdbt.com/signup). @@ -81,7 +81,7 @@ This guide outlines what you need to do in order to switch from dbt Core to dbt - To set these variables at the job level, navigate to **Deploy** > **Jobs** > **Select your job** > **Settings** > **Advanced settings**. - To set these variables at the personal override level, navigate to **Profile Settings** > **Credentials** > **Select your project** > **Environment variables**. -## Data platform set up +## Data platform setup _*Time to complete: Approximately 10-30 mins_ This section explains the considerations and methods to connect your data platform to dbt Cloud. @@ -102,7 +102,7 @@ This section explains the considerations and methods to connect your data platfo ### Advanced configuration 3. Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enables secure authentication using your data platform’s SSO. -## Git set up +## Git setup _*Time to complete: Approximately 10 to 30 mins_ Your existing dbt project source code should live in a Git repository. In this step, you should connect your existing dbt project source code from Git to dbt Cloud. @@ -118,13 +118,13 @@ Your existing dbt project source code should live in a Git repository. In this s Set up groups for dbt project access with those configured for repository access to streamline permissions. -## Developer set up +## Developer setup _*Time to complete: Approximately 30 mins to 1 hour_ This section highlights the development configurations you’ll need for your dbt Cloud project. The following categories are covered in this section: - [dbt Cloud environments](/guides/core-to-cloud-1?step=7#dbt-cloud-environments) -- [Initial set up steps](/guides/core-to-cloud-1?step=7#initial-set-up-steps) +- [Initial setup steps](/guides/core-to-cloud-1?step=7#initial-set-up-steps) - [Advanced configuration](/guides/core-to-cloud-1?step=7#initial-set-up-steps) - [dbt Cloud commands](/guides/core-to-cloud-1?step=7#dbt-cloud-commands) @@ -136,7 +136,7 @@ The main difference between `target` in dbt Core and a dbt Cloud environment is This difference streamlines the process of switching between development, staging, and production contexts, removing the need to manually edit a **`profiles.yml`** file. dbt Cloud environments also integrate with additional features such as job scheduling, version control, and more, making it easier to manage the full lifecycle of your dbt projects within a single platform You can [set up](/reference/dbt-jinja-functions/target) or [customize](/docs/build/custom-target-names) target names in dbt Cloud. -### Initial set up steps +### Initial setup steps 1. **Set up development environment** — Set up your [development](/docs/dbt-cloud-environments#create-a-development-environment) environment and [development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide). You’ll need this to access your dbt project and start developing. 2. **dbt Core version** — In your dbt Cloud environment and credentials, use the same dbt Core version you use locally. You can run `dbt --version` in the command line to find out which version of dbt Core you’re using. @@ -153,20 +153,20 @@ This difference streamlines the process of switching between development, stagin ### dbt Cloud commands 7. Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. -## Orchestration set up +## Orchestration setup _*Time to complete: Approximately 30 mins to 1 hour_ This section outlines the considerations and methods to set up your dbt Cloud environments and jobs for orchestration. The following categories are covered in this section: - [dbt Cloud environments](/guides/core-to-cloud-1?step=8#dbt-cloud-environments-1) -- [Initial set up steps](/guides/core-to-cloud-1?step=8#initial-set-up-steps-1) +- [Initial setup steps](/guides/core-to-cloud-1?step=8#initial-set-up-steps-1) - [Additional configurations](/guides/core-to-cloud-1?step=8#additional-configurations) -- [CI/CD set up](/guides/core-to-cloud-1?step=8#cicd-set-up) +- [CI/CD setup](/guides/core-to-cloud-1?step=8#cicd-set-up) ### dbt Cloud environments To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one environment as the production environment. This would be the [deployment](/docs/deploy/deploy-environments) environment. You can set up multiple environments for different stages of your deployment pipeline, such as development, staging/QA, and production. -### Initial set up steps +### Initial setup steps 1. **dbt Core version** — In your environment settings, configure dbt Cloud with the same dbt Core version. - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. @@ -184,7 +184,7 @@ To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one envir 7. **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. -### CI/CD set up +### CI/CD setup Building a custom solution to efficiently check code upon pull requests is complicated. With dbt Cloud, you can enable [continuous integration / continuous deployment (CI/CD)](/docs/deploy/continuous-integration) and configure dbt Cloud to run your dbt projects in a temporary schema when new commits are pushed to open pull requests. From b8edb7c897cb2cf441f049b66d98d913c510174a Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 12 Mar 2024 16:32:21 +0000 Subject: [PATCH 24/89] update to 'move' --- website/docs/guides/core-to-cloud-1.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 01b3e654b2b..a8fdf6b771c 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -1,8 +1,8 @@ --- -title: Switch from dbt Core to dbt Cloud - Getting started +title: Move from dbt Core to dbt Cloud - Getting started id: core-to-cloud-1 -description: "Learn how to Switch from dbt Core to dbt Cloud and what you need to get started." -hoverSnippet: "Use this guide to learn how to switch from dbt Core to dbt Cloud." +description: "Learn how to move from dbt Core to dbt Cloud and what you need to get started." +hoverSnippet: "Use this guide to learn how to move from dbt Core to dbt Cloud." icon: 'guides' hide_table_of_contents: true tags: ['Migration','dbt Core','dbt Cloud'] @@ -11,7 +11,7 @@ recently_updated: true --- ## Introduction -Switching from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. +Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. - dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development, a [Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics, and domain ownership of data with multi-project “[dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro)” setups. @@ -21,7 +21,7 @@ Switching from dbt Core to dbt Cloud streamlines analytics engineering workflows ### What you'll learn -This guide outlines what you need to do in order to switch from dbt Core to dbt Cloud and highlights the necessary technical changes: . +This guide outlines what you need to do in order to move from dbt Core to dbt Cloud and highlights the necessary technical changes: . - [Account setup](/guides/core-to-cloud-1?step=3#account-set-up): Learn how to create a dbt Cloud account, invite team members, and configure it for your team. - [Environment variables](/guides/core-to-cloud-1?step=4#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. @@ -213,7 +213,7 @@ import ConfettiTrigger from '/src/components/confetti/index.js'; -Congrats on completing the first part of your switch to dbt Cloud 🎉! +Congrats on completing the first part of your move to dbt Cloud 🎉! You should have learned: - How to set up your dbt Cloud account @@ -221,10 +221,10 @@ You should have learned: - Configure your development, orchestration, and CI/CD environments - You’ve also set up your models and are ready to run your first job in dbt Cloud. -For next steps, we'll soon share other guides on how to manage your switch and trips. Stay tuned! +For next steps, we'll soon share other guides on how to manage your move and tips/faqs. Stay tuned! From 773b82547e01a2e3afa7a1e59621228b862b214a Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 12 Mar 2024 16:36:20 +0000 Subject: [PATCH 25/89] Update website/docs/docs/core/about-core-setup.md --- website/docs/docs/core/about-core-setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/core/about-core-setup.md b/website/docs/docs/core/about-core-setup.md index b4caf66ef1d..23463832179 100644 --- a/website/docs/docs/core/about-core-setup.md +++ b/website/docs/docs/core/about-core-setup.md @@ -19,4 +19,4 @@ You can also learn about [developing dbt projects in dbt Cloud](/docs/cloud/abou If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](https://docs.getdbt.com/guides). ## Related docs -- [Switch from dbt Core to dbt Cloud](/guides/core-to-cloud-1?step=1) +- [Move from dbt Core to dbt Cloud](/guides/core-to-cloud-1?step=1) From 464494a8a91638626d9453d4d85d0840c19ea5b9 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 12 Mar 2024 16:36:34 +0000 Subject: [PATCH 26/89] Update website/docs/docs/core/installation-overview.md --- website/docs/docs/core/installation-overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/core/installation-overview.md b/website/docs/docs/core/installation-overview.md index 726950a8406..2e112066048 100644 --- a/website/docs/docs/core/installation-overview.md +++ b/website/docs/docs/core/installation-overview.md @@ -52,4 +52,4 @@ Most command-line tools, including dbt, have a `--help` flag that you can use to ::: ## Related docs -- [Switch from dbt Core to dbt Cloud](/guides/core-to-cloud-1?step=1) +- [Move from dbt Core to dbt Cloud](/guides/core-to-cloud-1?step=1) From 8d22935124ec71dabe976b199769c6c0d01f3d81 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 12 Mar 2024 16:37:03 +0000 Subject: [PATCH 27/89] Update website/docs/guides/manual-install-qs.md --- website/docs/guides/manual-install-qs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/manual-install-qs.md b/website/docs/guides/manual-install-qs.md index 2cc12d1db92..24cf2fafc20 100644 --- a/website/docs/guides/manual-install-qs.md +++ b/website/docs/guides/manual-install-qs.md @@ -26,7 +26,7 @@ If you'd rather edit files and run projects using the web-based Integrated Devel After setting up BigQuery to work with dbt, you are ready to create a starter project with example models, before building your own models. ### Related docs -- [Switch from dbt Core to dbt Cloud](/guides/core-to-cloud-1?step=1) +- [Move from dbt Core to dbt Cloud](/guides/core-to-cloud-1?step=1) ## Create a repository From 22821b148fe8df1a058430d5f905d1830de6578c Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 13 Mar 2024 10:28:29 +0000 Subject: [PATCH 28/89] fix links --- website/docs/guides/core-to-cloud-1.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index a8fdf6b771c..fa3a1806e21 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -21,16 +21,16 @@ Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by ### What you'll learn -This guide outlines what you need to do in order to move from dbt Core to dbt Cloud and highlights the necessary technical changes: . - -- [Account setup](/guides/core-to-cloud-1?step=3#account-set-up): Learn how to create a dbt Cloud account, invite team members, and configure it for your team. -- [Environment variables](/guides/core-to-cloud-1?step=4#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. -- [Data platform setup](/guides/core-to-cloud-1?step=5#data-platform-set-up): Find out about connecting your data platform to dbt Cloud. -- [Git setup](/guides/core-to-cloud-1?step=6#git-set-up): Learn to link your dbt project's Git repository with dbt Cloud. -- [Developer setup:](/guides/core-to-cloud-1?step=7#developer-set-up) Understand the setup needed for developing in dbt Cloud. -- [Orchestration setup](/guides/core-to-cloud-1?step=8#orchestration-set-up): Learn how to prepare your dbt Cloud environment and jobs for orchestration. -- [Models configuration](/guides/core-to-cloud-1?step=9#models-configuration): Get insights on validating and running your models in dbt Cloud, using either the dbt Cloud IDE or dbt Cloud CLI. -- [What's next?](/guides/core-to-cloud-1?step=10#whats-next): Summarizes key takeaways and introduces what to expect in the following guides. +This guide outlines what you need to do in order to move from dbt Core to dbt Cloud and highlights the necessary technical changes: + +- [Account setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=3#account-setup): Learn how to create a dbt Cloud account, invite team members, and configure it for your team. +- [Environment variables](https://docs.getdbt.com/guides/core-to-cloud-1?step=4#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. +- [Data platform setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=5#data-platform-setup): Find out about connecting your data platform to dbt Cloud. +- [Git setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=6#git-setup): Learn to link your dbt project's Git repository with dbt Cloud. +- [Developer setup:](https://docs.getdbt.com/guides/core-to-cloud-1?step=7#developer-setup) Understand the setup needed for developing in dbt Cloud. +- [Orchestration setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=8#orchestration-setup): Learn how to prepare your dbt Cloud environment and jobs for orchestration. +- [Models configuration](https://docs.getdbt.com/guides/core-to-cloud-1?step=9#models-configuration): Get insights on validating and running your models in dbt Cloud, using either the dbt Cloud IDE or dbt Cloud CLI. +- [What's next?](https://docs.getdbt.com/guides/core-to-cloud-1?step=10#whats-next): Summarizes key takeaways and introduces what to expect in the following guides. ## Prerequisites From c37c7c24413f8ea2cd9103630bb7d4fa41acfc66 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 13 Mar 2024 14:02:18 +0000 Subject: [PATCH 29/89] amy's feedback --- website/docs/guides/core-to-cloud-1.md | 54 +++++++++++++++----------- 1 file changed, 32 insertions(+), 22 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index fa3a1806e21..d36bf832d41 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -59,28 +59,6 @@ This guide outlines what you need to do in order to move from dbt Core to dbt Cl 5. In **Account settings**, enable [Git repo caching](/docs/deploy/deploy-environments#git-repository-caching) to ensure job reliability, protection from third-party outages, and performance. -## Environment variables - -1. In dbt Cloud, [environment variables](/docs/build/environment-variables) are managed with a clear order of precedence, allowing users to define values at four levels (highest to lowest order of precedence): - - The job level (job override) or in the IDE for an individual developer (personal override). _(Highest precedence)_ - - The environment level, which can be overridden by the job level or personal override. - - A project-wide default value, which can be overridden by the environment level, job level, or personal override. - - The optional default argument supplied to the `env_var` Jinja function in code. _(Lowest precedence)_ - - - -2. dbt Cloud secures environment variables, offering additional measures for sensitive values, such as prefixing keys with `DBT_ENV_SECRET_` to obscure them in logs and the user interface (UI). - - - -3. Ensure that all [environment variables](/docs/build/environment-variables) start with `DBT_` or `DBT_ENV_SECRET_`. If your dbt Core environment variables don’t follow this naming convention, perform a “find and replace” in your project to make sure all references to these environment variables contain the proper naming conventions. - -### How to set environment variables in dbt Cloud - -- To set these variables for an entire project or specific environments, navigate to **Deploy** > **Environments** > **Environment variables** tab. -- To set these variables at the job level, navigate to **Deploy** > **Jobs** > **Select your job** > **Settings** > **Advanced settings**. -- To set these variables at the personal override level, navigate to **Profile Settings** > **Credentials** > **Select your project** > **Environment variables**. - ## Data platform setup _*Time to complete: Approximately 10-30 mins_ @@ -153,6 +131,38 @@ This difference streamlines the process of switching between development, stagin ### dbt Cloud commands 7. Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. +## Environment variables + +This section will help you understand how to manage environment variables in dbt Cloud and how to set them up for your project. The following categories are covered in this section: +- [Environment variables in dbt Cloud](/guides/core-to-cloud-1?step=7#dbt-cloud-environment-variables) +- [dbt Cloud environment variables order of precedence](/guides/core-to-cloud-1?step=7#dbt-cloud-environment-variables-order-of-precedence) +- [Set environment variables in dbt Cloud](/guides/core-to-cloud-1?step=7#set-environment-variables-in-dbt-cloud) + +In dbt Core, environment variables, or the [`env_var` function](/reference/dbt-jinja-functions/env_var), are defined in the `profiles.yml` file. +In dbt Cloud, you can set [environment variables](/docs/build/environment-variables) in the dbt Cloud user interface (UI). Read [Set up environment variables](#set-environment-variables-in-dbt-cloud) for more info. + +### dbt Cloud environment variables + - dbt Cloud environment variables must be prefixed with `DBT_` (including `DBT_ENV_CUSTOM_ENV_` or `DBT_ENV_SECRET_`). + - If your dbt Core environment variables don’t follow this naming convention, perform a “find and replace” in your project to make sure all references to these environment variables contain the proper naming conventions. +- dbt Cloud secures environment variables, offering additional measures for sensitive values, such as prefixing keys with `DBT_ENV_SECRET_` to obscure them in logs and the UI. + + + +### dbt Cloud environment variables order of precedence +Environment variables in dbt Cloud are managed with a clear order of precedence, allowing users to define values at four levels (highest to lowest order of precedence): + - The job level (job override) or in the IDE for an individual developer (personal override). _(Highest precedence)_ + - The environment level, which can be overridden by the job level or personal override. + - A project-wide default value, which can be overridden by the environment level, job level, or personal override. + - The optional default argument supplied to the `env_var` Jinja function in code. _(Lowest precedence)_ + + + +### How to set environment variables in dbt Cloud + +- To set these variables for an entire project or specific environments, navigate to **Deploy** > **Environments** > **Environment variables** tab. +- To set these variables at the job level, navigate to **Deploy** > **Jobs** > **Select your job** > **Settings** > **Advanced settings**. +- To set these variables at the personal override level, navigate to **Profile Settings** > **Credentials** > **Select your project** > **Environment variables**. + ## Orchestration setup _*Time to complete: Approximately 30 mins to 1 hour_ From 361df55b529115e0c1344e15a3221f922084f92c Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 13 Mar 2024 14:03:45 +0000 Subject: [PATCH 30/89] tweak --- website/docs/guides/core-to-cloud-1.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index d36bf832d41..2de8b1d3e23 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -134,14 +134,14 @@ This difference streamlines the process of switching between development, stagin ## Environment variables This section will help you understand how to manage environment variables in dbt Cloud and how to set them up for your project. The following categories are covered in this section: -- [Environment variables in dbt Cloud](/guides/core-to-cloud-1?step=7#dbt-cloud-environment-variables) +- [Environment variables in dbt Cloud](/guides/core-to-cloud-1?step=7#environment-variables-in-dbt-cloud) - [dbt Cloud environment variables order of precedence](/guides/core-to-cloud-1?step=7#dbt-cloud-environment-variables-order-of-precedence) - [Set environment variables in dbt Cloud](/guides/core-to-cloud-1?step=7#set-environment-variables-in-dbt-cloud) In dbt Core, environment variables, or the [`env_var` function](/reference/dbt-jinja-functions/env_var), are defined in the `profiles.yml` file. In dbt Cloud, you can set [environment variables](/docs/build/environment-variables) in the dbt Cloud user interface (UI). Read [Set up environment variables](#set-environment-variables-in-dbt-cloud) for more info. -### dbt Cloud environment variables +### Environment variables in dbt Cloud - dbt Cloud environment variables must be prefixed with `DBT_` (including `DBT_ENV_CUSTOM_ENV_` or `DBT_ENV_SECRET_`). - If your dbt Core environment variables don’t follow this naming convention, perform a “find and replace” in your project to make sure all references to these environment variables contain the proper naming conventions. - dbt Cloud secures environment variables, offering additional measures for sensitive values, such as prefixing keys with `DBT_ENV_SECRET_` to obscure them in logs and the UI. @@ -157,7 +157,7 @@ Environment variables in dbt Cloud are managed with a clear order of precedence, -### How to set environment variables in dbt Cloud +### Set environment variables in dbt Cloud - To set these variables for an entire project or specific environments, navigate to **Deploy** > **Environments** > **Environment variables** tab. - To set these variables at the job level, navigate to **Deploy** > **Jobs** > **Select your job** > **Settings** > **Advanced settings**. From 9c1441172012498c0d41b9fd0b3a764c21f251c0 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 13 Mar 2024 14:08:29 +0000 Subject: [PATCH 31/89] fold in Amy's feedback --- website/docs/guides/core-to-cloud-1.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 2de8b1d3e23..009a359f7d5 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -34,7 +34,6 @@ This guide outlines what you need to do in order to move from dbt Core to dbt Cl ## Prerequisites -- You have dbt Core installed. - You have an existing dbt Core project connected to a data platform and Git repository. - You have a dbt Cloud account. **[Don't have one? Start your free trial today](https://www.getdbt.com/signup)**! @@ -126,10 +125,8 @@ This difference streamlines the process of switching between development, stagin ### Advanced configuration 5. **Custom target names** — If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. -6. **API access** — Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. - ### dbt Cloud commands -7. Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. +6. Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. ## Environment variables @@ -192,7 +189,9 @@ To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one envir 6. **Monitoring tools** — Use [monitoring tools](/docs/deploy/monitor-jobs) like run history, job retries, job chaining, dashboard status tiles, and more for a seamless orchestration experience. -7. **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. +7. **API access** — Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. + +8. **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. ### CI/CD setup From 670faa2bd56b290d466f3016270a7738e2d94bbc Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 13 Mar 2024 14:25:13 +0000 Subject: [PATCH 32/89] tweaks --- website/docs/guides/core-to-cloud-1.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 009a359f7d5..b0378138bd0 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -191,7 +191,7 @@ To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one envir 7. **API access** — Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. -8. **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. +8. **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. ### CI/CD setup @@ -214,7 +214,7 @@ You’ll want to make sure you set up your [development environment and credenti - Make sure your project compiles correctly. - Run a few models in the dbt Cloud IDE or dbt Cloud CLI to ensure you’re experiencing accurate results in development. -2. Once your first job has successfully run in your production environment, use [dbt Explorer](/docs/collaborate/explore-projects) to view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their [lineage](/terms/data-lineage) to gain a better understanding of its latest production state. +2. Once your first job has successfully run in your production environment, use [dbt Explorer](/docs/collaborate/explore-projects) to view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their [lineage](/terms/data-lineage) to gain a better understanding of its latest production state. ## What’s next? From 03df5c2065a988246e29018c6ef0ba2f284bec67 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 13 Mar 2024 17:14:45 +0000 Subject: [PATCH 33/89] add time --- website/docs/guides/core-to-cloud-1.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index b0378138bd0..cb1bc1cc5a1 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -4,6 +4,7 @@ id: core-to-cloud-1 description: "Learn how to move from dbt Core to dbt Cloud and what you need to get started." hoverSnippet: "Use this guide to learn how to move from dbt Core to dbt Cloud." icon: 'guides' +time_to_complete: 'Total estimated time: 3-4 hours' hide_table_of_contents: true tags: ['Migration','dbt Core','dbt Cloud'] level: 'Intermediate' @@ -45,7 +46,6 @@ This guide outlines what you need to do in order to move from dbt Core to dbt Cl - Work with the [dbt Labs’ Professional Services](https://www.getdbt.com/dbt-labs/services) team to support your data organization and migration. ## Account setup -*Time to complete: Approximately 30 mins to 1 hour* 1. [Create your dbt Cloud account](https://www.getdbt.com/signup). @@ -59,7 +59,6 @@ This guide outlines what you need to do in order to move from dbt Core to dbt Cl 5. In **Account settings**, enable [Git repo caching](/docs/deploy/deploy-environments#git-repository-caching) to ensure job reliability, protection from third-party outages, and performance. ## Data platform setup -_*Time to complete: Approximately 10-30 mins_ This section explains the considerations and methods to connect your data platform to dbt Cloud. @@ -96,7 +95,6 @@ Your existing dbt project source code should live in a Git repository. In this s Set up groups for dbt project access with those configured for repository access to streamline permissions. ## Developer setup -_*Time to complete: Approximately 30 mins to 1 hour_ This section highlights the development configurations you’ll need for your dbt Cloud project. The following categories are covered in this section: @@ -161,7 +159,6 @@ Environment variables in dbt Cloud are managed with a clear order of precedence, - To set these variables at the personal override level, navigate to **Profile Settings** > **Credentials** > **Select your project** > **Environment variables**. ## Orchestration setup -_*Time to complete: Approximately 30 mins to 1 hour_ This section outlines the considerations and methods to set up your dbt Cloud environments and jobs for orchestration. The following categories are covered in this section: @@ -204,7 +201,6 @@ This build-on-PR functionality is a great way to catch bugs before deploying to 3. Run your jobs in a production environment to fully implement CI/CD. Future pull requests will also leverage the last production runs to compare against. ## Models configuration -_*Time to complete: Approximately 30 mins_ In this section, you’ll be able to validate whether your models run or compile correctly in your development tool of choice: [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). From dc2cac39c1c8a51eb058be86af965c1c5e35eda7 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 13 Mar 2024 17:33:46 +0000 Subject: [PATCH 34/89] remove --- website/docs/guides/core-to-cloud-1.md | 1 - 1 file changed, 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index cb1bc1cc5a1..0a9a1173c61 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -79,7 +79,6 @@ This section explains the considerations and methods to connect your data platfo 3. Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enables secure authentication using your data platform’s SSO. ## Git setup -_*Time to complete: Approximately 10 to 30 mins_ Your existing dbt project source code should live in a Git repository. In this step, you should connect your existing dbt project source code from Git to dbt Cloud. From bde29474a1e37291936e4534119781fdced0c285 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 13 Mar 2024 17:38:21 +0000 Subject: [PATCH 35/89] turn into bullets --- website/docs/guides/core-to-cloud-1.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 0a9a1173c61..fa6cd9eedcc 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -14,7 +14,10 @@ recently_updated: true Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. -- dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development, a [Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics, and domain ownership of data with multi-project “[dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro)” setups. +- dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports: + - The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development + - A [Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics + - Domain ownership of data with multi-project “[dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro)” setups. Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features). From a733961bc6057c67d06c36a99271b3c07c9c1131 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 17:53:10 +0000 Subject: [PATCH 36/89] Update website/docs/docs/core/about-core-setup.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/docs/core/about-core-setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/core/about-core-setup.md b/website/docs/docs/core/about-core-setup.md index 23463832179..16bfe18fc37 100644 --- a/website/docs/docs/core/about-core-setup.md +++ b/website/docs/docs/core/about-core-setup.md @@ -14,7 +14,7 @@ dbt Core is an [open-source](https://github.com/dbt-labs/dbt-core) tool that ena - [Connecting to a data platform](/docs/core/connect-data-platform/profiles.yml) - [How to run your dbt projects](/docs/running-a-dbt-project/run-your-dbt-projects) -You can also learn about [developing dbt projects in dbt Cloud](/docs/cloud/about-develop-dbt) — dbt Cloud provides a command line interface with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) that let you run dbt commands. The dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). +If you're interested in using a command line interface to [develop dbt projects in dbt Cloud](/docs/cloud/about-develop-dbt), the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) lets you run dbt commands locally. The dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](https://docs.getdbt.com/guides). From 5951481f5f6b090ac97194e441bbde51424219fb Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 17:54:15 +0000 Subject: [PATCH 37/89] Update core-to-cloud-1.md --- website/docs/guides/core-to-cloud-1.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index fa6cd9eedcc..40950ce8c66 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -16,8 +16,9 @@ Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by - dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports: - The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development - - A [Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics - - Domain ownership of data with multi-project “[dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro)” setups. + - [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics + - Domain ownership of data with multi-project “[dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro)” setups + - [dbt Explorer](/docs/collaborate/explore-projects) for easier data discovery and understanding Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features). From 8e8eb76fc1b38ff782228fcaf377720a21f38fa9 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 17:54:46 +0000 Subject: [PATCH 38/89] Update core-to-cloud-1.md --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 40950ce8c66..2b81af0c9cb 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -17,7 +17,7 @@ Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by - dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports: - The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development - [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics - - Domain ownership of data with multi-project “[dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro)” setups + - Domain ownership of data with multi-project [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) setups - [dbt Explorer](/docs/collaborate/explore-projects) for easier data discovery and understanding Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features). From e6898d586afd226649a186e99f0d691ac623ce29 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 17:58:03 +0000 Subject: [PATCH 39/89] Update website/docs/guides/manual-install-qs.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/manual-install-qs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/manual-install-qs.md b/website/docs/guides/manual-install-qs.md index 24cf2fafc20..b433649299f 100644 --- a/website/docs/guides/manual-install-qs.md +++ b/website/docs/guides/manual-install-qs.md @@ -12,7 +12,7 @@ hide_table_of_contents: true When you use dbt Core to work with dbt, you will be editing files locally using a code editor, and running projects using a command line interface (CLI). -If you'd rather edit files and run projects using the web-based Integrated Development Environment (IDE), you should refer to the [dbt Cloud quickstarts](/guides). You can also develop and run dbt commands using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) — a dbt Cloud powered command line. +If you want to edit files and run projects using the web-based dbt Integrated Development Environment (IDE), refer to the [dbt Cloud quickstarts](/guides). You can also develop and run dbt commands using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) — a dbt Cloud powered command line. ### Prerequisites From a782b008f6cf175279e106ec148cb197f5c4e424 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:00:24 +0000 Subject: [PATCH 40/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 2b81af0c9cb..a2ec96dae7a 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -1,5 +1,5 @@ --- -title: Move from dbt Core to dbt Cloud - Getting started +title: Get started with the move from dbt Core to dbt Cloud id: core-to-cloud-1 description: "Learn how to move from dbt Core to dbt Cloud and what you need to get started." hoverSnippet: "Use this guide to learn how to move from dbt Core to dbt Cloud." From 81df4cae22aa7e9e7c563654ff884d58ef6c14e3 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:00:33 +0000 Subject: [PATCH 41/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index a2ec96dae7a..0fa3022da7e 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -2,7 +2,7 @@ title: Get started with the move from dbt Core to dbt Cloud id: core-to-cloud-1 description: "Learn how to move from dbt Core to dbt Cloud and what you need to get started." -hoverSnippet: "Use this guide to learn how to move from dbt Core to dbt Cloud." +hoverSnippet: "Learn how to move from dbt Core to dbt Cloud." icon: 'guides' time_to_complete: 'Total estimated time: 3-4 hours' hide_table_of_contents: true From 9e21f5367d28e483651f3092b6a0e2c98bfd95ab Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:02:04 +0000 Subject: [PATCH 42/89] Update website/docs/guides/core-to-cloud-1.md --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 0fa3022da7e..2407fa976a5 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -1,5 +1,5 @@ --- -title: Get started with the move from dbt Core to dbt Cloud +title: Move from dbt Core to dbt Cloud - Get started id: core-to-cloud-1 description: "Learn how to move from dbt Core to dbt Cloud and what you need to get started." hoverSnippet: "Learn how to move from dbt Core to dbt Cloud." From 0aaeadcb2ccb66b6c16a0d3ac86d7344ef6b780a Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:08:55 +0000 Subject: [PATCH 43/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 2407fa976a5..4ee25b10528 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -14,7 +14,7 @@ recently_updated: true Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. -- dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports: +dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports: - The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development - [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics - Domain ownership of data with multi-project [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) setups From 8498bc2edab8ad40f8e019a2ce086d3d541bc9b8 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:09:04 +0000 Subject: [PATCH 44/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 4ee25b10528..667c83a9910 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -22,7 +22,7 @@ dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, dep Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features). -- dbt Core is an open-source tool that enables data teams to transform data following analytics engineering best practices using a command line interface. It must be self-hosted and maintained. +dbt Core is an open-source tool that enables data teams to transform data following analytics engineering best practices using a command line interface. It must be self-hosted and maintained. ### What you'll learn From 323c9a16b6023dee517c7a5daaa68f626fbbe68c Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:14:35 +0000 Subject: [PATCH 45/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 667c83a9910..62e55422ccf 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -26,7 +26,7 @@ dbt Core is an open-source tool that enables data teams to transform data follow ### What you'll learn -This guide outlines what you need to do in order to move from dbt Core to dbt Cloud and highlights the necessary technical changes: +This guide outlines the steps you need to take to move from dbt Core to dbt Cloud and highlights the necessary technical changes: - [Account setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=3#account-setup): Learn how to create a dbt Cloud account, invite team members, and configure it for your team. - [Environment variables](https://docs.getdbt.com/guides/core-to-cloud-1?step=4#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. From e1734dd33a66973fac5624ace6ca25b58b741010 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 13 Mar 2024 18:27:08 +0000 Subject: [PATCH 46/89] add version --- website/docs/guides/core-to-cloud-1.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 667c83a9910..76ef4879137 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -15,12 +15,12 @@ recently_updated: true Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports: - - The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development - - [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics - - Domain ownership of data with multi-project [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) setups - - [dbt Explorer](/docs/collaborate/explore-projects) for easier data discovery and understanding - - Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features). +- The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development +- [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics +- Domain ownership of data with multi-project [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) setups +- [dbt Explorer](/docs/collaborate/explore-projects) for easier data discovery and understanding + +Learn more about [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features). dbt Core is an open-source tool that enables data teams to transform data following analytics engineering best practices using a command line interface. It must be self-hosted and maintained. @@ -40,6 +40,7 @@ This guide outlines what you need to do in order to move from dbt Core to dbt Cl ## Prerequisites - You have an existing dbt Core project connected to a data platform and Git repository. +- You are dbt version 1.0 or later. - You have a dbt Cloud account. **[Don't have one? Start your free trial today](https://www.getdbt.com/signup)**! ### Related docs From fcd3e82f93d768210eac975884b3250089e8bc07 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:31:33 +0000 Subject: [PATCH 47/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 62e55422ccf..af5a890aece 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -39,7 +39,7 @@ This guide outlines the steps you need to take to move from dbt Core to dbt Clou ## Prerequisites -- You have an existing dbt Core project connected to a data platform and Git repository. +- You have an existing dbt Core project connected to a Git repository and data platform supported in [dbt Cloud](/docs/cloud/connect-data-platform/about-connections). - You have a dbt Cloud account. **[Don't have one? Start your free trial today](https://www.getdbt.com/signup)**! ### Related docs From 0e661652116f2f2d05f389245c531662b9593e68 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:31:56 +0000 Subject: [PATCH 48/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index af5a890aece..ada8ef3de05 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -58,7 +58,7 @@ This guide outlines the steps you need to take to move from dbt Core to dbt Clou 3. Configure [Single Sign-On (SSO)](/docs/cloud/manage-access/sso-overview) or [Role-based access control (RBAC)](/docs/cloud/manage-access/about-user-access#role-based-access-control) for easy and secure access. - This removes the need to save passwords and secret environment variables locally. -4. In **Account settings**, switch on [partial parsing](/docs/deploy/deploy-environments#partial-parsing) to only reparse changed files, saving time. +4. In **Account settings**, enable [partial parsing](/docs/deploy/deploy-environments#partial-parsing) to only reparse changed files, saving time. 5. In **Account settings**, enable [Git repo caching](/docs/deploy/deploy-environments#git-repository-caching) to ensure job reliability, protection from third-party outages, and performance. From d2a1c6c4e83c691f0aafc59adcbae743f9bbc3ac Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:32:06 +0000 Subject: [PATCH 49/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index ada8ef3de05..bfb8a70665b 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -64,7 +64,7 @@ This guide outlines the steps you need to take to move from dbt Core to dbt Clou ## Data platform setup -This section explains the considerations and methods to connect your data platform to dbt Cloud. +This section outlines the considerations and methods to connect your data platform(s) to dbt Cloud. 1. Set up your [data platform connections](/docs/cloud/connect-data-platform/about-connections) and [environment variables](/guides/core-to-cloud-1?step=4) in dbt Cloud. dbt Cloud can connect with a variety of data platform providers including: - [AlloyDB](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) From 2ec500a58c99ff087fdc28cc8ea0a9ea060bee62 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:35:34 +0000 Subject: [PATCH 50/89] Update website/docs/guides/core-to-cloud-1.md --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index bfb8a70665b..e00411782ca 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -12,7 +12,7 @@ recently_updated: true --- ## Introduction -Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed service. +Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed software service. dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports: - The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development From e23fd0ecf69d6760d242bd999c251ff880474499 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:36:29 +0000 Subject: [PATCH 51/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index e00411782ca..c48fe7e59cc 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -66,7 +66,7 @@ This guide outlines the steps you need to take to move from dbt Core to dbt Clou This section outlines the considerations and methods to connect your data platform(s) to dbt Cloud. -1. Set up your [data platform connections](/docs/cloud/connect-data-platform/about-connections) and [environment variables](/guides/core-to-cloud-1?step=4) in dbt Cloud. dbt Cloud can connect with a variety of data platform providers including: +1. Set up your [data platform connections](/docs/cloud/connect-data-platform/about-connections) and [environment variables](/docs/build/environment-variables) in dbt Cloud. dbt Cloud can connect with a variety of data platform providers including: - [AlloyDB](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) - [Amazon Redshift](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) - [Apache Spark](/docs/cloud/connect-data-platform/connect-apache-spark) From acc9a8bb3c84023b396f05ad1dc9f9f236025683 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:40:10 +0000 Subject: [PATCH 52/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index c48fe7e59cc..c8e47f80b76 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -84,7 +84,7 @@ This section outlines the considerations and methods to connect your data platfo ## Git setup -Your existing dbt project source code should live in a Git repository. In this step, you should connect your existing dbt project source code from Git to dbt Cloud. +Your existing dbt project source code should live in a Git repository. In this section, you will connect your existing dbt project source code from Git to dbt Cloud. 1. Ensure your dbt project is in a Git repository From 45e3dc3ca965a2e5d2331712f4aca24d321bbdd6 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 18:40:27 +0000 Subject: [PATCH 53/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index c8e47f80b76..6e8fc73bdf9 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -86,7 +86,7 @@ This section outlines the considerations and methods to connect your data platfo Your existing dbt project source code should live in a Git repository. In this section, you will connect your existing dbt project source code from Git to dbt Cloud. -1. Ensure your dbt project is in a Git repository +1. Ensure your dbt project is in a Git repository. 2. Once you’ve set up a dbt Cloud account, you can [connect and configure Git](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud to connect your Git repository: - Connect with one of the [native integrations](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud (such as GitHub, GitLab, and Azure DevOps). From 4fa766d036d2996bbda1d0ee51853ed16bb02b2d Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:13:26 +0000 Subject: [PATCH 54/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 73cacc9b3b9..b1950fb87fc 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -182,7 +182,7 @@ To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one envir - Note that alongside [dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. ### Additional configurations -3. **Custom target names** — You should set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. +3. **Custom target names** — Set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. 4. **dbt commands** — Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. From 63a1afc7fe441227d8003fe535b041a413ffb612 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:13:46 +0000 Subject: [PATCH 55/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index b1950fb87fc..55e97a3de69 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -178,7 +178,7 @@ To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one envir 1. **dbt Core version** — In your environment settings, configure dbt Cloud with the same dbt Core version. - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. -2. **[Configure your jobs](/docs/deploy/deploy-jobs)** — You can create jobs for automated or event-driven dbt jobs. You can use cron execution, manual, pull requests, or API triggers. +2. **[Configure your jobs](/docs/deploy/deploy-jobs)** — Create jobs for automated or event-driven dbt jobs. You can use cron execution, manual, pull requests, or API triggers. - Note that alongside [dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. ### Additional configurations From c46bc7d7f1e68a9c74df941fa6a95392c68f9cc2 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:14:04 +0000 Subject: [PATCH 56/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 55e97a3de69..f0cb2178efc 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -172,7 +172,7 @@ This section outlines the considerations and methods to set up your dbt Cloud en - [CI/CD setup](/guides/core-to-cloud-1?step=8#cicd-set-up) ### dbt Cloud environments -To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one environment as the production environment. This would be the [deployment](/docs/deploy/deploy-environments) environment. You can set up multiple environments for different stages of your deployment pipeline, such as development, staging/QA, and production. +To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one environment as the production environment. This is the [deployment](/docs/deploy/deploy-environments) environment. You can set up multiple environments for different stages of your deployment pipeline, such as development, staging/QA, and production. ### Initial setup steps 1. **dbt Core version** — In your environment settings, configure dbt Cloud with the same dbt Core version. From 812541a8bbcf0bda9205b85f242fa0d3a9c7bba3 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:14:29 +0000 Subject: [PATCH 57/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index f0cb2178efc..1780784ee98 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -152,7 +152,7 @@ Environment variables in dbt Cloud are managed with a clear order of precedence, - The job level (job override) or in the IDE for an individual developer (personal override). _(Highest precedence)_ - The environment level, which can be overridden by the job level or personal override. - A project-wide default value, which can be overridden by the environment level, job level, or personal override. - - The optional default argument supplied to the `env_var` Jinja function in code. _(Lowest precedence)_ + - The optional default argument supplied to the `env_var` Jinja function in the code. _Lowest precedence_ From 64f21495f848aed178127ccd216e29f99ae3e36c Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:14:57 +0000 Subject: [PATCH 58/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 1780784ee98..e6e3706898f 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -149,7 +149,7 @@ In dbt Cloud, you can set [environment variables](/docs/build/environment-variab ### dbt Cloud environment variables order of precedence Environment variables in dbt Cloud are managed with a clear order of precedence, allowing users to define values at four levels (highest to lowest order of precedence): - - The job level (job override) or in the IDE for an individual developer (personal override). _(Highest precedence)_ + - The job level (job override) or in the IDE for an individual developer (personal override). _Highest precedence_ - The environment level, which can be overridden by the job level or personal override. - A project-wide default value, which can be overridden by the environment level, job level, or personal override. - The optional default argument supplied to the `env_var` Jinja function in the code. _Lowest precedence_ From 0e524155eac31f7ffe071c6e3cd17fa5b0761ea0 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:15:42 +0000 Subject: [PATCH 59/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index e6e3706898f..37b06c22697 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -148,7 +148,7 @@ In dbt Cloud, you can set [environment variables](/docs/build/environment-variab ### dbt Cloud environment variables order of precedence -Environment variables in dbt Cloud are managed with a clear order of precedence, allowing users to define values at four levels (highest to lowest order of precedence): +Environment variables in dbt Cloud are managed with a clear [order of precedence](/docs/build/environment-variables#setting-and-overriding-environment-variables), allowing users to define values at four levels (highest to lowest order of precedence): - The job level (job override) or in the IDE for an individual developer (personal override). _Highest precedence_ - The environment level, which can be overridden by the job level or personal override. - A project-wide default value, which can be overridden by the environment level, job level, or personal override. From db4d19e3b42cafb34c37189d5d0c0c13e6c6aedd Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:16:08 +0000 Subject: [PATCH 60/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 37b06c22697..cd71bc34403 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -200,7 +200,7 @@ Building a custom solution to efficiently check code upon pull requests is compl This build-on-PR functionality is a great way to catch bugs before deploying to production, and an essential tool for data practitioners. -1. Set up an integration with a native git application (such as Azure DevOps, GitHub, GitLab) and a CI environment in dbt Cloud. +1. Set up an integration with a native Git application (such as Azure DevOps, GitHub, GitLab) and a CI environment in dbt Cloud. 2. Create [a CI/CD job](/docs/deploy/ci-jobs) to optimize workflows. 3. Run your jobs in a production environment to fully implement CI/CD. Future pull requests will also leverage the last production runs to compare against. From 54dcc8457d6b3d3041a220d2702877231737aa41 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:16:23 +0000 Subject: [PATCH 61/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index cd71bc34403..e60aa08249d 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -206,7 +206,7 @@ This build-on-PR functionality is a great way to catch bugs before deploying to ## Models configuration -In this section, you’ll be able to validate whether your models run or compile correctly in your development tool of choice: [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). +In this section, you’ll be able to validate whether your models run or compile correctly in your development tool of choice: The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). You’ll want to make sure you set up your [development environment and credentials](#developer-set-up). From 9a22428f10580427922d8e36948f3433fce5d4b2 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:17:38 +0000 Subject: [PATCH 62/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index e60aa08249d..abd9a292b64 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -210,7 +210,7 @@ In this section, you’ll be able to validate whether your models run or compile You’ll want to make sure you set up your [development environment and credentials](#developer-set-up). -1. In your [development tool](/docs/cloud/about-develop-dbt) of choice, you can review your dbt project and ensure your project is set up correctly and you’re able to run commands. This means: +1. In your [development tool](/docs/cloud/about-develop-dbt) of choice, you can review your dbt project and ensure your project is set up correctly and you’re able to run commands. This will: - Make sure your project compiles correctly. - Run a few models in the dbt Cloud IDE or dbt Cloud CLI to ensure you’re experiencing accurate results in development. From 5fe894ff29460736b8878c351bf16e4316054cb6 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:18:40 +0000 Subject: [PATCH 63/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index abd9a292b64..beca88f2267 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -222,7 +222,7 @@ import ConfettiTrigger from '/src/components/confetti/index.js'; -Congrats on completing the first part of your move to dbt Cloud 🎉! +Congratulations on completing the first part of your move to dbt Cloud 🎉! You should have learned: - How to set up your dbt Cloud account From 4c95ee66bb3c0f37f644a78792af8a0e348b1b0c Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:18:52 +0000 Subject: [PATCH 64/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index beca88f2267..a7411bb71d7 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -228,7 +228,7 @@ You should have learned: - How to set up your dbt Cloud account - Connect your data platform and Git repository - Configure your development, orchestration, and CI/CD environments -- You’ve also set up your models and are ready to run your first job in dbt Cloud. +You’ve set up your models and are ready to run your first job in dbt Cloud. For next steps, we'll soon share other guides on how to manage your move and tips/faqs. Stay tuned! From d631d9dfc3da4015deaf2021f1df803d18b9db21 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:19:12 +0000 Subject: [PATCH 65/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index a7411bb71d7..c09d1ad89b8 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -226,7 +226,7 @@ Congratulations on completing the first part of your move to dbt Cloud 🎉! You should have learned: - How to set up your dbt Cloud account -- Connect your data platform and Git repository +- How to connect your data platform and Git repository - Configure your development, orchestration, and CI/CD environments You’ve set up your models and are ready to run your first job in dbt Cloud. From 3ab72068eb81d7f97fe1e24a9818729bdf23be7e Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 19:19:24 +0000 Subject: [PATCH 66/89] Update core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index c09d1ad89b8..2b9d5805d99 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -227,7 +227,7 @@ Congratulations on completing the first part of your move to dbt Cloud 🎉! You should have learned: - How to set up your dbt Cloud account - How to connect your data platform and Git repository -- Configure your development, orchestration, and CI/CD environments +- How to configure your development, orchestration, and CI/CD environments You’ve set up your models and are ready to run your first job in dbt Cloud. For next steps, we'll soon share other guides on how to manage your move and tips/faqs. Stay tuned! From a35f71e99e4e6339d2487146a4147041f746516c Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 13 Mar 2024 20:50:17 +0000 Subject: [PATCH 67/89] Update index.js --- website/src/theme/MDXComponents/index.js | 1 + 1 file changed, 1 insertion(+) diff --git a/website/src/theme/MDXComponents/index.js b/website/src/theme/MDXComponents/index.js index aa66db52de6..4104632d28c 100644 --- a/website/src/theme/MDXComponents/index.js +++ b/website/src/theme/MDXComponents/index.js @@ -97,5 +97,6 @@ const MDXComponents = { Lifecycle: Lifecycle, detailsToggle: detailsToggle, expandable: expandable, + ConfettiTrigger: ConfettiTrigger, }; export default MDXComponents; From 4c5cc31acb34af08f2779e4ad81db5ddaf720b98 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 14 Mar 2024 11:24:16 +0000 Subject: [PATCH 68/89] Update saved-queries.md update yaml --- website/docs/docs/build/saved-queries.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/website/docs/docs/build/saved-queries.md b/website/docs/docs/build/saved-queries.md index 51efbb39da4..050aa242afc 100644 --- a/website/docs/docs/build/saved-queries.md +++ b/website/docs/docs/build/saved-queries.md @@ -16,7 +16,7 @@ Saved queries are distinct from [exports](/docs/use-dbt-semantic-layer/exports), | ----------- | ----------- | ---------------- | | **Availability** | Available on dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) plans on dbt versions 1.7 or newer.| Available in both dbt Core and dbt Cloud. | | **Purpose** | To materialize saved queries in your data platform and expose metrics and dimensions as a view or table. | To define and manage common Semantic Layer queries in YAML, which includes metrics and dimensions. | -| **Usage** | Automatically runs saved queries and materializes them within your data platform. Exports count towards [queried metrics](/docs/cloud/billing#what-counts-as-a-queried-metric) usage.

**Example**: Creating a weekly aggregated table for active user metrics, automatically updated and stored in the data platform. | Used for organizing and reusing common MetricFlow queries within dbt projects.


**Example**: Group related metrics together for better organization, and include commonly uses dimensions and filters. | For materializing query results in the data platform. | +| **Usage** | Automatically runs saved queries and materializes them within your data platform. Exports count towards [queried metrics](/docs/cloud/billing#what-counts-as-a-queried-metric) usage.

**Example**: Creating a weekly aggregated table for active user metrics, automatically updated and stored in the data platform. | Used for organizing and reusing common MetricFlow queries within dbt projects.


**Example**: Group related metrics together for better organization, and include commonly used dimensions and filters. | For materializing query results in the data platform. | | **Integration** | Must have the dbt Semantic Layer configured in your dbt project.

Tightly integrated with the [MetricFlow Server](/docs/use-dbt-semantic-layer/sl-architecture#components) and dbt Cloud's job scheduler. | Integrated into the dbt and managed alongside other dbt nodes. | | **Configuration** | Defined within the `saved_queries` configuration. Set up within the dbt Cloud environment and job scheduler settings. | Defined in YAML format within dbt project files. | @@ -34,7 +34,7 @@ saved_queries: label: Test saved query config: cache: - enabled: True | False + enabled: true # Or false if you want it disabled by default query_params: metrics: - simple_metric @@ -62,7 +62,7 @@ To define a saved query, refer to the following parameters: | `description` | String | Required | A description of the saved query. | | `label` | String | Required | The display name for your saved query. This value will be shown in downstream tools. | | `config` | String | Required | A config section for any parameters specifying the saved query. | -| `config::cache` | String | Optional | A boolean to specify if a saved query should be used to populate the cache. Accepts `True` or `False`. Defaults to `False` | +| `config::cache` | String | Optional | A boolean to specify if a saved query should be used to populate the cache. Accepts `true` or `false`. Defaults to `false` | | `query_params` | Structure | Required | Contains the query parameters. | | `query_params::metrics` | List or String | Optional | A list of the metrics to be used in the query as specified in the command line interface. | | `query_params::group_by` | List or String | Optional | A list of the Entities and Dimensions to be used in the query, which include the `Dimension` or `TimeDimension`. | @@ -72,7 +72,7 @@ To define a saved query, refer to the following parameters: | `exports::config` | List or Structure | Required | A config section for any parameters specifying the export. | | `exports::config::export_as` | String | Required | The type of export to run. Options include table or view currently and cache in the near future. | | `exports::config::schema` | String | Optional | The schema for creating the table or view. This option cannot be used for caching. | -| `exports::config::alias` | String | Optional | The table alias to use to write the table or view. This option cannot be used for caching. | +| `exports::config::alias` | String | Optional | The table alias used to write to the table or view. This option cannot be used for caching. | @@ -121,7 +121,7 @@ To define a saved query, refer to the following parameters: | `exports::config` | List or Structure | Required | A config section for any parameters specifying the export. | | `exports::config::export_as` | String | Required | The type of export to run. Options include table or view currently and cache in the near future. | | `exports::config::schema` | String | Optional | The schema for creating the table or view. This option cannot be used for caching. | -| `exports::config::alias` | String | Optional | The table alias to use to write the table or view. This option cannot be used for caching. | +| `exports::config::alias` | String | Optional | The table alias used to write to the table or view. This option cannot be used for caching. | From 5982112ad001da41b3085089d06239946a163d27 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 14 Mar 2024 11:52:20 +0000 Subject: [PATCH 69/89] Update website/docs/guides/core-to-cloud-1.md Co-authored-by: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 2b9d5805d99..9ea130b94cd 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -109,7 +109,7 @@ This section highlights the development configurations you’ll need for your db ### dbt Cloud environments -The concept of a `target` in dbt Core is the same as a [dbt Cloud environment](/docs/environments-in-dbt). +The concept of an [environment](/docs/environments-in-dbt) in dbt Cloud is the same as a `target` in dbt Core. The main difference between `target` in dbt Core and a dbt Cloud environment is that you can make these configurations through the dbt Cloud UI, as opposed to within your **`profiles.yml`** file. From f214ceb50ed0e96e3ce3300564643bc607ea1ce4 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 14 Mar 2024 12:06:55 +0000 Subject: [PATCH 70/89] fold Matt's feedback --- website/docs/guides/core-to-cloud-1.md | 68 +++++++++++++------------- 1 file changed, 35 insertions(+), 33 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 2b9d5805d99..1c780bae833 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -1,5 +1,5 @@ --- -title: Move from dbt Core to dbt Cloud - Get started +title: 'Move from dbt Core to dbt Cloud: Get started' id: core-to-cloud-1 description: "Learn how to move from dbt Core to dbt Cloud and what you need to get started." hoverSnippet: "Learn how to move from dbt Core to dbt Cloud." @@ -14,7 +14,7 @@ recently_updated: true Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed software service. -dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports: +dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports: - The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development - [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics - Domain ownership of data with multi-project [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) setups @@ -80,8 +80,8 @@ This section outlines the considerations and methods to connect your data platfo 2. You can verify your data platform connections by clicking the **Test connection** button in your deployment and development credentials settings. -### Advanced configuration -3. Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enables secure authentication using your data platform’s SSO. +### Additional configuration +- Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enables secure authentication using your data platform’s SSO. ## Git setup @@ -89,12 +89,12 @@ Your existing dbt project source code should live in a Git repository. In this s 1. Ensure your dbt project is in a Git repository. -2. Once you’ve set up a dbt Cloud account, you can [connect and configure Git](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud to connect your Git repository: +2. In **Account settings**, select **Integrations** and [connect and configure Git](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud to connect your Git repository: - Connect with one of the [native integrations](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud (such as GitHub, GitLab, and Azure DevOps). - [Import a git repository](/docs/cloud/git/import-a-project-by-git-url) from any valid git URL that points to a dbt project. -### Advanced configuration -3. Log into dbt Cloud using [OAuth connections](/docs/cloud/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. +### Additional configuration +- Log into dbt Cloud using [OAuth connections](/docs/cloud/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. Set up groups for dbt project access with those configured for repository access to streamline permissions. @@ -103,36 +103,36 @@ Your existing dbt project source code should live in a Git repository. In this s This section highlights the development configurations you’ll need for your dbt Cloud project. The following categories are covered in this section: - [dbt Cloud environments](/guides/core-to-cloud-1?step=7#dbt-cloud-environments) -- [Initial setup steps](/guides/core-to-cloud-1?step=7#initial-set-up-steps) -- [Advanced configuration](/guides/core-to-cloud-1?step=7#initial-set-up-steps) +- [Initial setup steps](/guides/core-to-cloud-1?step=7#initial-setup-steps) +- [Additional configuration](/guides/core-to-cloud-1?step=7#additional-configuration-2) - [dbt Cloud commands](/guides/core-to-cloud-1?step=7#dbt-cloud-commands) ### dbt Cloud environments The concept of a `target` in dbt Core is the same as a [dbt Cloud environment](/docs/environments-in-dbt). -The main difference between `target` in dbt Core and a dbt Cloud environment is that you can make these configurations through the dbt Cloud UI, as opposed to within your **`profiles.yml`** file. +The primary difference between a dbt Cloud environment and a `target` in dbt Core is that you can make these configurations through the dbt Cloud UI instead of within the `profiles.yml` file. -This difference streamlines the process of switching between development, staging, and production contexts, removing the need to manually edit a **`profiles.yml`** file. dbt Cloud environments also integrate with additional features such as job scheduling, version control, and more, making it easier to manage the full lifecycle of your dbt projects within a single platform You can [set up](/reference/dbt-jinja-functions/target) or [customize](/docs/build/custom-target-names) target names in dbt Cloud. +This difference streamlines the process of switching between development, staging, and production contexts, removing the need to manually edit the `profiles.yml` file. dbt Cloud environments also integrate with additional features such as job scheduling, version control, and more, making it easier to manage the full lifecycle of your dbt projects within a single platform. You can [set up](/reference/dbt-jinja-functions/target) or [customize](/docs/build/custom-target-names) target names in dbt Cloud. ### Initial setup steps 1. **Set up development environment** — Set up your [development](/docs/dbt-cloud-environments#create-a-development-environment) environment and [development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide). You’ll need this to access your dbt project and start developing. 2. **dbt Core version** — In your dbt Cloud environment and credentials, use the same dbt Core version you use locally. You can run `dbt --version` in the command line to find out which version of dbt Core you’re using. - - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. + - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest features and more. -3. **Connect to your data platform** — When using dbt Cloud, you can [connect to your data platform](/docs/cloud/connect-data-platform/about-connections) directly in the UI and don't need a `profiles.yml` file. Each environment is roughly equivalent to an entry in your `profiles.yml` file. +3. **Connect to your data platform** — When using dbt Cloud, you can [connect to your data platform](/docs/cloud/connect-data-platform/about-connections) directly in the UI. + - Each environment is roughly equivalent to an entry in your `profiles.yml` file. This means you don't need a `profiles.yml` file in your project. 4. **Development tools** — Set up your development workspace with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) to edit and develop your dbt code in your tool of choice. -### Advanced configuration -5. **Custom target names** — If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. +### Additional configuration +- **Custom target names** — If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. ### dbt Cloud commands -6. Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. +- Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. ## Environment variables - -This section will help you understand how to manage environment variables in dbt Cloud and how to set them up for your project. The following categories are covered in this section: +This section will help you understand how to set up and manage dbt Cloud environment variables for your project. The following categories are covered: - [Environment variables in dbt Cloud](/guides/core-to-cloud-1?step=7#environment-variables-in-dbt-cloud) - [dbt Cloud environment variables order of precedence](/guides/core-to-cloud-1?step=7#dbt-cloud-environment-variables-order-of-precedence) - [Set environment variables in dbt Cloud](/guides/core-to-cloud-1?step=7#set-environment-variables-in-dbt-cloud) @@ -142,7 +142,7 @@ In dbt Cloud, you can set [environment variables](/docs/build/environment-variab ### Environment variables in dbt Cloud - dbt Cloud environment variables must be prefixed with `DBT_` (including `DBT_ENV_CUSTOM_ENV_` or `DBT_ENV_SECRET_`). - - If your dbt Core environment variables don’t follow this naming convention, perform a “find and replace” in your project to make sure all references to these environment variables contain the proper naming conventions. + - If your dbt Core environment variables don’t follow this naming convention, perform a [“find and replace”](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#dbt-cloud-ide-features) in your project to make sure all references to these environment variables contain the proper naming conventions. - dbt Cloud secures environment variables, offering additional measures for sensitive values, such as prefixing keys with `DBT_ENV_SECRET_` to obscure them in logs and the UI. @@ -167,32 +167,32 @@ Environment variables in dbt Cloud are managed with a clear [order of precedence This section outlines the considerations and methods to set up your dbt Cloud environments and jobs for orchestration. The following categories are covered in this section: - [dbt Cloud environments](/guides/core-to-cloud-1?step=8#dbt-cloud-environments-1) -- [Initial setup steps](/guides/core-to-cloud-1?step=8#initial-set-up-steps-1) -- [Additional configurations](/guides/core-to-cloud-1?step=8#additional-configurations) -- [CI/CD setup](/guides/core-to-cloud-1?step=8#cicd-set-up) +- [Initial setup steps](/guides/core-to-cloud-1?step=8#initial-setup-steps-1) +- [Additional configuration](/guides/core-to-cloud-1?step=8#additional-configuration-3) +- [CI/CD setup](/guides/core-to-cloud-1?step=8#cicd-setup) ### dbt Cloud environments To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one environment as the production environment. This is the [deployment](/docs/deploy/deploy-environments) environment. You can set up multiple environments for different stages of your deployment pipeline, such as development, staging/QA, and production. ### Initial setup steps 1. **dbt Core version** — In your environment settings, configure dbt Cloud with the same dbt Core version. - - Once your full migration is complete, consider upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest fixes and more. + - Once your full migration is complete, we recommend upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest features and more. 2. **[Configure your jobs](/docs/deploy/deploy-jobs)** — Create jobs for automated or event-driven dbt jobs. You can use cron execution, manual, pull requests, or API triggers. - Note that alongside [dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. -### Additional configurations -3. **Custom target names** — Set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. +### Additional configuration +- **Custom target names** — Set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. -4. **dbt commands** — Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. +- **dbt commands** — Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. -5. **Notifications** — Set up [notifications](/docs/deploy/job-notifications) by configuring email and Slack alerts to monitor your jobs. +- **Notifications** — Set up [notifications](/docs/deploy/job-notifications) by configuring email and Slack alerts to monitor your jobs. -6. **Monitoring tools** — Use [monitoring tools](/docs/deploy/monitor-jobs) like run history, job retries, job chaining, dashboard status tiles, and more for a seamless orchestration experience. +- **Monitoring tools** — Use [monitoring tools](/docs/deploy/monitor-jobs) like run history, job retries, job chaining, dashboard status tiles, and more for a seamless orchestration experience. -7. **API access** — Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. +- **API access** — Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. -8. **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. +- **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. ### CI/CD setup @@ -222,9 +222,9 @@ import ConfettiTrigger from '/src/components/confetti/index.js'; -Congratulations on completing the first part of your move to dbt Cloud 🎉! +Congratulations on completing the first part of your move to dbt Cloud 🎉! -You should have learned: +You have learned: - How to set up your dbt Cloud account - How to connect your data platform and Git repository - How to configure your development, orchestration, and CI/CD environments @@ -238,7 +238,9 @@ For next steps, we'll soon share other guides on how to manage your move and tip --> ### Related docs - +- [Learn dbt Cloud](https://courses.getdbt.com/collections) +- [Develop with dbt Cloud](/docs/cloud/about-develop-dbt) +- [Deploy jobs](/docs/deploy/deployments) - Book [expert-led demos](https://www.getdbt.com/resources/dbt-cloud-demos-with-experts) and insights - Work with the [dbt Labs’ Professional Services](https://www.getdbt.com/dbt-labs/services) team to support your data organization and migration. From a84746c5a184ae6a957b7b9ad02af72db2e8854b Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 14 Mar 2024 12:34:15 +0000 Subject: [PATCH 71/89] final tweaks --- website/docs/guides/core-to-cloud-1.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index f068cc0e040..2af489d4879 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -14,7 +14,7 @@ recently_updated: true Moving from dbt Core to dbt Cloud streamlines analytics engineering workflows by allowing teams to develop, test, deploy, and explore data products using a single, fully managed software service. -dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, deploy, and explore data products using a single, fully managed service. It also supports: +dbt Cloud is the fastest and most reliable way to deploy dbt. It enables you to develop, test, deploy, and explore data products using a single, fully managed service. It also supports: - The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or command line with [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) for development - [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) for consistent metrics - Domain ownership of data with multi-project [dbt Mesh](/best-practices/how-we-mesh/mesh-1-intro) setups @@ -61,7 +61,7 @@ This guide outlines the steps you need to take to move from dbt Core to dbt Clou 4. In **Account settings**, enable [partial parsing](/docs/deploy/deploy-environments#partial-parsing) to only reparse changed files, saving time. -5. In **Account settings**, enable [Git repo caching](/docs/deploy/deploy-environments#git-repository-caching) to ensure job reliability, protection from third-party outages, and performance. +5. In **Account settings**, enable [Git repo caching](/docs/deploy/deploy-environments#git-repository-caching) for job reliability & third-party outage protection. ## Data platform setup @@ -89,9 +89,9 @@ Your existing dbt project source code should live in a Git repository. In this s 1. Ensure your dbt project is in a Git repository. -2. In **Account settings**, select **Integrations** and [connect and configure Git](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud to connect your Git repository: +2. In **Account settings**, select **Integrations**, and [connect and configure Git](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud to connect your Git repository: - Connect with one of the [native integrations](/docs/cloud/git/git-configuration-in-dbt-cloud) in dbt Cloud (such as GitHub, GitLab, and Azure DevOps). - - [Import a git repository](/docs/cloud/git/import-a-project-by-git-url) from any valid git URL that points to a dbt project. + - [Import a Git repository](/docs/cloud/git/import-a-project-by-git-url) from any valid Git URL that points to a dbt project. ### Additional configuration - Log into dbt Cloud using [OAuth connections](/docs/cloud/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. @@ -172,14 +172,14 @@ This section outlines the considerations and methods to set up your dbt Cloud en - [CI/CD setup](/guides/core-to-cloud-1?step=8#cicd-setup) ### dbt Cloud environments -To use the [dbt Cloud's scheduler](/docs/deploy/job-scheduler), set up one environment as the production environment. This is the [deployment](/docs/deploy/deploy-environments) environment. You can set up multiple environments for different stages of your deployment pipeline, such as development, staging/QA, and production. +To use the [dbt Cloud's job scheduler](/docs/deploy/job-scheduler), set up one environment as the production environment. This is the [deployment](/docs/deploy/deploy-environments) environment. You can set up multiple environments for different stages of your deployment pipeline, such as development, staging/QA, and production. ### Initial setup steps 1. **dbt Core version** — In your environment settings, configure dbt Cloud with the same dbt Core version. - Once your full migration is complete, we recommend upgrading your environments to [Keep on latest version](/docs/dbt-versions/upgrade-dbt-version-in-cloud#keep-on-latest-version-) to always get the latest features and more. -2. **[Configure your jobs](/docs/deploy/deploy-jobs)** — Create jobs for automated or event-driven dbt jobs. You can use cron execution, manual, pull requests, or API triggers. - - Note that alongside [dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. +2. **Configure your jobs** — [Create jobs](/docs/deploy/deploy-jobs#create-and-schedule-jobs) for automated or event-driven dbt jobs. You can use cron execution, manual, pull requests, or API triggers. + - Note that alongside [jobs in dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of other tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. ### Additional configuration - **Custom target names** — Set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. @@ -208,13 +208,13 @@ This build-on-PR functionality is a great way to catch bugs before deploying to In this section, you’ll be able to validate whether your models run or compile correctly in your development tool of choice: The [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [dbt Cloud CLI](/docs/cloud/cloud-cli-installation). -You’ll want to make sure you set up your [development environment and credentials](#developer-set-up). +You’ll want to make sure you set up your [development environment and credentials](/docs/dbt-cloud-environments#set-developer-credentials). 1. In your [development tool](/docs/cloud/about-develop-dbt) of choice, you can review your dbt project and ensure your project is set up correctly and you’re able to run commands. This will: - Make sure your project compiles correctly. - Run a few models in the dbt Cloud IDE or dbt Cloud CLI to ensure you’re experiencing accurate results in development. -2. Once your first job has successfully run in your production environment, use [dbt Explorer](/docs/collaborate/explore-projects) to view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their [lineage](/terms/data-lineage) to gain a better understanding of its latest production state. +2. Once your first job has successfully run in your production environment, use [dbt Explorer](/docs/collaborate/explore-projects) to view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their   to gain a better understanding of its latest production state. ## What’s next? From 98f7f0ff3eec81099695d7d277454ba4c8c1c4c7 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 14 Mar 2024 12:35:15 +0000 Subject: [PATCH 72/89] Update website/src/components/confetti/index.js --- website/src/components/confetti/index.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/src/components/confetti/index.js b/website/src/components/confetti/index.js index 5ae2aa1b09d..aa63178b776 100644 --- a/website/src/components/confetti/index.js +++ b/website/src/components/confetti/index.js @@ -5,7 +5,7 @@ const ConfettiTrigger = ({ children }) => { const triggerRef = useRef(null); // Use a ref to refer to the component's section useEffect(() => { - const triggerConfetti = () => { + const triggerConfetti = (event?.target?.closest) => { if (event.target.closest('a, h2, h3')) { // Do nothing if the click is on these elements return; From 98a13766b3f3f7e6cddbfbbc32affe3145dc6b61 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 14 Mar 2024 12:49:53 +0000 Subject: [PATCH 73/89] jason's feedback --- website/docs/guides/core-to-cloud-1.md | 2 - website/src/components/confetti/index.js | 51 ++++++++---------------- 2 files changed, 16 insertions(+), 37 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 2af489d4879..1626d09ce79 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -218,8 +218,6 @@ You’ll want to make sure you set up your [development environment and credenti ## What’s next? -import ConfettiTrigger from '/src/components/confetti/index.js'; - Congratulations on completing the first part of your move to dbt Cloud 🎉! diff --git a/website/src/components/confetti/index.js b/website/src/components/confetti/index.js index aa63178b776..85aeaef46fb 100644 --- a/website/src/components/confetti/index.js +++ b/website/src/components/confetti/index.js @@ -1,57 +1,38 @@ -import React, { useRef, useEffect } from 'react'; +import React from 'react'; import confetti from 'canvas-confetti'; const ConfettiTrigger = ({ children }) => { - const triggerRef = useRef(null); // Use a ref to refer to the component's section - - useEffect(() => { - const triggerConfetti = (event?.target?.closest) => { - if (event.target.closest('a, h2, h3')) { + const triggerConfetti = (event) => { + if (event?.target?.closest('a, h2, h3')) { // Do nothing if the click is on these elements return; } - // Configuration for the confetti - const confettiCount = 200; - const spread = 70; - const startVelocity = 30; - const scalar = 0.7; - const gravity = 1.5; - const decay = 0.9; + // config for the confetti w spread operator + const confettiCount = 200; + const confettiSettings = { + spread: 70, + startVelocity: 30, + scalar: 0.7, + gravity: 1.5, + decay: 0.9, + } // Launch confetti from multiple points for (let i = 0; i < 5; i++) { confetti({ angle: 60, - spread: spread, particleCount: confettiCount / 5, origin: { y: 0.6, x: (i + 1) * 0.2 - 0.1 }, - startVelocity: startVelocity, - scalar: scalar, - gravity: gravity, - decay: decay, zIndex: 9999, + ...confettiSettings, }); } }; - - const triggerElement = triggerRef.current; - // Add event listener to the section - if (triggerElement) { - triggerElement.addEventListener('click', triggerConfetti); - } - - // Clean up the event listener on component unmount - return () => { - if (triggerElement) { - triggerElement.removeEventListener('click', triggerConfetti); - } - }; - }, []); - + // add an OnClick event to trigger the confetti instead of listing EventListeners with useEffect return ( -
+
{children} {/* Render children passed to the component */} -
+
); }; From ff9ecd760ff3ac3d162a3c7e04cd580a962c23ae Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 14 Mar 2024 13:00:36 +0000 Subject: [PATCH 74/89] typos --- website/docs/guides/core-to-cloud-1.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 1626d09ce79..a534c590771 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -40,7 +40,7 @@ This guide outlines the steps you need to take to move from dbt Core to dbt Clou ## Prerequisites - You have an existing dbt Core project connected to a Git repository and data platform supported in [dbt Cloud](/docs/cloud/connect-data-platform/about-connections). -- You are dbt version 1.0 or later. +- You are using dbt version 1.0 or later. - You have a dbt Cloud account. **[Don't have one? Start your free trial today](https://www.getdbt.com/signup)**! ### Related docs @@ -113,7 +113,7 @@ The concept of an [environment](/docs/environments-in-dbt) in dbt Cloud is the s The primary difference between a dbt Cloud environment and a `target` in dbt Core is that you can make these configurations through the dbt Cloud UI instead of within the `profiles.yml` file. -This difference streamlines the process of switching between development, staging, and production contexts, removing the need to manually edit the `profiles.yml` file. dbt Cloud environments also integrate with additional features such as job scheduling, version control, and more, making it easier to manage the full lifecycle of your dbt projects within a single platform. You can [set up](/reference/dbt-jinja-functions/target) or [customize](/docs/build/custom-target-names) target names in dbt Cloud. +This difference streamlines the process of switching between development, staging, and production contexts, removing the need to manually edit the `profiles.yml` file. dbt Cloud environments also integrate with additional features such as job scheduling, version control, and more — making it easier to manage the full lifecycle of your dbt projects within a single platform. You can [set up](/reference/dbt-jinja-functions/target) or [customize](/docs/build/custom-target-names) target names in dbt Cloud. ### Initial setup steps 1. **Set up development environment** — Set up your [development](/docs/dbt-cloud-environments#create-a-development-environment) environment and [development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide). You’ll need this to access your dbt project and start developing. From a2c7f690c7bef26b7e99fddf84b6076e726f26db Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 14 Mar 2024 14:26:48 +0000 Subject: [PATCH 75/89] add bullet --- website/docs/guides/core-to-cloud-1.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index a534c590771..ef0527fd79b 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -81,7 +81,7 @@ This section outlines the considerations and methods to connect your data platfo 2. You can verify your data platform connections by clicking the **Test connection** button in your deployment and development credentials settings. ### Additional configuration -- Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enables secure authentication using your data platform’s SSO. +1. Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enables secure authentication using your data platform’s SSO. ## Git setup @@ -94,7 +94,7 @@ Your existing dbt project source code should live in a Git repository. In this s - [Import a Git repository](/docs/cloud/git/import-a-project-by-git-url) from any valid Git URL that points to a dbt project. ### Additional configuration -- Log into dbt Cloud using [OAuth connections](/docs/cloud/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. +1. Log into dbt Cloud using [OAuth connections](/docs/cloud/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. Set up groups for dbt project access with those configured for repository access to streamline permissions. @@ -126,7 +126,7 @@ This difference streamlines the process of switching between development, stagin 4. **Development tools** — Set up your development workspace with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) to edit and develop your dbt code in your tool of choice. ### Additional configuration -- **Custom target names** — If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. +1. **Custom target names** — If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. ### dbt Cloud commands - Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. @@ -182,7 +182,7 @@ To use the [dbt Cloud's job scheduler](/docs/deploy/job-scheduler), set up one e - Note that alongside [jobs in dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of other tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. ### Additional configuration -- **Custom target names** — Set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. +1. **Custom target names** — Set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. - **dbt commands** — Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. From d52b3c7439248de5ca504e03ba20082a95556871 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 14 Mar 2024 14:56:51 +0000 Subject: [PATCH 76/89] final tweaks --- website/docs/guides/core-to-cloud-1.md | 28 +++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index ef0527fd79b..d484e07b5b7 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -52,9 +52,11 @@ This guide outlines the steps you need to take to move from dbt Core to dbt Clou ## Account setup +This section outlines the steps to set up your dbt Cloud account and configure it for your team. + 1. [Create your dbt Cloud account](https://www.getdbt.com/signup). -2. Learn about [access](/docs/cloud/manage-access/about-user-access) and [invite users](/docs/cloud/manage-access/about-user-access) to your dbt Cloud account and dbt Cloud project. +2. Provide user [access](/docs/cloud/manage-access/about-user-access) and [invite users](/docs/cloud/manage-access/about-user-access) to your dbt Cloud account and project. 3. Configure [Single Sign-On (SSO)](/docs/cloud/manage-access/sso-overview) or [Role-based access control (RBAC)](/docs/cloud/manage-access/about-user-access#role-based-access-control) for easy and secure access. - This removes the need to save passwords and secret environment variables locally. @@ -65,9 +67,9 @@ This guide outlines the steps you need to take to move from dbt Core to dbt Clou ## Data platform setup -This section outlines the considerations and methods to connect your data platform(s) to dbt Cloud. +This section outlines the considerations and methods to connect your data platform to dbt Cloud. -1. Set up your [data platform connections](/docs/cloud/connect-data-platform/about-connections) and [environment variables](/docs/build/environment-variables) in dbt Cloud. dbt Cloud can connect with a variety of data platform providers including: +1. In dbt Cloud, set up your [data platform connections](/docs/cloud/connect-data-platform/about-connections) and [environment variables](/docs/build/environment-variables). dbt Cloud can connect with a variety of data platform providers including: - [AlloyDB](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) - [Amazon Redshift](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) - [Apache Spark](/docs/cloud/connect-data-platform/connect-apache-spark) @@ -81,6 +83,9 @@ This section outlines the considerations and methods to connect your data platfo 2. You can verify your data platform connections by clicking the **Test connection** button in your deployment and development credentials settings. ### Additional configuration + +Explore these additional configurations to optimize your data platform setup further: + 1. Use [OAuth connections](/docs/cloud/manage-access/set-up-snowflake-oauth), which enables secure authentication using your data platform’s SSO. ## Git setup @@ -94,6 +99,8 @@ Your existing dbt project source code should live in a Git repository. In this s - [Import a Git repository](/docs/cloud/git/import-a-project-by-git-url) from any valid Git URL that points to a dbt project. ### Additional configuration +Explore these additional configurations to optimize Git setup further: + 1. Log into dbt Cloud using [OAuth connections](/docs/cloud/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. Set up groups for dbt project access with those configured for repository access to streamline permissions. @@ -126,10 +133,11 @@ This difference streamlines the process of switching between development, stagin 4. **Development tools** — Set up your development workspace with the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) to edit and develop your dbt code in your tool of choice. ### Additional configuration +Explore these additional configurations to optimize your developer setup further: 1. **Custom target names** — If you’re using a [`custom target.name`](/reference/dbt-jinja-functions/target) in your project, we recommend you set them using [environment variables](/docs/build/environment-variables). Alternatively, you can update it at the developer credentials level. ### dbt Cloud commands -- Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. +1. Review the [dbt commands](/reference/dbt-commands) supported for dbt Cloud development. For example, `dbt debug` isn’t needed in dbt Cloud since the UI displays logs for your viewing. ## Environment variables This section will help you understand how to set up and manage dbt Cloud environment variables for your project. The following categories are covered: @@ -182,17 +190,19 @@ To use the [dbt Cloud's job scheduler](/docs/deploy/job-scheduler), set up one e - Note that alongside [jobs in dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of other tools. Refer to [Integrate with other tools](/docs/deploy/deployment-tools) for more information. ### Additional configuration +Explore these additional configurations to optimize your dbt Cloud orchestration setup further: + 1. **Custom target names** — Set a `custom target.name` for every single [corresponding dbt Cloud job](/docs/build/custom-target-names). We recommend modifying the code to use [environment variables](/docs/build/environment-variables) instead since those can be set at the environment level. -- **dbt commands** — Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. +2. **dbt commands** — Add any relevant [dbt commands](/docs/deploy/job-commands) to execute your dbt Cloud jobs runs. -- **Notifications** — Set up [notifications](/docs/deploy/job-notifications) by configuring email and Slack alerts to monitor your jobs. +3. **Notifications** — Set up [notifications](/docs/deploy/job-notifications) by configuring email and Slack alerts to monitor your jobs. -- **Monitoring tools** — Use [monitoring tools](/docs/deploy/monitor-jobs) like run history, job retries, job chaining, dashboard status tiles, and more for a seamless orchestration experience. +4. **Monitoring tools** — Use [monitoring tools](/docs/deploy/monitor-jobs) like run history, job retries, job chaining, dashboard status tiles, and more for a seamless orchestration experience. -- **API access** — Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. +5. **API access** — Create [API auth tokens](/docs/dbt-cloud-apis/authentication) and access to [dbt Cloud APIs](/docs/dbt-cloud-apis/overview) as needed. -- **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. +6. **dbt Explorer** — If you use [dbt Explorer](/docs/collaborate/explore-projects) and run production jobs with an external orchestrator, ensure your production jobs run `dbt run` or `dbt build` to update and view resources and its metadata in dbt Explorer. Running `dbt compile` will not update resources and its metadata. ### CI/CD setup From 5eda6d684ad186cb100958188c2f8ef51f732a77 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 14 Mar 2024 15:05:13 +0000 Subject: [PATCH 77/89] add your --- website/docs/guides/core-to-cloud-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index d484e07b5b7..170705437a7 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -99,7 +99,7 @@ Your existing dbt project source code should live in a Git repository. In this s - [Import a Git repository](/docs/cloud/git/import-a-project-by-git-url) from any valid Git URL that points to a dbt project. ### Additional configuration -Explore these additional configurations to optimize Git setup further: +Explore these additional configurations to optimize your Git setup further: 1. Log into dbt Cloud using [OAuth connections](/docs/cloud/git/connect-github) to integrate with your source code platform. It automatically links to the repository using one of the native integrations set at the account level. From 2ae257d3f83360d31259b5f385655e49b1f271b6 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Thu, 14 Mar 2024 11:33:14 -0400 Subject: [PATCH 78/89] Update website/docs/docs/build/saved-queries.md --- website/docs/docs/build/saved-queries.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/saved-queries.md b/website/docs/docs/build/saved-queries.md index 050aa242afc..b4a31391c10 100644 --- a/website/docs/docs/build/saved-queries.md +++ b/website/docs/docs/build/saved-queries.md @@ -77,7 +77,7 @@ To define a saved query, refer to the following parameters: - + From 9ac19bdc8f8a39c2a001a0dca70f754ff339c6d6 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 14 Mar 2024 09:24:02 -0700 Subject: [PATCH 79/89] Update and rename new idea issue template (#5075) ## What are you changing in this pull request and why? Make this issue template more inclusive. --------- Co-authored-by: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> --- .../contribute-to-developer-blog.yml | 54 ------------------- .../ISSUE_TEMPLATE/propose-new-content.yml | 43 +++++++++++++++ 2 files changed, 43 insertions(+), 54 deletions(-) delete mode 100644 .github/ISSUE_TEMPLATE/contribute-to-developer-blog.yml create mode 100644 .github/ISSUE_TEMPLATE/propose-new-content.yml diff --git a/.github/ISSUE_TEMPLATE/contribute-to-developer-blog.yml b/.github/ISSUE_TEMPLATE/contribute-to-developer-blog.yml deleted file mode 100644 index 037da98dc6f..00000000000 --- a/.github/ISSUE_TEMPLATE/contribute-to-developer-blog.yml +++ /dev/null @@ -1,54 +0,0 @@ -name: Propose a dbt Developer Blog idea -description: > - For proposing a new post on the dbt Developer Blog. -labels: ["content","developer blog"] -body: - - type: markdown - attributes: - value: | - We're excited to hear about your idea for the dbt Developer Blog. This template will help lay out the proposed post and then we will work with on next steps! - - - type: input - id: contact - attributes: - label: Contact Details - description: How can we get in touch with you? - placeholder: your preferred email and/or dbt Slack handle - validations: - required: true - - - type: checkboxes - id: read-contribution - attributes: - label: I have read the dbt Developer Blog contribution guidelines. - description: You can find the contribution guide [here](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/developer-blog.md) - options: - - label: I have read the dbt Developer Blog contribution guidelines. - validations: - required: true - - - type: checkboxes - id: author_type - attributes: - label: Which of these best describes you? - options: - - label: I am a dbt Community member or partner contributing to the Developer Blog - - label: I work for dbt Labs and am creating this issue for a community or marketing approved piece. - validations: - - - - type: textarea - attributes: - label: > - What is the topic of your post? - description: | - Please provide a short (~ 1 paragraph) summary as well as who this post is targeted towards (ie people interested in learning more about dbt snapshots or advanced Jinja users) - validations: - required: true - - - type: textarea - attributes: - label: Link to an initial outline. - description: Please link to a short outline in Notion, or Google Docs - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE/propose-new-content.yml b/.github/ISSUE_TEMPLATE/propose-new-content.yml new file mode 100644 index 00000000000..1dc2816d311 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/propose-new-content.yml @@ -0,0 +1,43 @@ +name: "Propose new content" +description: "For proposing new content, such as a new guide or a new page." +title: "[idea] " +labels: ["content","idea"] +body: +- type: markdown + attributes: + value: | + Thank you for sharing your idea for the dbt product documentation! Here are a few things to consider: + * You can submit ideas or suggest changes to our content by opening an [Issue](https://github.com/dbt-labs/docs.getdbt.com/issues/new/choose). + * Before filing an issue, please [search our current issues](https://github.com/dbt-labs/docs.getdbt.com/issues) to avoid duplicates. + * Please read the [Contributing guide](https://github.com/dbt-labs/docs.getdbt.com#contributing) if you want to open a pull request. + +- type: checkboxes + id: author_type + attributes: + label: "Which of these best describes you?" + options: + - label: "dbt Community member" + - label: "Partner" + - label: "dbt Labs employee" + - label: "Other" + validations: + required: false + +- type: textarea + id: content_idea + attributes: + label: "What's your idea for new content?" + description: | + - Give as much detail as you can to help us understand your idea. + - Why do you think this content is important? + - Who will this new content help? + validations: + required: true + +- type: textarea + id: location + attributes: + label: Where would you recommend this content live on the docs.getdbt.com? + description: "Please link to the page or pages you think best fit." + validations: + required: false From d4145e87a337e6890252b6f386a3c40572a5e5b8 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 14 Mar 2024 09:25:12 -0700 Subject: [PATCH 80/89] Rename propose-new-content.yml to b-propose-new-content.yml --- .../{propose-new-content.yml => b-propose-new-content.yml} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename .github/ISSUE_TEMPLATE/{propose-new-content.yml => b-propose-new-content.yml} (100%) diff --git a/.github/ISSUE_TEMPLATE/propose-new-content.yml b/.github/ISSUE_TEMPLATE/b-propose-new-content.yml similarity index 100% rename from .github/ISSUE_TEMPLATE/propose-new-content.yml rename to .github/ISSUE_TEMPLATE/b-propose-new-content.yml From 26d8460f0d885d7fe4c7af1234f049546a395ad4 Mon Sep 17 00:00:00 2001 From: "Leona B. Campbell" <3880403+runleonarun@users.noreply.github.com> Date: Thu, 14 Mar 2024 09:28:25 -0700 Subject: [PATCH 81/89] Copy pasta fix --- .github/ISSUE_TEMPLATE/external-core-team.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/external-core-team.yml b/.github/ISSUE_TEMPLATE/external-core-team.yml index 51460952c6c..7e90a2840d0 100644 --- a/.github/ISSUE_TEMPLATE/external-core-team.yml +++ b/.github/ISSUE_TEMPLATE/external-core-team.yml @@ -1,5 +1,5 @@ name: dbt Core - Request changes to docs -description: File a docs update request that is not already tracked in Orch team's Release Plans (Notion database). +description: "File a docs update or feature request related to dbt Core content." title: "[Core] <title>" labels: ["content","dbt Core"] body: From 890fe28a39eab5abf0d2e7712ba53c8f25f39cc7 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 14 Mar 2024 16:52:19 +0000 Subject: [PATCH 82/89] Update core-to-cloud-1.md fix links --- website/docs/guides/core-to-cloud-1.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/docs/guides/core-to-cloud-1.md b/website/docs/guides/core-to-cloud-1.md index 170705437a7..b1737cb14f4 100644 --- a/website/docs/guides/core-to-cloud-1.md +++ b/website/docs/guides/core-to-cloud-1.md @@ -29,10 +29,10 @@ dbt Core is an open-source tool that enables data teams to transform data follow This guide outlines the steps you need to take to move from dbt Core to dbt Cloud and highlights the necessary technical changes: - [Account setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=3#account-setup): Learn how to create a dbt Cloud account, invite team members, and configure it for your team. -- [Environment variables](https://docs.getdbt.com/guides/core-to-cloud-1?step=4#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. -- [Data platform setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=5#data-platform-setup): Find out about connecting your data platform to dbt Cloud. -- [Git setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=6#git-setup): Learn to link your dbt project's Git repository with dbt Cloud. -- [Developer setup:](https://docs.getdbt.com/guides/core-to-cloud-1?step=7#developer-setup) Understand the setup needed for developing in dbt Cloud. +- [Data platform setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=4#data-platform-setup): Find out about connecting your data platform to dbt Cloud. +- [Git setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=5#git-setup): Learn to link your dbt project's Git repository with dbt Cloud. +- [Developer setup:](https://docs.getdbt.com/guides/core-to-cloud-1?step=6#developer-setup) Understand the setup needed for developing in dbt Cloud. +- [Environment variables](https://docs.getdbt.com/guides/core-to-cloud-1?step=7#environment-variables): Discover how to manage environment variables in dbt Cloud, including their priority. - [Orchestration setup](https://docs.getdbt.com/guides/core-to-cloud-1?step=8#orchestration-setup): Learn how to prepare your dbt Cloud environment and jobs for orchestration. - [Models configuration](https://docs.getdbt.com/guides/core-to-cloud-1?step=9#models-configuration): Get insights on validating and running your models in dbt Cloud, using either the dbt Cloud IDE or dbt Cloud CLI. - [What's next?](https://docs.getdbt.com/guides/core-to-cloud-1?step=10#whats-next): Summarizes key takeaways and introduces what to expect in the following guides. From 59769b5e60ad0c26209d2cde3bee40e2ce134ecd Mon Sep 17 00:00:00 2001 From: Joel Labes <joel.labes@dbtlabs.com> Date: Fri, 15 Mar 2024 15:26:48 +1300 Subject: [PATCH 83/89] Remove webhook triggering run, redirect to trigger --- website/docs/guides/zapier-new-cloud-job.md | 98 --------------------- website/vercel.json | 7 +- 2 files changed, 6 insertions(+), 99 deletions(-) delete mode 100644 website/docs/guides/zapier-new-cloud-job.md diff --git a/website/docs/guides/zapier-new-cloud-job.md b/website/docs/guides/zapier-new-cloud-job.md deleted file mode 100644 index b16fa94bc21..00000000000 --- a/website/docs/guides/zapier-new-cloud-job.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -title: "Trigger a dbt Cloud job after a run finishes" -id: zapier-new-cloud-job -description: Use Zapier to trigger a dbt Cloud job once a run completes. -hoverSnippet: Learn how to use Zapier to trigger a dbt Cloud job once a run completes. -# time_to_complete: '30 minutes' commenting out until we test -icon: 'guides' -hide_table_of_contents: true -tags: ['Webhooks'] -level: 'Advanced' -recently_updated: true ---- - -## Introduction - -This guide will show you how to trigger a dbt Cloud job based on the successful completion of a different job. This can be useful when you need to trigger a job in a different project. Remember that dbt works best when it understands the whole context of the <Term id="dag"/> it has been asked to run, so use this ability judiciously. - -### Prerequisites - -In order to set up the integration, you should have familiarity with: -- [dbt Cloud Webhooks](/docs/deploy/webhooks) -- Zapier - -## Create a new Zap in Zapier -Use **Webhooks by Zapier** as the Trigger, and **Catch Raw Hook** as the Event. If you don't intend to [validate the authenticity of your webhook](/docs/deploy/webhooks#validate-a-webhook) (not recommended!) then you can choose **Catch Hook** instead. - -Press **Continue**, then copy the webhook URL. - -![Screenshot of the Zapier UI, showing the webhook URL ready to be copied](/img/guides/orchestration/webhooks/zapier-common/catch-raw-hook.png) - -## Configure a new webhook in dbt Cloud -See [Create a webhook subscription](/docs/deploy/webhooks#create-a-webhook-subscription) for full instructions. Your event should be **Run completed**, and you need to change the **Jobs** list to only contain the job you want to trigger the next run. - -Make note of the Webhook Secret Key for later. - -Once you've tested the endpoint in dbt Cloud, go back to Zapier and click **Test Trigger**, which will create a sample webhook body based on the test event dbt Cloud sent. - -The sample body's values are hard-coded and not reflective of your project, but they give Zapier a correctly-shaped object during development. - -## Store secrets -In the next step, you will need the Webhook Secret Key from the prior step, and a dbt Cloud [user token](https://docs.getdbt.com/docs/dbt-cloud-apis/user-tokens) or [service account token](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens). - -Zapier allows you to [store secrets](https://help.zapier.com/hc/en-us/articles/8496293271053-Save-and-retrieve-data-from-Zaps), which prevents your keys from being displayed in plaintext in the Zap code. You will be able to access them via the [StoreClient utility](https://help.zapier.com/hc/en-us/articles/8496293969549-Store-data-from-code-steps-with-StoreClient). - -<Snippet path="webhook_guide_zapier_secret_store" /> - -## Add a code action -Select **Code by Zapier** as the App, and **Run Python** as the Event. - -In the **Set up action** area, add two items to **Input Data**: `raw_body` and `auth_header`. Map those to the `1. Raw Body` and `1. Headers Http Authorization` fields from the **Catch Raw Hook** step above. - -![Screenshot of the Zapier UI, showing the mappings of raw_body and auth_header](/img/guides/orchestration/webhooks/zapier-common/run-python.png) - -In the **Code** field, paste the following code, replacing `YOUR_SECRET_HERE` with the secret you created when setting up the Storage by Zapier integration. Remember that this is not your dbt Cloud secret. - -The code below will validate the authenticity of the request, then send a [`trigger run` command to the dbt Cloud API](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#tag/Jobs/operation/triggerRun) for the given job ID. - -```python -import hashlib -import hmac -import json - -#replace with the Job ID you want to run -target_job_id = 12345 - -auth_header = input_data['auth_header'] -raw_body = input_data['raw_body'] - -# Access secret credentials -secret_store = StoreClient('YOUR_SECRET_HERE') -hook_secret = secret_store.get('DBT_WEBHOOK_KEY') -api_token = secret_store.get('DBT_CLOUD_SERVICE_TOKEN') - -# Validate the webhook came from dbt Cloud -signature = hmac.new(hook_secret.encode('utf-8'), raw_body.encode('utf-8'), hashlib.sha256).hexdigest() - -if signature != auth_header: - raise Exception("Calculated signature doesn't match contents of the Authorization header. This webhook may not have been sent from dbt Cloud.") - -full_body = json.loads(raw_body) -hook_data = full_body['data'] - -if hook_data['runStatus'] == "Success": - - # Trigger a new run with the dbt Cloud Admin API - url = f'https://cloud.getdbt.com/api/v2/accounts/{full_body['accountId']}/jobs/{target_job_id}/run' - - body = {'cause': f"Triggered by Zapier because {hook_data['jobName']} Run #{hook_data['runId']} completed successfully"} - headers = {'Authorization': f'Token {api_token}'} - response = requests.post(url, json=body, headers=headers) - response.raise_for_status() - -return -``` - -## Test and deploy - -When you're happy with it, remember to ensure that your `account_id` is no longer hardcoded, then publish your Zap. diff --git a/website/vercel.json b/website/vercel.json index 377cc635561..fc275cff56c 100644 --- a/website/vercel.json +++ b/website/vercel.json @@ -574,7 +574,12 @@ }, { "source": "/guides/orchestration/webhooks/zapier-new-cloud-job", - "destination": "/guides/zapier-new-cloud-job", + "destination": "/docs/deploy/deploy-jobs#trigger-on-job-completion--", + "permanent": true + }, + { + "source": "/guides/zapier-new-cloud-job", + "destination": "/docs/deploy/deploy-jobs#trigger-on-job-completion--", "permanent": true }, { From eb5e403b65a61fa1bdf5d4276371393f3a115967 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 15 Mar 2024 09:41:08 +0000 Subject: [PATCH 84/89] Update website/vercel.json --- website/vercel.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/vercel.json b/website/vercel.json index fc275cff56c..f67c45f9c6d 100644 --- a/website/vercel.json +++ b/website/vercel.json @@ -574,7 +574,7 @@ }, { "source": "/guides/orchestration/webhooks/zapier-new-cloud-job", - "destination": "/docs/deploy/deploy-jobs#trigger-on-job-completion--", + "destination": "/docs/deploy/deploy-jobs#trigger-on-job-completion", "permanent": true }, { From 398ff052b47598da7957c030918f8066bab18584 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 15 Mar 2024 09:41:20 +0000 Subject: [PATCH 85/89] Update website/vercel.json --- website/vercel.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/vercel.json b/website/vercel.json index f67c45f9c6d..443482a1fc1 100644 --- a/website/vercel.json +++ b/website/vercel.json @@ -579,7 +579,7 @@ }, { "source": "/guides/zapier-new-cloud-job", - "destination": "/docs/deploy/deploy-jobs#trigger-on-job-completion--", + "destination": "/docs/deploy/deploy-jobs#trigger-on-job-completion", "permanent": true }, { From ef5f22ede5232aac7ea524a97d717c8e8a44ae58 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 15 Mar 2024 09:56:33 +0000 Subject: [PATCH 86/89] Update deploy-jobs.md --- website/docs/docs/deploy/deploy-jobs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/deploy-jobs.md b/website/docs/docs/deploy/deploy-jobs.md index a0c04b9a6b8..63c2dd98103 100644 --- a/website/docs/docs/deploy/deploy-jobs.md +++ b/website/docs/docs/deploy/deploy-jobs.md @@ -105,7 +105,7 @@ Here are examples of cron job schedules. The dbt Cloud job scheduler supports us - `0 7 L * 5`: At 07:00 AM, on the last day of the month, and on Friday. - `30 14 L * *`: At 02:30 PM, on the last day of the month. -### Trigger on job completion <Lifecycle status="team,enterprise" /> + To _chain_ deploy jobs together, enable the **Run when another job finishes** option and specify the upstream (parent) job that, when it completes, will trigger your job. You can also use the [Create Job API](/dbt-cloud/api-v2#/operations/Create%20Job) to do this. From 343256afa21740f451e41dac35e215767a3b4e71 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 15 Mar 2024 09:56:51 +0000 Subject: [PATCH 87/89] Update deploy-jobs.md --- website/docs/docs/deploy/deploy-jobs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/deploy/deploy-jobs.md b/website/docs/docs/deploy/deploy-jobs.md index 63c2dd98103..ee703895f7a 100644 --- a/website/docs/docs/deploy/deploy-jobs.md +++ b/website/docs/docs/deploy/deploy-jobs.md @@ -105,7 +105,7 @@ Here are examples of cron job schedules. The dbt Cloud job scheduler supports us - `0 7 L * 5`: At 07:00 AM, on the last day of the month, and on Friday. - `30 14 L * *`: At 02:30 PM, on the last day of the month. - +### Trigger on job completion <Lifecycle status="team,enterprise" /> To _chain_ deploy jobs together, enable the **Run when another job finishes** option and specify the upstream (parent) job that, when it completes, will trigger your job. You can also use the [Create Job API](/dbt-cloud/api-v2#/operations/Create%20Job) to do this. From 37751b66155ebce9b7a01287ab3db946e421f9ae Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 15 Mar 2024 10:41:42 +0000 Subject: [PATCH 88/89] Update website/vercel.json --- website/vercel.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/vercel.json b/website/vercel.json index 443482a1fc1..f67c45f9c6d 100644 --- a/website/vercel.json +++ b/website/vercel.json @@ -579,7 +579,7 @@ }, { "source": "/guides/zapier-new-cloud-job", - "destination": "/docs/deploy/deploy-jobs#trigger-on-job-completion", + "destination": "/docs/deploy/deploy-jobs#trigger-on-job-completion--", "permanent": true }, { From eec8eb624b505e01a67e43b3a4d101800fbc32e4 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 15 Mar 2024 10:41:54 +0000 Subject: [PATCH 89/89] Update website/vercel.json --- website/vercel.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/vercel.json b/website/vercel.json index f67c45f9c6d..fc275cff56c 100644 --- a/website/vercel.json +++ b/website/vercel.json @@ -574,7 +574,7 @@ }, { "source": "/guides/orchestration/webhooks/zapier-new-cloud-job", - "destination": "/docs/deploy/deploy-jobs#trigger-on-job-completion", + "destination": "/docs/deploy/deploy-jobs#trigger-on-job-completion--", "permanent": true }, {