From 305f9ffc7c8f06a6de35318eb1f2e8f264a788d7 Mon Sep 17 00:00:00 2001 From: Maxime Armstrong Date: Mon, 5 Aug 2024 16:30:55 -0400 Subject: [PATCH] Update course after moving DbtProject to project.py --- .../3-representing-the-dbt-project-in-dagster.md | 4 ++-- .../lesson-3/4-creating-a-dbt-resource-in-dagster.md | 2 +- .../5-loading-dbt-models-into-dagster-as-assets.md | 12 +++++------- .../lesson-4/2-speeding-up-the-development-cycle.md | 2 +- .../2-connecting-dbt-models-to-dagster-assets.md | 9 ++------- .../lesson-6/3-creating-a-partitioned-dbt-asset.md | 9 ++------- .../4-creating-the-manifest-during-deployment.md | 2 +- .../lesson-7/5-preparing-for-a-successful-run.md | 6 +++--- 8 files changed, 17 insertions(+), 29 deletions(-) diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-3/3-representing-the-dbt-project-in-dagster.md b/docs/dagster-university/pages/dagster-dbt/lesson-3/3-representing-the-dbt-project-in-dagster.md index 3cd80ce31fba1..37b8232360333 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-3/3-representing-the-dbt-project-in-dagster.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-3/3-representing-the-dbt-project-in-dagster.md @@ -8,7 +8,7 @@ lesson: '3' As you’ll frequently point your Dagster code to the `target/manifest.json` file and your dbt project in this course, it’ll be helpful to keep a reusable representation of the dbt project. This can be easily done using the `DbtProject` class. -In the `assets` directory, create a new `dbt.py` file and add the following imports: +In the `dagster_university` directory, create a new `project.py` file and add the following imports: ```python from pathlib import Path @@ -22,7 +22,7 @@ After the import, add the following code: ```python dbt_project = DbtProject( - project_dir=Path(__file__).joinpath("..", "..", "..", "analytics").resolve(), + project_dir=Path(__file__).joinpath("..", "..", "analytics").resolve(), ) ``` diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-3/4-creating-a-dbt-resource-in-dagster.md b/docs/dagster-university/pages/dagster-dbt/lesson-3/4-creating-a-dbt-resource-in-dagster.md index d7bba3af0a570..fb3cfdf1e1812 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-3/4-creating-a-dbt-resource-in-dagster.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-3/4-creating-a-dbt-resource-in-dagster.md @@ -20,7 +20,7 @@ Navigate to the `dagster_university/resources/__init__.py`, which is where other ```python from dagster_dbt import DbtCliResource -from ..assets.dbt import dbt_project +from ..project import dbt_project # the import lines go at the top of the file # this can be defined anywhere below the imports diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-3/5-loading-dbt-models-into-dagster-as-assets.md b/docs/dagster-university/pages/dagster-dbt/lesson-3/5-loading-dbt-models-into-dagster-as-assets.md index bfb1f7bff20c7..19d4fb7140226 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-3/5-loading-dbt-models-into-dagster-as-assets.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-3/5-loading-dbt-models-into-dagster-as-assets.md @@ -28,13 +28,15 @@ We’ll only create one `@dbt_assets` definition for now, but in a later lesson, ## Loading the models as assets -1. Open the `assets/dbt.py` file. +1. Create a new file in the `assets` directory called `dbt.py`. 2. Add the following imports to the top of the file: ```python from dagster import AssetExecutionContext from dagster_dbt import dbt_assets, DbtCliResource + + from ..project import dbt_project ``` 3. Next, we'll use the `@dbt_assets` decorator to create a new asset function and provide it with a reference to the project's manifest file: @@ -66,14 +68,10 @@ We’ll only create one `@dbt_assets` definition for now, but in a later lesson, At this point, `dbt.py` should look like this: ```python -from pathlib import Path - from dagster import AssetExecutionContext -from dagster_dbt import DbtCliResource, DbtProject, dbt_assets +from dagster_dbt import DbtCliResource, dbt_assets -dbt_project = DbtProject( - project_dir=Path(__file__).joinpath("..", "..", "..", "analytics").resolve(), -) +from ..project import dbt_project @dbt_assets( diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-4/2-speeding-up-the-development-cycle.md b/docs/dagster-university/pages/dagster-dbt/lesson-4/2-speeding-up-the-development-cycle.md index 035ec6f26f5ff..5b31e3659a05a 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-4/2-speeding-up-the-development-cycle.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-4/2-speeding-up-the-development-cycle.md @@ -16,7 +16,7 @@ Before we move on, we’ll reduce the number of steps in the feedback loop. We'l The first detail is that the `dbt_project` doesn’t need to be part of an asset to be executed. This means that once a `dbt_project` is defined, you can use it to execute commands when your code location is being built. Rather than manually running `dbt parse`, let’s use the `dbt_project` to prepare the manifest file for us. -In `dbt.py`, after the code initializing `dbt_project`, add the following code: +In `project.py`, after the code initializing `dbt_project`, add the following code: ```python dbt_project.prepare_if_dev() diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-5/2-connecting-dbt-models-to-dagster-assets.md b/docs/dagster-university/pages/dagster-dbt/lesson-5/2-connecting-dbt-models-to-dagster-assets.md index ea401ed2262d8..6f0851e9b674f 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-5/2-connecting-dbt-models-to-dagster-assets.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-5/2-connecting-dbt-models-to-dagster-assets.md @@ -105,15 +105,10 @@ Open the `assets/dbt.py` file and do the following: At this point, your `dbt.py` file should match the following: ```python -from pathlib import Path - from dagster import AssetExecutionContext, AssetKey -from dagster_dbt import DagsterDbtTranslator, DbtCliResource, DbtProject, dbt_assets +from dagster_dbt import DagsterDbtTranslator, DbtCliResource, dbt_assets -dbt_project = DbtProject( - project_dir=Path(__file__).joinpath("..", "..", "..", "analytics").resolve(), -) -dbt_project.prepare_if_dev() +from ..project import dbt_project class CustomizedDagsterDbtTranslator(DagsterDbtTranslator): diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-6/3-creating-a-partitioned-dbt-asset.md b/docs/dagster-university/pages/dagster-dbt/lesson-6/3-creating-a-partitioned-dbt-asset.md index 70e52a2acd617..98df443878c20 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-6/3-creating-a-partitioned-dbt-asset.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-6/3-creating-a-partitioned-dbt-asset.md @@ -122,20 +122,15 @@ At this point, the `dagster_university/assets/dbt.py` file should look like this ```python import json -from pathlib import Path from dagster import AssetExecutionContext, AssetKey -from dagster_dbt import DagsterDbtTranslator, DbtCliResource, DbtProject, dbt_assets +from dagster_dbt import DagsterDbtTranslator, DbtCliResource, dbt_assets from ..partitions import daily_partition +from ..project import dbt_project INCREMENTAL_SELECTOR = "config.materialized:incremental" -dbt_project = DbtProject( - project_dir=Path(__file__).joinpath("..", "..", "..", "analytics").resolve(), -) -dbt_project.prepare_if_dev() - class CustomizedDagsterDbtTranslator(DagsterDbtTranslator): def get_asset_key(self, dbt_resource_props): diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-7/4-creating-the-manifest-during-deployment.md b/docs/dagster-university/pages/dagster-dbt/lesson-7/4-creating-the-manifest-during-deployment.md index 60378031a1051..762413ea6c33f 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-7/4-creating-the-manifest-during-deployment.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-7/4-creating-the-manifest-during-deployment.md @@ -26,7 +26,7 @@ To get our deployment working, we need to add a step to our GitHub Actions workf pip install pip --upgrade cd project-repo pip install . --upgrade --upgrade-strategy eager - dagster-dbt project prepare-and-package --file dagster_university/assets/dbt.py + dagster-dbt project prepare-and-package --file dagster_university/project.py shell: bash ``` diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-7/5-preparing-for-a-successful-run.md b/docs/dagster-university/pages/dagster-dbt/lesson-7/5-preparing-for-a-successful-run.md index b544eef9bee74..d7db4993d1e0b 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-7/5-preparing-for-a-successful-run.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-7/5-preparing-for-a-successful-run.md @@ -63,7 +63,7 @@ Because we’re still using a DuckDB-backed database, our `type` will also be `d ## Adding a prod target to DbtProject -Next, we need to update the `DbtProject` object in `dagster_university/assets/dbt.py` to specify what profile to target. To optimize the developer experience, let’s use an environment variable to specify the profile to target. +Next, we need to update the `DbtProject` object in `dagster_university/project.py` to specify what profile to target. To optimize the developer experience, let’s use an environment variable to specify the profile to target. 1. In the `.env` file, define an environment variable named `DBT_TARGET` and set it to `dev`: @@ -71,7 +71,7 @@ Next, we need to update the `DbtProject` object in `dagster_university/assets/db DBT_TARGET=dev ``` -2. Next, import the `os` module at the top of the `dbt.py` file so the environment variable is accessible: +2. Next, import the `os` module at the top of the `project.py` file so the environment variable is accessible: ```python import os @@ -81,7 +81,7 @@ Next, we need to update the `DbtProject` object in `dagster_university/assets/db ```python dbt_project = DbtProject( - project_dir=Path(__file__).joinpath("..", "..", "..", "analytics").resolve(), + project_dir=Path(__file__).joinpath("..", "..", "analytics").resolve(), target=os.getenv("DBT_TARGET") ) ```