From 1f824b5eef73bf3c21a1ef8b57e418383c6430d4 Mon Sep 17 00:00:00 2001 From: Erin Cochran Date: Tue, 27 Feb 2024 15:56:24 -0500 Subject: [PATCH] [daggy-u- [dbt] - Minor corrections (#20103) ## Summary & Motivation ## How I Tested These Changes --------- Co-authored-by: Tim Castillo --- docs/dagster-university/pages/dagster-dbt.md | 1 + .../dagster-dbt/lesson-2/1-requirements.md | 2 +- ...ing-the-dbt-project-location-in-dagster.md | 2 ++ .../dagster-dbt/lesson-3/knowledge-check.md | 8 +++--- .../coding-practice-grouping-models.md | 25 +++++++++++++++++++ .../3-creating-a-partitioned-dbt-asset.md | 6 +++-- .../5-preparing-for-a-successful-run.md | 5 ++++ 7 files changed, 43 insertions(+), 6 deletions(-) create mode 100644 docs/dagster-university/pages/dagster-dbt/lesson-5/coding-practice-grouping-models.md diff --git a/docs/dagster-university/pages/dagster-dbt.md b/docs/dagster-university/pages/dagster-dbt.md index 72542cdbc3574..ac6bad04cb661 100644 --- a/docs/dagster-university/pages/dagster-dbt.md +++ b/docs/dagster-university/pages/dagster-dbt.md @@ -34,6 +34,7 @@ title: Dagster + dbt - [Connecting dbt models to Dagster assets](/dagster-dbt/lesson-5/2-connecting-dbt-models-to-dagster-assets) - [Creating assets that depend on dbt models](/dagster-dbt/lesson-5/3-creating-assets-that-depend-on-dbt-models) - [Automating dbt models in Dagster](/dagster-dbt/lesson-5/4-automating-dbt-models-in-dagster) + - [Practice: Grouping dbt models by layer](/dagster-dbt/lesson-5/coding-practice-grouping-models) - Lesson 6: Using Dagster to partition dbt models - [Overview](/dagster-dbt/lesson-6/1-overview) - [Creating an incremental model](/dagster-dbt/lesson-6/2-creating-a-simple-incremental-model) diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-2/1-requirements.md b/docs/dagster-university/pages/dagster-dbt/lesson-2/1-requirements.md index 92a5d882427a0..1f5c952ea35c2 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-2/1-requirements.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-2/1-requirements.md @@ -30,5 +30,5 @@ Even if you’ve already completed the Dagster Essentials course, you should sti Run the following to clone the project: ```bash -git clone https://github.com/dagster-io/project-dagster-university -b module/dagster-and-dbt/starter +git clone https://github.com/dagster-io/project-dagster-university -b module/dagster-and-dbt-starter ``` \ No newline at end of file diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-3/3-defining-the-dbt-project-location-in-dagster.md b/docs/dagster-university/pages/dagster-dbt/lesson-3/3-defining-the-dbt-project-location-in-dagster.md index 611ebd9e119fa..a86895867b221 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-3/3-defining-the-dbt-project-location-in-dagster.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-3/3-defining-the-dbt-project-location-in-dagster.md @@ -21,6 +21,8 @@ The `Path` class from the `pathlib` standard library will help us create an accu DBT_DIRECTORY = Path(__file__).joinpath("..", "..", "..", "analytics").resolve() ``` +TODO: Tim to update as these instructions as the old instructions use `os.path.join` + This line creates a new constant called `DBT_DIRECTORY`. This line might look a little complicated, so let’s break it down: - It uses `constants.py`'s file location (via `__file__`) as a point of reference for finding the dbt project diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-3/knowledge-check.md b/docs/dagster-university/pages/dagster-dbt/lesson-3/knowledge-check.md index d401fda83e85e..6b6820279e487 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-3/knowledge-check.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-3/knowledge-check.md @@ -20,8 +20,10 @@ lesson: '3' 3. In the Dagster UI, re-materialize both of the dbt models. -4. Navigate to the details page for the run you just started. - -5. Navigate to the logs for the run. +4. Navigate to the details page for the run you just started, then look at the logs. When finished, proceed to the next page. + +{% callout %} +> **Important!** Before continuing, change `dbt_analytics` back to use `dbt run`. +{% /callout %} diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-5/coding-practice-grouping-models.md b/docs/dagster-university/pages/dagster-dbt/lesson-5/coding-practice-grouping-models.md new file mode 100644 index 0000000000000..ef83729ff2571 --- /dev/null +++ b/docs/dagster-university/pages/dagster-dbt/lesson-5/coding-practice-grouping-models.md @@ -0,0 +1,25 @@ +--- +title: 'Practice: Grouping dbt models by layer' +module: 'dagster_essentials' +lesson: '5' +--- + +# Practice: Grouping dbt models by layer + +Override the `get_group_name` method in your `CustomizedDagsterDbtTranslator` to group each dbt model by their layer (`marts` and `staging`). + +**Hint:** `dbt_resource_props` + +--- + +## Check your work + +The method you built should look similar to the following code. Click **View answer** to view it. + +**If there are differences**, compare what you wrote to the code below and change them, as this method will be used as-is in future lessons. + +```python {% obfuscated="true" %} +@classmethod +def get_group_name(cls, dbt_resource_props): + return dbt_resource_props["fqn"][1] +``` \ No newline at end of file diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-6/3-creating-a-partitioned-dbt-asset.md b/docs/dagster-university/pages/dagster-dbt/lesson-6/3-creating-a-partitioned-dbt-asset.md index 7aec98b549d0b..854c4059f7310 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-6/3-creating-a-partitioned-dbt-asset.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-6/3-creating-a-partitioned-dbt-asset.md @@ -154,11 +154,13 @@ Finally, we’ll modify the `daily_metrics.sql` file to reflect that dbt knows w In `analytics/models/marts/daily_metrics.sql`, update the model's incremental logic to the following: +`{% if is_incremental() %}` ```python -{% if is_incremental() %} where date_of_business >= strptime('{{ var('min_date') }}', '%c') and date_of_business < strptime('{{ var('max_date') }}', '%c') -{% endif %} ``` +`{% endif %}` + +**TODO: AWARE THIS IS BROKEN - NEEDS A MARKDOC FIX** Here, we’ve changed the logic to say that we only want to select rows between the `min_date` and the `max_date`. Note that we are turning the variables into timestamps using `strptime` because they’re loaded as strings. diff --git a/docs/dagster-university/pages/dagster-dbt/lesson-7/5-preparing-for-a-successful-run.md b/docs/dagster-university/pages/dagster-dbt/lesson-7/5-preparing-for-a-successful-run.md index b833b87ea70eb..64c7e2195ea30 100644 --- a/docs/dagster-university/pages/dagster-dbt/lesson-7/5-preparing-for-a-successful-run.md +++ b/docs/dagster-university/pages/dagster-dbt/lesson-7/5-preparing-for-a-successful-run.md @@ -121,6 +121,11 @@ The following table contains the environment variables we need to create in Dags - `AWS_REGION` - The region the S3 bucket is located in. +--- + +- `S3_BUCKET_NAME` +- The name of the S3 bucket, by default `"s3://dagster-university/"` + {% /table %} ### Creating environment variables