Skip to content

Commit

Permalink
[daggy-u- [dbt] - Minor corrections (#20103)
Browse files Browse the repository at this point in the history
## Summary & Motivation

## How I Tested These Changes

---------

Co-authored-by: Tim Castillo <[email protected]>
  • Loading branch information
2 people authored and PedramNavid committed Mar 28, 2024
1 parent 4277064 commit 1f824b5
Show file tree
Hide file tree
Showing 7 changed files with 43 additions and 6 deletions.
1 change: 1 addition & 0 deletions docs/dagster-university/pages/dagster-dbt.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ title: Dagster + dbt
- [Connecting dbt models to Dagster assets](/dagster-dbt/lesson-5/2-connecting-dbt-models-to-dagster-assets)
- [Creating assets that depend on dbt models](/dagster-dbt/lesson-5/3-creating-assets-that-depend-on-dbt-models)
- [Automating dbt models in Dagster](/dagster-dbt/lesson-5/4-automating-dbt-models-in-dagster)
- [Practice: Grouping dbt models by layer](/dagster-dbt/lesson-5/coding-practice-grouping-models)
- Lesson 6: Using Dagster to partition dbt models
- [Overview](/dagster-dbt/lesson-6/1-overview)
- [Creating an incremental model](/dagster-dbt/lesson-6/2-creating-a-simple-incremental-model)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,5 @@ Even if you’ve already completed the Dagster Essentials course, you should sti
Run the following to clone the project:

```bash
git clone https://github.com/dagster-io/project-dagster-university -b module/dagster-and-dbt/starter
git clone https://github.com/dagster-io/project-dagster-university -b module/dagster-and-dbt-starter
```
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ The `Path` class from the `pathlib` standard library will help us create an accu
DBT_DIRECTORY = Path(__file__).joinpath("..", "..", "..", "analytics").resolve()
```

TODO: Tim to update as these instructions as the old instructions use `os.path.join`

This line creates a new constant called `DBT_DIRECTORY`. This line might look a little complicated, so let’s break it down:

- It uses `constants.py`'s file location (via `__file__`) as a point of reference for finding the dbt project
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,10 @@ lesson: '3'

3. In the Dagster UI, re-materialize both of the dbt models.

4. Navigate to the details page for the run you just started.

5. Navigate to the logs for the run.
4. Navigate to the details page for the run you just started, then look at the logs.

When finished, proceed to the next page.

{% callout %}
> **Important!** Before continuing, change `dbt_analytics` back to use `dbt run`.
{% /callout %}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
title: 'Practice: Grouping dbt models by layer'
module: 'dagster_essentials'
lesson: '5'
---

# Practice: Grouping dbt models by layer

Override the `get_group_name` method in your `CustomizedDagsterDbtTranslator` to group each dbt model by their layer (`marts` and `staging`).

**Hint:** `dbt_resource_props`

---

## Check your work

The method you built should look similar to the following code. Click **View answer** to view it.

**If there are differences**, compare what you wrote to the code below and change them, as this method will be used as-is in future lessons.

```python {% obfuscated="true" %}
@classmethod
def get_group_name(cls, dbt_resource_props):
return dbt_resource_props["fqn"][1]
```
Original file line number Diff line number Diff line change
Expand Up @@ -154,11 +154,13 @@ Finally, we’ll modify the `daily_metrics.sql` file to reflect that dbt knows w

In `analytics/models/marts/daily_metrics.sql`, update the model's incremental logic to the following:

`{% if is_incremental() %}`
```python
{% if is_incremental() %}
where date_of_business >= strptime('{{ var('min_date') }}', '%c') and date_of_business < strptime('{{ var('max_date') }}', '%c')
{% endif %}
```
`{% endif %}`

**TODO: AWARE THIS IS BROKEN - NEEDS A MARKDOC FIX**

Here, we’ve changed the logic to say that we only want to select rows between the `min_date` and the `max_date`. Note that we are turning the variables into timestamps using `strptime` because they’re loaded as strings.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,11 @@ The following table contains the environment variables we need to create in Dags
- `AWS_REGION`
- The region the S3 bucket is located in.

---

- `S3_BUCKET_NAME`
- The name of the S3 bucket, by default `"s3://dagster-university/"`

{% /table %}

### Creating environment variables
Expand Down

0 comments on commit 1f824b5

Please sign in to comment.