-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add existing Dagster project example to dbt quickstart #21817
Add existing Dagster project example to dbt quickstart #21817
Conversation
This stack of pull requests is managed by Graphite. Learn more about stacking. Join @maximearmstrong and the rest of your teammates on Graphite |
5ff29d3
to
0ddd004
Compare
3753f75
to
58b6518
Compare
|
||
### Option 2: Use an existing Dagster project | ||
|
||
You can use an existing Dagster project to run your dbt project. To do so, you'll need to add some objects using the [dagster-dbt library](/\_apidocs/libraries/dagster-dbt) and update your `Definitions` object. This example assumes that your existing Dagster project includes both `assets.py` and `definitions.py` files. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should show the file structure that we expect as a directory tree to be explicit about where we expect files to be.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in ce9d1a1
assets=[my_dbt_assets], | ||
resources={ | ||
"dbt": DbtCliResource(project_dir=my_project), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should have placeholders for their existing assets, jobs, resources, etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in ce9d1a1
@@ -58,6 +58,67 @@ dagster-dbt project scaffold --project-name my-dagster-project --dbt-project-dir | |||
|
|||
This creates a directory called `my-dagster-project/` inside the current directory. The `my-dagster-project/` directory contains a set of files that define a Dagster project. | |||
|
|||
</TabItem> | |||
<TabItem name="Option 2: Use an existing Dagster project"> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
<TabItem name="Option 2: Use an existing Dagster project"> | |
<TabItem name="Option 2: Load your dbt project in an existing Dagster project"> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in ce9d1a1
0ddd004
to
6e2bb9f
Compare
58b6518
to
76415f8
Compare
6e2bb9f
to
b75e81f
Compare
76415f8
to
2878c6f
Compare
b75e81f
to
ca9f008
Compare
ce9d1a1
to
a5952c4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See comment on how to format the placeholder
defs = Definitions( | ||
assets=[ | ||
# your other assets here, | ||
my_dbt_assets | ||
], | ||
jobs=[ | ||
# your jobs here | ||
], | ||
resources={ | ||
# your other resources here, | ||
"dbt": DbtCliResource(project_dir=my_project), | ||
}, | ||
) | ||
# end_dbt_definitions_example |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use ...
defs = Definitions( | |
assets=[ | |
# your other assets here, | |
my_dbt_assets | |
], | |
jobs=[ | |
# your jobs here | |
], | |
resources={ | |
# your other resources here, | |
"dbt": DbtCliResource(project_dir=my_project), | |
}, | |
) | |
# end_dbt_definitions_example | |
defs = Definitions( | |
..., | |
assets=[ | |
..., | |
# Add the dbt assets alongside your other assets | |
my_dbt_assets | |
], | |
resources={ | |
... | |
# Add the dbt resource alongside your other resources | |
"dbt": DbtCliResource(project_dir=my_project), | |
}, | |
) | |
# end_dbt_definitions_example |
a5952c4
to
0fd3a50
Compare
927bbc5
to
cb5b70d
Compare
0fd3a50
to
d86c381
Compare
b79360c
into
maxime/ds-169/add-full-copy-pasteable-example-to-dbt-integrate-page
Summary & Motivation
This PR updates the quickstart example for dbt + Dagster - it adds the steps to update an existing Dagster project to wrap a dbt project.
This work is the continuation of #21240 and #21816
How I Tested These Changes
BK
+
local