Skip to content

Commit

Permalink
[docs] add dbt code references guide (#22318)
Browse files Browse the repository at this point in the history
## Summary

Brief expansion of the code references guide to include info on generating dbt code references.

## Test Plan

vercel
  • Loading branch information
benpankow authored Jul 8, 2024
1 parent 25b79c5 commit 4b30462
Show file tree
Hide file tree
Showing 6 changed files with 74 additions and 1 deletion.
Binary file modified docs/content/api/modules.json.gz
Binary file not shown.
Binary file modified docs/content/api/searchindex.json.gz
Binary file not shown.
Binary file modified docs/content/api/sections.json.gz
Binary file not shown.
8 changes: 7 additions & 1 deletion docs/content/guides/dagster/code-references.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ To complete the steps in this guide, you'll need:

## Automatically attaching code references to asset definitions

To automatically attach code references to the asset's function definition, you can use the `dagster._core.definitions.metadata.with_source_code_references` utility. Any asset definitions passed to the utility will have their source file attached as metadata.
### Assets defined in Python

To automatically attach code references to Python assets' function definitions, you can use the `dagster._core.definitions.metadata.with_source_code_references` utility. Any asset definitions passed to the utility will have their source file attached as metadata.

```python file=/guides/dagster/code_references/with_source_code_references.py
from dagster import Definitions, asset, with_source_code_references
Expand Down Expand Up @@ -57,6 +59,10 @@ A link to the asset's source in `with_source_code_references.py` will then be vi
/>
</center>

### dbt assets

Dagster's dbt integration can automatically attach references to the SQL files backing your dbt assets. For more information, see the [dagster-dbt integration reference](/integrations/dbt/reference#attaching-code-reference-metadata).

---

## Manually attaching code references to asset definitions
Expand Down
37 changes: 37 additions & 0 deletions docs/content/integrations/dbt/reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -520,6 +520,43 @@ def my_dbt_assets(context: AssetExecutionContext, dbt: DbtCliResource):

Dagster also supports fetching additional metadata at dbt execution time to attach to asset materializations. For more information, see the [Customizing asset materialization metadata](#customizing-asset-materialization-metadata) section.

#### Attaching code reference metadata

Dagster's dbt integration can automatically attach [code reference](/guides/dagster/code-references) metadata to the SQL files backing your dbt assets. To enable this feature, set the `enable_code_references` parameter to `True` in the <PyObject module="dagster_dbt" object="DagsterDbtTranslatorSettings" /> passed to your <PyObject module="dagster_dbt" object="DagsterDbtTranslator" />:

```python file=/guides/dagster/code_references/with_dbt_code_references.py
from pathlib import Path
from dagster_dbt import (
DagsterDbtTranslator,
DagsterDbtTranslatorSettings,
DbtCliResource,
DbtProject,
dbt_assets,
)
from dagster import AssetExecutionContext, Definitions, with_source_code_references
my_project = DbtProject(project_dir=Path("path/to/dbt_project"))
# links to dbt model source code from assets
dagster_dbt_translator = DagsterDbtTranslator(
settings=DagsterDbtTranslatorSettings(enable_code_references=True)
)
@dbt_assets(
manifest=my_project.manifest_path,
dagster_dbt_translator=dagster_dbt_translator,
project=my_project,
)
def my_dbt_assets(context: AssetExecutionContext, dbt: DbtCliResource):
yield from dbt.cli(["build"], context=context).stream()
defs = Definitions(assets=with_source_code_references([my_dbt_assets]))
```

### Customizing tags

<Note>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
from pathlib import Path

from dagster_dbt import (
DagsterDbtTranslator,
DagsterDbtTranslatorSettings,
DbtCliResource,
DbtProject,
dbt_assets,
)

from dagster import AssetExecutionContext, Definitions, with_source_code_references

my_project = DbtProject(project_dir=Path("path/to/dbt_project"))

# links to dbt model source code from assets
dagster_dbt_translator = DagsterDbtTranslator(
settings=DagsterDbtTranslatorSettings(enable_code_references=True)
)


@dbt_assets(
manifest=my_project.manifest_path,
dagster_dbt_translator=dagster_dbt_translator,
project=my_project,
)
def my_dbt_assets(context: AssetExecutionContext, dbt: DbtCliResource):
yield from dbt.cli(["build"], context=context).stream()


defs = Definitions(assets=with_source_code_references([my_dbt_assets]))

1 comment on commit 4b30462

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deploy preview for dagster-docs ready!

✅ Preview
https://dagster-docs-j4y6k03m7-elementl.vercel.app
https://master.dagster.dagster-docs.io

Built with commit 4b30462.
This pull request is being automatically deployed with vercel-action

Please sign in to comment.