Skip to content

Commit

Permalink
[docs] add dbt code references guide
Browse files Browse the repository at this point in the history
  • Loading branch information
benpankow committed Jun 6, 2024
1 parent d4e8f7b commit 918f3c4
Show file tree
Hide file tree
Showing 5 changed files with 70 additions and 1 deletion.
Binary file modified docs/content/api/modules.json.gz
Binary file not shown.
Binary file modified docs/content/api/searchindex.json.gz
Binary file not shown.
Binary file modified docs/content/api/sections.json.gz
Binary file not shown.
41 changes: 40 additions & 1 deletion docs/content/guides/dagster/code-references.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ To complete the steps in this guide, you'll need:

## Automatically attaching code references to asset definitions

To automatically attach code references to the asset's function definition, you can use the `dagster._core.definitions.metadata.with_source_code_references` utility. Any asset definitions passed to the utility will have their source file attached as metadata.
### Assets defined in Python

To automatically attach code references to Python assets' function definitions, you can use the `dagster._core.definitions.metadata.with_source_code_references` utility. Any asset definitions passed to the utility will have their source file attached as metadata.

```python file=/guides/dagster/code_references/with_source_code_references.py
from dagster import Definitions, asset
Expand Down Expand Up @@ -58,6 +60,43 @@ A link to the asset's source in `with_source_code_references.py` will then be vi
/>
</center>

### dbt assets

Dagster's dbt integration can automatically attach references to the SQL files backing your dbt assets. To enable this feature, set the `enable_code_references` parameter to `True` in the <PyObject module="dagster_dbt" object="DagsterDbtTranslatorSettings" /> passed to your <PyObject module="dagster_dbt" object="DagsterDbtTranslator" />:

```python file=/guides/dagster/code_references/with_dbt_code_references.py
from pathlib import Path

from dagster_dbt import (
DagsterDbtTranslator,
DagsterDbtTranslatorSettings,
DbtCliResource,
dbt_assets,
)

from dagster import AssetExecutionContext, Definitions
from dagster._core.definitions.metadata import with_source_code_references

manifest_path = Path("path/to/dbt_project/target/manifest.json")

# links to dbt model source code from assets
dagster_dbt_translator = DagsterDbtTranslator(
settings=DagsterDbtTranslatorSettings(enable_code_references=True)
)


@dbt_assets(
manifest=manifest_path,
dagster_dbt_translator=dagster_dbt_translator,
)
def my_dbt_assets(context: AssetExecutionContext, dbt: DbtCliResource):
yield from dbt.cli(["build"], context=context).stream()


# optionally, add references to the Python source with with_source_code_references
defs = Definitions(assets=with_source_code_references([my_dbt_assets]))
```

---

## Manually attaching code references to asset definitions
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
from pathlib import Path

from dagster_dbt import (
DagsterDbtTranslator,
DagsterDbtTranslatorSettings,
DbtCliResource,
dbt_assets,
)

from dagster import AssetExecutionContext, Definitions
from dagster._core.definitions.metadata import with_source_code_references

manifest_path = Path("path/to/dbt_project/target/manifest.json")

# links to dbt model source code from assets
dagster_dbt_translator = DagsterDbtTranslator(
settings=DagsterDbtTranslatorSettings(enable_code_references=True)
)


@dbt_assets(
manifest=manifest_path,
dagster_dbt_translator=dagster_dbt_translator,
)
def my_dbt_assets(context: AssetExecutionContext, dbt: DbtCliResource):
yield from dbt.cli(["build"], context=context).stream()


# optionally, add references to the Python source with with_source_code_references
defs = Definitions(assets=with_source_code_references([my_dbt_assets]))

0 comments on commit 918f3c4

Please sign in to comment.