Skip to content

Commit

Permalink
Merge in the on_configuration_change config on relevant pages (#5940)
Browse files Browse the repository at this point in the history
## What are you changing in this pull request and why?
* I added the order in which materializations are run to the
materialized.md reference page. It's important and helpful information.
I do not know how to make sure the same snippet occurs on multiple doc
pages in a DRY way.
* I added `on_configuration_change` to the possible model configurations
list.
* I added links to the `on_configuration_change` page to other pages
that may have an interaction with it.

I did not:
* Attempt to fully integrate `on_configuration_change` into the doc
pages. I linked to it so it's at least on those pages. If you want to
edit them more, they are easily gathered for you in this PR. I don't
promise it's a complete list.

Questions I think are relevant to include or clarify:
1. What counts as a configuration change? If it's only for materialized
views, then you probably can't mean materialization type changes.
2. How does `on_configuration_change` interact with `--defer`?
3. When you say "model does not exist in the provided path, do you mean
database structure or file structure? Does it understand `identifier`
and `alias`? If it's database structure and the database object name
already exists because it was added manually outside of dbt will it
drop/delete that object even though it has no information about it(and
hence can't know whether it's allowed to have the power to drop/delete
it)?
4. When you say "when the model exists, but has a different type" do you
mean materialization type?
5. "Determine whether to apply the configuration changes" is both vague
and disconcerting. dbt could decide not to follow the
`on_configuration_change` specified by the dbt AE?
6. How does this interact with `grants`?
7. I assume it has no interaction with pre-hooks, post-hooks,
on-run-start, on-run-end, correct?
8. What's the error message and where is it displayed if you specify
`on_configuration_change` and your adapter doesn't support it?

Please go ahead and change it however you want as I'm unlikely to get
back to this PR to make any changes myself in a timely manner.

## Checklist
- [x] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.
- [x] For [docs
versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#about-versioning),
review how to [version a whole
page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
and [version a block of
content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content).
- [x] Add a checklist item for anything that needs to happen before this
PR is merged, such as "needs technical review" or "change base branch."
- [ ] May need technical review as is. Will need technical review if any
of the questions posed have answers added in this PR.

---------

Co-authored-by: Matt Shaver <[email protected]>
  • Loading branch information
alison985 and matthewshaver authored Aug 27, 2024
1 parent 0e5b136 commit f5c7d27
Show file tree
Hide file tree
Showing 5 changed files with 19 additions and 1 deletion.
3 changes: 3 additions & 0 deletions website/docs/reference/model-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ models:
[<resource-path>](/reference/resource-configs/resource-path):
[+](/reference/resource-configs/plus-prefix)[materialized](/reference/resource-configs/materialized): <materialization_name>
[+](/reference/resource-configs/plus-prefix)[sql_header](/reference/resource-configs/sql_header): <string>
[+](/reference/resource-configs/plus-prefix)[on_configuration_change](/reference/resource-configs/on_configuration_change): apply | continue | fail #only for materialized views on supported adapters

```

Expand All @@ -55,6 +56,7 @@ models:
config:
[materialized](/reference/resource-configs/materialized): <materialization_name>
[sql_header](/reference/resource-configs/sql_header): <string>
[on_configuration_change](/reference/resource-configs/on_configuration_change): apply | continue | fail #only for materialized views on supported adapters

```

Expand All @@ -72,6 +74,7 @@ models:
{{ config(
[materialized](/reference/resource-configs/materialized)="<materialization_name>",
[sql_header](/reference/resource-configs/sql_header)="<string>"
[on_configuration_change](/reference/resource-configs/on_configuration_change): apply | continue | fail #only for materialized views for supported adapters
) }}
```
Expand Down
1 change: 1 addition & 0 deletions website/docs/reference/node-selection/defer.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,4 +223,5 @@ dbt will check to see if `dev_alice.model_a` exists. If it doesn't exist, dbt wi
## Related docs

- [Using defer in dbt Cloud](/docs/cloud/about-cloud-develop-defer)
- [on_configuration_change](/reference/resource-configs/on_configuration_change)

3 changes: 3 additions & 0 deletions website/docs/reference/resource-configs/full_refresh.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,3 +85,6 @@ This logic is encoded in the [`should_full_refresh()`](https://github.com/dbt-la

## Recommendation
Set `full_refresh: false` for models of especially large datasets, which you would _never_ want dbt to fully drop and recreate.

## Reference docs
* [on_configuration_change](/reference/resource-configs/on_configuration_change)
11 changes: 11 additions & 0 deletions website/docs/reference/resource-configs/materialized.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,14 @@ select ...

You can also configure [custom materializations](/guides/create-new-materializations?step=1) in dbt. Custom materializations are a powerful way to extend dbt's functionality to meet your specific needs.

## Creation Precedence
<!-- This text is copied from /reference/resource-configs/on_configuration_change.md -->
Materializations are implemented following this "drop through" life cycle:

1. If a model does not exist with the provided path, create the new model.
2. If a model exists, but has a different type, drop the existing model and create the new model.
3. If [`--full-refresh`](/reference/resource-configs/full_refresh) is supplied, replace the existing model regardless of configuration changes and the [`on_configuration_change`](/reference/resource-configs/on_configuration_change) setting.
4. If there are no configuration changes, perform the default action for that type (e.g. apply refresh for a materialized view).
5. Determine whether to apply the configuration changes according to the `on_configuration_change` setting.


Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,6 @@ models:
Materializations are implemented following this "drop through" life cycle:
1. If a model does not exist with the provided path, create the new model.
2. If a model exists, but has a different type, drop the existing model and create the new model.
3. If `--full-refresh` is supplied, replace the existing model regardless of configuration changes and the `on_configuration_change` setting.
3. If [`--full-refresh`](/reference/resource-configs/full_refresh) is supplied, replace the existing model regardless of configuration changes and the `on_configuration_change` setting.
4. If there are no configuration changes, perform the default action for that type (e.g. apply refresh for a materialized view).
5. Determine whether to apply the configuration changes according to the `on_configuration_change` setting.

0 comments on commit f5c7d27

Please sign in to comment.