Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Core] 1.9 upgrade guide #6184

Merged
merged 37 commits into from
Oct 3, 2024
Merged

[Core] 1.9 upgrade guide #6184

merged 37 commits into from
Oct 3, 2024

Conversation

runleonarun
Copy link
Collaborator

@runleonarun runleonarun commented Oct 1, 2024 •
edited by dbeatty10
Loading

Closes #6159

What are you changing in this pull request and why?

First draft of upgrade guide. This is the most current content!

Checklist

  • I have reviewed the Content style guide so my content adheres to these guidelines.
  • The topic I'm writing about is for specific dbt version(s) and I have versioned it according to the version a whole page and/or version a block of content guidelines.
  • I have added checklist item(s) to this list for anything anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch."

@runleonarun runleonarun requested a review from a team as a code owner October 1, 2024 01:01
@vercel Vercel
Copy link

vercel bot commented Oct 1, 2024 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
docs-getdbt-com ✅ Ready (Inspect) Visit Preview Oct 3, 2024 6:04pm

@github-actions github-actions bot added content Improvements or additions to content size: medium This change will take up to a week to address Docs team Authored by the Docs team @dbt Labs labels Oct 1, 2024
@runleonarun runleonarun changed the title adding upgrade guide [Core] 1.9 upgrade guide Oct 1, 2024
jtcohen6
jtcohen6 reviewed Oct 1, 2024
View reviewed changes
Copy link
Collaborator

@jtcohen6 jtcohen6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @runleonarun - we're doing it! 🎉

website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
runleonarun
runleonarun commented Oct 1, 2024
View reviewed changes
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md

While this works for many use-cases, there’s a clear limitation with this approach: *Some datasets are just too big to fit into one query.*

Starting in Core 1.9, you can use the new microbatch strategy to optimize your largest datasets -- **process your event data in discrete periods with their own SQL queries, rather than all at once.** The benefits include:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Starting in Core 1.9, you can use the new microbatch strategy to optimize your largest datasets -- **process your event data in discrete periods with their own SQL queries, rather than all at once.** The benefits include:
Starting in Core 1.9, you can use the new [microbatch strategy](/docs/build/incremental-microbatch) to optimize your largest datasets -- **process your event data in discrete periods with their own SQL queries, rather than all at once.** The benefits include:

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blocked on #6194

graciegoheen
graciegoheen approved these changes Oct 3, 2024
View reviewed changes
Copy link
Collaborator

@graciegoheen graciegoheen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple small comments - but otherwise this is ready to go

website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated Show resolved Hide resolved
@runleonarun runleonarun enabled auto-merge (squash) October 3, 2024 17:59
@runleonarun runleonarun merged commit fc5d053 into current Oct 3, 2024
6 checks passed
@runleonarun runleonarun deleted the upgrade-1.9-guide branch October 3, 2024 18:04
dbeatty10
dbeatty10 reviewed Oct 4, 2024
View reviewed changes
website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated
@@ -71,7 +71,6 @@ Read more about [Snapshots meta fields](/docs/build/snapshots#snapshot-meta-fiel
We’ve made improvements to `state:modified` behaviors to help reduce the risk of false positives and negatives. Read more about [the `state:modified` behavior flags](#managing-changes-to-legacy-behaviors) that leverage these improvements:

- Added environment-aware enhancements for environments where the logic purposefully differs (for example, materializing as a table in `prod` but a `view` in dev).
- Models that use `var` or `env_var` in their definition are included in `state:modified` when their values change.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to #6222

website/docs/docs/dbt-versions/core-upgrade/06-upgrading-to-v1.9.md Outdated
@@ -80,7 +79,6 @@ dbt Core v1.9 has a handful of new flags for [managing changes to legacy behavi
You can read more about each of these behavior changes in the following links:

- (Introduced, disabled by default) [`state_modified_compare_more_unrendered_values`](/reference/global-configs/behavior-changes#behavior-change-flags). Set to `True` to start persisting unrendered_database and unrendered_schema configs during source parsing, and do comparison on unrendered values during `state:modified` checks.
- (Introduced, disabled by default) [`state_modified_compare_vars`](/reference/global-configs/behavior-changes#behavior-change-flags). Set to `True` if a model uses a `var` or `env_var` in its definition. dbt will be able to identify its lineage to include the model in `state:modified` because the var or env_var value has changed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added to #6222

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matthewshaver matthewshaver matthewshaver left review comments

@graciegoheen graciegoheen graciegoheen approved these changes

@amychen1776 amychen1776 amychen1776 left review comments

@dbeatty10 dbeatty10 dbeatty10 left review comments

@jtcohen6 jtcohen6 jtcohen6 left review comments

Assignees
No one assigned
Labels
content Improvements or additions to content Docs team Authored by the Docs team @dbt Labs size: medium This change will take up to a week to address
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Core] Add content to v1.9 upgrade guide for beta
7 participants
@runleonarun @jtcohen6 @dbeatty10 @amychen1776 @graciegoheen @matthewshaver @mirnawong1