Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug with calculation of last_run stats for incremental models #384

Merged
merged 1 commit into from
Sep 21, 2023

Conversation

glsdown
Copy link
Contributor

@glsdown glsdown commented Sep 18, 2023

Overview

As noted in #372 the way that last_run_* stats are calculated in dim_dbt__current_models is incorrect. This aims to fix them, as well as provide 4 additional fields:

  • last_incremental_run_completed_at
  • last_incremental_run_total_runtime
  • last_incremental_run_rows_affected
  • last_incremental_run_bytes_processed - BQ only

Update type - breaking / non-breaking

  • Minor bug fix
  • Documentation improvements
  • Quality of Life improvements
  • New features (non-breaking change)
  • New features (breaking change)
  • Other (non-breaking change)
  • Other (breaking change)
  • Release preparation

What does this solve?

Outstanding questions

What databases have you tested with?

  • Snowflake
  • Google BigQuery
  • Databricks
  • Spark
  • N/A

@glsdown glsdown added the bug Something isn't working label Sep 18, 2023
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:25 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:25 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:25 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:29 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:29 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:29 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:29 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:29 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:29 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:29 — with GitHub Actions Inactive
@glsdown glsdown temporarily deployed to Approve Integration Tests September 18, 2023 15:29 — with GitHub Actions Inactive
@glsdown glsdown merged commit 3bde046 into main Sep 21, 2023
12 checks passed
@glsdown glsdown deleted the fix/fix-last-run-stats-incremental branch September 21, 2023 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Invalid rows_affected, bytes_processed and runtime on incremental models in current_models
2 participants