-
Notifications
You must be signed in to change notification settings - Fork 978
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updates for unit testing docs #6345
base: current
Are you sure you want to change the base?
Changes from all commits
a3af151
19ff5b2
69715ac
8488b99
4c8513a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -29,6 +29,7 @@ | |
- Unit tests must be defined in a YML file in your `models/` directory. | ||
- Table names must be [aliased](/docs/build/custom-aliases) in order to unit test `join` logic. | ||
- Redshift customers need to be aware of a [limitation when building unit tests](/reference/resource-configs/redshift-configs#unit-test-limitations) that requires a workaround. | ||
- All references (`ref()`) used in your model must be included in the unit test configuration as input fixtures, even if they do not directly affect the logic being tested. If these references are missing, you may encounter "node not found" errors during compilation. | ||
|
||
Read the [reference doc](/reference/resource-properties/unit-tests) for more details about formatting your unit tests. | ||
|
||
|
@@ -56,6 +57,8 @@ | |
|
||
## Unit testing a model | ||
|
||
When defining mock data for a unit test, it's crucial to include all necessary input values that satisfy the entire model logic. This means including values that fulfill any `WHERE` clauses, `JOIN` conditions, or other constraints present in the model, even if they do not seem directly related to the specific logic being tested. Failing to do so may lead to errors or unexpected null values in the unit test results. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We probably don't want to lead with this paragraph. Instead, we'd probably want to move it somewhere else. Possibly under a new "pitfalls" section? |
||
|
||
This example creates a new `dim_customers` model with a field `is_valid_email_address` that calculates whether or not the customer’s email is valid: | ||
|
||
<file name='dim_customers.sql'> | ||
|
@@ -319,6 +322,36 @@ | |
Learn about [exit codes](/reference/exit-codes) for more information. | ||
|
||
|
||
### Common Pitfalls | ||
Check warning on line 325 in website/docs/docs/build/unit-tests.md GitHub Actions / vale[vale] website/docs/docs/build/unit-tests.md#L325
Raw output
|
||
> - **Missing Fixtures for Referenced Models**: When creating a unit test, all referenced models must be declared as mock inputs. Missing any referenced model, even if it isn't directly involved in the specific logic being tested, will lead to compilation errors such as "node not found." | ||
> - **Not Satisfying `WHERE` or `JOIN` Logic**: Ensure that the mock data meets all conditions in the model, such as `WHERE` clauses or `JOIN` requirements. If these conditions are not met, the unit test will either return null rows or fail to execute properly. This often involves adding rows for auxiliary data tables, like locations or transactions, to satisfy joins and filters. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We'd probably want to give a more detailed example here. Otherwise, it's hard to determine what exactly we mean here. Maybe it would be best to create a post in Discourse with all the details and then link to it? |
||
|
||
### How Unit Tests Compile | ||
Check warning on line 329 in website/docs/docs/build/unit-tests.md GitHub Actions / vale[vale] website/docs/docs/build/unit-tests.md#L329
Raw output
|
||
> During a unit test, dbt creates Common Table Expressions (CTEs) for all dependencies of the model using the mock input data you provide. These CTEs replace the actual references (`ref()`) in the model and allow dbt to run your SQL logic against the mock data. | ||
> | ||
> For example, when you provide a reference such as `ref('stg_transactions')`, dbt creates a CTE named `__dbt__cte__stg_transactions` that contains the mocked data. The entire compiled SQL might look something like this: | ||
> ```sql | ||
> with | ||
> __dbt__cte__stg_transactions as ( | ||
> -- fixture for stg_transactions | ||
> -- contains unions to create "test inputs" corresponding to all rows | ||
> ), | ||
> __dbt__cte__stg_locations as ( | ||
> -- fixture for stg_locations | ||
> -- contains select statement that "mocks" stg_locations | ||
> ), | ||
> applied_donations as ( | ||
> select | ||
> transaction_id, | ||
> sum(cash_value) as donated_cash | ||
> from __dbt__cte__stg_donations | ||
> group by transaction_id | ||
> ) | ||
> select * from __dbt__cte__stg_transactions; | ||
> ``` | ||
> Understanding this process will help ensure you configure your unit tests correctly and avoid common issues. | ||
|
||
|
||
Comment on lines
+329
to
+354
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is interesting implementation detail that helps in troubleshooting when there is some kind of error. In general, this kind of detail doesn't feel like it should go in product docs though because the maintainers are free to change the details at will. Maybe we move it to Markdown docs within the dbt-core or dbt-adapters repo instead? |
||
## Additional resources | ||
|
||
- [Unit testing reference page](/reference/resource-properties/unit-tests) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be
ref
orsource
.See #6331 for a related docs issue asking for a unified super concept that describes both
ref
andsource
calls.