Skip to content

Commit

Permalink
Update job-memory-limits.md (#5791)
Browse files Browse the repository at this point in the history
  • Loading branch information
mirnawong1 authored Jul 25, 2024
1 parent 054e6f6 commit 022efc3
Showing 1 changed file with 7 additions and 4 deletions.
11 changes: 7 additions & 4 deletions website/docs/faqs/Troubleshooting/job-memory-limits.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,15 @@ Some common reasons for higher memory usage are:

## Resolution

Try the following to resolve this:
There are various reasons why you could be experiencing this error. We recommend you review your data models to see if there are any opportunities to optimize or refactor them. For example, you can try to reduce the number of columns being selected, use `group` or `where` clauses to filter data early in the query, or use `limit` clauses to reduce the amount of data being processed.

1. **Use incremental models**: Try using [incremental models](/docs/build/incremental-models-overview) to reduce the amount of data being processed in each run. Incremental models only process new or updated data, which can help reduce the memory usage of your jobs.
2. **Refactor your data model**: Review your data models to see if there are any opportunities to optimize or refactor them. For example, you can try to reduce the number of columns being selected, use `where` clauses to filter data early in the query or use `limit` clauses to reduce the amount of data being processed.
<!-- add url link when available
:::tip Check out video example
As an additional resource, check out this example video add URL here, which demonstrates how to refactor the sample code by reducing the number of columns returned.
:::
-->

If you've tried the earlier suggestions and are still experiencing failed job runs with this error about hitting the memory limits of your account, please [reach out to support](mailto:[email protected]) and we can try increasing your account's memory. We're happy to help!
If you've tried the earlier suggestions and are still experiencing failed job runs with this error about hitting the memory limits of your account, please [reach out to support](mailto:[email protected]). We're happy to help!

## Additional resources
- [Blog post on how we shaved 90 mins off](https://docs.getdbt.com/blog/how-we-shaved-90-minutes-off-model)

0 comments on commit 022efc3

Please sign in to comment.