Skip to content

Commit

Permalink
Merge branch 'current' into ly-docs-preview-adv-ci
Browse files Browse the repository at this point in the history
  • Loading branch information
nghi-ly authored Aug 7, 2024
2 parents 0115497 + 8054b9e commit e90a865
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 27 deletions.
27 changes: 5 additions & 22 deletions website/docs/reference/node-selection/syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,36 +193,19 @@ The state and result selectors can also be combined in a single invocation of db
dbt run --select "result:<status>+" state:modified+ --defer --state ./<dbt-artifact-path>
```

### Fresh rebuilds

Only supported by v1.1 or newer.

When a job is selected, dbt Cloud will surface the artifacts from that job's most recent successful run. dbt will then use those artifacts to determine the set of fresh sources. In your job commands, you can signal to dbt to run and test only on these fresher sources and their children by including the `source_status:fresher+` argument. This requires both previous and current state to have the `sources.json` artifact be available. Or plainly said, both job states need to run `dbt source freshness`.

As example:

```bash
# Command step order
dbt source freshness
dbt build --select "source_status:fresher+"
```


For more example commands, refer to [Pro-tips for workflows](/best-practices/best-practice-workflows#pro-tips-for-workflows).

### The "source_status" status

Only supported by v1.1 or newer.

Another element of job state is the `source_status` of a prior dbt invocation. After executing `dbt source freshness`, for example, dbt creates the `sources.json` artifact which contains execution times and `max_loaded_at` dates for dbt sources. You can read more about `sources.json` on the ['sources'](/reference/artifacts/sources-json) page.

The following dbt commands produce `sources.json` artifacts whose results can be referenced in subsequent dbt invocations:
- `dbt source freshness`
The `dbt source freshness` command produces a `sources.json` artifact whose results can be referenced in subsequent dbt invocations.

When a job is selected, dbt Cloud will surface the artifacts from that job's most recent successful run. dbt will then use those artifacts to determine the set of fresh sources. In your job commands, you can signal dbt to run and test only on the fresher sources and their children by including the `source_status:fresher+` argument. This requires both the previous and current states to have the `sources.json` artifact available. Or plainly said, both job states need to run `dbt source freshness`.

After issuing one of the above commands, you can reference the source freshness results by adding a selector to a subsequent command as follows:
After issuing the `dbt source freshness` command, you can reference the source freshness results by adding a selector to a subsequent command:

```bash
# You can also set the DBT_ARTIFACT_STATE_PATH environment variable instead of the --state flag.
dbt source freshness # must be run again to compare current to previous state
dbt build --select "source_status:fresher+" --state path/to/prod/artifacts
```
For more example commands, refer to [Pro-tips for workflows](/best-practices/best-practice-workflows#pro-tips-for-workflows).
12 changes: 7 additions & 5 deletions website/docs/reference/resource-configs/redshift-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,31 +27,33 @@ All of these strategies are inherited from dbt-postgres.
Tables in Amazon Redshift have two powerful optimizations to improve query performance: distkeys and sortkeys. Supplying these values as model-level configurations apply the corresponding settings in the generated `CREATE TABLE` <Term id="ddl" />. Note that these settings will have no effect on models set to `view` or `ephemeral` models.

- `dist` can have a setting of `all`, `even`, `auto`, or the name of a key.
- `sort` accepts a list of sort keys, for example: `['timestamp', 'userid']`. dbt will build the sort key in the same order the fields are supplied.
- `sort` accepts a list of sort keys, for example: `['reporting_day', 'category']`. dbt will build the sort key in the same order the fields are supplied.
- `sort_type` can have a setting of `interleaved` or `compound`. if no setting is specified, sort_type defaults to `compound`.

When working with sort keys, it's highly recommended you follow [Redshift's best practices](https://docs.aws.amazon.com/prescriptive-guidance/latest/query-best-practices-redshift/best-practices-tables.html#sort-keys) on sort key effectiveness and cardinality.

Sort and dist keys should be added to the `{{ config(...) }}` block in model `.sql` files, eg:

<File name='my_model.sql'>

```sql
-- Example with one sort key
{{ config(materialized='table', sort='id', dist='received_at') }}
{{ config(materialized='table', sort='reporting_day', dist='unique_id') }}

select ...


-- Example with multiple sort keys
{{ config(materialized='table', sort=['id', 'category'], dist='received_at') }}
{{ config(materialized='table', sort=['category', 'region', 'reporting_day'], dist='received_at') }}

select ...


-- Example with interleaved sort keys
{{ config(materialized='table',
sort_type='interleaved'
sort=['id', 'category'],
dist='received_at')
sort=['category', 'region', 'reporting_day'],
dist='unique_id')
}}

select ...
Expand Down

0 comments on commit e90a865

Please sign in to comment.