Skip to content

Commit

Permalink
Merge branch 'current' into docusaurus-v3
Browse files Browse the repository at this point in the history
  • Loading branch information
JKarlavige authored Aug 7, 2024
2 parents 518bede + fc185c4 commit f817e8b
Show file tree
Hide file tree
Showing 9 changed files with 63 additions and 86 deletions.
18 changes: 11 additions & 7 deletions website/docs/docs/build/custom-aliases.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,22 @@ id: "custom-aliases"

## Overview

When dbt runs a model, it will generally create a relation (either a `table` or a `view`) in the database. By default, dbt uses the filename of the model as the identifier for this relation in the database. This identifier can optionally be overridden using the [`alias`](/reference/resource-configs/alias) model configuration.
When dbt runs a model, it will generally create a relation (either a <Term id="table" /> or a <Term id="view" /> ) in the database, except in the case of an [ephemeral model](/docs/build/materializations), when it will create a <Term id="cte" /> for use in another model. By default, dbt uses the model's filename as the identifier for the relation or CTE it creates. This identifier can be overridden using the [`alias`](/reference/resource-configs/alias) model configuration.

### Why alias model names?
The names of schemas and tables are effectively the "user interface" of your <Term id="data-warehouse" />. Well-named schemas and tables can help provide clarity and direction for consumers of this data. In combination with [custom schemas](/docs/build/custom-schemas), model aliasing is a powerful mechanism for designing your warehouse.

### Usage
The `alias` config can be used to change the name of a model's identifier in the database. The following <Term id="table" /> shows examples of database identifiers for models both with, and without, a supplied `alias`.
The file naming scheme that you use to organize your models may also interfere with your data platform's requirements for identifiers. For example, you might wish to namespace your files using a period (`.`), but your data platform's SQL dialect may interpret periods to indicate a separation between schema names and table names in identifiers, or it may forbid periods from being used at all in CTE identifiers. In cases like these, model aliasing can allow you to retain flexibility in the way you name your model files without violating your data platform's identifier requirements.

| Model | Config | Database Identifier |
| ----- | ------ | ------------------- |
| ga_sessions.sql | &lt;None&gt; | "analytics"."ga_sessions" |
| ga_sessions.sql | \{\{ config(alias='sessions') \}\} | "analytics"."sessions" |
### Usage
The `alias` config can be used to change the name of a model's identifier in the database. The following table shows examples of database identifiers for models both with and without a supplied `alias`, and with different materializations.

| Model | Config | Relation Type | Database Identifier |
| ----- | ------ | --------------| ------------------- |
| ga_sessions.sql | {{ config(materialization='view') }} | <Term id="view" /> | "analytics"."ga_sessions" |
| ga_sessions.sql | {{ config(materialization='view', alias='sessions') }} | <Term id="view" /> | "analytics"."sessions" |
| ga_sessions.sql | {{ config(materialization='ephemeral') }} | <Term id="cte" /> | "\__dbt\__cte\__ga_sessions" |
| ga_sessions.sql | {{ config(materialization='ephemeral', alias='sessions') }} | <Term id="cte" /> | "\__dbt\__cte\__sessions" |

To configure an alias for a model, supply a value for the model's `alias` configuration parameter. For example:

Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/build/incremental-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ from {{ ref('app_data_events') }}

-- this filter will only be applied on an incremental run
-- (uses >= to include records arriving later on the same day as the last run of this model)
where date_day >= (select coalesce(max(event_time), '1900-01-01') from {{ this }})
where date_day >= (select coalesce(max(date_day), '1900-01-01') from {{ this }})

{% endif %}

Expand Down
3 changes: 2 additions & 1 deletion website/docs/docs/build/materializations.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,8 @@ When using the `table` materialization, your model is rebuilt as a <Term id="tab
* Use incremental models when your `dbt run`s are becoming too slow (i.e. don't start with incremental models)

### Ephemeral
`ephemeral` models are not directly built into the database. Instead, dbt will interpolate the code from this model into dependent models as a common <Term id="table" /> expression.
`ephemeral` models are not directly built into the database. Instead, dbt will interpolate the code from an ephemeral model into its dependent models using a common table expression (<Term id="cte" />). You can control the identifier for this CTE using a [model alias](/docs/build/custom-aliases), but dbt will always prefix the model identifier with `__dbt__cte__`.

* **Pros:**
* You can still write reusable logic
- Ephemeral models can help keep your <Term id="data-warehouse" /> clean by reducing clutter (also consider splitting your models across multiple schemas by [using custom schemas](/docs/build/custom-schemas)).
Expand Down
26 changes: 11 additions & 15 deletions website/docs/docs/core/connect-data-platform/firebolt-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,47 +40,43 @@ To connect to Firebolt from dbt, you'll need to add a [profile](https://docs.get
outputs:
<target-name>:
type: firebolt
user: "<username>"
password: "<password>"
client_id: "<id>"
client_secret: "<secret>"
database: "<database-name>"
engine_name: "<engine-name>"
account_name: "<account-name>"
schema: <tablename-prefix>
threads: 1
#optional fields
jar_path: <path-to-local-jdbc-driver>
host: "<hostname>"
account_name: "<account-name>"
```
</File>
#### Description of Firebolt Profile Fields
To specify values as environment variables, use the format `{{ env_var('<variable_name>' }}`. For example, `{{ env_var('DATABASE_NAME' }}`.
To specify values as environment variables, use the format `{{ env_var('<variable_name>' }}`. For example, `{{ env_var('DATABASE_NAME' }}`.

| Field | Description |
|--------------------------|--------------------------------------------------------------------------------------------------------|
| `type` | This must be included either in `profiles.yml` or in the `dbt_project.yml` file. Must be set to `firebolt`. |
| `user` | Required. A Firebolt username with adequate permissions to access the specified `engine_name`. |
| `password` | Required. The password associated with the specified `user`. |
| `client_id` | Required. Your [service account](https://docs.firebolt.io/godocs/Guides/managing-your-organization/service-accounts.html) id. |
| `client_secret` | Required. The secret associated with the specified `client_id`. |
| `database` | Required. The name of the Firebolt database to connect to. |
| `engine_name` | Required in version 0.21.10 and later. Optional in earlier versions. The name (not the URL) of the Firebolt engine to use in the specified `database`. This must be a general purpose read-write engine and the engine must be running. If omitted in earlier versions, the default engine for the specified `database` is used. |
| `account_name` | Required. Specifies the account name under which the specified `database` exists. |
| `schema` | Recommended. A string to add as a prefix to the names of generated tables when using the [custom schemas workaround](https://docs.getdbt.com/reference/warehouse-profiles/firebolt-profile#supporting-concurrent-development). |
| `threads` | Required. Must be set to `1`. Multi-threading is not currently supported. |
| `jar_path` | Required only with versions earlier than 0.21.0. Ignored in 0.21.0 and later. The path to your JDBC driver on your local drive. |
| `threads` | Required. Set to higher number to improve performance. |
| `host` | Optional. The host name of the connection. For all customers it is `api.app.firebolt.io`, which will be used if omitted. |
| `account_name` | Required if more than one account is associated with the specified `user1`. Specifies the account name (not the account ID) under which the specified `database` exists. If omitted, the default account is assumed. |


#### Troubleshooting Connections

If you encounter issues connecting to Firebolt from dbt, make sure the following criteria are met:
- The engine must be a general-purpose read-write engine, not an analytics engine.
- You must have adequate permissions to access the engine.
- You must have adequate permissions to access the engine and the database.
- Your service account must be attached to a user.
- The engine must be running.
- If you're not using the default engine for the database, you must specify an engine name.
- If there is more than one account associated with your credentials, you must specify an account.


## Supporting Concurrent Development
Expand Down
6 changes: 5 additions & 1 deletion website/docs/docs/deploy/ci-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,11 @@ If you're on a Virtual Private dbt Enterprise plan using security features like

When you start a CI job, the pull request status should show as `pending` while it waits for an update from dbt. Once the CI job finishes, dbt sends the status to Azure DevOps (ADO), and the status will change to either `succeeded` or `failed`.

If the status doesn't get updated after the job runs, check if there are any git branch policies in place that's blocking ADO from receiving these updates. You can find relevant information here:
If the status doesn't get updated after the job runs, check if there are any git branch policies in place blocking ADO from receiving these updates.

One potential issue is the **Reset conditions** under **Status checks** in the ADO repository branch policy. If you enable the **Reset status whenever there are new changes** checkbox (under **Reset conditions**), it can prevent dbt from updating ADO about your CI job run status.
You can find relevant information here:
- [Azure DevOps Services Status checks](https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops&tabs=browser#status-checks)
- [Azure DevOps Services Pull Request Stuck Waiting on Status Update](https://support.hashicorp.com/hc/en-us/articles/18670331556627-Azure-DevOps-Services-Pull-Request-Stuck-Waiting-on-Status-Update-from-Terraform-Cloud-Enterprise-Run)
- [Pull request status](https://learn.microsoft.com/en-us/azure/devops/repos/git/pull-request-status?view=azure-devops#pull-request-status)

Expand Down
1 change: 1 addition & 0 deletions website/docs/reference/global-configs/warnings.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ DBT_WARN_ERROR_OPTIONS='{"include": ["NoNodesForSelectionCriteria"]}' dbt run
...
```

Values for `error`, `warn`, and/or `silence` should be passed on as arrays. For example, `dbt --warn-error-options '{"error": "all", "warn": ["NoNodesForSelectionCriteria"]}' run` not `dbt --warn-error-options '{"error": "all", "warn": "NoNodesForSelectionCriteria"}' run`.


<File name='profiles.yml'>
Expand Down
2 changes: 2 additions & 0 deletions website/docs/reference/resource-configs/alias.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,5 +116,7 @@ The standard behavior of dbt is:
* If a custom alias is _not_ specified, the identifier of the relation is the resource name (i.e. the filename).
* If a custom alias is specified, the identifier of the relation is the `{{ alias }}` value.

**Note** With an [ephemeral model](/docs/build/materializations), dbt will always apply the prefix `__dbt__cte__` to the <Term id="cte" /> identifier. This means that if an alias is set on an ephemeral model, then its CTE identifier will be `__dbt__cte__{{ alias }}`, but if no alias is set then its identifier will be `__dbt__cte__{{ filename }}`.

To learn more about changing the way that dbt generates a relation's `identifier`, read [Using Aliases](/docs/build/custom-aliases).

Loading

0 comments on commit f817e8b

Please sign in to comment.