Skip to content

Commit

Permalink
Merge branch 'current' into update/enterprise-permissions-table
Browse files Browse the repository at this point in the history
  • Loading branch information
mirnawong1 authored Jul 17, 2024
2 parents 0759117 + 76ca8f1 commit 1b063b5
Show file tree
Hide file tree
Showing 44 changed files with 316 additions and 43 deletions.
2 changes: 1 addition & 1 deletion website/blog/2022-08-31-august-product-update.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ You’ll hear more in [Tristan’s keynote](https://coalesce.getdbt.com/agenda/k

## **What's new**

- **dbt Core v1.3 beta:** Do you use Python for analytics? The first beta prerelease of dbt Core v1.3—including support for dbt models written in Python—is [ready to explore](https://docs.getdbt.com/docs/dbt-versions/core-upgrade/upgrading-to-v1.3)! Check it out, and read more about dbt supported Python models [in our docs](/docs/build/python-models).
- **dbt Core v1.3 beta:** Do you use Python for analytics? The first beta prerelease of dbt Core v1.3—including support for dbt models written in Python—is [ready to explore](https://docs.getdbt.com/docs/dbt-versions/core-upgrade/older%20versions/upgrading-to-v1.3)! Check it out, and read more about dbt supported Python models [in our docs](/docs/build/python-models).
- **Technology Partner Program:** We just launched our new [Technology Partner Program](https://www.getdbt.com/blog/dbt-labs-technology-partner-program/) with 40+ friends in the Modern Data Stack to provide consistent support for seamless integrations joint-users can trust. Check our new [dbt Cloud integrations page](http://www.getdbt.com/product/integrations) for what’s available today!
- **Single-tenant users:** dbt Cloud v1.1.60 is now available on dbt Cloud Enterprise.

Expand Down
4 changes: 4 additions & 0 deletions website/dbt-versions.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
exports.versions = [
{
version: "1.9",
isPrerelease: true,
},
{
version: "1.8",
EOLDate: "2025-04-15",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ pagination_next: null

- 🗺️ Use these best practices to map out your team's plan to **incrementally adopt the Semantic Layer**.
- 🤗 Get involved in the community and ask questions, **help craft best practices**, and share your progress in building a dbt Semantic Layer.
- [Validate semantic nodes in CI](/docs/deploy/ci-jobs#semantic-validations-in-ci) to ensure code changes made to dbt models don't break these metrics.

The dbt Semantic Layer is the biggest paradigm shift thus far in the young practice of analytics engineering. It's ready to provide value right away, but is most impactful if you move your project towards increasing normalization, and allow MetricFlow to do the denormalization for you with maximum dimensionality.

Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/build/saved-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,5 +230,5 @@ To include all saved queries in the dbt build run, use the [`--resource-type` fl
</detailsToggle>

## Related docs

- [Validate semantic nodes in a CI job](/docs/deploy/ci-jobs#semantic-validations-in-ci)
- Configure [caching](/docs/use-dbt-semantic-layer/sl-cache)
31 changes: 24 additions & 7 deletions website/docs/docs/cloud/configure-cloud-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,17 +110,34 @@ As a tip, most command-line tools have a `--help` flag to show available command
- `dbt run --help`: Lists the flags available for the `run` command
:::

### SQLFluff integration
The dbt Cloud CLI supports [SQLFluff](https://sqlfluff.com/), a modular and configuration SQL linter, which warns you of complex functions, syntax, formatting, and compilation errors.
### Lint SQL files

To get started, run `dbt sqlfluff -h` to see the list of supported commands and flags, such as `dbt sqlfluff lint` to lint SQL files.
From the dbt Cloud CLI, you can invoke [SQLFluff](https://sqlfluff.com/) which is a modular and configurable SQL linter that warns you of complex functions, syntax, formatting, and compilation errors. Many of the same flags that you can pass to SQLFluff are available from the dbt Cloud CLI.

The available SQLFluff commands are:

- `lint` &mdash; Lint SQL files by passing a list of files or from standard input (stdin).
- `fix` &mdash; Fix SQL files.
- `format` &mdash; Autoformat SQL files.


To lint SQL files, run the command as follows:

```shell
dbt sqlfluff lint [PATHS]... [flags]
```

When no path is set, dbt lints all SQL files in the current project. To lint a specific SQL file or a directory, set `PATHS` to the path of the SQL file(s) or directory of files. To lint multiple files or directories, pass multiple `PATHS` flags.

To show detailed information on all the dbt supported commands and flags, run the `dbt sqlfluff -h` command.

#### Considerations
Keep the following points in mind when using SQLFluff with the dbt Cloud:

- When you run `dbt sqlfluff`, it picks up changes to your local .sqlfluff config.
- To use SQLFluff in continuous integration/continuous development, you need to have a `dbt_cloud.yml` file in your project and run commands from a valid dbt project.
- SQLFluff commands in the dbt Cloud CLI do not return exit codes yet.
When running `dbt sqlfluff` from the dbt Cloud CLI, the following are important behaviors to consider:

- dbt reads the `.sqlfluff` file, if it exists, for any custom configurations you might have.
- For continuous integration/continuous development (CI/CD) workflows, your project must have a `dbt_cloud.yml` file and you have successfully run commands from within this dbt project.
- An SQLFluff command will return an exit code of 0 if it ran without any file violations. This dbt behavior differs from SQLFluff behavior, where a linting violation returns a non-zero exit code. dbt Labs plans on addressing this in a later release.

## FAQs
<Expandable alt_header="How to create a .dbt directory and move your file">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ The user's [license](/docs/cloud/manage-access/seats-and-users) type always over

## How to set up RBAC Groups in dbt Cloud

Role-Based Access Control (RBAC) is helpful for automatically assigning permissions to dbt admins based on their SSO provider group associations.
Role-Based Access Control (RBAC) is helpful for automatically assigning permissions to dbt admins based on their SSO provider group associations. RBAC does not apply to [model groups](/docs/collaborate/govern/model-access#groups).

1. Click the gear icon to the top right and select **Account Settings**. Click **Groups & Licenses**

Expand Down
45 changes: 30 additions & 15 deletions website/docs/docs/collaborate/explore-multiple-projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,46 @@ sidebar_label: "Explore multiple projects"
description: "Learn about project-level lineage in dbt Explorer and its uses."
---

You can also view all the different projects and public models in the account, where the public models are defined, and how they are used to gain a better understanding about your cross-project resources.
View all the projects and public models in your account (where public models are defined) and gain a better understanding of your cross-project resources and how they're used.

The resource-level lineage graph for a given project displays the cross-project relationships in the DAG. The different icons indicate whether you’re looking at an upstream producer project (parent) or a downstream consumer project (child).
The resource-level lineage graph for a project displays the cross-project relationships in the DAG, with a **PRJ** icon indicating whether or not it's a project resource. That icon is located to the left side of the node name.

When you view an upstream (parent) project, its public models display a counter icon in the upper right corner indicating how many downstream (child) projects depend on them. Selecting a model reveals the lineage indicating the projects dependent on that model. These counts include all projects listing the upstream one as a dependency in its `dependencies.yml`, even without a direct `{{ ref() }}`. Selecting a project node from a public model opens its detailed lineage graph, which is subject to your [permission](/docs/cloud/manage-access/enterprise-permissions).
To view the project-level lineage graph, click the **View lineage** icon in the upper right corner from the main overview page:
- This view displays all the projects in your account and their relationships.
- Viewing an upstream (parent) project displays the downstream (child) projects that depend on it.
- Selecting a model reveals its dependent projects in the lineage.
- Click on an upstream (parent) project to view the other projects that reference it in the **Relationships** tab, showing the number of downstream (child) projects that depend on them.
- This includes all projects listing the upstream one as a dependency in its `dependencies.yml` file, even without a direct `{{ ref() }}`.
- Selecting a project node from a public model opens its detailed lineage graph if you have the [permissions](/docs/cloud/manage-access/enterprise-permissions) to do so.

<Lightbox src="/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png" width="70%" height="100" title="Cross-project lineage in a parent project"/>
<Lightbox src="/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png" width="100%" height="100" title="View your cross-project lineage in a parent project and the other projects that reference it by clicking the 'Relationships' tab."/>

When viewing a downstream (child) project that imports and refs public models from upstream (parent) projects, public models will show up in the lineage graph and display an icon on the graph edge that indicates what the relationship is to a model from another project. Hovering over this icon indicates the specific dbt Cloud project that produces that model. Double-clicking on a model from another project opens the resource-level lineage graph of the parent project, which is subject to your permissions.
When viewing a downstream (child) project that imports and refs public models from upstream (parent) projects:
- Public models will show up in the lineage graph and you can click on them to view the model details.
- Clicking on a model opens a side panel containing general information about the model, such as the specific dbt Cloud project that produces that model, description, package, and more.
- Double-clicking on a model from another project opens the resource-level lineage graph of the parent project, if you have the permissions to do so.


<Lightbox src="/img/docs/collaborate/dbt-explorer/cross-project-lineage-child.png" width="85%" height="100" title="Cross-project lineage in a child project"/>
<Lightbox src="/img/docs/collaborate/dbt-explorer/cross-project-child.png" width="100%" height="100" title="View a downstream (child) project that importants and refs public models from the upstream (parent) project."/>

## Explore the project-level lineage graph

For cross-project collaboration, you can interact with the DAG in all the same ways as described in [Explore your project's lineage](/docs/collaborate/explore-projects#project-lineage) but you can also interact with it at the project level and view the details.
For cross-project collaboration, you can interact with the DAG in all the same ways as described in [Explore your project's lineage](/docs/collaborate/explore-projects#project-lineage) but you can also interact with it at the project level and view the details.

If you have permissions for a project in the account, you can view all public models used across the entire account. However, you can only view full public model details and private models if you have permissions for the specific project where those models are defined.

To get a list view of all the projects, select the account name at the top of the **Explore** page near the navigation bar. This view includes a public model list, project list, and a search bar for project searches. You can also view the project-level lineage graph by clicking the Lineage view icon in the page's upper right corner.
To view all the projects in your account (displayed as a lineage graph or list view):
- Navigate to the top left section of the **Explore** page, near the navigation bar.
- Hover over the project name and select the account name. This takes you to a account-level lineage graph page, where you can view all the projects in the account, including dependencies and relationships between different projects.
- Click the **List view** icon in the page's upper right corner to see a list view of all the projects in the account.
- The list view page displays a public model list, project list, and a search bar for project searches.
- Click the **Lineage view** icon in the page's upper right corner to view the account-level lineage graph.

If you have permissions for a project in the account, you can view all public models used across the entire account. However, you can only view full public model details and private models if you have permissions for a project where the models are defined.
<Lightbox src="/img/docs/collaborate/dbt-explorer/account-level-lineage.gif" width="100%" title="View a downstream (child) project, which imports and refs public models from upstream (parent) projects."/>

From the project-level lineage graph, you can:
From the account-level lineage graph, you can:

- Click the Lineage view icon (in the graph’s upper right corner) to view the cross-project lineage graph.
- Click the List view icon (in the graph’s upper right corner) to view the project list.
- Click the **Lineage view** icon (in the graph’s upper right corner) to view the cross-project lineage graph.
- Click the **List view** icon (in the graph’s upper right corner) to view the project list.
- Select a project from the **Projects** tab to switch to that project’s main **Explore** page.
- Select a model from the **Public Models** tab to view the [model’s details page](/docs/collaborate/explore-projects#view-resource-details).
- Perform searches on your projects with the search bar.
Expand All @@ -40,6 +55,6 @@ When you select a project node in the graph, a project details panel opens on th
- View a list of its public models, if any.
- View a list of other projects that uses the project, if any.
- Click **Open Project Lineage** to switch to the project’s lineage graph.
- Click the Share icon to copy the project panel link to your clipboard so you can share the graph with someone.
- Click the **Share** icon to copy the project panel link to your clipboard so you can share the graph with someone.

<LoomVideo id='606f02e1cce343eba7e1061d6273ff0a?t=1' />
<Lightbox src="/img/docs/collaborate/dbt-explorer/multi-project-overview.gif" width="95%" title="Select a downstream (child) project to open the project details panel for resource counts, public models associated, and more. "/>
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
title: "Upgrading to v1.9 (beta)"
id: upgrading-to-v1.9
description: New features and changes in dbt Core v1.9
displayed_sidebar: "docs"
---

## Resources

- Changelog (coming soon)
- [dbt Core CLI Installation guide](/docs/core/installation-overview)
- [Cloud upgrade guide](/docs/dbt-versions/upgrade-dbt-version-in-cloud) &mdash; dbt Cloud is now versionless. dbt v1.9 will not appear in the version dropdown. Select "Keep on latest version" to get all the latest features and functionality in your dbt Cloud account.

## What to know before upgrading

dbt Labs is committed to providing backward compatibility for all versions 1.x, except for any changes explicitly mentioned on this page. If you encounter an error upon upgrading, please let us know by [opening an issue](https://github.com/dbt-labs/dbt-core/issues/new).


## New and changed features and functionality

Features and functionality new in dbt v1.9.

**Coming soon**

## Quick hits

**Coming soon**
4 changes: 4 additions & 0 deletions website/docs/docs/dbt-versions/release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ Release notes are grouped by month for both multi-tenant and virtual private clo

[^*] The official release date for this new format of release notes is May 15th, 2024. Historical release notes for prior dates may not reflect all available features released earlier this year or their tenancy availability.

## July 2024
- **New**: The ability to lint your SQL files from the dbt Cloud CLI is now available. To learn more, refer to [Lint SQL files](/docs/cloud/configure-cloud-cli#lint-sql-files).
- **New**: Introduced Semantic validations in CI pipelines. Automatically test your semantic nodes (metrics, semantic models, and saved queries) during code reviews by adding warehouse validation checks in your CI job using the `dbt sl validate` command. You can also validate modified semantic nodes to guarantee code changes made to dbt models don't break these metrics. Refer to [Semantic validations in CI](/docs/deploy/ci-jobs#semantic-validations-in-ci) to learn about the additional commands and use cases.

## June 2024
- **New:** Introduced new granularity support for cumulative metrics in MetricFlow. Granularity options for cumulative metrics are slightly different than granularity for other metric types. For other metrics, we use the `date_trunc` function to implement granularity. However, because cumulative metrics are non-additive (values can't be added up), we can't use the `date_trunc` function to change their time grain granularity.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ clean-targets:
- Do you have custom scripts that parse dbt <Term id="json" /> artifacts?
- (BigQuery only) Do you use dbt's legacy capabilities around ingestion-time-partitioned tables?

If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade/upgrading-to-v1.0).
If you believe your project might be affected, read more details in the migration guide [here](/docs/dbt-versions/core-upgrade/older%20versions/upgrading-to-v1.0).

</details>

Expand Down
Loading

0 comments on commit 1b063b5

Please sign in to comment.