Skip to content

Commit

Permalink
add to docs
Browse files Browse the repository at this point in the history
  • Loading branch information
mirnawong1 committed Aug 6, 2024
1 parent 8b4e386 commit a5d1e36
Show file tree
Hide file tree
Showing 7 changed files with 54 additions and 59 deletions.
38 changes: 19 additions & 19 deletions website/docs/docs/cloud-integrations/configure-auto-exposures.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: "Import and auto-generate exposures from dashboards and understand

# Configure auto-exposures <Lifecycle status='beta' />

As a data team, it’s critical that you have context into the downstream use cases and users of your data products. [Auto-exposures](/docs/collaborate/auto-exposures) integrates natively with Tableau (Power BI coming soon) and auto-generates downstream lineage in dbt Explorer for a richer experience.
As a data team, it’s critical that you have context into the downstream use cases and users of your data products. [Auto-exposures](/docs/collaborate/auto-exposures) integrates natively with Tableau (Power BI coming soon) and [auto-generates downstream lineage](/docs/collaborate/auto-exposures#view-auto-exposures-in-dbt-explorer) in dbt Explorer for a richer experience.

:::info Available in beta
Auto-exposures is currently in beta. If you have any feedback or would like to take part in the beta, please reach out to your account manager.
Expand All @@ -25,50 +25,50 @@ To access the features, you should meet the following:

1. You have a dbt Cloud account on the [Enterprise plan](https://www.getdbt.com/pricing/).
2. You have set up a [production](/docs/deploy/deploy-environments#set-as-production-environment) deployment environment for each project you want to explore, with at least one successful job run.
3. You have admin permissions in dbt Cloud to edit project settings or production environment settings
3. You have [admin permissions](/docs/cloud/manage-access/enterprise-permissions) in dbt Cloud to edit project settings or production environment settings
4. Use Tableau as your BI tool and can enable metadata permissions or work with an admin to do so. Compatible with Tableau Cloud or Tableau Server with the Metadata API enabled.

## Set up in Tableau

This section of the document explains the steps you need to set up the auto-exposures integration with Tableau.
This section of the document explains the steps you need to set up the auto-exposures integration with Tableau. Once you've set this up in Tableau and dbt Cloud, you can view the [auto-exposures](/docs/collaborate/auto-exposures#view-auto-exposures-in-dbt-explorer) in dbt Explorer.

To set up personal access tokens (PATs) needed for auto exposures, ask a site admin to configure it for the account.

1. First, ensure you or a site admin enable PATs for the account in Tableau.
1. Ensure you or a site admin enable PATs for the account in Tableau.
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/tableau-enable-pat.jpg" title="Enable PATs for the account in Tableau"/>

2. Next, create a PAT that you can add to dbt Cloud to pull in Tableau metadata for auto exposures.
2. Create a PAT that you can add to dbt Cloud to pull in Tableau metadata for auto exposures.
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/tableau-create-pat.jpg" title="Create PATs for the account in Tableau"/>

3. Copy the **Secret** and the **Token name.** Enter them in dbt Cloud. For security, you can save them in a password manager if desired.
3. Copy the **Secret** and the **Token name** and enter them in dbt Cloud. The secret is only displayed once, so store it in a safe location (like a password manager).
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/tableau-copy-token.jpg" title="Copy the secret and token name to enter them in dbt Cloud"/>

4. You will also need the **Server URL** and **Sitename**. You can find these in the URL while logged into Tableau.
4. Copy the **Server URL** and **Sitename**. You can find these in the URL while logged into Tableau.
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/tablueau-serverurl.jpg" title="Locate the Server URL and Sitename in Tableau"/>

For example, if the full URL is: `10az.online.tableau.com/#/site/dbtlabspartner/explore`:
- The **Server URL** is the first part of the URL, in this case: `10az.online.tableau.com`
- The **Sitename** is right after the `site` in the URL, in this case: `dbtlabspartner`

5. You should now be ready to set up Auto Exposures in dbt Cloud after having the four following items: ServerURL, Sitename, Token name, and Secret
5. You should now be ready to set up auto-exposures in dbt Cloud after copying the following items, which you'll need during the dbt Cloud setup: ServerURL, Sitename, Token name, and Secret.

## Set up in dbt Cloud
## Set up in dbt Cloud <Lifecycle status="enterprise"/>

1. First, navigate to the project you want to add the Auto Exposures to and then select **Settings**.
2. Select **Add Integration** to add the Tableau connection.
1. In dbt Cloud, navigate to the project you want to add the auto-exposures to and then select **Settings**.
2. Under the **Exposures** section, select **Add integration** to add the Tableau connection.
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/cloud-add-integration.jpg" title="Select Add Integration to add the Tableau connection."/>
3. Enter the details for the exposure connection you gathered from Tableau in step 4 earlier (all fields are case-sensitive) and click **Continue**.
3. Enter the details for the exposure connection you collected from Tableau in the [previous step](#set-up-in-tableau) and click **Continue**. Note that all fields are case-sensitive.
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/cloud-integration-details.jpg" title="Enter the details for the exposure connection."/>
4. Select the collections you want to include for auto exposures. dbt Cloud automatically imports and syncs any workbook within the selected collections. New additions to the collections will be added to the lineage in dbt Cloud during the next automatic sync (usually once per day).
4. Select the collections you want to include for auto exposures.

dbt Cloud automatically imports and syncs any workbook within the selected collections. New additions to the collections will be added to the lineage in dbt Cloud during the next automatic sync (usually once per day).
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/cloud-select-collections.jpg" title="Select the collections you want to include for auto exposures."/>
5. Click **Save**. Everything in this collection will be imported and you can continue to view them in Explorer using the steps below.
5. Click **Save**. dbt Cloud imports everything in this collection and you can continue to view them in Explorer using the next steps.

import ViewExposures from '/snippets/_auto-exposures-view.md';
For more information on how to view and use auto-exposures, refer to the [Auto-exposures](/docs/collaborate/auto-exposures) page.

<ViewExposures/>
## Refresh auto-exposures in jobs

## Later: Refresh the auto-exposures via your jobs

:::info
:::info Coming soon
Soon, you’ll also be able to use auto-exposures trigger refresh of the data used in your Tableau dashboards from within dbt Cloud. Stay tuned for more on this soon!
:::
2 changes: 1 addition & 1 deletion website/docs/docs/cloud-integrations/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Many data applications integrate with dbt Cloud, enabling you to leverage the po
<div className="grid--3-col">

<Card
title="Configure auto-exposures (beta)"
title="Auto-exposures (beta)"
body="Import and auto-generate exposures from dashboards to understand how models are used in downstream tools for a richer downstream lineage."
link="/docs/cloud-integrations/configure-auto-exposures"
icon="dbt-bit"/>
Expand Down
13 changes: 5 additions & 8 deletions website/docs/docs/collaborate/auto-exposures.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,23 @@
title: "Auto-exposures"
sidebar_label: "Auto-exposures"
description: "Import and auto-generate exposures from dashboards and understand how models are used in downstream tools for a richer lineage."
pagination_prev: null
pagination_next: "docs/collaborate/model-query-history"
---

# About auto-exposures <Lifecycle status='beta' />

# Auto-exposures <Lifecycle status='beta' />

As a data team, it’s critical that you have context into the downstream use cases and users of your data products. Auto-exposures integrates natively with Tableau (Power BI coming soon) and auto-generates downstream lineage in dbt Explorer for a richer experience.

:::info Available in beta
Auto-exposures is currently in beta. If you have any feedback or would like to take part in the beta, please reach out to your account manager.
:::

Auto-exposures help data teams optimize their efficiency and ensure data quality by:

- Helping users understand how their models are used in downstream analytics tools to inform investments and reduce incidents — ultimately building trust and confidence in data products.
- Importing and auto-generating exposures based on Tableau dashboards, with user-defined curation.
- Enabling the active exposure work to run models based on when exposures are updated or need to be updated, improving timeliness and reducing costs.
Auto-exposures helps users understand how their models are used in downstream analytics tools to inform investments and reduce incidents — ultimately building trust and confidence in data products. It imports and auto-generates exposures based on Tableau dashboards, with user-defined curation.

Auto-exposures is available to [dbt Cloud Enterprise plans](https://www.getdbt.com/pricing/).

For more info on how to set up auto-exposures, [prerequisites](/docs/cloud-integrations/configure-auto-exposures#prerequisites), and more &mdash; refer to [configure auto-exposures in dbt Cloud](/docs/cloud-integrations/configure-auto-exposures)
For more info on how to set up auto-exposures, prerequisites, and more &mdash; refer to [configure auto-exposures in Tableau and dbt Cloud](/docs/cloud-integrations/configure-auto-exposures).

import ViewExposures from '/snippets/_auto-exposures-view.md';

Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/collaborate/dbt-explorer-faqs.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: "dbt Explorer FAQs"
sidebar_label: "dbt Explorer FAQs"
description: "Learn more with the FAQs about dbt Explorer, how it works, how to interact with it, and more."
pagination_next: null
pagination_next: "docs/collaborate/auto-exposures"
---

[dbt Explorer](/docs/collaborate/explore-projects) is dbt Cloud’s new knowledge base and lineage visualization experience. It offers an interactive and high-level view of your company’s entire data estate, where you can dive deep into the context you need to understand and improve lineage so your teams can trust the data they’re using to make decisions.
Expand Down
4 changes: 2 additions & 2 deletions website/docs/docs/collaborate/explore-projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: "Discover data with dbt Explorer"
sidebar_label: "Discover data with dbt Explorer"
description: "Learn about dbt Explorer and how to interact with it to understand, improve, and leverage your dbt projects."
pagination_next: "docs/collaborate/column-level-lineage"
pagination_next: "docs/collaborate/access-from-dbt-cloud"
pagination_prev: null
---

Expand All @@ -25,7 +25,7 @@ Navigate the dbt Explorer overview page to access your project's resources and m
- **Latest updates** &mdash; View the latest changes or issues related to your project's resources, including the most recent job runs, changed properties, lineage, and issues.
- **Marts and public models** &mdash; View the [marts](/best-practices/how-we-structure/1-guide-overview#guide-structure-overview) and [public models](/docs/collaborate/govern/model-access#access-modifiers) in your project.
- **Model query history** &mdash; Use [model query history](/docs/collaborate/model-query-history) to track the history of queries on your models for deeper insights.
- **Auto-exposures** &mdash; [Set up auto-exposures](/docs/collaborate/auto-exposures) to automatically expose relevant data models from your BI tool to enhance visibility.
- **Auto-exposures** &mdash; [Set up and view auto-exposures](/docs/collaborate/auto-exposures) to automatically expose relevant data models from your BI tool to enhance visibility.

<Lightbox src="/img/docs/collaborate/dbt-explorer/explorer-main-page.gif" width="100%" title="Access dbt Explorer from dbt Cloud by clicking Explore in the navigation."/>

Expand Down
37 changes: 17 additions & 20 deletions website/docs/docs/collaborate/model-query-history.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: "Model query history"
sidebar_label: "Model query history"
description: "Import and auto-generate exposures from dashboards and understand how models are used in downstream tools for a richer lineage."
pagination_next: null
---

# About model query history <Lifecycle status='beta' />
Expand All @@ -14,40 +15,37 @@ The model query history tile allows you to:

Model query history is powered by a single query of query log table in your data warehouse aggregated on a daily basis. It filters down to `select` statements only to gauge model consumption and excludes dbt model build and test executions.

:::info
Available for Snowflake and BigQuery users initially.
:::

## Prerequisites

To access the features, you should meet the following:

1. You have a dbt Cloud account on the [Enterprise plan](https://www.getdbt.com/pricing/).
2. You have set up a [production](https://docs.getdbt.com/docs/deploy/deploy-environments#set-as-production-environment) deployment environment for each project you want to explore, with at least one successful job run.
3. You have admin permissions in dbt Cloud to edit project settings or production environment settings.
3. You have [admin permissions](/docs/cloud/manage-access/enterprise-permissions) in dbt Cloud to edit project settings or production environment settings.
4. Use Snowflake or BigQuery as your data warehouse and can enable query history permissions or work with an admin to do so. (Support for additional data platforms coming soon)

## Credential permissions

This section of the document explains the permissions and steps you need to enable and view model query history in dbt Explorer.
This section explains the permissions and steps you need to enable and view model query history in dbt Explorer.

The model query history feature uses the credentials in your production environment to gather metadata from your data warehouse’s query logs. This means you may need elevated permissions with the warehouse. Before making any changes to your data platform permissions, confirm the configured permissions in dbt Cloud:

1. Navigate to **Deploy** and then **Environments**.
2. Select the Environment marked **PROD** and click **Settings**.
3. Look at the information under **Deployment credentials**.
- Note: Querying query history entails warehouse costs / uses credits.
4. Navigate to **Deploy** and then **Environments**.
5. Select the Environment marked **PROD** and click **Settings**.
6. Look at the information under **Deployment credentials**.
- Note: Querying query history entails warehouse costs / uses credits.

<Lightbox src="/img/docs/collaborate/dbt-explorer/model-query-credentials.jpg" width="50%" title="Confirm your deployment credentials in your environment settings page." />

7. You can cross reference those credential permissions with the warehouse permissions and grant your user the right permissions.
4. Copy or cross reference those credential permissions with the warehouse permissions and grant your user the right permissions.

<Expandable alt_header="For Snowflake">

This feature makes use of metadata tables available to Snowflake Enterprise tier accounts or higher, `QUERY_HISTORY` and `ACCESS_HISTORY`. The snowflake user used in the production environment must have `GOVERNANCE_VIEWER` permissions to view the data.

This can be granted to this user by your `ACCOUNTADMIN` user in Snowflake. For more details, view the snowflake docs [here](https://docs.snowflake.com/en/sql-reference/account-usage#enabling-other-roles-to-use-schemas-in-the-snowflake-database).

</Expandable>
Expand All @@ -64,26 +62,25 @@ The model query history feature uses the credentials in your production environm
## Enable query history in dbt Cloud

:::info
During the beta, the dbt Labs team will manually enable query history for your dbt Cloud projects. Later on, you’ll be able to do it yourself.
During beta, the dbt Labs team will manually enable query history for your dbt Cloud projects. Later on, you’ll be able to do it yourself.

:::

<!--
1. Navigate to **Deploy** and then **Environments**.
2. Select the environment marked **PROD** and click **Settings**.
3. Enable the checkbox for query history within **General Settings**.
3. Enable the checkbox for query history in **General Settings**.
4. Click the **Test** button to validate the deployment credentials permissions are sufficient to support query history.
-->

## View query history in Explorer

To enhance your discovery, you can view your model query history in various locations within dbt Explorer.

For details on how to access model query history in each location, expand the following toggles:
To enhance your discovery, you can view your model query history in various locations within dbt Explorer. For details on how to access model query history in each location, expand the following toggles:

<Expandable alt_header="Performance charts">
<Expandable alt_header="View from Performance charts">

1. Navigate to dbt Explorer by clicking on the **Explore** link in the navigation.
2. In the main **Overview** page, under **Project** click **Performance** and scroll down to view the most queried models
3. Use the dropdown menu to select the desired time period, with options available for up to the past 3 months.
3. Use the dropdown menu on the right to select the desired time period, with options available for up to the past 3 months.

<Lightbox src="/img/docs/collaborate/dbt-explorer/model-query-queried-models.jpg" width="70%" title="View most queried models in 'Performance' page in dbt Explorer." />

Expand All @@ -92,7 +89,7 @@ For details on how to access model query history in each location, expand the fo

</Expandable>

<Expandable alt_header="Project lineage">
<Expandable alt_header="View from Project lineage">

1. To view your model in your project lineage, go to the main **Overview page** and click on **Project lineage.**
2. In the lower left of your lineage, click on **Lenses** and select **Usage queries**.
Expand All @@ -102,9 +99,9 @@ For details on how to access model query history in each location, expand the fo

</Expandable>

<Expandable alt_header="Model list">
<Expandable alt_header="View from Model list">

1. To view your model in your project lineage, go to the main **Overview page** and click on **Project lineage.**
1. To view your model in your project lineage, go to the main **Overview page**.
2. In the left navigation, go to the **Resources** tab and click on **Models** to view the models list.
3. You can view the usage query count for the models and sort by most or least queried. The query number for each model represents the query history over the last 30 days.
<Lightbox src="/img/docs/collaborate/dbt-explorer/model-query-list.jpg" width="85%" title="View models query history in the 'Models' list page under the 'Usage' column." />
Expand Down
Loading

0 comments on commit a5d1e36

Please sign in to comment.