+
@@ -121,9 +128,13 @@ Use `is_partition: True` to show that a dimension exists over a specific time wi
You can also use `is_partition` for [categorical](#categorical) dimensions as well.
-MetricFlow enables metric aggregation during query time. For example, you can aggregate the `messages_per_month` measure. If you originally had a `time_granularity` for the time dimensions `metric_time`, you can specify a yearly granularity for aggregation in your CLI query:
+MetricFlow enables metric aggregation during query time. For example, you can aggregate the `messages_per_month` measure. If you originally had a `time_granularity` for the time dimensions `metric_time`, you can specify a yearly granularity for aggregation in your query:
```bash
+# dbt Cloud users
+dbt sl query --metrics messages_per_month --dimensions metric_time --order metric_time --time-granularity year
+
+# dbt Core users
mf query --metrics messages_per_month --dimensions metric_time --order metric_time --time-granularity year
```
@@ -344,7 +355,11 @@ In the sales tier example, if sales_person_id 456 is Tier 2 from 2022-03-08 onwa
The following command or code represents how to return the count of transactions generated by each sales tier per month:
-```
+```bash
+# dbt Cloud users
+dbt sl query --metrics transactions --dimensions metric_time__month,sales_person__tier --order metric_time__month --order sales_person__tier
+
+# dbt Core users
mf query --metrics transactions --dimensions metric_time__month,sales_person__tier --order metric_time__month --order sales_person__tier
```
diff --git a/website/docs/docs/build/enhance-your-code.md b/website/docs/docs/build/enhance-your-code.md
new file mode 100644
index 00000000000..5f2d48f6f5a
--- /dev/null
+++ b/website/docs/docs/build/enhance-your-code.md
@@ -0,0 +1,38 @@
+---
+title: "Enhance your code"
+description: "Learn how you can enhance your code"
+pagination_next: "docs/build/project-variables"
+pagination_prev: null
+---
+
+
+
\ No newline at end of file diff --git a/website/docs/docs/build/materializations.md b/website/docs/docs/build/materializations.md index ae75b575d5f..79fe2e1b7c5 100644 --- a/website/docs/docs/build/materializations.md +++ b/website/docs/docs/build/materializations.md @@ -2,6 +2,7 @@ title: "Materializations" description: "Read this tutorial to learn how to use materializations when building in dbt." id: "materializations" +pagination_next: "docs/build/incremental-models" --- ## Overview diff --git a/website/docs/docs/build/measures.md b/website/docs/docs/build/measures.md index ba82a4aa4a5..e06b5046976 100644 --- a/website/docs/docs/build/measures.md +++ b/website/docs/docs/build/measures.md @@ -234,6 +234,15 @@ metrics: We can query the semi-additive metrics using the following syntax: +For dbt Cloud: + +```bash +dbt sl query --metrics mrr_by_end_of_month --dimensions metric_time__month --order metric_time__month +dbt sl query --metrics mrr_by_end_of_month --dimensions metric_time__week --order metric_time__week +``` + +For dbt Core: + ```bash mf query --metrics mrr_by_end_of_month --dimensions metric_time__month --order metric_time__month mf query --metrics mrr_by_end_of_month --dimensions metric_time__week --order metric_time__week diff --git a/website/docs/docs/build/metricflow-cli.md b/website/docs/docs/build/metricflow-commands.md similarity index 62% rename from website/docs/docs/build/metricflow-cli.md rename to website/docs/docs/build/metricflow-commands.md index 2650b2215ae..049b415d40e 100644 --- a/website/docs/docs/build/metricflow-cli.md +++ b/website/docs/docs/build/metricflow-commands.md @@ -1,69 +1,142 @@ --- -title: MetricFlow CLI -id: metricflow-cli -description: "Query metrics and metadata in your dbt project with the metricflow cli" -sidebar_label: "MetricFlow CLI commands" +title: MetricFlow commands +id: metricflow-commands +description: "Query metrics and metadata in your dbt project with the MetricFlow commands." +sidebar_label: "MetricFlow commands" tags: [Metrics, Semantic Layer] --- -Once you define metrics in your dbt project, you can query metrics, dimensions, dimension values, and validate your configs using the MetricFlow command line (CLI). +Once you define metrics in your dbt project, you can query metrics, dimensions, dimension values, and validate your configs using the MetricFlow commands. -# Installation +MetricFlow allows you to define and query metrics in your dbt project in the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation), [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud), or [dbt Core](/docs/core/installation). To experience the power of the universal [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) and dynamically query those metrics in downstream tools, you'll need a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account. -You can install the [MetricFlow CLI](https://github.com/dbt-labs/metricflow#getting-started) from [PyPI](https://pypi.org/project/dbt-metricflow/). You need to use `pip` to install the MetricFlow CLI on Windows or Linux operating systems: +MetricFlow is compatible with Python versions 3.8, 3.9, 3.10 and 3.11. -1. Create or activate your virtual environment.`python -m venv venv` + +## MetricFlow + +MetricFlow is a dbt package that allows you to define and query metrics in your dbt project. You can use MetricFlow to query metrics in your dbt project in the dbt Cloud CLI, dbt Cloud IDE, or dbt Core. + +**Note** — MetricFlow commands aren't supported in dbt Cloud jobs yet. However, you can add MetricFlow validations with your git provider (such as GitHub Actions) by installing MetricFlow (`pip install metricflow`). This allows you to run MetricFlow commands as part of your continuous integration checks on PRs. + +
+
+
+
+MetricFlow commands are embedded in the dbt Cloud CLI, which means you can immediately run them once you install the dbt Cloud CLI.
+
+A benefit to using the dbt Cloud CLI or dbt Cloud IDE is that you won't need to manage versioning — your dbt Cloud account will automatically manage the versioning for you.
+
+
+
+
+
+:::info
+You can create metrics using MetricFlow in the dbt Cloud IDE. However, support for running MetricFlow commands in the IDE will be available soon.
+:::
+
+
+
+
+
+
+:::info Use dbt Cloud CLI for semantic layer development
+
+Use the dbt Cloud CLI for the experience in defining and querying metrics in your dbt project on dbt Cloud or dbt Core with MetricFlow.
+
+A benefit to using the dbt Cloud CLI or dbt Cloud IDE is that you won't need to manage versioning — your dbt Cloud account will automatically manage the versioning for you.
+:::
+
+
+You can install [MetricFlow](https://github.com/dbt-labs/metricflow#getting-started) from [PyPI](https://pypi.org/project/dbt-metricflow/). You need to use `pip` to instal MetricFlow on Windows or Linux operating systems:
+
+1. Create or activate your virtual environment`python -m venv venv`
2. Run `pip install dbt-metricflow`
+ * You can install MetricFlow using PyPI as an extension of your dbt adapter in the command line. To install the adapter, run `pip install "dbt-metricflow[your_adapter_name]"` and add the adapter name at the end of the command. For example, for a Snowflake adapter run `pip install "dbt-metricflow[snowflake]"`
- * You can install MetricFlow using PyPI as an extension of your dbt adapter in the CLI. To install the adapter, run `pip install "dbt-metricflow[your_adapter_name]"` and add the adapter name at the end of the command. For example, for a Snowflake adapter run `pip install "dbt-metricflow[snowflake]"`
+**Note**, you'll need to manage versioning between dbt Core, your adapter, and MetricFlow.
-The MetricFlow CLI is compatible with Python versions 3.8, 3.9, 3.10 and 3.11
+
+
+
-# CLI commands
-The MetricFlow CLI provides the following commands to retrieve metadata and query metrics.
+## MetricFlow commands
-To execute the commands, use the `mf` prefix before the command name. For example, to list all metrics, run `mf list metrics`:
+MetricFlow provides the following commands to retrieve metadata and query metrics.
+
+
+
+
+Use the `dbt sl` prefix before the command name to execute them in dbt Cloud. For example, to list all metrics, run `dbt sl list metrics`.
- [`list`](#list) — Retrieves metadata values.
- [`list metrics`](#list-metrics) — Lists metrics with dimensions.
- [`list dimensions`](#list) — Lists unique dimensions for metrics.
- [`list dimension-values`](#list-dimension-values) — List dimensions with metrics.
- [`list entities`](#list-entities) — Lists all unique entities.
+- [`query`](#query) — Query metrics and dimensions you want to see in the command line interface. Refer to [query examples](#query-examples) to help you get started.
+
+
+
+
-## List
+
+
+Use the `mf` prefix before the command name to execute them in dbt Core. For example, to list all metrics, run `mf list metrics`.
+
+- [`list`](#list) — Retrieves metadata values.
+- [`list metrics`](#list-metrics) — Lists metrics with dimensions.
+- [`list dimensions`](#list) — Lists unique dimensions for metrics.
+- [`list dimension-values`](#list-dimension-values) — List dimensions with metrics.
+- [`list entities`](#list-entities) — Lists all unique entities.
+- [`validate-configs`](#validate-configs) — Validates semantic model configurations.
+- [`health-checks`](#health-checks) — Performs data platform health check.
+- [`tutorial`](#tutorial) — Dedicated MetricFlow tutorial to help get you started.
+- [`query`](#query) — Query metrics and dimensions you want to see in the command line interface. Refer to [query examples](#query-examples) to help you get started.
+
+
+
+
+### List
This command retrieves metadata values related to [Metrics](/docs/build/metrics-overview), [Dimensions](/docs/build/dimensions), and [Entities](/docs/build/entities) values.
-## List metrics
+### List metrics
```bash
-mf list
-
+dbt sl list # In dbt Cloud
+mf list # In dbt Core
+```
This command lists the metrics with their available dimensions:
```bash
-mf list metrics
+dbt sl list metrics # In dbt Cloud
+
+mf list metrics # In dbt Core
+
Options:
--search TEXT Filter available metrics by this search term
--show-all-dimensions Show all dimensions associated with a metric.
--help Show this message and exit.
```
-## List dimensions
+### List dimensions
This command lists all unique dimensions for a metric or multiple metrics. It displays only common dimensions when querying multiple metrics:
```bash
-mf list dimensions --metrics
+dbt sl list dimensions --metrics # In dbt Cloud
+
+mf list dimensions --metrics # In dbt Core
+
Options:
- --metrics SEQUENCE List dimensions by given metrics (intersection). Ex.
- --metrics bookings,messages
+ --metrics SEQUENCE List dimensions by given metrics (intersection). Ex. --metrics bookings,messages
--help Show this message and exit.
```
@@ -72,7 +145,10 @@ Options:
This command lists all dimension values with the corresponding metric:
```bash
-mf list dimension-values --metrics --dimension
+dbt sl list dimension-values --metrics --dimension # In dbt Cloud
+
+mf list dimension-values --metrics --dimension # In dbt Core
+
Options:
--dimension TEXT Dimension to query values from [required]
--metrics SEQUENCE Metrics that are associated with the dimension
@@ -83,24 +159,30 @@ Options:
of the data (inclusive)
--help Show this message and exit.
```
-## List entities
+
+### List entities
This command lists all unique entities:
```bash
-mf list entities --metrics
+dbt sl list entities --metrics # In dbt Cloud
+
+mf list entities --metrics # In dbt Core
+
Options:
- --metrics SEQUENCE List entities by given metrics (intersection). Ex.
- --metrics bookings,messages
+ --metrics SEQUENCE List entities by given metrics (intersection). Ex. --metrics bookings,messages
--help Show this message and exit.
```
-## Validate-configs
+### Validate-configs
This command performs validations against the defined semantic model configurations:
```bash
-mf validate-configs
+dbt sl validate-configs # In dbt Cloud
+
+mf validate-configs # In dbt Core
+
Options:
--dw-timeout INTEGER Optional timeout for data warehouse
validation steps. Default None.
@@ -118,28 +200,34 @@ Options:
--help Show this message and exit.
```
-## Health checks
+### Health checks
This command performs a health check against the data platform you provided in the configs:
```bash
-mf health-checks
+dbt sl health-checks #in dbt Cloud
+
+mf health-checks #in dbt Core
```
-## Tutorial
+### Tutorial
Follow the dedicated MetricFlow tutorial to help you get started:
```bash
-mf tutorial
+dbt sl tutorial # In dbt Cloud
+
+mf tutorial # In dbt Core
```
-## Query
+### Query
Create a new query with MetricFlow, execute that query against the user's data platform, and return the result:
```bash
-mf query --metrics --group-by
+dbt sl query --metrics --group-by # In dbt Cloud
+
+mf query --metrics --group-by # In dbt Core
Options:
@@ -170,8 +258,9 @@ Options:
--csv FILENAME Provide filepath for data frame output to csv
- --explain In the query output, show the query that was
- executed against the data warehouse
+ --compile (dbt Cloud) In the query output, show the query that was
+ --explain (dbt Core) executed against the data warehouse
+
--show-dataflow-plan Display dataflow plan in explain output
@@ -186,7 +275,7 @@ Options:
```
-## Query examples
+### Query examples
The following tabs present various different types of query examples that you can use to query metrics and dimensions. Select the tab that best suits your needs:
@@ -198,7 +287,9 @@ Use the example to query metrics by dimension and return the `order_total` metri
**Query**
```bash
-mf query --metrics order_total --group-by metric_time
+dbt sl query --metrics order_total --group-by metric_time # In dbt Cloud
+
+mf query --metrics order_total --group-by metric_time # In dbt Core
```
**Result**
@@ -221,7 +312,9 @@ You can include multiple dimensions in a query. For example, you can group by th
**Query**
```bash
-mf query --metrics order_total --group-by metric_time, is_food_order
+dbt sl query --metrics order_total --group-by metric_time, is_food_order # In dbt Cloud
+
+mf query --metrics order_total --group-by metric_time, is_food_order # In dbt Core
```
**Result**
@@ -248,7 +341,11 @@ You can add order and limit functions to filter and present the data in a readab
**Query**
```bash
-mf query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time
+# In dbt Cloud
+dbt sl query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time
+
+# In dbt Core
+mf query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time
```
**Result**
@@ -273,7 +370,11 @@ You can further filter the data set by adding a `where` clause to your query.
**Query**
```bash
-mf query --metrics order_total --group-by metric_time --where "{{Dimension('order_id__is_food_order')}} = True"
+# In dbt Cloud
+dbt sl query --metrics order_total --group-by metric_time --where "{{ Dimension('order_id__is_food_order') }} = True"
+
+# In dbt Core
+mf query --metrics order_total --group-by metric_time --where "{{ Dimension('order_id__is_food_order') }} = True"
```
**Result**
@@ -301,7 +402,12 @@ To filter by time, there are dedicated start and end time options. Using these o
**Query**
```bash
- mf query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time --where "is_food_order = True" --start-time '2017-08-22' --end-time '2017-08-27'
+
+# In dbt Cloud
+dbt sl query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time --where "is_food_order = True" --start-time '2017-08-22' --end-time '2017-08-27'
+
+# In dbt Core
+mf query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time --where "is_food_order = True" --start-time '2017-08-22' --end-time '2017-08-27'
```
**Result**
@@ -331,20 +437,24 @@ The following tabs present additional query examples, like exporting to a CSV. S
-
+
-Add `--explain` to your query to view the SQL generated by MetricFlow.
+Add `--compile` (or `--explain` for dbt Core users) to your query to view the SQL generated by MetricFlow.
**Query**
```bash
- mf query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time --where "is_food_order = True" --start-time '2017-08-22' --end-time '2017-08-27' --explain
+# In dbt Cloud
+dbt sl query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time --where "is_food_order = True" --start-time '2017-08-22' --end-time '2017-08-27' --compile
+
+# In dbt Core
+mf query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time --where "is_food_order = True" --start-time '2017-08-22' --end-time '2017-08-27' --explain
```
**Result**
```bash
✔ Success 🦄 - query completed after 0.28 seconds
-🔎 SQL (remove --explain to see data or add --show-dataflow-plan to see the generated dataflow plan):
+🔎 SQL (remove --compile to see data or add --show-dataflow-plan to see the generated dataflow plan):
SELECT
metric_time
, is_food_order
@@ -374,6 +484,10 @@ Add the `--csv file_name.csv` flag to export the results of your query to a csv.
**Query**
```bash
+# In dbt Cloud
+dbt sl query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time --where "is_food_order = True" --start-time '2017-08-22' --end-time '2017-08-27' --csv query_example.csv
+
+# In dbt Core
mf query --metrics order_total --group-by metric_time,is_food_order --limit 10 --order -metric_time --where "is_food_order = True" --start-time '2017-08-22' --end-time '2017-08-27' --csv query_example.csv
```
@@ -386,14 +500,16 @@ mf query --metrics order_total --group-by metric_time,is_food_order --limit 10 -
-## Time granularity
+### Time granularity
Optionally, you can specify the time granularity you want your data to be aggregated at by appending two underscores and the unit of granularity you want to `metric_time`, the global time dimension. You can group the granularity by: `day`, `week`, `month`, `quarter`, and `year`.
Below is an example for querying metric data at a monthly grain:
```bash
-mf query --metrics revenue --group-by metric_time__month
+dbt sl query --metrics revenue --group-by metric_time__month # In dbt Cloud
+
+mf query --metrics revenue --group-by metric_time__month # In dbt Core
```
## FAQs
@@ -403,7 +519,7 @@ mf query --metrics revenue --group-by metric_time__month
To add a dimension filter to a where filter, you have to indicate that the filter item is part of your model and use a template wrapper:
Before using the template wrapper, however, you will need to set up your terminal to escape curly braces for the filter template to work. +Here's an example query:
Before using the template wrapper, however, you will need to set up your terminal to escape curly braces for the filter template to work.
+ `dependencies.yml` or `packages.yml` to your dbt project. This should be at the same level as your `dbt_project.yml` file.
2. Specify the package(s) you wish to add using one of the supported syntaxes, for example:
@@ -366,3 +372,4 @@ packages:
```
+
diff --git a/website/docs/docs/build/project-variables.md b/website/docs/docs/build/project-variables.md
index a69132d6a3b..1fe45b18ac0 100644
--- a/website/docs/docs/build/project-variables.md
+++ b/website/docs/docs/build/project-variables.md
@@ -1,6 +1,7 @@
---
title: "Project variables"
id: "project-variables"
+pagination_next: "docs/build/environment-variables"
---
dbt provides a mechanism, [variables](/reference/dbt-jinja-functions/var), to provide data to models for
diff --git a/website/docs/docs/build/projects.md b/website/docs/docs/build/projects.md
index 0d7dd889fa6..b4b04e3334d 100644
--- a/website/docs/docs/build/projects.md
+++ b/website/docs/docs/build/projects.md
@@ -1,6 +1,8 @@
---
title: "About dbt projects"
id: "projects"
+pagination_next: null
+pagination_prev: null
---
A dbt project informs dbt about the context of your project and how to transform your data (build your data sets). By design, dbt enforces the top-level structure of a dbt project such as the `dbt_project.yml` file, the `models` directory, the `snapshots` directory, and so on. Within the directories of the top-level, you can organize your project in any way that meets the needs of your organization and data pipeline.
diff --git a/website/docs/docs/build/saved-queries.md b/website/docs/docs/build/saved-queries.md
new file mode 100644
index 00000000000..39a4b2e52fd
--- /dev/null
+++ b/website/docs/docs/build/saved-queries.md
@@ -0,0 +1,43 @@
+---
+title: Saved queries
+id: saved-queries
+description: "Saved queries are a way to save commonly used queries in MetricFlow. They can be used to save time and avoid writing the same query over and over again."
+sidebar_label: "Saved queries"
+tags: [Metrics, Semantic Layer]
+---
+
+:::info Saved queries coming soon
+Saved queries isn't currently available in MetricFlow but support is coming soon.
+:::
+
+Saved queries are a way to save commonly used queries in MetricFlow. You can group metrics, dimensions, and filters that are logically related into a saved query.
+
+To define a saved query, refer to the following specification:
+
+ Parameter | Description | Type |
+| --------- | ----------- | ---- |
+| `name` | The name of the metric. | Required |
+| `description` | The description of the metric. | Optional |
+| `metrics` | The metrics included in the saved query. | Required |
+| `group_bys` | The value displayed in downstream tools. | Required |
+| `where` | Filter applied to the query. | Optional |
+
+The following is an example of a saved query:
+
+```yaml
+saved_query:
+ name: p0_booking
+ description: Booking-related metrics that are of the highest priority.
+ metrics:
+ - bookings
+ - instant_bookings
+ group_bys:
+ - TimeDimension('metric_time', 'day')
+ - Dimension('listing__capacity_latest')
+ where:
+ - "{{ Dimension('listing__capacity_latest') }} > 3"
+```
+
+### FAQs
+
+* All metrics in a saved query need to use the same dimensions in the `group_by` or `where` clauses.
diff --git a/website/docs/docs/build/semantic-models.md b/website/docs/docs/build/semantic-models.md
index 3341f49609a..226c138b545 100644
--- a/website/docs/docs/build/semantic-models.md
+++ b/website/docs/docs/build/semantic-models.md
@@ -6,6 +6,7 @@ keywords:
- dbt metrics layer
sidebar_label: Semantic models
tags: [Metrics, Semantic Layer]
+pagination_next: "docs/build/dimensions"
---
Semantic models are the foundation for data definition in MetricFlow, which powers the dbt Semantic Layer:
diff --git a/website/docs/docs/build/simple.md b/website/docs/docs/build/simple.md
index 7022ca9d007..1803e952a69 100644
--- a/website/docs/docs/build/simple.md
+++ b/website/docs/docs/build/simple.md
@@ -4,6 +4,7 @@ id: simple
description: "Use simple metrics to directly reference a single measure."
sidebar_label: Simple
tags: [Metrics, Semantic Layer]
+pagination_next: null
---
Simple metrics are metrics that directly reference a single measure, without any additional measures involved. They are aggregations over a column in your data platform and can be filtered by one or multiple dimensions.
diff --git a/website/docs/docs/build/sl-getting-started.md b/website/docs/docs/build/sl-getting-started.md
index c0bf59ae0c2..34c3479cbc0 100644
--- a/website/docs/docs/build/sl-getting-started.md
+++ b/website/docs/docs/build/sl-getting-started.md
@@ -8,38 +8,33 @@ meta:
api_name: dbt Semantic Layer APIs
---
-import InstallMetricFlow from '/snippets/_sl-install-metricflow.md';
import CreateModel from '/snippets/_sl-create-semanticmodel.md';
import DefineMetrics from '/snippets/_sl-define-metrics.md';
import ConfigMetric from '/snippets/_sl-configure-metricflow.md';
import TestQuery from '/snippets/_sl-test-and-query-metrics.md';
+import ConnectQueryAPI from '/snippets/_sl-connect-and-query-api.md';
+import RunProdJob from '/snippets/_sl-run-prod-job.md';
-This getting started page presents a sample workflow to help you create your first metrics in dbt Cloud or the command-line interface (CLI). It uses the [Jaffle shop example project](https://github.com/dbt-labs/jaffle-sl-template) as the project data source and is available for you to use.
+This getting started page presents a sample workflow to help you create your first metrics in dbt Cloud or the command line interface (CLI). It uses the [Jaffle shop example project](https://github.com/dbt-labs/jaffle-sl-template) as the project data source and is available for you to use.
If you prefer, you can create semantic models and metrics for your own dbt project. This page will guide you on how to:
- [Create a semantic model](#create-a-semantic-model) using MetricFlow
- [Define metrics](#define-metrics) using MetricFlow
-- [Test and query metrics locally](#test-and-query-metrics) using MetricFlow
+- [Test and query metrics](#test-and-query-metrics) using MetricFlow
- [Run a production job](#run-a-production-job) in dbt Cloud
- [Set up dbt Semantic Layer](#set-up-dbt-semantic-layer) in dbt Cloud
- [Connect to and query the API](#connect-and-query-api) with dbt Cloud
-
-MetricFlow allows users to define metrics in their dbt project whether in dbt Cloud or in dbt Core. dbt Core users can use the [MetricFlow CLI](/docs/build/metricflow-cli) to define metrics in their local dbt Core project.
+MetricFlow allows you to define metrics in your dbt project and query them whether in dbt Cloud or dbt Core with [MetricFlow commands](/docs/build/metricflow-commands).
However, to experience the power of the universal [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) and query those metrics in downstream tools, you'll need a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account.
## Prerequisites
-- Have an understanding of key concepts in [MetricFlow](/docs/build/about-metricflow), which powers the revamped dbt Semantic Layer.
-- Have both your production and development environments running dbt version 1.6 or higher. Refer to [upgrade in dbt Cloud](/docs/dbt-versions/upgrade-core-in-cloud) for more info.
-- Use Snowflake, BigQuery, Databricks, Redshift, or Postgres (Postgres available in the CLI only, dbt Cloud support coming soon).
-- Create a successful run in the environment where you configure the Semantic Layer.
- - **Note:** Semantic Layer currently supports the Deployment environment for querying. (_development querying experience coming soon_)
-- Set up the [Semantic Layer API](/docs/dbt-cloud-apis/sl-api-overview) in the integrated tool to import metric definitions.
- - **Note:** To access the API and query metrics in downstream tools, you must have a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account. dbt Core or Developer accounts can define metrics using [MetricFlow CLI](/docs/build/metricflow-cli) or the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud).
-- Understand [MetricFlow's](/docs/build/about-metricflow) key concepts, which powers the revamped dbt Semantic Layer. +import SetUp from '/snippets/_v2-sl-prerequisites.md'; + +
- - * To authenticate, you need to [generate a service token](/docs/dbt-cloud-apis/service-tokens) with Semantic Layer Only and Metadata Only permissions. - * Refer to the [SQL query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the API. - -- To learn more about the sophisticated integrations that connect to the dbt Semantic Layer, refer to [Available integrations](/docs/use-dbt-semantic-layer/avail-sl-integrations) for more info. +
+
+ ```yml
+dever-env-id: '123456'
+```
+
+
+
+
+
+
+```yml
+dbt_cloud:
+ dever-env-id: '123456'
+```
+
+
diff --git a/website/docs/docs/cloud/about-cloud-develop.md b/website/docs/docs/cloud/about-cloud-develop.md
new file mode 100644
index 00000000000..9f864ede5ca
--- /dev/null
+++ b/website/docs/docs/cloud/about-cloud-develop.md
@@ -0,0 +1,33 @@
+---
+title: About developing in dbt Cloud
+id: about-cloud-develop
+description: "Learn how to develop your dbt projects using dbt Cloud."
+sidebar_label: "About developing in dbt Cloud"
+pagination_next: "docs/cloud/cloud-cli-installation"
+hide_table_of_contents: true
+---
+
+dbt Cloud offers a fast and reliable way to work on your dbt project. It runs dbt Core in a hosted (single or multi-tenant) environment. You can develop in your browser using an integrated development environment (IDE) or in a dbt Cloud-powered command line interface (CLI):
+
+
+ +The following sections provide detailed instructions on setting up the dbt Cloud CLI and dbt Cloud IDE. To get started with dbt development, you'll need a [developer](/docs/cloud/manage-access/seats-and-users) account. For a more comprehensive guide about developing in dbt, refer to our [quickstart guides](/quickstarts). + + +--------- +**Note**: The dbt Cloud CLI and the open-sourced dbt Core are both command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). + diff --git a/website/docs/docs/cloud/about-cloud-setup.md b/website/docs/docs/cloud/about-cloud-setup.md index baa2465472e..7b68b52a45a 100644 --- a/website/docs/docs/cloud/about-cloud-setup.md +++ b/website/docs/docs/cloud/about-cloud-setup.md @@ -3,6 +3,8 @@ title: About dbt Cloud setup id: about-cloud-setup description: "Configuration settings for dbt Cloud." sidebar_label: "About dbt Cloud setup" +pagination_next: "docs/dbt-cloud-environments" +pagination_prev: null --- dbt Cloud is the fastest and most reliable way to deploy your dbt jobs. It contains a myriad of settings that can be configured by admins, from the necessities (data platform integration) to security enhancements (SSO) and quality-of-life features (RBAC). This portion of our documentation will take you through the various settings found by clicking on the gear icon in the dbt Cloud UI, including: @@ -11,6 +13,8 @@ dbt Cloud is the fastest and most reliable way to deploy your dbt jobs. It conta - Configuring access to [GitHub](/docs/cloud/git/connect-github), [GitLab](/docs/cloud/git/connect-gitlab), or your own [git repo URL](/docs/cloud/git/import-a-project-by-git-url). - [Managing users and licenses](/docs/cloud/manage-access/seats-and-users) - [Configuring secure access](/docs/cloud/manage-access/about-user-access) +- Configuring the [dbt Cloud IDE](/docs/cloud/about-cloud-develop) +- Installing and configuring the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) These settings are intended for dbt Cloud administrators. If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](/quickstarts). diff --git a/website/docs/docs/cloud/about-cloud/dbt-cloud-features.md b/website/docs/docs/cloud/about-cloud/about-dbt-cloud.md similarity index 78% rename from website/docs/docs/cloud/about-cloud/dbt-cloud-features.md rename to website/docs/docs/cloud/about-cloud/about-dbt-cloud.md index f1d8b32cdb1..71f3175a108 100644 --- a/website/docs/docs/cloud/about-cloud/dbt-cloud-features.md +++ b/website/docs/docs/cloud/about-cloud/about-dbt-cloud.md @@ -4,86 +4,95 @@ id: "dbt-cloud-features" sidebar_label: "dbt Cloud features" description: "Explore dbt Cloud's features and learn why dbt Cloud is the fastest way to deploy dbt" hide_table_of_contents: true +pagination_next: "docs/cloud/about-cloud/architecture" +pagination_prev: null --- -dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, schedule, document, and investigate data models all in one browser-based UI. In addition to providing a hosted architecture for running dbt across your organization, dbt Cloud comes equipped with turnkey support for scheduling jobs, CI/CD, hosting documentation, monitoring & alerting, and an integrated development environment (IDE). +dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, schedule, document, and investigate data models all in one browser-based UI. + +In addition to providing a hosted architecture for running dbt across your organization, dbt Cloud comes equipped with turnkey support for scheduling jobs, CI/CD, hosting documentation, monitoring and alerting, an integrated development environment (IDE), and allows you to develop and run dbt commands from your local command line interface (CLI) or code editor. dbt Cloud's [flexible plans](https://www.getdbt.com/pricing/) and features make it well-suited for data teams of any size — sign up for your [free 14-day trial](https://www.getdbt.com/signup/)!
*These features are available on [selected plans](https://www.getdbt.com/pricing/). diff --git a/website/docs/docs/cloud/about-cloud/browsers.md b/website/docs/docs/cloud/about-cloud/browsers.md index 2fc5a8b4b4d..12665bc7b72 100644 --- a/website/docs/docs/cloud/about-cloud/browsers.md +++ b/website/docs/docs/cloud/about-cloud/browsers.md @@ -2,6 +2,7 @@ title: "Supported browsers" id: "browsers" description: "dbt Cloud supports the latest browsers like Chrome and Firefox." +pagination_next: null --- To have the best experience with dbt Cloud, we recommend using the latest versions of the following browsers: diff --git a/website/docs/docs/cloud/billing.md b/website/docs/docs/cloud/billing.md index 61251f6e41d..1d71d33e9a1 100644 --- a/website/docs/docs/cloud/billing.md +++ b/website/docs/docs/cloud/billing.md @@ -3,28 +3,81 @@ title: "Billing" id: billing description: "dbt Cloud billing information." sidebar_label: Billing +pagination_next: null +pagination_prev: null --- dbt Cloud offers a variety of [plans and pricing](https://www.getdbt.com/pricing/) to fit your organization’s needs. With flexible billing options that appeal to large enterprises and small businesses and [server availability](/docs/cloud/about-cloud/regions-ip-addresses) worldwide, dbt Cloud is the fastest and easiest way to begin transforming your data. ## How does dbt Cloud pricing work? -As a customer, you pay for the number of seats you have and the amount of usage consumed each month. Usage is based on the number of Successful Models Built, and seats are billed primarily on the amount of Developer licenses purchased. All billing computations are conducted in Coordinated Universal Time (UTC). +As a customer, you pay for the number of seats you have and the amount of usage consumed each month. Seats are billed primarily on the amount of Developer and Read licenses purchased. Usage is based on the number of [Successful Models Built](#what-counts-as-a-successful-model-built) and, if purchased and used, Semantic Layer Query Units subject to reasonable usage. All billing computations are conducted in Coordinated Universal Time (UTC). + +### What counts as a seat license? + +There are three types of possible seat licenses: + +* **Developer** — for roles and permissions that require interaction with the dbt Cloud environment day-to-day. +* **Read-Only** — for access to view certain documents and reports. +* **IT** — for access to specific features related to account management (for example, configuring git integration). ### What counts as a Successful Model Built? -dbt Cloud considers a Successful Model Built as any model that is successfully built via a run through dbt Cloud’s orchestration functionality in a dbt Cloud deployment environment. Models are counted when built and run. This includes any jobs run via dbt Cloud's scheduler, CI builds (jobs triggered by pull requests), runs kicked off via the dbt Cloud API, and any successor dbt Cloud tools with similar functionality. This also includes models that are successfully built even when a run may fail to complete. For example, you may have a job that contains 100 models and on one of its runs, 51 models are successfully built and then the job fails. In this situation, only 51 models would be counted. +dbt Cloud considers a Successful Model Built as anymodel that is successfully built via a run through dbt Cloud’s orchestration functionality in a dbt Cloud deployment environment. Models are counted when built and run. This includes any jobs run via dbt Cloud's scheduler, CI builds (jobs triggered by pull requests), runs kicked off via the dbt Cloud API, and any successor dbt Cloud tools with similar functionality. This also includes models that are successfully built even when a run may fail to complete. For example, you may have a job that contains 100 models and on one of its runs, 51 models are successfully built and then the job fails. In this situation, only 51 models would be counted.
Any models built in a dbt Cloud development environment (for example, via the IDE) do not count towards your usage. Tests, seeds, ephemeral models, and snapshots also do not count.
+| What counts towards Successful Models Built | |
+|---------------------------------------------|---------------------|
+| View | ✅ |
+| Table | ✅ |
+| Incremental | ✅ |
+| Ephemeral Models | ❌ |
+| Tests | ❌ |
+| Seeds | ❌ |
+| Snapshots | ❌ |
-### What counts as a seat license?
+### What counts as a Query Unit?
-There are three types of possible seat licenses:
+The dbt Semantic Layer, powered by MetricFlow, measures usage in distinct query units. Every successful request you make to render or run SQL to the Semantic Layer API counts as at least one query unit, even if no data is returned. If the query calculates or renders SQL for multiple metrics, each calculated metric will be counted as a query unit.
+If a request to run a query is not executed successfully in the data platform or if a query results in an error without completion, it is not counted as a query unit. Requests for metadata from the Semantic Layer are also not counted as query units.
-* **Developer** — for roles and permissions that require interaction with the dbt Cloud environment day-to-day.
-* **Read-Only** — for access to view certain documents and reports.
-* **IT** — for access to specific features related to account management (for example, configuring git integration).
+Examples of query units include:
+
+Querying one metric, grouping by one dimension → 1 query unit
+
+```shell
+dbt sl query --metrics revenue --group_by metric_time
+```
+Querying one metric, grouping by two dimensions → 1 query unit
+
+```shell
+dbt sl query --metrics revenue --group_by metric_time,user__country
+```
+
+Querying two metrics, grouping by two dimensions → 2 query units
+
+```shell
+dbt sl query --metrics revenue,gross_sales --group_by metric_time,user__country
+```
+
+Running an explain for one metric → 1 query unit
+
+```shell
+dbt sl query --metrics revenue --group_by metric_time --explain
+```
+
+Running an explain for two metrics → 2 query units
+
+```shell
+dbt sl query --metrics revenue,gross_sales --group_by metric_time --explain
+```
+
+Running a query for only dimensions such as dimension_values or a query with no metrics → 1 query unit
+
+```shell
+bt sl list dimension-values --dimension user__country
+```
### Viewing usage in the product
@@ -59,7 +112,7 @@ All included successful models built numbers above reflect our most current pric
Team customers pay monthly via credit card for seats and usage, and accounts include 15,000 models monthly. Seats are charged upfront at the beginning of the month. If you add seats during the month, seats will be prorated and charged on the same day. Seats removed during the month will be reflected on the next invoice and are not eligible for refunds. You can change the credit card information and the number of seats from the billings section anytime. Accounts will receive one monthly invoice that includes the upfront charge for the seats and the usage charged in arrears from the previous month.
-Usage is calculated and charged in arrears for the previous month. If you exceed 15,000 models in any month, you will be billed for additional usage on your next invoice. Additional use is billed at the rates on our [pricing page](https://www.getdbt.com/pricing).
+Usage is calculated and charged in arrears for the previous month. If you exceed 15,000 models in any month, you will be billed for additional usage on your next invoice. Additional usage is billed at the rates on our [pricing page](https://www.getdbt.com/pricing).
Included models that are not consumed do not roll over to future months. You can estimate your bill with a simple formula:
@@ -68,15 +121,22 @@ Included models that are not consumed do not roll over to future months. You can
All included successful models built numbers above reflect our most current pricing and packaging. Based on your usage terms when you signed up for the Team Plan, the included model entitlements may be different from what’s reflected above.
-:::note Legacy pricing plans
-
-Customers who purchased the dbt Cloud Team plan before August 11, 2023, remain on a legacy pricing plan as long as their account is in good standing. The legacy pricing plan is based on seats and includes unlimited models subject to reasonable use. dbt Labs may institute use limits if reasonable use is exceeded. Additional features, upgrades, or updates may be subject to separate charges. Any changes to your current plan pricing will be communicated in advance according to our Terms of Use.
+### Enterprise plan billing
+
+As an Enterprise customer, you pay annually via invoice, monthly in arrears for additional usage (if applicable), and may benefit from negotiated usage rates. Please refer to your order form or contract for your specific pricing details, or [contact the account team](https://www.getdbt.com/contact-demo) with any questions.
+
+### Legacy plans
+
+Customers who purchased the dbt Cloud Team plan before August 11, 2023, remain on a legacy pricing plan as long as your account is in good standing. The legacy pricing plan is based on seats and includes unlimited models, subject to reasonable use.
+
+:::note Legacy Semantic Layer
+
+For customers using the legacy Semantic Layer with dbt_metrics package, this product will be deprecated in December 2023. Legacy users may choose to upgrade at any time to the revamped version, Semantic Layer powered by MetricFlow. The revamped version is available to most customers (see [prerequisites](/docs/use-dbt-semantic-layer/quickstart-sl#prerequisites)) for a limited time on a free trial basis, subject to reasonable use.
:::
-### Enterprise plan billing
+dbt Labs may institute use limits if reasonable use is exceeded. Additional features, upgrades, or updates may be subject to separate charges. Any changes to your current plan pricing will be communicated in advance according to our Terms of Use.
-As an Enterprise customer, you pay annually via invoice, monthly in arrears for additional usage (if applicable), and may benefit from negotiated usage rates. Please refer to your order form or contract for your specific pricing details, or [contact the account team](https://www.getdbt.com/contact-demo) with any questions.
## Managing usage
@@ -191,3 +251,10 @@ _Yes. Your dbt Cloud account will be upgraded without impacting your existing pr
* How do I determine the right plan for me?
_The best option is to consult with our sales team. They'll help you figure out what is right for your needs. We also offer a free two-week trial on the Team plan._
+
+* What are the Semantic Layer trial terms?
+_Team and Enterprise customers can sign up for a free trial of the dbt Semantic Layer, powered by MetricFlow, for use of up to 1,000 query units per month. The trial will be available at least through January 2024. dbt Labs may extend the trial period in its sole discretion. During the trial period, we may reach out to discuss pricing options or ask for feedback. At the end of the trial, free access may be removed and a purchase may be required to continue use. dbt Labs reserves the right to change limits in a free trial or institute pricing when required or at any time in its sole discretion._
+
+* What is the reasonable use limitation for the dbt Semantic Layer powered by MetricFlow during the trial?
+_Each account will be limited to 1,000 Queried Metrics per month during the trial period and may be changed at the sole discretion of dbt Labs._
+
diff --git a/website/docs/docs/cloud/cloud-cli-installation.md b/website/docs/docs/cloud/cloud-cli-installation.md
index 44d411bbf2d..6d3060d9c23 100644
--- a/website/docs/docs/cloud/cloud-cli-installation.md
+++ b/website/docs/docs/cloud/cloud-cli-installation.md
@@ -1,16 +1,14 @@
---
-title: Installing the dbt Cloud CLI (Alpha)
+title: Install dbt Cloud CLI
+sidebar_label: "Install dbt Cloud CLI"
id: cloud-cli-installation
description: "Instructions for installing and configuring dbt Cloud CLI"
+pagination_next: "docs/cloud/configure-cloud-cli"
---
-:::warning Alpha functionality
+import CloudCLIFlag from '/snippets/_cloud-cli-flag.md';
-The following installation instructions are for the dbt Cloud CLI, currently in Alpha (actively in development and being tested).
-
-These instructions are not intended for general audiences at this time.
-
-:::
+
+
+
+
+:::info Use native packages or a virtual environment to avoid overriding dbt Core
+
+Installing the dbt Cloud CLI with pip replaces dbt Core. This change can be avoided by using the native install method and configuring your PATH or by creating a new virtual environment.
+
+Otherwise, to switch back to dbt Core, uninstall the dbt Cloud CLI and follow the dbt Core installation instructions.
+
+:::
+
+Before installing the dbt Cloud CLI, make sure you have Python installed and your virtual environment venv or pyenv . If you already have a Python environment configured, you can skip to the [pip installation step](#install-dbt-cloud-cli-in-pip).
+
+### Install a virtual environment
+
+We recommend using virtual environments (venv) to namespace `cloud-cli`.
+
+1. Create a new venv:
+ ```shell
+ python3 -m venv dbt-cloud
+ ```
+
+2. Activate the virtual environment each time you create a shell window or session:
+ ```shell
+ source dbt-cloud/bin/activate # activate the environment for Mac and Linux OR
+ dbt-env\Scripts\activate # activate the environment for Windows
+ ```
+
+3. (Mac and Linux only) Create an alias to activate your dbt environment with every new shell window or session. You can add the following to your shell's configuration file (for example, $HOME/.bashrc, $HOME/.zshrc) while replacing `` with the path to your virtual environment configuration:
+ ```shell
+ alias env_dbt='source /bin/activate'
+ ```
+
+### Install dbt Cloud CLI in pip
+
+1. (Optional) If you already have dbt Core installed, this installation will override that package. Note your dbt Core version in case you need to reinstall it later:
+
+ ```bash
+ dbt --version
+ ```
+
+2. Make sure you're in your virtual environment and run the following command to install the dbt Cloud CLI:
+
+ ```bash
+ pip3 install dbt
+ ```
+
+3. (Optional) To revert back to dbt Core, first uninstall both the dbt Cloud CLI and dbt Core
+4. Reinstall dbt Core using the version from Step 2.
+
+ ```bash
+ pip3 uninstall dbt-core dbt
+ pip3 install dbt-core==VERSION
+ ```
+
+
-
+
+
+Before you begin, make sure you have [Homebrew installed](http://brew.sh/) in your code editor or command line terminal. Refer to the [FAQs](#faqs) if your operating system runs into path conflicts.
-Before you begin, make sure you have [Homebrew installed](http://brew.sh/) in your code editor or command line terminal. If your operating system runs into path conflicts, refer to the [FAQs](#faqs).
1. Run the following command to verify that there is no conflict with a dbt Core installation on your system:
-```bash
-which dbt
-```
+ ```bash
+ which dbt
+ ```
- This should return a `dbt not found`. If the dbt help text appears, use `pip uninstall dbt` to deactivate dbt Core from your machine.
2. Install the dbt Cloud CLI with Homebrew:
-```bash
-brew tap dbt-labs/dbt-cli
-brew install dbt-cloud-cli
-```
+ ```bash
+ brew untap dbt-labs/dbt
+ brew tap dbt-labs/dbt-cli
+ brew install dbt
+ ```
3. Verify the installation by running `dbt --help` from the command line. If the help text doesn't indicate that you're using the dbt Cloud CLI, make sure you've deactivated your pyenv or venv and don't have a version of dbt globally installed.
-
-* You no longer need to use the `dbt deps` command. Previously, you had to run that command.
+ * You don't have to run the `dbt deps` command when your environment starts. Previously, you had to do it during initialization. However, you'll still need to run `dbt deps` if you make changes to your `packages.yml` file.
-
+
-If your operating system runs into path conflicts, refer to the [FAQs](#faqs).
+Refer to the [FAQs](#faqs) if your operating system runs into path conflicts.
1. Download the latest Windows release for your platform from [GitHub](https://github.com/dbt-labs/dbt-cli/releases).
-2. Extract the `dbt.exe` executeable into the same folder as your dbt project.
+2. Extract the `dbt.exe` executable into the same folder as your dbt project.
:::info
Advanced users can configure multiple projects to use the same dbt Cloud CLI by placing the executable in the Program Files folder and [adding it to their Windows PATH environment variable](https://medium.com/@kevinmarkvi/how-to-add-executables-to-your-path-in-windows-5ffa4ce61a53).
-Note that if you are using VS Code, you'll need to restart it to pick up modified environment variables.
+Note that if you are using VS Code, you must restart it to pick up modified environment variables.
:::
3. Verify the installation by running `./dbt --help` from the command line. If the help text doesn't indicate that you're using the dbt Cloud CLI, make sure you've deactivated your pyenv or venv and don't have a version of dbt globally installed.
-
-* You no longer need to use the `dbt deps` command. Previously, you had to run that command.
+ * You don't have to run the `dbt deps` command when your environment starts. Previously, you had to do it during initialization. However, you'll still need to run `dbt deps` if you make changes to your `packages.yml` file.
-
+
Refer to the [FAQs](#faqs) if your operating system runs into path conflicts.
@@ -83,10 +142,10 @@ Refer to the [FAQs](#faqs) if your operating system runs into path conflicts.
2. Extract the `dbt-cloud-cli` binary to the same folder as your dbt project.
-```bash
-tar -xf dbt_0.29.9_linux_amd64.tar.gz
-./dbt --version
-```
+ ```bash
+ tar -xf dbt_0.29.9_linux_amd64.tar.gz
+ ./dbt --version
+ ```
:::info
@@ -95,89 +154,57 @@ Advanced users can configure multiple projects to use the same Cloud CLI executa
:::
3. Verify the installation by running `./dbt --help` from the command line. If the help text doesn't indicate that you're using the dbt Cloud CLI, make sure you've deactivated your pyenv or venv and don't have a version of dbt globally installed.
-
-* You no longer need to use the `dbt deps` command. Previously, you had to run that command.
+ * You don't have to run the `dbt deps` command when your environment starts. Previously, you had to do it during initialization. However, you'll still need to run `dbt deps` if you make changes to your `packages.yml` file.
+
## Update dbt Cloud CLI
The following instructions explain how to update the dbt CLoud CLI to the latest version depending on your operating system.
-
-
-
-
-To update the dbt Cloud CLI, run `brew upgrade dbt-cloud-cli`.
+During the public preview period, we recommend updating before filing a bug report. This is because the API is subject to breaking changes.
-
-
-To update, follow the same process explained in [Install manually (Windows)](/docs/cloud/cloud-cli-installation?install=windows#install-dbt-cloud-cli) and replace the existing `dbt.exe` executable with the new one.
+
+
+To update:
+- Make sure you're in your virtual environment
+- Run `pip install --upgrade dbt`.
+
-
-
-## Configure the dbt Cloud CLI
-
-After installation, you can configure the dbt Cloud CLI for your dbt Cloud project and use it to run [dbt commands](/reference/dbt-commands) similar to dbt Core. For example, you can execute the following command to compile a project using dbt Cloud:
+
-```bash
-dbt compile
-```
+To update the dbt Cloud CLI, run `brew upgrade dbt`. (You can also use `brew install dbt`).
-**Prerequisites**
-
-- You must set up a project in dbt Cloud.
-- You must have your [personal development credentials](/docs/dbt-cloud-environments#set-developer-credentials) set for that project. The dbt Cloud CLI will use these credentials, stored securely in dbt Cloud, to communicate with your data platform.
-- You must [enroll](/docs/dbt-versions/experimental-features) in the dbt Cloud beta features.
- - To enroll, navigate to your **Profile Settings** and enable the **Beta** flag under **Experimental Features**.
-
-Once you install the dbt Cloud CLI, you need to configure it to connect to a dbt Cloud project.
-
-1. Ensure you meet the prerequisites above.
-2. Create an environment variable with your [dbt Cloud API key](/docs/dbt-cloud-apis/user-tokens):
- - On MacOS, Linux, or Windows add an environment variable:
-
- ```bash
- export DBT_CLOUD_API_KEY="1234" # Replace 1234 with your API key
- ```
-
- - In Powershell, add an environment variable:
- - Note that this variable resets if you restart your shell. To add an environment variable permanently, add a system environment variable in your platform.
+
-3. Navigate to a dbt project in your terminal:
+
-```bash
-cd ~/dbt-projects/jaffle_shop
-```
+To update, follow the same process explained in [Windows](/docs/cloud/cloud-cli-installation?install=windows#install-dbt-cloud-cli) and replace the existing `dbt.exe` executable with the new one.
-4. In your `dbt_project.yml` file, ensure there is a section titled `dbt-cloud`. This section is required to have a `project-id` field with a valid project ID.
+
-```yaml
-# dbt_project.yml
-name:
+
-version:
-...
+To update, follow the same process explained in [Windows](/docs/cloud/cloud-cli-installation?install=linux#install-dbt-cloud-cli) and replace the existing `dbt` executable with the new one.
-dbt-cloud:
- project-id: PROJECT_ID
-```
+
-- To find your project ID, go to **Develop** in the navigation menu. Select the dbt Cloud project URL, such as `https://cloud.getdbt.com/develop/26228/projects123456`, where the project ID is `123456`.
+
-## Use the dbt Cloud CLI
+## Next steps
-The dbt Cloud CLI shares the same set of commands as dbt Core. When you invoke a dbt command, that command is sent to dbt Cloud for processing.
+After installation, you can [configure](/docs/cloud/configure-cloud-cli) the dbt Cloud CLI for your dbt Cloud project and use it to run [dbt commands](/reference/dbt-commands) similar to dbt Core.
-The dbt Cloud CLI supports [project dependencies](/docs/collaborate/govern/project-dependencies), which is an exciting way to depend on another project using the metadata service in dbt Cloud. It instantly resolves references (or `ref`) to public models defined in other projects. You don't need to execute or analyze these upstream models yourself. Instead, you treat them as an API that returns a dataset.
+For example, you can execute `dbt compile` to compile a project using dbt Cloud.
-Share feedback or request features you'd like to see on the [dbt community Slack](https://getdbt.slack.com/archives/C05M77P54FL).
+Note, that if you're using the dbt Cloud CLI, you can connect to your data platform directly in the dbt Cloud interface and don't need a [`profiles.yml`](/docs/core/connect-data-platform/profiles.yml) file.
## FAQs
@@ -190,11 +217,16 @@ The dbt Cloud CLI and dbt Core
diff --git a/website/docs/docs/cloud/configure-cloud-cli.md b/website/docs/docs/cloud/configure-cloud-cli.md
new file mode 100644
index 00000000000..f13a4d267c3
--- /dev/null
+++ b/website/docs/docs/cloud/configure-cloud-cli.md
@@ -0,0 +1,92 @@
+---
+title: Configure dbt Cloud CLI
+id: configure-cloud-cli
+description: "Instructions on how to configure the dbt Cloud CLI"
+sidebar_label: "Configure dbt Cloud CLI"
+pagination_next: null
+---
+
+import CloudCLIFlag from '/snippets/_cloud-cli-flag.md';
+
+"
+ active-host: ""
+ defer-env-id: ""
+ projects:
+ - project-id: ""
+ account-host: ""
+ api-key: ""
+
+ - project-id: ""
+ account-host: ""
+ api-key: ""
+
+ ```
+
+4. After downloading the config file, navigate to a dbt project in your terminal:
+
+ ```bash
+ cd ~/dbt-projects/jaffle_shop
+ ```
+
+5. In your `dbt_project.yml` file, ensure you have or include a `dbt-cloud` section with a `project-id` field. The `project-id` field contains the dbt Cloud project ID you want to use.
+
+ ```yaml
+ # dbt_project.yml
+ name:
+
+ version:
+ ...
+
+ dbt-cloud:
+ project-id: PROJECT_ID
+ ```
+
+ - To find your project ID, select **Develop** in the dbt Cloud navigation menu. You can use the URL to find the project ID. For example, in `https://cloud.getdbt.com/develop/26228/projects/123456`, the project ID is `123456`.
+
+## Use the dbt Cloud CLI
+
+- The dbt Cloud CLI shares the same set of [dbt commands](/reference/dbt-commands) as dbt Core and processes the commands you invoke.
+- It allows you to use automatic deferral of build artifacts to your Cloud project's production environment.
+- It also supports [project dependencies](/docs/collaborate/govern/project-dependencies), which allows you to depend on another project using the metadata service in dbt Cloud.
+ - Project dependencies instantly connects to and references (or `ref`) public models defined in other projects. This means you don't need to execute or analyze these upstream models yourself. Instead, you treat them as an API that returns a dataset.
+
+:::tip Use the
+- `dbt run --help`: Lists the flags available for the `run` command +::: diff --git a/website/docs/docs/cloud/connect-data-platform/about-connections.md b/website/docs/docs/cloud/connect-data-platform/about-connections.md index 65bfac3a90d..1fe89c7273c 100644 --- a/website/docs/docs/cloud/connect-data-platform/about-connections.md +++ b/website/docs/docs/cloud/connect-data-platform/about-connections.md @@ -3,6 +3,8 @@ title: "About data platform connections" id: about-connections description: "Information about data platform connections" sidebar_label: "About data platform connections" +pagination_next: "docs/cloud/connect-data-platform/connect-starburst-trino" +pagination_prev: null --- dbt Cloud can connect with a variety of data platform providers including: - [Amazon Redshift](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) @@ -13,6 +15,10 @@ dbt Cloud can connect with a variety of data platform providers including: - [Snowflake](/docs/cloud/connect-data-platform/connect-snowflake) - [Starburst or Trino](/docs/cloud/connect-data-platform/connect-starburst-trino) +import MSCallout from '/snippets/_microsoft-adapters-soon.md'; + +
+
+
+
:::tip Disable ad blockers
@@ -24,21 +33,16 @@ To improve your experience using dbt Cloud, we suggest that you turn off ad bloc
:::
-## Develop in the Cloud IDE
-
-The Cloud IDE is a powerful tool that can help streamline and govern your data platform development process. It offers a range of [editing features](/docs/cloud/dbt-cloud-ide/ide-user-interface#editing-features) that can help make your data platform development process faster and more efficient. Some of the editing features include:
-
-- The IDE has syntax highlighting for SQL. This makes it easy to visually distinguish between different parts of your code. This helps prevent syntax errors and improve readability.
-- Use the IDE built-in auto-completion, which suggests table names, arguments, and column names as you type. This saves time and reduces the likelihood of typos or errors in your code.
-- The code [formatting and linting](/docs/cloud/dbt-cloud-ide/lint-format) tools allow you to standardize and fix your SQL code with ease.
-- The IDE has a range of navigation tools, making it easy to move around your code with ease. You can quickly jump to specific lines of code, find and replace text, and navigate between different files in your project.
-- Use the version control menu and features to version-control your code with just a few clicks.
+## Prerequisites
-All of these [features](#cloud-ide-features) work together to create a powerful editing environment that can help you write and maintain high-quality SQL code in less time. Whether you're a seasoned developer or just starting out, the Cloud IDE has everything you need to be productive, collaborative, and efficient.
+- A [dbt Cloud account](https://cloud.getdbt.com/) and [Developer seat license](/docs/cloud/manage-access/seats-and-users)
+- A git repository set up and git provider must have `write` access enabled. See [Connecting your GitHub Account](/docs/cloud/git/connect-github) or [Importing a project by git URL](/docs/cloud/git/import-a-project-by-git-url) for detailed setup instructions
+- A dbt project connected to a [data platform](/docs/cloud/connect-data-platform/about-connections)
+- A [development environment and development credentials](#access-the-cloud-ide) set up
+- The environment must be on dbt version 1.0 or higher
-
+
+
++
+
+
\ No newline at end of file
diff --git a/website/docs/docs/build/enhance-your-models.md b/website/docs/docs/build/enhance-your-models.md
new file mode 100644
index 00000000000..46e7fa74353
--- /dev/null
+++ b/website/docs/docs/build/enhance-your-models.md
@@ -0,0 +1,23 @@
+---
+title: "Enhance your models"
+description: "Learn how you can enhance your models"
+pagination_next: "docs/build/materializations"
+pagination_prev: null
+---
+
+
+
+
+\ No newline at end of file diff --git a/website/docs/docs/build/materializations.md b/website/docs/docs/build/materializations.md index ae75b575d5f..79fe2e1b7c5 100644 --- a/website/docs/docs/build/materializations.md +++ b/website/docs/docs/build/materializations.md @@ -2,6 +2,7 @@ title: "Materializations" description: "Read this tutorial to learn how to use materializations when building in dbt." id: "materializations" +pagination_next: "docs/build/incremental-models" --- ## Overview diff --git a/website/docs/docs/build/measures.md b/website/docs/docs/build/measures.md index ba82a4aa4a5..e06b5046976 100644 --- a/website/docs/docs/build/measures.md +++ b/website/docs/docs/build/measures.md @@ -234,6 +234,15 @@ metrics: We can query the semi-additive metrics using the following syntax: +For dbt Cloud: + +```bash +dbt sl query --metrics mrr_by_end_of_month --dimensions metric_time__month --order metric_time__month +dbt sl query --metrics mrr_by_end_of_month --dimensions metric_time__week --order metric_time__week +``` + +For dbt Core: + ```bash mf query --metrics mrr_by_end_of_month --dimensions metric_time__month --order metric_time__month mf query --metrics mrr_by_end_of_month --dimensions metric_time__week --order metric_time__week diff --git a/website/docs/docs/build/metricflow-cli.md b/website/docs/docs/build/metricflow-commands.md similarity index 62% rename from website/docs/docs/build/metricflow-cli.md rename to website/docs/docs/build/metricflow-commands.md index 2650b2215ae..049b415d40e 100644 --- a/website/docs/docs/build/metricflow-cli.md +++ b/website/docs/docs/build/metricflow-commands.md @@ -1,69 +1,142 @@ --- -title: MetricFlow CLI -id: metricflow-cli -description: "Query metrics and metadata in your dbt project with the metricflow cli" -sidebar_label: "MetricFlow CLI commands" +title: MetricFlow commands +id: metricflow-commands +description: "Query metrics and metadata in your dbt project with the MetricFlow commands." +sidebar_label: "MetricFlow commands" tags: [Metrics, Semantic Layer] --- -Once you define metrics in your dbt project, you can query metrics, dimensions, dimension values, and validate your configs using the MetricFlow command line (CLI). +Once you define metrics in your dbt project, you can query metrics, dimensions, dimension values, and validate your configs using the MetricFlow commands. -# Installation +MetricFlow allows you to define and query metrics in your dbt project in the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation), [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud), or [dbt Core](/docs/core/installation). To experience the power of the universal [dbt Semantic Layer](/docs/use-dbt-semantic-layer/dbt-sl) and dynamically query those metrics in downstream tools, you'll need a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account. -You can install the [MetricFlow CLI](https://github.com/dbt-labs/metricflow#getting-started) from [PyPI](https://pypi.org/project/dbt-metricflow/). You need to use `pip` to install the MetricFlow CLI on Windows or Linux operating systems: +MetricFlow is compatible with Python versions 3.8, 3.9, 3.10 and 3.11. -1. Create or activate your virtual environment.`python -m venv venv` + +## MetricFlow + +MetricFlow is a dbt package that allows you to define and query metrics in your dbt project. You can use MetricFlow to query metrics in your dbt project in the dbt Cloud CLI, dbt Cloud IDE, or dbt Core. + +**Note** — MetricFlow commands aren't supported in dbt Cloud jobs yet. However, you can add MetricFlow validations with your git provider (such as GitHub Actions) by installing MetricFlow (`pip install metricflow`). This allows you to run MetricFlow commands as part of your continuous integration checks on PRs. + +
{{Dimension('primary_entity__dimension_name')}}.
-Here's an example query: mf query --metrics order_total --group-by metric_time --where "{{Dimension('order_id__is_food_order')}} = True".Before using the template wrapper, however, you will need to set up your terminal to escape curly braces for the filter template to work. +Here's an example query:
dbt sl query --metrics order_total --group-by metric_time --where "{{Dimension('order_id__is_food_order')}} = True".Before using the template wrapper, however, you will need to set up your terminal to escape curly braces for the filter template to work.
How to set up your terminal to escape curly braces?
diff --git a/website/docs/docs/build/metrics-overview.md b/website/docs/docs/build/metrics-overview.md index b699bf4d959..81af149a7d9 100644 --- a/website/docs/docs/build/metrics-overview.md +++ b/website/docs/docs/build/metrics-overview.md @@ -4,6 +4,7 @@ id: metrics-overview description: "Metrics can be defined in the same or separate YAML files from semantic models within the same dbt project repo." sidebar_label: "Creating metrics" tags: [Metrics, Semantic Layer] +pagination_next: "docs/build/cumulative" --- Once you've created your semantic models, it's time to start adding metrics! Metrics can be defined in the same YAML files as your semantic models, or split into separate YAML files into any other subdirectories (provided that these subdirectories are also within the same dbt project repo) @@ -34,7 +35,7 @@ metrics: config: here for `enabled` ## Optional label: The display name for your metric. This value will be shown in downstream tools. ## Required filter: | ## Optional - {{ Dimension('entity__name') }} > 0 and {{ Dimension(' entity__another name') }} is not + {{ Dimension('entity__name') }} > 0 and {{ Dimension(' entity__another_name') }} is not null ``` diff --git a/website/docs/docs/build/models.md b/website/docs/docs/build/models.md index e0683158e6d..d10eb5ed01a 100644 --- a/website/docs/docs/build/models.md +++ b/website/docs/docs/build/models.md @@ -2,6 +2,8 @@ title: "About dbt models" description: "Read this tutorial to learn how to use models when building in dbt." id: "models" +pagination_next: "docs/build/sql-models" +pagination_prev: null --- ## Overview diff --git a/website/docs/docs/build/organize-your-outputs.md b/website/docs/docs/build/organize-your-outputs.md new file mode 100644 index 00000000000..ad5efeda1c7 --- /dev/null +++ b/website/docs/docs/build/organize-your-outputs.md @@ -0,0 +1,38 @@ +--- +title: "Organize your outputs" +description: "Learn how you can organize your outputs" +pagination_next: "docs/build/custom-schemas" +pagination_prev: null +--- + +
+
+
++
+
+
\ No newline at end of file
diff --git a/website/docs/docs/build/packages.md b/website/docs/docs/build/packages.md
index 74e25262994..8d18a55e949 100644
--- a/website/docs/docs/build/packages.md
+++ b/website/docs/docs/build/packages.md
@@ -3,7 +3,7 @@ title: "Packages"
id: "packages"
---
-## What is a package?
+
Software engineers frequently modularize code into libraries. These libraries help programmers operate with leverage: they can spend more time focusing on their unique business logic, and less time implementing code that someone else has already spent the time perfecting.
In dbt, libraries like these are called _packages_. dbt's packages are so powerful because so many of the analytic problems we encountered are shared across organizations, for example:
@@ -22,13 +22,19 @@ dbt _packages_ are in fact standalone dbt projects, with models and macros that
* Models in the package will be materialized when you `dbt run`.
* You can use `ref` in your own models to refer to models from the package.
* You can use macros in the package in your own project.
+* It's important to note that defining and installing dbt packages is different from [defining and installing Python packages](/docs/build/python-models#using-pypi-packages)
-:::note Using Python packages
-Defining and installing dbt packages is different from [defining and installing Python packages](/docs/build/python-models#using-pypi-packages).
+:::info `dependencies.yml` has replaced `packages.yml`
+Starting from dbt v1.6, `dependencies.yml` has replaced `packages.yml`. This file can now contain both types of dependencies: "package" and "project" dependencies.
+- "Package" dependencies lets you add source code from someone else's dbt project into your own, like a library.
+- "Project" dependencies provide a different way to build on top of someone else's work in dbt. Refer to [Project dependencies](/docs/collaborate/govern/project-dependencies) for more info.
+-
+You can rename `packages.yml` to `dependencies.yml`, _unless_ you need to use Jinja within your packages specification. This could be necessary, for example, if you want to add an environment variable with a git token in a private git package specification.
:::
+
## How do I add a package to my project?
1. Add a file named -- Understand [MetricFlow's](/docs/build/about-metricflow) key concepts, which powers the revamped dbt Semantic Layer. +import SetUp from '/snippets/_v2-sl-prerequisites.md'; + +
- - * To authenticate, you need to [generate a service token](/docs/dbt-cloud-apis/service-tokens) with Semantic Layer Only and Metadata Only permissions. - * Refer to the [SQL query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the API. - -- To learn more about the sophisticated integrations that connect to the dbt Semantic Layer, refer to [Available integrations](/docs/use-dbt-semantic-layer/avail-sl-integrations) for more info. +
+
+
+ +The following sections provide detailed instructions on setting up the dbt Cloud CLI and dbt Cloud IDE. To get started with dbt development, you'll need a [developer](/docs/cloud/manage-access/seats-and-users) account. For a more comprehensive guide about developing in dbt, refer to our [quickstart guides](/quickstarts). + + +--------- +**Note**: The dbt Cloud CLI and the open-sourced dbt Core are both command line tools that let you run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features). + diff --git a/website/docs/docs/cloud/about-cloud-setup.md b/website/docs/docs/cloud/about-cloud-setup.md index baa2465472e..7b68b52a45a 100644 --- a/website/docs/docs/cloud/about-cloud-setup.md +++ b/website/docs/docs/cloud/about-cloud-setup.md @@ -3,6 +3,8 @@ title: About dbt Cloud setup id: about-cloud-setup description: "Configuration settings for dbt Cloud." sidebar_label: "About dbt Cloud setup" +pagination_next: "docs/dbt-cloud-environments" +pagination_prev: null --- dbt Cloud is the fastest and most reliable way to deploy your dbt jobs. It contains a myriad of settings that can be configured by admins, from the necessities (data platform integration) to security enhancements (SSO) and quality-of-life features (RBAC). This portion of our documentation will take you through the various settings found by clicking on the gear icon in the dbt Cloud UI, including: @@ -11,6 +13,8 @@ dbt Cloud is the fastest and most reliable way to deploy your dbt jobs. It conta - Configuring access to [GitHub](/docs/cloud/git/connect-github), [GitLab](/docs/cloud/git/connect-gitlab), or your own [git repo URL](/docs/cloud/git/import-a-project-by-git-url). - [Managing users and licenses](/docs/cloud/manage-access/seats-and-users) - [Configuring secure access](/docs/cloud/manage-access/about-user-access) +- Configuring the [dbt Cloud IDE](/docs/cloud/about-cloud-develop) +- Installing and configuring the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) These settings are intended for dbt Cloud administrators. If you need a more detailed first-time setup guide for specific data platforms, read our [quickstart guides](/quickstarts). diff --git a/website/docs/docs/cloud/about-cloud/dbt-cloud-features.md b/website/docs/docs/cloud/about-cloud/about-dbt-cloud.md similarity index 78% rename from website/docs/docs/cloud/about-cloud/dbt-cloud-features.md rename to website/docs/docs/cloud/about-cloud/about-dbt-cloud.md index f1d8b32cdb1..71f3175a108 100644 --- a/website/docs/docs/cloud/about-cloud/dbt-cloud-features.md +++ b/website/docs/docs/cloud/about-cloud/about-dbt-cloud.md @@ -4,86 +4,95 @@ id: "dbt-cloud-features" sidebar_label: "dbt Cloud features" description: "Explore dbt Cloud's features and learn why dbt Cloud is the fastest way to deploy dbt" hide_table_of_contents: true +pagination_next: "docs/cloud/about-cloud/architecture" +pagination_prev: null --- -dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, schedule, document, and investigate data models all in one browser-based UI. In addition to providing a hosted architecture for running dbt across your organization, dbt Cloud comes equipped with turnkey support for scheduling jobs, CI/CD, hosting documentation, monitoring & alerting, and an integrated development environment (IDE). +dbt Cloud is the fastest and most reliable way to deploy dbt. Develop, test, schedule, document, and investigate data models all in one browser-based UI. + +In addition to providing a hosted architecture for running dbt across your organization, dbt Cloud comes equipped with turnkey support for scheduling jobs, CI/CD, hosting documentation, monitoring and alerting, an integrated development environment (IDE), and allows you to develop and run dbt commands from your local command line interface (CLI) or code editor. dbt Cloud's [flexible plans](https://www.getdbt.com/pricing/) and features make it well-suited for data teams of any size — sign up for your [free 14-day trial](https://www.getdbt.com/signup/)!
+
*These features are available on [selected plans](https://www.getdbt.com/pricing/). diff --git a/website/docs/docs/cloud/about-cloud/browsers.md b/website/docs/docs/cloud/about-cloud/browsers.md index 2fc5a8b4b4d..12665bc7b72 100644 --- a/website/docs/docs/cloud/about-cloud/browsers.md +++ b/website/docs/docs/cloud/about-cloud/browsers.md @@ -2,6 +2,7 @@ title: "Supported browsers" id: "browsers" description: "dbt Cloud supports the latest browsers like Chrome and Firefox." +pagination_next: null --- To have the best experience with dbt Cloud, we recommend using the latest versions of the following browsers: diff --git a/website/docs/docs/cloud/billing.md b/website/docs/docs/cloud/billing.md index 61251f6e41d..1d71d33e9a1 100644 --- a/website/docs/docs/cloud/billing.md +++ b/website/docs/docs/cloud/billing.md @@ -3,28 +3,81 @@ title: "Billing" id: billing description: "dbt Cloud billing information." sidebar_label: Billing +pagination_next: null +pagination_prev: null --- dbt Cloud offers a variety of [plans and pricing](https://www.getdbt.com/pricing/) to fit your organization’s needs. With flexible billing options that appeal to large enterprises and small businesses and [server availability](/docs/cloud/about-cloud/regions-ip-addresses) worldwide, dbt Cloud is the fastest and easiest way to begin transforming your data. ## How does dbt Cloud pricing work? -As a customer, you pay for the number of seats you have and the amount of usage consumed each month. Usage is based on the number of Successful Models Built, and seats are billed primarily on the amount of Developer licenses purchased. All billing computations are conducted in Coordinated Universal Time (UTC). +As a customer, you pay for the number of seats you have and the amount of usage consumed each month. Seats are billed primarily on the amount of Developer and Read licenses purchased. Usage is based on the number of [Successful Models Built](#what-counts-as-a-successful-model-built) and, if purchased and used, Semantic Layer Query Units subject to reasonable usage. All billing computations are conducted in Coordinated Universal Time (UTC). + +### What counts as a seat license? + +There are three types of possible seat licenses: + +* **Developer** — for roles and permissions that require interaction with the dbt Cloud environment day-to-day. +* **Read-Only** — for access to view certain documents and reports. +* **IT** — for access to specific features related to account management (for example, configuring git integration). ### What counts as a Successful Model Built? -dbt Cloud considers a Successful Model Built as any model that is successfully built via a run through dbt Cloud’s orchestration functionality in a dbt Cloud deployment environment. Models are counted when built and run. This includes any jobs run via dbt Cloud's scheduler, CI builds (jobs triggered by pull requests), runs kicked off via the dbt Cloud API, and any successor dbt Cloud tools with similar functionality. This also includes models that are successfully built even when a run may fail to complete. For example, you may have a job that contains 100 models and on one of its runs, 51 models are successfully built and then the job fails. In this situation, only 51 models would be counted. +dbt Cloud considers a Successful Model Built as any
-
+1. Install using [pip](/docs/cloud/cloud-cli-installation?install=pip#install-dbt-cloud-cli).
+2. Install natively, but ensure that you deactivate your Python environment or uninstall it using `pip uninstall dbt` before proceeding.
+
+3. (Advanced users) Install natively, but modify the $PATH environment variable to correctly point to the dbt Cloud CLI binary to use both dbt Cloud CLI and dbt Core together.
+
+You can always uninstall the dbt Cloud CLI to return to using dbt Core.
+How do I solve for path conflicts
-For compatibility, both the dbt Cloud CLI and dbt Core are invoked by running `dbt`. This can create path conflicts if your operating system selects one over the other based on your $PATH environment variable (settings). +How do I run both the dbt Cloud CLI and dbt Core?
+For compatibility, both the dbt Cloud CLI and dbt Core are invoked by runningdbt. This can create path conflicts if your operating system selects one over the other based on your $PATH environment variable (settings).
-If you have dbt Core installed locally, ensure that you deactivate your Python environment or uninstall it using `pip uninstall dbt` before proceeding. Alternatively, advanced users can modify the $PATH environment variable to correctly point to the dbt Cloud CLI binary to use both dbt Cloud CLI and dbt Core together.
+If you have dbt Core installed locally, either:
-You can always uninstall the Cloud CLI to return to using dbt Core.
---help flag
+As a tip, most command-line tools have a `--help` flag to show available commands and arguments. Use the `--help` flag with dbt in two ways:
+- `dbt --help`: Lists the commands available for dbt+- `dbt run --help`: Lists the flags available for the `run` command +::: diff --git a/website/docs/docs/cloud/connect-data-platform/about-connections.md b/website/docs/docs/cloud/connect-data-platform/about-connections.md index 65bfac3a90d..1fe89c7273c 100644 --- a/website/docs/docs/cloud/connect-data-platform/about-connections.md +++ b/website/docs/docs/cloud/connect-data-platform/about-connections.md @@ -3,6 +3,8 @@ title: "About data platform connections" id: about-connections description: "Information about data platform connections" sidebar_label: "About data platform connections" +pagination_next: "docs/cloud/connect-data-platform/connect-starburst-trino" +pagination_prev: null --- dbt Cloud can connect with a variety of data platform providers including: - [Amazon Redshift](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb) @@ -13,6 +15,10 @@ dbt Cloud can connect with a variety of data platform providers including: - [Snowflake](/docs/cloud/connect-data-platform/connect-snowflake) - [Starburst or Trino](/docs/cloud/connect-data-platform/connect-starburst-trino) +import MSCallout from '/snippets/_microsoft-adapters-soon.md'; + +
+
+
++
+
\ No newline at end of file
diff --git a/website/docs/docs/cloud/dbt-cloud-ide/dbt-cloud-tips.md b/website/docs/docs/cloud/dbt-cloud-ide/dbt-cloud-tips.md
index cfae00b960e..39db7832d79 100644
--- a/website/docs/docs/cloud/dbt-cloud-ide/dbt-cloud-tips.md
+++ b/website/docs/docs/cloud/dbt-cloud-ide/dbt-cloud-tips.md
@@ -3,6 +3,7 @@ title: "Tips and tricks"
id: dbt-cloud-tips
description: "Check out any dbt Cloud and IDE-related tips."
sidebar_label: "Tips and tricks"
+pagination_next: null
---
# dbt Cloud tips
@@ -16,7 +17,7 @@ There are default keyboard shortcuts that can help make development more product
- Press Fn-F1 to view a full list of the editor shortcuts
- Command-O on macOS or Control-O on Windows to select a file to open
- Command-P/Command-Shift-P on macOS or Control-P/Control-Shift-P on Windows to see the command palette
-- Hold Option-click-on-area on macOS or Hold-Alt-click-on-area on Windows to select multiple lines and perform a multi-edit. You can also press Command-E to perform this operation on the command line.
+- Hold Option-click-on-area or press Shift-Option-Command on macOS or Hold-Alt-click-on-area on Windows to select multiple lines and perform a multi-edit. You can also press Command-E to perform this operation on the command line.
- Command-Enter on macOS or Control-Enter on Windows to Preview your code
- Command-Shift-Enter on macOS or Control-Shift-Enter on Windows to Compile
- Highlight a portion of code and use the above shortcuts to Preview or Compile code
diff --git a/website/docs/docs/cloud/dbt-cloud-ide/develop-in-the-cloud.md b/website/docs/docs/cloud/dbt-cloud-ide/develop-in-the-cloud.md
index c55e67cf93e..9fc382f0217 100644
--- a/website/docs/docs/cloud/dbt-cloud-ide/develop-in-the-cloud.md
+++ b/website/docs/docs/cloud/dbt-cloud-ide/develop-in-the-cloud.md
@@ -1,22 +1,31 @@
---
-title: "Develop in the IDE"
+title: "About the dbt Cloud IDE"
id: develop-in-the-cloud
description: "Develop, test, run, and build in the Cloud IDE. With the Cloud IDE, you can compile dbt code into SQL and run it against your database directly"
-sidebar_label: Develop in the IDE
+sidebar_label: About the IDE
tags: [IDE]
+pagination_next: "docs/cloud/dbt-cloud-ide/ide-user-interface"
+pagination_prev: null
---
-The dbt Cloud integrated development environment (IDE) is a single interface for building, testing, running, and version-controlling dbt projects from your browser. With the Cloud IDE, you can compile dbt code into SQL and run it against your database directly.
+The dbt Cloud integrated development environment (IDE) is a single web-based interface for building, testing, running, and version-controlling dbt projects. It compiles dbt code into SQL and executes it directly on your database.
-## Prerequisites
+The dbt Cloud IDE offers several [editing features](/docs/cloud/dbt-cloud-ide/ide-user-interface#editing-features) for faster and more efficient data platform development and governance:
-To develop in the Cloud IDE, make sure you have the following:
+- Syntax highlighting for SQL: Makes it easy to distinguish different parts of your code, reducing syntax errors and enhancing readability.
+- Auto-completion: Suggests table names, arguments, and column names as you type, saving time and reducing typos.
+- Code [formatting and linting](/docs/cloud/dbt-cloud-ide/lint-format): Help standardize and fix your SQL code effortlessly.
+- Navigation tools: Easily move around your code, jump to specific lines, find and replace text, and navigate between project files.
+- Version control: Manage code versions with a few clicks.
-- A [dbt Cloud account](https://cloud.getdbt.com/) and [Developer seat license](/docs/cloud/manage-access/seats-and-users)
-- A git repository set up and git provider must have `write` access enabled. See [Connecting your GitHub Account](/docs/cloud/git/connect-github) or [Importing a project by git URL](/docs/cloud/git/import-a-project-by-git-url) for detailed setup instructions
-- A dbt project connected to a [data platform](/docs/cloud/connect-data-platform/about-connections)
-- A [development environment and development credentials](#access-the-cloud-ide) set up
-- The environment must be on dbt version 1.0 or higher
+These [features](#dbt-cloud-ide-features) create a powerful editing environment for efficient SQL coding, suitable for both experienced and beginner developers.
+
+
-
What is the difference between developing on the Cloud IDE and on the CLI?
+What is the difference between developing on the dbt Cloud IDE, the dbt Cloud CLI, and dbt Core?
-
+
+
+lineage to gain a better understanding of its latest production state. Navigate and manage your projects within dbt Cloud to help your data consumers discover and leverage your dbt resources.
+With dbt Explorer, you can view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their lineage to gain a better understanding of its latest production state. Navigate and manage your projects within dbt Cloud to help you and other data developers, analysts, and consumers discover and leverage your dbt resources.
-To display the details about your [project state](/docs/dbt-cloud-apis/project-state), dbt Explorer utilizes the metadata provided through the [Discovery API](/docs/dbt-cloud-apis/discovery-api). The metadata that's available on your project depends on the [deployment environment](/docs/deploy/deploy-environments) you've designated as _production_ in your dbt Cloud project. dbt Explorer automatically retrieves the metadata updates after each job run in the production deployment environment so it will always have the latest state on your project. The metadata it displays depends on the [commands executed by the jobs](/docs/deploy/job-commands). For instance:
+:::tip Public preview
-- To update model details or results, you must run `dbt run` or `dbt build` on a given model within a job in the environment.
-- To view catalog statistics and columns, you must run `dbt docs generate` within a job in the environment.
-- To view test results, you must run `dbt test` or `dbt build` within a job in the environment.
-- To view source freshness check results, you must run `dbt source freshness` within a job in the environment.
-
-The need to run these commands will diminish, and richer, more timely metadata will become available as the Discovery API and its underlying platform evolve.
-
-:::tip Join the beta
-
-dbt Explorer is a [beta feature](/docs/dbt-versions/product-lifecycles#dbt-cloud) and subject to change without notification. More updates to this feature coming soon.
-
-If you’re interested in joining the beta, please contact your account team.
+Try dbt Explorer! It's available in [Public Preview](/docs/dbt-versions/product-lifecycles#dbt-cloud) as of October 17, 2023 for dbt Cloud customers. More updates coming soon.
:::
@@ -28,115 +19,218 @@ If you’re interested in joining the beta, please contact your account team.
- You have a [multi-tenant](/docs/cloud/about-cloud/tenancy#multi-tenant) or AWS single-tenant dbt Cloud account on the [Team or Enterprise plan](https://www.getdbt.com/pricing/).
- You have set up a [production deployment environment](/docs/deploy/deploy-environments#set-as-production-environment-beta) for each project you want to explore.
- There has been at least one successful job run in the production deployment environment.
-- You are on the dbt Explorer page. This requires the feature to be enabled for your account.
- - To go to the page, select **Explore (Beta)** from the top navigation bar in dbt Cloud.
+- You are on the dbt Explorer page. To do this, select **Explore** from the top navigation bar in dbt Cloud.
+
+
+## Generate metadata
+
+dbt Explorer uses the metadata provided by the [Discovery API](/docs/dbt-cloud-apis/discovery-api) to display the details about [the state of your project](/docs/dbt-cloud-apis/project-state). The metadata that's available depends on the [deployment environment](/docs/deploy/deploy-environments) you've designated as _production_ in your dbt Cloud project. dbt Explorer automatically retrieves the metadata updates after each job run in the production deployment environment so it always has the latest results for your project.
+
+To view a resource and its metadata, you must define the resource in your project and run a job in the production environment. The resulting metadata depends on the [commands executed by the jobs](/docs/deploy/job-commands).
+
+For a richer experience with dbt Explorer, you must:
+
+- Run [dbt run](/reference/commands/run) or [dbt build](/reference/commands/build) on a given model within a job in the environment to update model details or results.
+- Run [dbt docs generate](/reference/commands/cmd-docs) within a job in the environment to view catalog statistics and columns for models, sources, and snapshots.
+- Run [dbt test](/reference/commands/test) or [dbt build](/reference/commands/build) within a job in the environment to view test results.
+- Run [dbt source freshness](/reference/commands/source#dbt-source-freshness) within a job in the environment to view source freshness data.
+- Run [dbt snapshot](/reference/commands/snapshot) or [dbt build](/reference/commands/build) within a job in the environment to view snapshot details.
+
+Richer and more timely metadata will become available as dbt, the Discovery API, and the underlying dbt Cloud platform evolves.
-## Explore the project’s lineage
+## Explore your project's lineage graph {#project-lineage}
-dbt Explorer provides a visualization of your project’s DAG that you can interact with. To start, select **Overview** in the left sidebar and click the **Explore Lineage** button on the main (center) section of the page.
+dbt Explorer provides a visualization of your project’s DAG that you can interact with. To access the project's full lineage graph, select **Overview** in the left sidebar and click the **Explore Lineage** button on the main (center) section of the page.
-If you don't see the lineage graph immediately, click **Render Lineage**. It can take some time for the graph to render depending on the size of your project and your computer’s available memory. The graph of very large projects might not render so, instead, you can select a subset of nodes by using selectors.
+If you don't see the project lineage graph immediately, click **Render Lineage**. It can take some time for the graph to render depending on the size of your project and your computer’s available memory. The graph of very large projects might not render so you can select a subset of nodes by using selectors, instead.
-The nodes in the lineage graph represent the project’s resources and the edges represent the relationships between the nodes. Resources like tests and macros display in the lineage within their [resource details pages](#view-resource-details) but not within the overall project lineage graph. Nodes are color-coded and include iconography according to their resource type.
+The nodes in the lineage graph represent the project’s resources and the edges represent the relationships between the nodes. Nodes are color-coded and include iconography according to their resource type.
-To interact with the lineage graph, you can:
+To explore the lineage graphs of tests and macros, view [their resource details pages](#view-resource-details). By default, dbt Explorer excludes these resources from the full lineage graph unless a search query returns them as results.
+
+To interact with the full lineage graph, you can:
- Hover over any item in the graph to display the resource’s name and type.
- Zoom in and out on the graph by mouse-scrolling.
-- Grab and move the graph.
-- Click on a resource to highlight its relationship with other resources in your project.
-- [Search and select specific resources](#search-resources) or a subset of the DAG using selectors and lineage (for example, `+[YOUR_RESOURCE_NAME]` displays all nodes upstream of a particular resource).
-- [View resource details](#view-resource-details) by selecting a node in the graph (double-clicking).
+- Grab and move the graph and the nodes.
+- Select a resource to highlight its relationship with other resources in your project. A panel opens on the graph’s right-hand side that displays a high-level summary of the resource’s details. The side panel includes a **General** tab for information like description, materialized type, and other details.
+ - Click the Share icon in the side panel to copy the graph’s link to your clipboard.
+ - Click the View Resource icon in the side panel to [view the resource details](#view-resource-details).
+- [Search and select specific resources](#search-resources) or a subset of the DAG using selectors and graph operators. For example:
+ - `+[RESOURCE_NAME]` — Displays all parent nodes of the resource
+ - `resource_type:model [RESOURCE_NAME]` — Displays all models matching the name search
+- [View resource details](#view-resource-details) by selecting a node (double-clicking) in the graph.
+- Click the List view icon in the graph's upper right corner to return to the main **Explore** page.
-
-- **group:** — Find models defined within a specified [group](/reference/node-selection/methods#the-group-method).
-- **access:** — Find models based on their [access](/reference/node-selection/methods#the-access-method) property.
+- `group:` — Find models defined within a specified [group](/reference/node-selection/methods#the-group-method).
+- `access:` — Find models based on their [access](/reference/node-selection/methods#the-access-method) property.
-
diff --git a/website/docs/docs/collaborate/govern/project-dependencies.md b/website/docs/docs/collaborate/govern/project-dependencies.md
index 7785e428678..9a1d8b59b68 100644
--- a/website/docs/docs/collaborate/govern/project-dependencies.md
+++ b/website/docs/docs/collaborate/govern/project-dependencies.md
@@ -3,18 +3,17 @@ title: "Project dependencies"
id: project-dependencies
sidebar_label: "Project dependencies"
description: "Reference public models across dbt projects"
+pagination_next: null
---
-:::caution Closed Beta - dbt Cloud Enterprise
-"Project" dependencies and cross-project `ref` are features of dbt Cloud Enterprise, currently in Closed Beta. To access these features while they are in beta, please contact your account team at dbt Labs.
+:::info Available in Public Preview for dbt Cloud Enterprise accounts
-**Prerequisites:** In order to add project dependencies and resolve cross-project `ref`, you must:
-- Have the feature enabled (speak to your account team)
-- Use dbt v1.6 for **both** the upstream ("producer") project and the downstream ("consumer") project.
-- Have a deployment environment in the upstream ("producer") project [that is set to be your production environment](/docs/deploy/deploy-environments#set-as-production-environment-beta)
-- Have a successful run of the upstream ("producer") project
+Project dependencies and cross-project `ref` are features available in [dbt Cloud Enterprise](https://www.getdbt.com/pricing), currently in [Public Preview](/docs/dbt-versions/product-lifecycles#dbt-cloud).
+
+Enterprise users can use these features by designating a [public model](/docs/collaborate/govern/model-access) and adding a [cross-project ref](#how-to-use-ref).
:::
+
For a long time, dbt has supported code reuse and extension by installing other projects as [packages](/docs/build/packages). When you install another project as a package, you are pulling in its full source code, and adding it to your own. This enables you to call macros and run models defined in that other project.
While this is a great way to reuse code, share utility macros, and establish a starting point for common transformations, it's not a great way to enable collaboration across teams and at scale, especially at larger organizations.
@@ -23,6 +22,33 @@ This year, dbt Labs is introducing an expanded notion of `dependencies` across m
- **Packages** — Familiar and pre-existing type of dependency. You take this dependency by installing the package's full source code (like a software library).
- **Projects** — A _new_ way to take a dependency on another project. Using a metadata service that runs behind the scenes, dbt Cloud resolves references on-the-fly to public models defined in other projects. You don't need to parse or run those upstream models yourself. Instead, you treat your dependency on those models as an API that returns a dataset. The maintainer of the public model is responsible for guaranteeing its quality and stability.
+
+Starting in dbt v1.6 or higher, `packages.yml` has been renamed to `dependencies.yml`. However, if you need use Jinja within your packages config, such as an environment variable for your private package, you need to keep using `packages.yml` for your packages for now. Refer to the [FAQs](#faqs) for more info.
+
+## Prerequisites
+
+In order to add project dependencies and resolve cross-project `ref`, you must:
+- Use dbt v1.6 or higher for **both** the upstream ("producer") project and the downstream ("consumer") project.
+- Have a deployment environment in the upstream ("producer") project [that is set to be your production environment](/docs/deploy/deploy-environments#set-as-production-environment-beta)
+- Have a successful run of the upstream ("producer") project
+- Have a multi-tenant or single-tenant [dbt Cloud Enterprise](https://www.getdbt.com/pricing) account (Azure ST is not supported but coming soon)
+
+
## Example
As an example, let's say you work on the Marketing team at the Jaffle Shop. The name of your team's project is `jaffle_marketing`:
@@ -36,7 +62,7 @@ name: jaffle_marketing
As part of your modeling of marketing data, you need to take a dependency on two other projects:
-- `dbt_utils` as a [package](#packages-use-case): An collection of utility macros that you can use while writing the SQL for your own models. This package is, open-source public, and maintained by dbt Labs.
+- `dbt_utils` as a [package](#packages-use-case): A collection of utility macros that you can use while writing the SQL for your own models. This package is, open-source public, and maintained by dbt Labs.
- `jaffle_finance` as a [project use-case](#projects-use-case): Data models about the Jaffle Shop's revenue. This project is private and maintained by your colleagues on the Finance team. You want to select from some of this project's final models, as a starting point for your own work.
@@ -66,7 +92,7 @@ When you're building on top of another team's work, resolving the references in
- You don't need to mirror any conditional configuration of the upstream project such as `vars`, environment variables, or `target.name`. You can reference them directly wherever the Finance team is building their models in production. Even if the Finance team makes changes like renaming the model, changing the name of its schema, or [bumping its version](/docs/collaborate/govern/model-versions), your `ref` would still resolve successfully.
- You eliminate the risk of accidentally building those models with `dbt run` or `dbt build`. While you can select those models, you can't actually build them. This prevents unexpected warehouse costs and permissions issues. This also ensures proper ownership and cost allocation for each team's models.
-### Usage
+### How to use ref
**Writing `ref`:** Models referenced from a `project`-type dependency must use [two-argument `ref`](/reference/dbt-jinja-functions/ref#two-argument-variant), including the project name:
@@ -87,6 +113,8 @@ with monthly_revenue as (
**Cycle detection:** Currently, "project" dependencies can only go in one direction, meaning that the `jaffle_finance` project could not add a new model that depends, in turn, on `jaffle_marketing.roi_by_channel`. dbt will check for cycles across projects and raise errors if any are detected. We are considering support for this pattern in the future, whereby dbt would still check for node-level cycles while allowing cycles at the project level.
+For more guidance on how to use dbt Mesh, refer to the dedicated [dbt Mesh guide](/guides/best-practices/how-we-mesh/mesh-1-intro).
+
### Comparison
If you were to instead install the `jaffle_finance` project as a `package` dependency, you would instead be pulling down its full source code and adding it to your runtime environment. This means:
@@ -99,7 +127,7 @@ There are a few cases where installing another internal project as a package can
- Unified deployments — In a production environment, if the central data platform team of Jaffle Shop wanted to schedule the deployment of models across both `jaffle_finance` and `jaffle_marketing`, they could use dbt's [selection syntax](/reference/node-selection/syntax) to create a new "passthrough" project that installed both projects as packages.
- Coordinated changes — In development, if you wanted to test the effects of a change to a public model in an upstream project (`jaffle_finance.monthly_revenue`) on a downstream model (`jaffle_marketing.roi_by_channel`) _before_ introducing changes to a staging or production environment, you can install the `jaffle_finance` package as a package within `jaffle_marketing`. The installation can point to a specific git branch, however, if you find yourself frequently needing to perform end-to-end testing across both projects, we recommend you re-examine if this represents a stable interface boundary.
-These are the exceptions, rather than the rule. Installing another team's project as a package adds complexity, latency, and risk of unnecessary costs. By defining clear interface boundaries across teams, by serving one team's public models as "APIs" to another, and by enabling practitioners to develop with a more narrowly-defined scope, we can enable more people to contribute, with more confidence, while requiring less context upfront.
+These are the exceptions, rather than the rule. Installing another team's project as a package adds complexity, latency, and risk of unnecessary costs. By defining clear interface boundaries across teams, by serving one team's public models as "APIs" to another, and by enabling practitioners to develop with a more narrowly defined scope, we can enable more people to contribute, with more confidence, while requiring less context upfront.
## FAQs
@@ -108,3 +136,7 @@ These are the exceptions, rather than the rule. Installing another team's projec
If you're using private packages with the [git token method](/docs/build/packages#git-token-method), you must define them in the `packages.yml` file instead of the `dependencies.yml` file. This is because conditional rendering (like Jinja-in-yaml) is not supported.
+
+
+## Related docs
+- Refer to the [dbt Mesh](/guides/best-practices/how-we-mesh/mesh-1-intro) guide for more guidance on how to use dbt Mesh.
diff --git a/website/docs/docs/connect-adapters.md b/website/docs/docs/connect-adapters.md
index f45da732abb..77ead34e51d 100644
--- a/website/docs/docs/connect-adapters.md
+++ b/website/docs/docs/connect-adapters.md
@@ -11,9 +11,9 @@ This section provides more details on different ways you can connect dbt to an a
Explore the fastest and most reliable way to deploy dbt using dbt Cloud, a hosted architecture that runs dbt Core across your organization. dbt Cloud lets you seamlessly [connect](/docs/cloud/about-cloud-setup) with a variety of [verified](/docs/supported-data-platforms) data platform providers directly in the dbt Cloud UI.
-### Install using the CLI
+### Install with dbt Core
-Install dbt Core, which is an open-source tool, locally using the CLI. dbt communicates with a number of different data platforms by using a dedicated adapter plugin for each. When you install dbt Core, you'll also need to install the specific adapter for your database, [connect to dbt Core](/docs/core/about-core-setup), and set up a `profiles.yml` file.
+Install dbt Core, an open-source tool, locally using the command line. dbt communicates with a number of different data platforms by using a dedicated adapter plugin for each. When you install dbt Core, you'll also need to install the specific adapter for your database, [connect to dbt Core](/docs/core/about-core-setup), and set up a `profiles.yml` file.
With a few exceptions [^1], you can install all [Verified adapters](/docs/supported-data-platforms) from PyPI using `pip install adapter-name`. For example to install Snowflake, use the command `pip install dbt-snowflake`. The installation will include `dbt-core` and any other required dependencies, which may include both other dependencies and even other adapter plugins. Read more about [installing dbt](/docs/core/installation).
diff --git a/website/docs/docs/contribute-core-adapters.md b/website/docs/docs/contribute-core-adapters.md
index 6e66a5d28ff..553361ee1a2 100644
--- a/website/docs/docs/contribute-core-adapters.md
+++ b/website/docs/docs/contribute-core-adapters.md
@@ -1,6 +1,7 @@
---
title: "Contribute to adapters"
id: "contribute-core-adapters"
+pagination_next: null
---
The dbt Community exists to allow analytics practitioners share their knowledge, help others and collectively to drive forward the discipline of analytics engineering. There are opportunities here for everyone to contribute whether you're at the beginning your analytics engineering journey or you are a seasoned data professional.
diff --git a/website/docs/docs/core/about-core-setup.md b/website/docs/docs/core/about-core-setup.md
index 0408e529b2d..a4d5ff09ee3 100644
--- a/website/docs/docs/core/about-core-setup.md
+++ b/website/docs/docs/core/about-core-setup.md
@@ -3,13 +3,15 @@ title: About dbt Core setup
id: about-core-setup
description: "Configuration settings for dbt Core."
sidebar_label: "About dbt Core setup"
+pagination_next: "docs/core/about-dbt-core"
+pagination_prev: null
---
dbt Core is an [open-source](https://github.com/dbt-labs/dbt-core) tool that enables data teams to transform data using analytics engineering best practices. You can install dbt locally in your environment and use dbt Core on the command line. It can communicate with databases through adapters.
This section of our docs will guide you through various settings to get started:
-- [About the CLI](/docs/core/about-the-cli)
+- [About dbt Core](/docs/core/about-dbt-core)
- [Installing dbt](/docs/core/installation)
- [Connecting to a data platform](/docs/core/connect-data-platform/profiles.yml)
- [How to run your dbt projects](/docs/running-a-dbt-project/run-your-dbt-projects)
diff --git a/website/docs/docs/core/about-dbt-core.md b/website/docs/docs/core/about-dbt-core.md
new file mode 100644
index 00000000000..a35d92420f3
--- /dev/null
+++ b/website/docs/docs/core/about-dbt-core.md
@@ -0,0 +1,25 @@
+---
+title: "About dbt Core"
+id: "about-dbt-core"
+sidebar_label: "About dbt Core"
+---
+
+[dbt Core](https://github.com/dbt-labs/dbt-core) is an open sourced project where you can develop from the command line and run your dbt project.
+
+To use dbt Core, your workflow generally looks like:
+
+1. **Build your dbt project in a code editor —** popular choices include VSCode and Atom.
+
+2. **Run your project from the command line —** macOS ships with a default Terminal program, however you can also use iTerm or the command line prompt within a code editor to execute dbt commands.
+
+:::info How we set up our computers for working on dbt projects
+
+We've written a [guide](https://discourse.getdbt.com/t/how-we-set-up-our-computers-for-working-on-dbt-projects/243) for our recommended setup when running dbt projects using dbt Core.
+
+:::
+
+If you're using the command line, we recommend learning some basics of your terminal to help you work more effectively. In particular, it's important to understand `cd`, `ls` and `pwd` to be able to navigate through the directory structure of your computer easily.
+
+You can find more information on installing and setting up the dbt Core [here](/docs/core/installation).
+
+**Note** — dbt supports a dbt Cloud CLI and dbt Core, both command line interface tools that enable you to run dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features).
diff --git a/website/docs/docs/core/about-the-cli.md b/website/docs/docs/core/about-the-cli.md
deleted file mode 100644
index d05fb514dfa..00000000000
--- a/website/docs/docs/core/about-the-cli.md
+++ /dev/null
@@ -1,22 +0,0 @@
----
-title: "About the CLI"
-id: "about-the-cli"
-sidebar_label: "About the CLI"
----
-
-dbt ships with a command line interface (CLI) for running your dbt project. This way of running dbt and a dbt project is free and open source.
-
-To use the CLI, your workflow generally looks like:
-1. **Build your dbt project in a code editor —** popular choices include VSCode and Atom.
-
-1. **Run your project from the command line —** macOS ships with a default Terminal program, however you can also use iTerm or the command line prompt within a code editor to execute dbt commands.
-
-:::info How we set up our computers for working on dbt projects
-
-We've written a [guide](https://discourse.getdbt.com/t/how-we-set-up-our-computers-for-working-on-dbt-projects/243) for our recommended setup when running dbt projects using the CLI.
-
-:::
-
-If you're using the CLI, we recommend learning some basics of your terminal to help you work more effectively. In particular, it's important to understand `cd`, `ls` and `pwd` to be able to navigate through the directory structure of your computer easily.
-
-You can find more information on installing and setting up the dbt CLI [here](/dbt-cli/cli-overview).
diff --git a/website/docs/docs/core/connect-data-platform/about-core-connections.md b/website/docs/docs/core/connect-data-platform/about-core-connections.md
index 802e197514c..a85a32cc031 100644
--- a/website/docs/docs/core/connect-data-platform/about-core-connections.md
+++ b/website/docs/docs/core/connect-data-platform/about-core-connections.md
@@ -4,6 +4,8 @@ id: "about-core-connections"
description: "Information about data platform connections in dbt Core"
sidebar_label: "About data platform connections in dbt Core"
hide_table_of_contents: true
+pagination_next: "docs/core/connect-data-platform/profiles.yml"
+pagination_prev: null
---
dbt Core can connect with a variety of data platform providers including:
diff --git a/website/docs/docs/core/connect-data-platform/profiles.yml.md b/website/docs/docs/core/connect-data-platform/profiles.yml.md
index 67b0eb15fbe..97254dda1c4 100644
--- a/website/docs/docs/core/connect-data-platform/profiles.yml.md
+++ b/website/docs/docs/core/connect-data-platform/profiles.yml.md
@@ -3,7 +3,7 @@ title: "About profiles.yml"
id: profiles.yml
---
-If you're using dbt from the [command line (CLI)](/docs/core/about-the-cli), you'll need a `profiles.yml` file that contains the connection details for your data platform. When you run dbt from the CLI, it reads your `dbt_project.yml` file to find the `profile` name, and then looks for a profile with the same name in your `profiles.yml` file. This profile contains all the information dbt needs to connect to your data platform.
+If you're using [dbt Core](/docs/core/about-dbt-core), you'll need a `profiles.yml` file that contains the connection details for your data platform. When you run dbt Core from the command line, it reads your `dbt_project.yml` file to find the `profile` name, and then looks for a profile with the same name in your `profiles.yml` file. This profile contains all the information dbt needs to connect to your data platform.
For detailed info, you can refer to the [Connection profiles](/docs/core/connect-data-platform/connection-profiles).
diff --git a/website/docs/docs/core/connect-data-platform/upsolver-setup.md b/website/docs/docs/core/connect-data-platform/upsolver-setup.md
index 68cfa3045cd..6b2f410fc07 100644
--- a/website/docs/docs/core/connect-data-platform/upsolver-setup.md
+++ b/website/docs/docs/core/connect-data-platform/upsolver-setup.md
@@ -14,6 +14,7 @@ meta:
slack_channel_link: 'https://join.slack.com/t/upsolvercommunity/shared_invite/zt-1zo1dbyys-hj28WfaZvMh4Z4Id3OkkhA'
platform_name: 'Upsolver'
config_page: '/reference/resource-configs/upsolver-configs'
+pagination_next: null
---
+— `dbt --help`: Lists the commands available for dbt
+— `dbt run --help`: Lists the flags available for the `run` command + +::: + ## Upgrading dbt Core dbt provides a number of resources for understanding [general best practices](/blog/upgrade-dbt-without-fear) while upgrading your dbt project as well as detailed [migration guides](/guides/migration/versions/upgrading-to-v1.4) highlighting the changes required for each minor and major release, and [core versions](/docs/dbt-versions/core) diff --git a/website/docs/docs/core/pip-install.md b/website/docs/docs/core/pip-install.md index a35ad5f0d77..44fac00e493 100644 --- a/website/docs/docs/core/pip-install.md +++ b/website/docs/docs/core/pip-install.md @@ -5,7 +5,7 @@ description: "You can use pip to install dbt Core and adapter plugins from the c You need to use `pip` to install dbt Core on Windows or Linux operating systems. You can use `pip` or [Homebrew](/docs/core/homebrew-install) for installing dbt Core on a MacOS. -You can install dbt Core and plugins using `pip` because they are Python modules distributed on [PyPI](https://pypi.org/project/dbt/). +You can install dbt Core and plugins using `pip` because they are Python modules distributed on [PyPI](https://pypi.org/project/dbt-core/).DAG and the data it produces.
diff --git a/website/docs/docs/dbt-cloud-apis/discovery-querying.md b/website/docs/docs/dbt-cloud-apis/discovery-querying.md
index ba1365e632b..35c092adb4b 100644
--- a/website/docs/docs/dbt-cloud-apis/discovery-querying.md
+++ b/website/docs/docs/dbt-cloud-apis/discovery-querying.md
@@ -2,6 +2,7 @@
title: "Query the Discovery API"
id: "discovery-querying"
sidebar_label: "Query the Discovery API"
+pagination_next: "docs/dbt-cloud-apis/discovery-schema-environment"
---
The Discovery API supports ad-hoc queries and integrations. If you are new to the API, refer to [About the Discovery API](/docs/dbt-cloud-apis/discovery-api) for an introduction.
diff --git a/website/docs/docs/dbt-cloud-apis/schema-discovery-job.mdx b/website/docs/docs/dbt-cloud-apis/schema-discovery-job.mdx
index bb30786e19d..8b02c5601ad 100644
--- a/website/docs/docs/dbt-cloud-apis/schema-discovery-job.mdx
+++ b/website/docs/docs/dbt-cloud-apis/schema-discovery-job.mdx
@@ -2,6 +2,8 @@
title: "Job object schema"
sidebar_label: "Job"
id: "discovery-schema-job"
+pagination_next: "docs/dbt-cloud-apis/discovery-schema-job-model"
+pagination_prev: null
---
import { QueryArgsTable, SchemaTable } from "./schema";
diff --git a/website/docs/docs/dbt-cloud-apis/sl-api-overview.md b/website/docs/docs/dbt-cloud-apis/sl-api-overview.md
index 42416765904..3ddbf76d152 100644
--- a/website/docs/docs/dbt-cloud-apis/sl-api-overview.md
+++ b/website/docs/docs/dbt-cloud-apis/sl-api-overview.md
@@ -4,6 +4,7 @@ id: sl-api-overview
description: "Integrate and query metrics and dimensions in downstream tools using the Semantic Layer APIs"
tags: [Semantic Layer, API]
hide_table_of_contents: true
+pagination_next: "docs/dbt-cloud-apis/sl-jdbc"
---
@@ -31,10 +32,8 @@ You can use the dbt Semantic Layer for a variety of tools and applications of da
import Features from '/snippets/_sl-plan-info.md'
There are two main ways to develop with dbt: using the web-based IDE in dbt Cloud or using the command-line interface (CLI) in dbt Core:
- — - dbt Cloud IDE dbt Cloud is a web-based application that allows you to develop dbt projects with the IDE, includes a purpose-built scheduler, and provides an easier way to share your dbt documentation with your team. The IDE is a faster and more reliable way to deploy your dbt models and provides a real-time editing and execution environment for your dbt project.
- — - dbt Core CLI The command line interface (CLI) uses dbt Core, an open-source software that’s freely available. You can build your dbt project in a code editor, like Jetbrains or VSCode, and run dbt commands from the command line. +
diff --git a/website/docs/docs/cloud/dbt-cloud-ide/ide-user-interface.md b/website/docs/docs/cloud/dbt-cloud-ide/ide-user-interface.md
index de643413a8a..05910b23e7f 100644
--- a/website/docs/docs/cloud/dbt-cloud-ide/ide-user-interface.md
+++ b/website/docs/docs/cloud/dbt-cloud-ide/ide-user-interface.md
@@ -36,11 +36,13 @@ The IDE streamlines your workflow, and features a popular user interface layout
* Added (A) — The IDE detects added files
* Deleted (D) — The IDE detects deleted files.
-- — - dbt Cloud IDE dbt Cloud is a web-based application that allows you to develop dbt projects with the IDE, includes a purpose-built scheduler, and provides an easier way to share your dbt documentation with your team. The IDE is a faster and more reliable way to deploy your dbt models and provides a real-time editing and execution environment for your dbt project.
- — - dbt Core CLI The command line interface (CLI) uses dbt Core, an open-source software that’s freely available. You can build your dbt project in a code editor, like Jetbrains or VSCode, and run dbt commands from the command line. +
You can develop dbt using the web-based IDE in dbt Cloud or on the command line interface using the dbt Cloud CLI or open-source dbt Core, all of which enable you to execute dbt commands. The key distinction between the dbt Cloud CLI and dbt Core is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its features.
+ — + dbt Cloud IDE: dbt Cloud is a web-based application that allows you to develop dbt projects with the IDE, includes a purpose-built scheduler, and provides an easier way to share your dbt documentation with your team. The IDE is a faster and more reliable way to deploy your dbt models and provides a real-time editing and execution environment for your dbt project.
+ — + dbt Cloud CLI: The dbt Cloud CLI allows you to run dbt commands against your dbt Cloud development environment from your local command line or code editor. It supports cross-project ref, speedier, lower-cost builds, automatic deferral of build artifacts, and more.
+ — + dbt Core: dbt Core is an open-sourced software that’s freely available. You can build your dbt project in a code editor, and run dbt commands from the command line.
+ — + dbt Cloud IDE: dbt Cloud is a web-based application that allows you to develop dbt projects with the IDE, includes a purpose-built scheduler, and provides an easier way to share your dbt documentation with your team. The IDE is a faster and more reliable way to deploy your dbt models and provides a real-time editing and execution environment for your dbt project.
+ — + dbt Cloud CLI: The dbt Cloud CLI allows you to run dbt commands against your dbt Cloud development environment from your local command line or code editor. It supports cross-project ref, speedier, lower-cost builds, automatic deferral of build artifacts, and more.
+ — + dbt Core: dbt Core is an open-sourced software that’s freely available. You can build your dbt project in a code editor, and run dbt commands from the command line.
+
+
++
+
\ No newline at end of file
diff --git a/website/docs/docs/cloud/git/import-a-project-by-git-url.md b/website/docs/docs/cloud/git/import-a-project-by-git-url.md
index ba53baa33ea..83846bb1f0b 100644
--- a/website/docs/docs/cloud/git/import-a-project-by-git-url.md
+++ b/website/docs/docs/cloud/git/import-a-project-by-git-url.md
@@ -1,6 +1,8 @@
---
title: "Import a project by git URL"
id: "import-a-project-by-git-url"
+pagination_next: "docs/cloud/git/connect-github"
+pagination_prev: null
---
In dbt Cloud, you can import a git repository from any valid git URL that points to a dbt project. There are some important considerations to keep in mind when doing this.
diff --git a/website/docs/docs/cloud/git/setup-azure.md b/website/docs/docs/cloud/git/setup-azure.md
index 9eca77d7014..843371be6ea 100644
--- a/website/docs/docs/cloud/git/setup-azure.md
+++ b/website/docs/docs/cloud/git/setup-azure.md
@@ -93,7 +93,7 @@ Once you connect your Azure AD app and Azure DevOps, you need to provide dbt Clo
- **Directory(tenant) ID:** Found in the Azure AD App.
+
+
++
+
\ No newline at end of file
diff --git a/website/docs/docs/collaborate/cloud-build-and-view-your-docs.md b/website/docs/docs/collaborate/cloud-build-and-view-your-docs.md
index 36f4781bfde..a54b8fcdc2b 100644
--- a/website/docs/docs/collaborate/cloud-build-and-view-your-docs.md
+++ b/website/docs/docs/collaborate/cloud-build-and-view-your-docs.md
@@ -2,6 +2,7 @@
title: "Build and view your docs with dbt Cloud"
id: "build-and-view-your-docs"
description: "Automatically generate project documentation as you run jobs."
+pagination_next: null
---
dbt enables you to generate documentation for your project and data warehouse, and renders the documentation in a website. For more information, see [Documentation](/docs/collaborate/documentation).
diff --git a/website/docs/docs/collaborate/collaborate-with-others.md b/website/docs/docs/collaborate/collaborate-with-others.md
new file mode 100644
index 00000000000..7875a8044b6
--- /dev/null
+++ b/website/docs/docs/collaborate/collaborate-with-others.md
@@ -0,0 +1,38 @@
+---
+title: "Collaborate with others"
+description: "Learn how dbt Cloud makes it easier to collaborate with others"
+pagination_next: "docs/collaborate/explore-projects"
+pagination_prev: null
+---
+
+
+
+
++
+
+
\ No newline at end of file
diff --git a/website/docs/docs/collaborate/documentation.md b/website/docs/docs/collaborate/documentation.md
index 429b5187152..0fa00c7cca2 100644
--- a/website/docs/docs/collaborate/documentation.md
+++ b/website/docs/docs/collaborate/documentation.md
@@ -2,6 +2,8 @@
title: "About documentation"
description: "Learn how good documentation for your dbt models helps stakeholders discover and understand your datasets."
id: "documentation"
+pagination_next: "docs/collaborate/build-and-view-your-docs"
+pagination_prev: null
---
## Related documentation
diff --git a/website/docs/docs/collaborate/explore-projects.md b/website/docs/docs/collaborate/explore-projects.md
index a4c914259ef..b041cd0c915 100644
--- a/website/docs/docs/collaborate/explore-projects.md
+++ b/website/docs/docs/collaborate/explore-projects.md
@@ -1,25 +1,16 @@
---
-title: "Explore your dbt projects (beta)"
-sidebar_label: "Explore dbt projects (beta)"
+title: "Explore your dbt projects"
+sidebar_label: "Explore dbt projects"
description: "Learn about dbt Explorer and how to interact with it to understand, improve, and leverage your data pipelines."
+pagination_next: null
+pagination_prev: null
---
-With dbt Explorer, you can view your project's [resources](/docs/build/projects) (such as models, tests, and metrics) and their Overview of {frontMatter.meta.pypi_package}
diff --git a/website/docs/docs/core/dbt-core-environments.md b/website/docs/docs/core/dbt-core-environments.md index 5daf17bddf9..c7f340557fd 100644 --- a/website/docs/docs/core/dbt-core-environments.md +++ b/website/docs/docs/core/dbt-core-environments.md @@ -1,6 +1,7 @@ --- title: "dbt Core environments" id: "dbt-core-environments" +pagination_next: "docs/running-a-dbt-project/run-your-dbt-projects" --- dbt makes it easy to maintain separate production and development environments through the use of [targets](/reference/dbt-jinja-functions/target.md) within a [profile](/docs/core/connect-data-platform/profiles.yml). A typical profile, when using dbt locally (for example, running from your command line), will have a target named `dev` and have this set as the default. This means that while making changes, your objects will be built in your _development_ target without affecting production queries made by your end users. Once you are confident in your changes, you can deploy the code to _production_, by running your dbt project with a _prod_ target. diff --git a/website/docs/docs/core/installation-overview.md b/website/docs/docs/core/installation-overview.md index f1fdb800fdf..25628b76358 100644 --- a/website/docs/docs/core/installation-overview.md +++ b/website/docs/docs/core/installation-overview.md @@ -2,6 +2,8 @@ title: "About installing dbt" id: "installation" description: "You can install dbt Core using a few different tested methods." +pagination_next: "docs/core/homebrew-install" +pagination_prev: null --- You can install dbt Core on the command line by using one of these methods: @@ -11,6 +13,14 @@ You can install dbt Core on the command line by using one of these methods: - [Use a Docker image to install dbt](/docs/core/docker-install) - [Install dbt from source](/docs/core/source-install) +:::tip Pro tip: Using the --help flag + +Most command-line tools, including dbt, have a `--help` flag that you can use to show available commands and arguments. For example, you can use the `--help` flag with dbt in two ways:+— `dbt --help`: Lists the commands available for dbt
+— `dbt run --help`: Lists the flags available for the `run` command + +::: + ## Upgrading dbt Core dbt provides a number of resources for understanding [general best practices](/blog/upgrade-dbt-without-fear) while upgrading your dbt project as well as detailed [migration guides](/guides/migration/versions/upgrading-to-v1.4) highlighting the changes required for each minor and major release, and [core versions](/docs/dbt-versions/core) diff --git a/website/docs/docs/core/pip-install.md b/website/docs/docs/core/pip-install.md index a35ad5f0d77..44fac00e493 100644 --- a/website/docs/docs/core/pip-install.md +++ b/website/docs/docs/core/pip-install.md @@ -5,7 +5,7 @@ description: "You can use pip to install dbt Core and adapter plugins from the c You need to use `pip` to install dbt Core on Windows or Linux operating systems. You can use `pip` or [Homebrew](/docs/core/homebrew-install) for installing dbt Core on a MacOS. -You can install dbt Core and plugins using `pip` because they are Python modules distributed on [PyPI](https://pypi.org/project/dbt/). +You can install dbt Core and plugins using `pip` because they are Python modules distributed on [PyPI](https://pypi.org/project/dbt-core/).
+
+
\ No newline at end of file
diff --git a/website/docs/docs/dbt-cloud-apis/discovery-api.md b/website/docs/docs/dbt-cloud-apis/discovery-api.md
index e4441aa55a2..747128cf7bc 100644
--- a/website/docs/docs/dbt-cloud-apis/discovery-api.md
+++ b/website/docs/docs/dbt-cloud-apis/discovery-api.md
@@ -1,5 +1,6 @@
---
title: "About the Discovery API"
+pagination_next: "docs/dbt-cloud-apis/discovery-use-cases-and-examples"
---
Every time dbt Cloud runs a project, it generates and stores information about the project. The metadata includes details about your project’s models, sources, and other nodes along with their execution results. With the dbt Cloud Discovery API, you can query this comprehensive information to gain a better understanding of your
diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md
index 02d26229794..4d0d4f6d1a2 100644
--- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md
+++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md
@@ -5,7 +5,6 @@ description: "Integrate and use the JDBC API to query your metrics."
tags: [Semantic Layer, API]
---
-
import LegacyInfo from '/snippets/_legacy-sl-callout.md';
@@ -59,11 +58,13 @@ jdbc:arrow-flight-sql://semantic-layer.cloud.getdbt.com:443?&environmentId=20233
## Querying the API for metric metadata
-The Semantic Layer JDBC API has built-in metadata calls which can provide a user with information about their metrics and dimensions. Here are some metadata commands and examples:
+The Semantic Layer JDBC API has built-in metadata calls which can provide a user with information about their metrics and dimensions.
+
+Refer to the following tabs for metadata commands and examples:
-
+
Use this query to fetch all defined metrics in your dbt project:
@@ -74,7 +75,7 @@ select * from {{
```
-
+
Use this query to fetch all dimensions for a metric.
@@ -87,7 +88,7 @@ select * from {{
-
+
Use this query to fetch dimension values for one or multiple metrics and single dimension.
@@ -100,7 +101,7 @@ semantic_layer.dimension_values(metrics=['food_order_amount'], group_by=['custom
-
+
Use this query to fetch queryable granularities for a list of metrics. This API request allows you to only show the time granularities that make sense for the primary time dimension of the metrics (such as `metric_time`), but if you want queryable granularities for other time dimensions, you can use the `dimensions()` call, and find the column queryable_granularities.
@@ -113,6 +114,9 @@ select * from {{
+
+
+
@@ -144,9 +148,10 @@ select NAME, QUERYABLE_GRANULARITIES from {{
-
+
It may be useful in your application to expose the names of the time dimensions that represent `metric_time` or the common thread across all metrics.
+
You can first query the `metrics()` argument to fetch a list of measures, then use the `measures()` call which will return the name(s) of the time dimensions that make up metric time.
```bash
@@ -167,12 +172,13 @@ To query metric values, here are the following parameters that are available:
| `metrics` | The metric name as defined in your dbt metric configuration | `metrics=['revenue']` | Required |
| `group_by` | Dimension names or entities to group by. We require a reference to the entity of the dimension (other than for the primary time dimension), which is pre-appended to the front of the dimension name with a double underscore. | `group_by=['user__country', 'metric_time']` | Optional |
| `grain` | A parameter specific to any time dimension and changes the grain of the data from the default for the metric. | `group_by=[Dimension('metric_time')`
`grain('week\|day\|month\|quarter\|year')]` | Optional | -| `where` | A where clause that allows you to filter on dimensions and entities using parameters - comes with `TimeDimension`, `Dimension`, and `Entity` objects. Granularity is required with `TimeDimension` | `"{{ where=Dimension('customer__country') }} = 'US')"` | Optional | +| `where` | A where clause that allows you to filter on dimensions and entities using parameters. This takes a filter list OR string. Inputs come with `Dimension`, and `Entity` objects. Granularity is required if the `Dimension` is a time dimension | `"{{ where=Dimension('customer__country') }} = 'US')"` | Optional | | `limit` | Limit the data returned | `limit=10` | Optional | -|`order` | Order the data returned | `order_by=['-order_gross_profit']` (remove `-` for ascending order) | Optional | +|`order` | Order the data returned by a particular field | `order_by=['order_gross_profit']`, use `-` for descending, or full object notation if the object is operated on: `order_by=[Metric('order_gross_profit').descending(True)`] | Optional | | `compile` | If true, returns generated SQL for the data platform but does not execute | `compile=True` | Optional | + ## Note on time dimensions and `metric_time` You will notice that in the list of dimensions for all metrics, there is a dimension called `metric_time`. `Metric_time` is a reserved keyword for the measure-specific aggregation time dimensions. For any time-series metric, the `metric_time` keyword should always be available for use in queries. This is a common dimension across *all* metrics in a semantic graph. @@ -246,13 +252,13 @@ select * from {{ Where filters in API allow for a filter list or string. We recommend using the filter list for production applications as this format will realize all benefits from the
diff --git a/website/docs/docs/dbt-cloud-apis/user-tokens.md b/website/docs/docs/dbt-cloud-apis/user-tokens.md
index f0d694f5edd..77e536b12a5 100644
--- a/website/docs/docs/dbt-cloud-apis/user-tokens.md
+++ b/website/docs/docs/dbt-cloud-apis/user-tokens.md
@@ -1,6 +1,7 @@
---
title: "User tokens"
id: "user-tokens"
+pagination_next: "docs/dbt-cloud-apis/service-tokens"
---
## User API tokens
diff --git a/website/docs/docs/dbt-cloud-environments.md b/website/docs/docs/dbt-cloud-environments.md
index f61ec5ef72b..8fa4522d47c 100644
--- a/website/docs/docs/dbt-cloud-environments.md
+++ b/website/docs/docs/dbt-cloud-environments.md
@@ -2,9 +2,10 @@
title: "dbt Cloud environments"
id: "dbt-cloud-environments"
description: "Learn about dbt Cloud's development environment to execute your project in the IDE"
+pagination_next: null
---
-An environment determines how dbt Cloud will execute your project in both the dbt Cloud IDE (for development) and scheduled jobs (for deployment).
+An environment determines how dbt Cloud will execute your project in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) (for development) and scheduled jobs (for deployment).
Critically, in order to execute dbt, environments define three variables:
@@ -34,7 +35,7 @@ To create a new dbt Cloud development environment:
### Set developer credentials
-To use the IDE, each developer will need to set up [personal development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide) to your warehouse connection in their **Profile Settings**. This allows you to set separate target information and maintain individual credentials to connect to your warehouse via the dbt Cloud IDE.
+To use the dbt Cloud IDE or dbt Cloud CLI, each developer will need to set up [personal development credentials](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud#access-the-cloud-ide) to your warehouse connection in their **Profile Settings**. This allows you to set separate target information and maintain individual credentials to connect to your warehouse.
_` (for example, `dbt_cloud_pr_1862_1704`) and can be found in the run details for the given run, as shown in the following image:
diff --git a/website/docs/docs/deploy/deployment-overview.md b/website/docs/docs/deploy/deployment-overview.md
index 553dca923a5..29934663544 100644
--- a/website/docs/docs/deploy/deployment-overview.md
+++ b/website/docs/docs/deploy/deployment-overview.md
@@ -4,6 +4,8 @@ id: "deployments"
sidebar: "Use dbt Cloud's capabilities to seamlessly run a dbt job in production."
hide_table_of_contents: true
tags: ["scheduler"]
+pagination_next: "docs/deploy/job-scheduler"
+pagination_prev: null
---
Use dbt Cloud's capabilities to seamlessly run a dbt job in production or staging environments. Rather than run dbt commands manually from the command line, you can leverage the [dbt Cloud's in-app scheduling](/docs/deploy/job-scheduler) to automate how and when you execute dbt.
diff --git a/website/docs/docs/deploy/deployment-tools.md b/website/docs/docs/deploy/deployment-tools.md
index 80622880c2c..6fba9caf6e8 100644
--- a/website/docs/docs/deploy/deployment-tools.md
+++ b/website/docs/docs/deploy/deployment-tools.md
@@ -2,6 +2,7 @@
title: "Integrate with other orchestration tools"
id: "deployment-tools"
sidebar_label: "Integrate with other tools"
+pagination_next: null
---
Alongside [dbt Cloud](/docs/deploy/jobs), discover other ways to schedule and run your dbt jobs with the help of tools such as Airflow, Prefect, Dagster, automation server, Cron, and Azure Data Factory (ADF),
diff --git a/website/docs/docs/deploy/job-commands.md b/website/docs/docs/deploy/job-commands.md
index ca26182fc7b..db284c78a05 100644
--- a/website/docs/docs/deploy/job-commands.md
+++ b/website/docs/docs/deploy/job-commands.md
@@ -41,8 +41,7 @@ For every job, you have the option to select the [Generate docs on run](/docs/co
### Command list
-You can add or remove as many [dbt commands](/reference/dbt-commands) as necessary for every job. However, you need to have at least one dbt command.
-Commands under the "CLI" tab in the [dbt Command reference doc](/reference/dbt-commands) page are meant for use in the [CLI](/docs/core/about-the-cli) only and are not available in dbt Cloud.
+You can add or remove as many [dbt commands](/reference/dbt-commands) as necessary for every job. However, you need to have at least one dbt command. There are few commands listed as "dbt Core" in the [dbt Command reference doc](/reference/dbt-commands) page. This means they are meant for use in [dbt Core](/docs/core/about-dbt-core) only and are not available in dbt Cloud.
:::tip Using selectors
diff --git a/website/docs/docs/deploy/job-scheduler.md b/website/docs/docs/deploy/job-scheduler.md
index 8ade670f1cc..fba76f677a7 100644
--- a/website/docs/docs/deploy/job-scheduler.md
+++ b/website/docs/docs/deploy/job-scheduler.md
@@ -85,7 +85,7 @@ The scheduler prevents queue clog by canceling runs that aren't needed, ensuring
To prevent over-scheduling, users will need to take action by either refactoring the job so it runs faster or modifying its [schedule](/docs/deploy/deploy-jobs#schedule-days).
## Related docs
-- [dbt Cloud architecture](/docs/cloud/about-cloud/architecture#about-dbt-cloud-architecture)
+- [dbt Cloud architecture](/docs/cloud/about-cloud/architecture#dbt-cloud-features-architecture)
- [Job commands](/docs/deploy/job-commands)
- [Job notifications](/docs/deploy/job-notifications)
- [Webhooks](/docs/deploy/webhooks)
diff --git a/website/docs/docs/deploy/jobs.md b/website/docs/docs/deploy/jobs.md
index e8ca864d65f..92390907cd0 100644
--- a/website/docs/docs/deploy/jobs.md
+++ b/website/docs/docs/deploy/jobs.md
@@ -1,8 +1,9 @@
---
title: "Jobs in dbt Cloud"
-sidebar_label: "Jobs"
+sidebar_label: "About Jobs"
description: "Learn about deploy jobs and continuous integration (CI) jobs in dbt Cloud and what their differences are."
tags: [scheduler]
+pagination_next: "docs/deploy/deploy-jobs"
---
In dbt Cloud, there are two types of jobs:
diff --git a/website/docs/docs/deploy/monitor-jobs.md b/website/docs/docs/deploy/monitor-jobs.md
index 35743adb422..45156bb341c 100644
--- a/website/docs/docs/deploy/monitor-jobs.md
+++ b/website/docs/docs/deploy/monitor-jobs.md
@@ -3,6 +3,7 @@ title: "Monitor jobs and alerts"
id: "monitor-jobs"
description: "Monitor your dbt Cloud job and set up alerts to ensure seamless orchestration and optimize your data transformations"
tags: ["scheduler"]
+pagination_next: "docs/deploy/run-visibility"
---
Monitor your dbt Cloud jobs to help identify improvement and set up alerts to proactively alert the right people or team.
diff --git a/website/docs/docs/environments-in-dbt.md b/website/docs/docs/environments-in-dbt.md
index 54eaa68f667..70bc096cf4f 100644
--- a/website/docs/docs/environments-in-dbt.md
+++ b/website/docs/docs/environments-in-dbt.md
@@ -2,6 +2,7 @@
title: "About environments"
id: "environments-in-dbt"
hide_table_of_contents: true
+pagination_next: null
---
In software engineering, environments are used to enable engineers to develop and test code without impacting the users of their software. Typically, there are two types of environments in dbt:
@@ -18,7 +19,7 @@ Configure environments to tell dbt Cloud or dbt Core how to build and execute yo
* Install these adapters using the CLI as they're not currently supported in dbt Cloud.
+
* Install these adapters using dbt Core as they're not currently supported in dbt Cloud.
diff --git a/website/docs/docs/trusted-adapters.md b/website/docs/docs/trusted-adapters.md index e19bb40785f..08191e8ea42 100644 --- a/website/docs/docs/trusted-adapters.md +++ b/website/docs/docs/trusted-adapters.md @@ -6,7 +6,7 @@ hide_table_of_contents: true Trusted adapters are adapters not maintained by dbt Labs, that we feel comfortable recommending to users for use in production. -Free and open-source tools for the data professional are increasingly abundant. This is by-and-large a *good thing*, however it requires due dilligence that wasn't required in a paid-license, closed-source software world. As a user, there are questions to answer important before taking a dependency on an open-source project. The trusted adapter designation is meant to streamline this process for end users. +Free and open-source tools for the data professional are increasingly abundant. This is by-and-large a *good thing*, however it requires due diligence that wasn't required in a paid-license, closed-source software world. As a user, there are questions to answer important before taking a dependency on an open-source project. The trusted adapter designation is meant to streamline this process for end users.
-import NewSLChanges from '/snippets/_new-sl-changes.md';
-
-
-import NewSLChanges from '/snippets/_new-sl-changes.md';
-
-
diff --git a/website/snippets/_cloud-cli-flag.md b/website/snippets/_cloud-cli-flag.md new file mode 100644 index 00000000000..523591a438c --- /dev/null +++ b/website/snippets/_cloud-cli-flag.md @@ -0,0 +1,5 @@ +:::info Public preview functionality + +The dbt Cloud CLI is currently in [public preview](/docs/dbt-versions/product-lifecycles#dbt-cloud). Share feedback or request features you'd like to see on the [dbt community Slack](https://getdbt.slack.com/archives/C05M77P54FL). + +::: diff --git a/website/snippets/_cloud-environments-info.md b/website/snippets/_cloud-environments-info.md index 5388379dc34..b0c8e86a2b5 100644 --- a/website/snippets/_cloud-environments-info.md +++ b/website/snippets/_cloud-environments-info.md @@ -3,17 +3,17 @@ In dbt Cloud, there are two types of environments: - Deployment environment — Determines the settings used when jobs created within that environment are executed. -- Development environment — Determines the settings used in the dbt Cloud IDE for that particular dbt Cloud project. +- Development environment — Determines the settings used in the dbt Cloud IDE or dbt Cloud CLI, for that particular project. Each dbt Cloud project can only have a single development environment but can have any number of deployment environments. | | Development Environments | Deployment Environments | | --- | --- | --- | -| Determines settings for | dbt Cloud IDE | dbt Cloud Job runs | +| Determines settings for | dbt Cloud IDE or dbt Cloud CLI | dbt Cloud Job runs | | How many can I have in my project? | 1 | Any number | :::note -For users familiar with development on the CLI, each environment is roughly analogous to an entry in your `profiles.yml` file, with some additional information about your repository to ensure the proper version of code is executed. More info on dbt core environments [here](/docs/core/dbt-core-environments). +For users familiar with development on dbt Core, each environment is roughly analogous to an entry in your `profiles.yml` file, with some additional information about your repository to ensure the proper version of code is executed. More info on dbt core environments [here](/docs/core/dbt-core-environments). ::: ## Common environment settings @@ -38,7 +38,7 @@ Both development and deployment environments have a section called **General Set By default, all environments will use the default branch in your repository (usually the `main` branch) when accessing your dbt code. This is overridable within each dbt Cloud Environment using the **Default to a custom branch** option. This setting have will have slightly different behavior depending on the environment type: -- **Development**: determines which branch in the dbt Cloud IDE developers create branches from and open PRs against +- **Development**: determines which branch in the dbt Cloud IDE or dbt Cloud CLI developers create branches from and open PRs against. - **Deployment:** determines the branch is cloned during job executions for each environment. For more info, check out this [FAQ page on this topic](/faqs/Environments/custom-branch-settings)! @@ -59,7 +59,7 @@ Something to note, Extended Attributes doesn't mask secret values. We recommend
-If you're developing in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [orchestrating job runs](/docs/deploy/deployments), Extended Attributes parses through the provided YAML and extracts the `profiles.yml` attributes. For each individual attribute: +If you're developing in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud), [dbt Cloud CLI](/docs/cloud/cloud-cli-installation), or [orchestrating job runs](/docs/deploy/deployments), Extended Attributes parses through the provided YAML and extracts the `profiles.yml` attributes. For each individual attribute: - If the attribute exists in another source (such as your project settings), it will replace its value (like environment-level values) in the profile. It also overrides any custom environment variables. diff --git a/website/snippets/_enterprise-permissions-table.md b/website/snippets/_enterprise-permissions-table.md index 779c5bcb479..3eb313e0f5b 100644 --- a/website/snippets/_enterprise-permissions-table.md +++ b/website/snippets/_enterprise-permissions-table.md @@ -7,7 +7,7 @@ Key: Permissions: * Account-level permissions — Permissions related to management of the dbt Cloud account. For example, billing and account settings. -* Project-level permissions — Permissions related to the projects in dbt Cloud. For example, repos and access to the IDE. +* Project-level permissions — Permissions related to the projects in dbt Cloud. For example, repos and access to the IDE or dbt Cloud CLI. ### Account roles Account roles enable you to manage the dbt Cloud account and manage the account settings (for example, generating service tokens, inviting users, configuring SSO). They also provide project-level permissions. The **Account Admin** role is the highest level of access you can assign. @@ -20,8 +20,10 @@ Account roles enable you to manage the dbt Cloud account and manage the account | Audit logs | R | | | R | | | Auth provider | W | | | W | R | | Billing | W | W | | | R | +| Groups | W | | R | W | R | | Invitations | W | | W | W | R | | IP restrictions | W | | | W | R | +| Licenses | W | | W | W | R | | Members | W | | W | W | R | | Project (create) | W | | W | | | | Public models | R | R | R | R | R | @@ -34,25 +36,23 @@ Account roles enable you to manage the dbt Cloud account and manage the account |:-------------------------|:-------------:|:-------------:|:---------------:|:--------------:|:------:| | Connections | W | | W | | R | | Credentials | W | | W | | R | -| Custom env. variables | W | | W | | R | +| Custom env. variables | W | | W | | R | | dbt adapters | W | | W | | R | -| Develop (IDE) | W | | W | | | +| Develop (IDE or dbt Cloud CLI) | W | | W | | | | Environments | W | | W | | R | -| Groups | W | | R | W | R | | Jobs | W | | W | | R | -| Licenses | W | | W | W | R | | Metadata | R | | R | | R | | Permissions | W | | W | W | R | | Profile | W | | W | | R | | Projects | W | | W | R | R | | Repositories | W | | W | | R | | Runs | W | | W | | R | -| Semantic Layer Config | W | | W | | R | +| Semantic Layer Config | W | | W | | R | ### Project role permissions -The project roles enable you to work within the projects in various capacities. They primarily provide access to project-level permissions such as repos and the IDE, but may also provide some account-level permissions. +The project roles enable you to work within the projects in various capacities. They primarily provide access to project-level permissions such as repos and the IDE or dbt Cloud CLI, but may also provide some account-level permissions. #### Account permissions for project roles @@ -61,12 +61,14 @@ The project roles enable you to work within the projects in various capacities. | Account settings | R | | R | | R | | | | | | R | | | Auth provider | | | | | | | | | | | | | | Billing | | | | | | | | | | | | | -| Invitations | W | R | R | R | R | R | R | | | R | R | | -| Members | W | | R | R | R | | | | | R | R | | +| Groups | R | | R | R | R | | | | | R | R | | +| Invitations | W | R | R | R | R | R | R | | | R | R | | +| Licenses | W | R | R | R | R | R | R | | | | R | | +| Members | W | | R | R | R | | | | | R | R | | | Project (create) | | | | | | | | | | | | | -| Public models | R | R | R | R | R | R | R | R | R | R | R | R | +| Public models | R | R | R | R | R | R | R | R | R | R | R | R | | Service tokens | | | | | | | | | | | | | -| Webhooks | W | | | W | | | | | | | | W | +| Webhooks | W | | | W | | | | | | | | W | #### Project permissions for project roles @@ -74,13 +76,11 @@ The project roles enable you to work within the projects in various capacities. |--------------------------|:-----:|:-------:|:--------------:|:---------:|:---------:|:---------:|:-----------:|:--------:|:--------------:|:-----------:|:----------:|:------:| | Connections | W | R | W | R | R | R | | | | R | R | | | Credentials | W | W | W | W | R | W | | | | R | R | | -| Custom env. variables | W | W | W | W | W | W | R | | | R | W | | +| Custom env. variables | W | W | W | W | W | W | R | | | R | W | | | dbt adapters | W | W | W | W | R | W | | | | R | R | | -| Develop (IDE) | W | W | | W | | | | | | | | | +| Develop (IDE or dbt Cloud CLI) | W | W | | W | | | | | | | | | | Environments | W | R | R | R | R | W | R | | | R | R | | -| Groups | R | | R | R | R | | | | | R | R | | | Jobs | W | R | R | W | R | W | R | | | R | R | | -| Licenses | W | R | R | R | R | R | R | | | | R | | | Metadata | R | R | R | R | R | R | R | R | | R | R | | | Permissions | W | | R | R | R | | | | | | W | | | Profile | W | R | W | R | R | R | | | | R | R | | diff --git a/website/snippets/_manifest-versions.md b/website/snippets/_manifest-versions.md new file mode 100644 index 00000000000..c9b3e7af6ec --- /dev/null +++ b/website/snippets/_manifest-versions.md @@ -0,0 +1,11 @@ + +| dbt Core version | Manifest version | +|------------------|---------------------------------------------------------------| +| v1.7 | [v11](https://schemas.getdbt.com/dbt/manifest/v11/index.html) | +| v1.6 | [v10](https://schemas.getdbt.com/dbt/manifest/v10/index.html) | +| v1.5 | [v9](https://schemas.getdbt.com/dbt/manifest/v9/index.html) | +| v1.4 | [v8](https://schemas.getdbt.com/dbt/manifest/v8/index.html) | +| v1.3 | [v7](https://schemas.getdbt.com/dbt/manifest/v7/index.html) | +| v1.2 | [v6](https://schemas.getdbt.com/dbt/manifest/v6/index.html) | +| v1.1 | [v5](https://schemas.getdbt.com/dbt/manifest/v5/index.html) | +| v1.0 | [v4](https://schemas.getdbt.com/dbt/manifest/v4/index.html) | \ No newline at end of file diff --git a/website/snippets/_microsoft-adapters-soon.md b/website/snippets/_microsoft-adapters-soon.md new file mode 100644 index 00000000000..c3f30ef0939 --- /dev/null +++ b/website/snippets/_microsoft-adapters-soon.md @@ -0,0 +1,3 @@ +:::tip Coming soon +dbt Cloud support for the Microsoft Fabric and Azure Synapse Analytics adapters is coming soon! +::: \ No newline at end of file diff --git a/website/snippets/_new-sl-setup.md b/website/snippets/_new-sl-setup.md index b802db9c5ae..ad248bc3ca9 100644 --- a/website/snippets/_new-sl-setup.md +++ b/website/snippets/_new-sl-setup.md @@ -1,13 +1,13 @@ You can set up the dbt Semantic Layer in dbt Cloud at the environment and project level. Before you begin: -- You must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment, hosted in North America. +- You must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment. Single-tenant coming soon. - You must be part of the Owner group, and have the correct [license](/docs/cloud/manage-access/seats-and-users) and [permissions](/docs/cloud/manage-access/self-service-permissions) to configure the Semantic Layer: * Enterprise plan — Developer license with Account Admin permissions. Or Owner with a Developer license, assigned Project Creator, Database Admin, or Admin permissions. * Team plan — Owner with a Developer license. - You must have a successful run in your new environment. :::tip -If you're using the legacy Semantic Layer, we **highly** recommend you [upgrade your dbt version](/docs/dbt-versions/upgrade-core-in-cloud) to dbt v1.6 or higher to use the new dbt Semantic Layer. Refer to the dedicated [migration guide](/guides/migration/sl-migration) for more info. +If you're using the legacy Semantic Layer, dbt Labs strongly recommends that you [upgrade your dbt version](/docs/dbt-versions/upgrade-core-in-cloud) to dbt version 1.6 or newer to use the latest dbt Semantic Layer. Refer to the dedicated [migration guide](/guides/migration/sl-migration) for details. ::: 1. In dbt Cloud, create a new [deployment environment](/docs/deploy/deploy-environments#create-a-deployment-environment) or use an existing environment on dbt 1.6 or higher. diff --git a/website/snippets/_sl-connect-and-query-api.md b/website/snippets/_sl-connect-and-query-api.md new file mode 100644 index 00000000000..429f41c3bf6 --- /dev/null +++ b/website/snippets/_sl-connect-and-query-api.md @@ -0,0 +1,10 @@ +You can query your metrics in a JDBC-enabled tool or use existing first-class integrations with the dbt Semantic Layer. + +You must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment. Single-tenant coming soon. + +- To learn how to use the JDBC or GraphQL API and what tools you can query it with, refer to [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview). + + * To authenticate, you need to [generate a service token](/docs/dbt-cloud-apis/service-tokens) with Semantic Layer Only and Metadata Only permissions. + * Refer to the [SQL query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the API. + +- To learn more about the sophisticated integrations that connect to the dbt Semantic Layer, refer to [Available integrations](/docs/use-dbt-semantic-layer/avail-sl-integrations) for more info. diff --git a/website/snippets/_sl-create-semanticmodel.md b/website/snippets/_sl-create-semanticmodel.md index bc4276efcb6..6e0376ab10b 100644 --- a/website/snippets/_sl-create-semanticmodel.md +++ b/website/snippets/_sl-create-semanticmodel.md @@ -1,4 +1,4 @@ -The following steps will walk you through setting up semantic models, which you can do with the dbt Cloud IDE or the CLI. Semantic models consist of [entities](/docs/build/entities), [dimensions](/docs/build/dimensions), and [measures](/docs/build/measures). +The following steps describe how to set up semantic models. Semantic models consist of [entities](/docs/build/entities), [dimensions](/docs/build/dimensions), and [measures](/docs/build/measures). We highly recommend you read the overview of what a [semantic model](/docs/build/semantic-models) is before getting started. If you're working in the [Jaffle shop example](https://github.com/dbt-labs/jaffle-sl-template), delete the `orders.yml` config or delete the .yml extension so it's ignored during parsing. **We'll be rebuilding it step by step in this example.** diff --git a/website/snippets/_sl-define-metrics.md b/website/snippets/_sl-define-metrics.md index 29af3f5b7c3..3734e819c1b 100644 --- a/website/snippets/_sl-define-metrics.md +++ b/website/snippets/_sl-define-metrics.md @@ -1,4 +1,4 @@ -Now that you've created your first semantic model, it's time to define your first metric! You can define metrics with the dbt Cloud IDE or CLI. +Now that you've created your first semantic model, it's time to define your first metric! You can define metrics with the dbt Cloud IDE or command line. MetricFlow supports different metric types like [simple](/docs/build/simple), [ratio](/docs/build/ratio), [cumulative](/docs/build/cumulative), and [derived](/docs/build/derived). It's recommended that you read the [metrics overview docs](/docs/build/metrics-overview) before getting started. diff --git a/website/snippets/_sl-install-metricflow.md b/website/snippets/_sl-install-metricflow.md deleted file mode 100644 index 73e60d34e85..00000000000 --- a/website/snippets/_sl-install-metricflow.md +++ /dev/null @@ -1,8 +0,0 @@ -Install the [MetricFlow CLI](/docs/build/metricflow-cli) as an extension of a dbt adapter from PyPI. The MetricFlow CLI is compatible with Python versions 3.8, 3.9, 3.10 and 3.11 - -Use pip install `metricflow` and your [dbt adapter](/docs/supported-data-platforms): - -- Create or activate your virtual environment. `python -m venv venv` or `source your-venv/bin/activate` -- Run `pip install "dbt-metricflow[your_adapter_name]"` - - You must specify `[your_adapter_name]`. - - For example, run `pip install "dbt-metricflow[snowflake]"` if you use a Snowflake adapter. diff --git a/website/snippets/_sl-partner-links.md b/website/snippets/_sl-partner-links.md index fa8146c6418..59e5b54b7c0 100644 --- a/website/snippets/_sl-partner-links.md +++ b/website/snippets/_sl-partner-links.md @@ -2,26 +2,61 @@ The following tools integrate with the dbt Semantic Layer:
-Before you connect to these tools, you'll need to first [set up the dbt Semantic Layer](/docs/use-dbt-semantic-layer/setup-sl) and [generate a service token](/docs/dbt-cloud-apis/service-tokens) to create **Semantic Layer Only** and **Metadata Only** permissions. - +Before you connect to these tools, you'll need to first [set up the dbt Semantic Layer](/docs/use-dbt-semantic-layer/setup-sl) and [generate a service token](/docs/dbt-cloud-apis/service-tokens) to create **Semantic Layer Only** and **Metadata Only** permissions. diff --git a/website/snippets/_sl-plan-info.md b/website/snippets/_sl-plan-info.md index 5fba18de6bb..083ab2209bc 100644 --- a/website/snippets/_sl-plan-info.md +++ b/website/snippets/_sl-plan-info.md @@ -1,2 +1,2 @@ -To define and query metrics with the {props.product}, you must be on a {props.plan} multi-tenant plan, {props.instance} (Additional region support coming soon).
The re-released dbt Semantic Layer is available on dbt v1.6 or higher. dbt Core users can use the MetricFlow CLI to define metrics in their local project, but won't be able to dynamically query them with integrated tools.
+To define and query metrics with the {props.product}, you must be on a {props.plan} multi-tenant plan .
diff --git a/website/snippets/_sl-run-prod-job.md b/website/snippets/_sl-run-prod-job.md new file mode 100644 index 00000000000..a637b0b431e --- /dev/null +++ b/website/snippets/_sl-run-prod-job.md @@ -0,0 +1,7 @@ +Once you’ve defined metrics in your dbt project, you can perform a job run in your deployment environment in dbt Cloud to materialize your metrics. The deployment environment is only supported for the dbt Semantic Layer currently. + +1. Select **Deploy** from the top navigation bar. +2. Select **Jobs** to rerun the job with the most recent code in the deployment environment. +3. Your metric should appear as a red node in the dbt Cloud IDE and dbt directed acyclic graphs (DAG). + +
-Before you begin, you'll need to install the [MetricFlow CLI](/docs/build/metricflow-cli) package and make sure you run at least one model.
-### Install MetricFlow
-import InstallMetricFlow from '/snippets/_sl-install-metricflow.md';
+
- --group-by ` to query the metrics and dimensions. For example, `mf query --metrics order_total --group-by metric_time`
-4. Verify that the metric values are what you expect. To further understand how the metric is being generated, you can view the generated SQL if you type `--explain` in the CLI.
-5. Run `mf validate-configs` to run validation on your semantic models and metrics.
-6. Commit and merge the code changes that contain the metric definitions.
+To get started:
+
+1. Make sure you've installed the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation).
+2. Navigate to your dbt project directory.
+3. Run a dbt command, such as `dbt parse`, `dbt run`, `dbt compile`, or `dbt build`. If you don't, you'll receive an error message that begins with: "ensure that you've ran an artifacts...."
+ - MetricFlow builds a semantic graph and generates a `semantic_manifest.json` file in dbt Cloud, which is stored in the `/target` directory. If using the Jaffle shop example, run `dbt seed && dbt run` to ensure the required data is in your data platform before proceeding.
+
+4. Run `dbt sl --help` to confirm you have MetricFlow installed and that you can view the available commands.
+5. Run `dbt sl query --metrics --group-by ` to query the metrics and dimensions. For example, `dbt sl query --metrics order_total --group-by metric_time`
+6. Verify that the metric values are what you expect. To further understand how the metric is being generated, you can view the generated SQL if you type `--compile` in the command line.
+7. Commit and merge the code changes that contain the metric definitions.
To streamline your metric querying process, you can connect to the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) to access your metrics programmatically. For SQL syntax, refer to [Querying the API for metric metadata](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the API.
+
+
+
+
+
+
+
+This step is for dbt Core users only. MetricFlow is compatible with Python versions 3.8, 3.9, 3.10 and 3.11. You need to use `pip` to install MetricFlow on Windows or Linux operating systems:
+
+:::note
+The dbt Cloud CLI is strongly recommended to define and query metrics for your dbt project in dbt Cloud or dbt Core with MetricFlow. If you're using dbt Core, you'll need to manage versioning between dbt Core, your adapter, and MetricFlow.
+:::
+
+
+1. Install [MetricFlow](/docs/build/metricflow-commands) as an extension of a dbt adapter from PyPI.
+2. Create or activate your virtual environment with `python -m venv venv` or `source your-venv/bin/activate`.
+3. Run `pip install dbt-metricflow`.
+ - You can install MetricFlow using PyPI as an extension of your dbt adapter in the command line. To install the adapter, run `pip install "dbt-metricflow[your_adapter_name]"` and add the adapter name at the end of the command. As an example for a Snowflake adapter, run `pip install "dbt-metricflow[snowflake]"`.
+ - You'll need to manage versioning between dbt Core, your adapter, and MetricFlow.
+4. Run `dbt parse`. This allows MetricFlow to build a semantic graph and generate a `semantic_manifest.json`.
+ - This creates the file in your `/target` directory. If you're working from the Jaffle shop example, run `dbt seed && dbt run` before proceeding to ensure the data exists in your warehouse.
+5. Run `mf --help` to confirm you have MetricFlow installed and that you can view the available commands.
+6. Run `mf query --metrics --group-by ` to query the metrics and dimensions. For example, `mf query --metrics order_total --group-by metric_time`.
+7. Verify that the metric values are what you expect. To further understand how the metric is being generated, you can view the generated SQL if you type `--explain` in the command line.
+8. Run `mf validate-configs` to run validation on your semantic models and metrics.
+9. Commit and merge the code changes that contain the metric definitions.
+
+To streamline your metric querying process, you can connect to the [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview) to access your metrics programmatically. For SQL syntax, refer to [Querying the API for metric metadata](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the API.
+
+
+
+
+
+
diff --git a/website/snippets/_v2-sl-prerequisites.md b/website/snippets/_v2-sl-prerequisites.md
index 9fdc3b53143..852730604d0 100644
--- a/website/snippets/_v2-sl-prerequisites.md
+++ b/website/snippets/_v2-sl-prerequisites.md
@@ -1,17 +1,14 @@
-To use the Semantic Layer, you must:
-
-- Have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment, hosted in North America.
+- Have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment. Single-tenant coming soon.
- Have both your production and development environments running dbt version 1.6 or higher. Refer to [upgrade in dbt Cloud](/docs/dbt-versions/upgrade-core-in-cloud) for more info.
-- Use Snowflake, BigQuery, Databricks, or Redshift (dbt Cloud Postgres support coming soon).
+- Use Snowflake, BigQuery, Databricks, or Redshift.
- Create a successful run in the environment where you configure the Semantic Layer.
- **Note:** Semantic Layer currently supports the Deployment environment for querying. (_development querying experience coming soon_)
- Set up the [Semantic Layer API](/docs/dbt-cloud-apis/sl-api-overview) in the integrated tool to import metric definitions.
- - **Note:** To access the API and query metrics in downstream tools, you must have a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account. dbt Core or Developer accounts can define metrics with the [MetricFlow CLI](/docs/build/metricflow-cli) or [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) but won't be able to dynamically query them.
-- Understand [MetricFlow's](/docs/build/about-metricflow) key concepts, which powers the revamped dbt Semantic Layer. - + - To access the API and query metrics in downstream tools, you must have a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account. dbt Core or Developer accounts can define metrics but won't be able to dynamically query them.
+- Understand [MetricFlow's](/docs/build/about-metricflow) key concepts, which powers the latest dbt Semantic Layer.
diff --git a/website/snippets/core-versions-table.md b/website/snippets/core-versions-table.md
index 431e1f08b4c..b08c23c84c5 100644
--- a/website/snippets/core-versions-table.md
+++ b/website/snippets/core-versions-table.md
@@ -6,7 +6,7 @@
| [**v1.6**](/guides/migration/versions/upgrading-to-v1.6) | Jul 31, 2023 | Active | Jul 30, 2024 |
| [**v1.5**](/guides/migration/versions/upgrading-to-v1.5) | Apr 27, 2023 | Critical | Apr 27, 2024 |
| [**v1.4**](/guides/migration/versions/upgrading-to-v1.4) | Jan 25, 2023 | Critical | Jan 25, 2024 |
-| [**v1.3**](/guides/migration/versions/upgrading-to-v1.3) | Oct 12, 2022 | Critical | Oct 12, 2023 |
+| [**v1.3**](/guides/migration/versions/upgrading-to-v1.3) | Oct 12, 2022 | End of Life* ⚠️ | Oct 12, 2023 |
| [**v1.2**](/guides/migration/versions/upgrading-to-v1.2) | Jul 26, 2022 | End of Life* ⚠️ | Jul 26, 2023 |
| [**v1.1**](/guides/migration/versions/upgrading-to-v1.1) ⚠️ | Apr 28, 2022 | Deprecated ⛔️ | Deprecated ⛔️ |
| [**v1.0**](/guides/migration/versions/upgrading-to-v1.0) ⚠️ | Dec 3, 2021 | Deprecated ⛔️ | Deprecated ⛔️ |
diff --git a/website/src/css/custom.css b/website/src/css/custom.css
index 972b86a48ca..fc51ef8a8ef 100644
--- a/website/src/css/custom.css
+++ b/website/src/css/custom.css
@@ -2022,6 +2022,18 @@ html[data-theme="dark"] .theme-doc-sidebar-container>div>button.button:hover {
display: none;
}
+.card-container {
+ position: relative;
+}
+
+.external-link {
+ position: absolute;
+ top: 0;
+ right: 0;
+ margin: 10px;
+ color: #818589; /* You can adjust the color as needed */
+}
+
@media (max-width: 996px) {
.quickstart-container {
flex-direction: column;
diff --git a/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-child.png b/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-child.png
new file mode 100644
index 00000000000..666db3384fa
Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-child.png differ
diff --git a/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png b/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png
new file mode 100644
index 00000000000..ee5d19de369
Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png differ
diff --git a/website/static/img/docs/dbt-cloud/cloud-ide/ide-command-bar.jpg b/website/static/img/docs/dbt-cloud/cloud-ide/ide-command-bar.jpg
new file mode 100644
index 00000000000..fe60ddd7e03
Binary files /dev/null and b/website/static/img/docs/dbt-cloud/cloud-ide/ide-command-bar.jpg differ
diff --git a/website/static/img/docs/dbt-cloud/dbt-cloud-enterprise/azure/azure-redirect-uri.png b/website/static/img/docs/dbt-cloud/dbt-cloud-enterprise/azure/azure-redirect-uri.png
index 7daaab4504d..3bb04467abd 100644
Binary files a/website/static/img/docs/dbt-cloud/dbt-cloud-enterprise/azure/azure-redirect-uri.png and b/website/static/img/docs/dbt-cloud/dbt-cloud-enterprise/azure/azure-redirect-uri.png differ
diff --git a/website/static/img/docs/dbt-cloud/defer-toggle.jpg b/website/static/img/docs/dbt-cloud/defer-toggle.jpg
new file mode 100644
index 00000000000..7bd5a1c1283
Binary files /dev/null and b/website/static/img/docs/dbt-cloud/defer-toggle.jpg differ
diff --git a/website/static/img/docs/dbt-cloud/using-dbt-cloud/dbt-cloud-enterprise/DBX-auth/dbt-databricks-oauth-user.png b/website/static/img/docs/dbt-cloud/using-dbt-cloud/dbt-cloud-enterprise/DBX-auth/dbt-databricks-oauth-user.png
new file mode 100644
index 00000000000..aecf99d726a
Binary files /dev/null and b/website/static/img/docs/dbt-cloud/using-dbt-cloud/dbt-cloud-enterprise/DBX-auth/dbt-databricks-oauth-user.png differ
diff --git a/website/static/img/docs/dbt-cloud/using-dbt-cloud/dbt-cloud-enterprise/DBX-auth/dbt-databricks-oauth.png b/website/static/img/docs/dbt-cloud/using-dbt-cloud/dbt-cloud-enterprise/DBX-auth/dbt-databricks-oauth.png
new file mode 100644
index 00000000000..bb32fab2afb
Binary files /dev/null and b/website/static/img/docs/dbt-cloud/using-dbt-cloud/dbt-cloud-enterprise/DBX-auth/dbt-databricks-oauth.png differ
diff --git a/website/static/img/icons/google-sheets-logo-icon.svg b/website/static/img/icons/google-sheets-logo-icon.svg
new file mode 100644
index 00000000000..d080c1dd53d
--- /dev/null
+++ b/website/static/img/icons/google-sheets-logo-icon.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/website/static/img/icons/hex.svg b/website/static/img/icons/hex.svg
new file mode 100755
index 00000000000..00431ffe299
--- /dev/null
+++ b/website/static/img/icons/hex.svg
@@ -0,0 +1,5 @@
+
diff --git a/website/static/img/icons/lightdash.svg b/website/static/img/icons/lightdash.svg
new file mode 100644
index 00000000000..96f4676e7ee
--- /dev/null
+++ b/website/static/img/icons/lightdash.svg
@@ -0,0 +1,96 @@
+
\ No newline at end of file
diff --git a/website/static/img/icons/mode.svg b/website/static/img/icons/mode.svg
new file mode 100644
index 00000000000..269c182cd8b
--- /dev/null
+++ b/website/static/img/icons/mode.svg
@@ -0,0 +1,165 @@
+
\ No newline at end of file
diff --git a/website/static/img/icons/tableau-software.svg b/website/static/img/icons/tableau-software.svg
new file mode 100644
index 00000000000..28996f1dadd
--- /dev/null
+++ b/website/static/img/icons/tableau-software.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/website/static/img/icons/white/google-sheets-logo-icon.svg b/website/static/img/icons/white/google-sheets-logo-icon.svg
new file mode 100644
index 00000000000..d080c1dd53d
--- /dev/null
+++ b/website/static/img/icons/white/google-sheets-logo-icon.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/website/static/img/icons/white/hex.svg b/website/static/img/icons/white/hex.svg
new file mode 100644
index 00000000000..00431ffe299
--- /dev/null
+++ b/website/static/img/icons/white/hex.svg
@@ -0,0 +1,5 @@
+
diff --git a/website/static/img/icons/white/lightdash.svg b/website/static/img/icons/white/lightdash.svg
new file mode 100644
index 00000000000..96f4676e7ee
--- /dev/null
+++ b/website/static/img/icons/white/lightdash.svg
@@ -0,0 +1,96 @@
+
\ No newline at end of file
diff --git a/website/static/img/icons/white/mode.svg b/website/static/img/icons/white/mode.svg
new file mode 100644
index 00000000000..269c182cd8b
--- /dev/null
+++ b/website/static/img/icons/white/mode.svg
@@ -0,0 +1,165 @@
+
\ No newline at end of file
diff --git a/website/static/img/icons/white/tableau-software.svg b/website/static/img/icons/white/tableau-software.svg
new file mode 100644
index 00000000000..28996f1dadd
--- /dev/null
+++ b/website/static/img/icons/white/tableau-software.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/website/vercel.json b/website/vercel.json
index a9187933980..14b3a0a6af0 100644
--- a/website/vercel.json
+++ b/website/vercel.json
@@ -2,6 +2,21 @@
"cleanUrls": true,
"trailingSlash": false,
"redirects": [
+ {
+ "source": "/docs/build/metricflow-cli",
+ "destination": "/docs/build/metricflow-commands",
+ "permanent": true
+ },
+ {
+ "source": "/docs/core/about-the-cli",
+ "destination": "/docs/core/about-dbt-core",
+ "permanent": true
+ },
+ {
+ "source": "/docs/cloud/about-cloud/about-cloud-ide",
+ "destination": "/docs/cloud/about-cloud-develop",
+ "permanent": true
+ },
{
"source": "/faqs/models/reference-models-in-another-project",
"destination": "/docs/collaborate/govern/project-dependencies",
`grain('week\|day\|month\|quarter\|year')]` | Optional | -| `where` | A where clause that allows you to filter on dimensions and entities using parameters - comes with `TimeDimension`, `Dimension`, and `Entity` objects. Granularity is required with `TimeDimension` | `"{{ where=Dimension('customer__country') }} = 'US')"` | Optional | +| `where` | A where clause that allows you to filter on dimensions and entities using parameters. This takes a filter list OR string. Inputs come with `Dimension`, and `Entity` objects. Granularity is required if the `Dimension` is a time dimension | `"{{ where=Dimension('customer__country') }} = 'US')"` | Optional | | `limit` | Limit the data returned | `limit=10` | Optional | -|`order` | Order the data returned | `order_by=['-order_gross_profit']` (remove `-` for ascending order) | Optional | +|`order` | Order the data returned by a particular field | `order_by=['order_gross_profit']`, use `-` for descending, or full object notation if the object is operated on: `order_by=[Metric('order_gross_profit').descending(True)`] | Optional | | `compile` | If true, returns generated SQL for the data platform but does not execute | `compile=True` | Optional | + ## Note on time dimensions and `metric_time` You will notice that in the list of dimensions for all metrics, there is a dimension called `metric_time`. `Metric_time` is a reserved keyword for the measure-specific aggregation time dimensions. For any time-series metric, the `metric_time` keyword should always be available for use in queries. This is a common dimension across *all* metrics in a semantic graph. @@ -246,13 +252,13 @@ select * from {{ Where filters in API allow for a filter list or string. We recommend using the filter list for production applications as this format will realize all benefits from the
* Install these adapters using the CLI as they're not currently supported in dbt Cloud.
+
* Install these adapters using dbt Core as they're not currently supported in dbt Cloud.
diff --git a/website/docs/docs/trusted-adapters.md b/website/docs/docs/trusted-adapters.md index e19bb40785f..08191e8ea42 100644 --- a/website/docs/docs/trusted-adapters.md +++ b/website/docs/docs/trusted-adapters.md @@ -6,7 +6,7 @@ hide_table_of_contents: true Trusted adapters are adapters not maintained by dbt Labs, that we feel comfortable recommending to users for use in production. -Free and open-source tools for the data professional are increasingly abundant. This is by-and-large a *good thing*, however it requires due dilligence that wasn't required in a paid-license, closed-source software world. As a user, there are questions to answer important before taking a dependency on an open-source project. The trusted adapter designation is meant to streamline this process for end users. +Free and open-source tools for the data professional are increasingly abundant. This is by-and-large a *good thing*, however it requires due diligence that wasn't required in a paid-license, closed-source software world. As a user, there are questions to answer important before taking a dependency on an open-source project. The trusted adapter designation is meant to streamline this process for end users.
Considerations for depending on an open-source project
diff --git a/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md b/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md index ea5833d586b..a398ace164e 100644 --- a/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md +++ b/website/docs/docs/use-dbt-semantic-layer/avail-sl-integrations.md @@ -11,10 +11,6 @@ meta:
@@ -59,8 +56,8 @@ instance="hosted in North America"
icon="dbt-bit"/>
-import NewSLChanges from '/snippets/_new-sl-changes.md';
-import InstallMetricFlow from '/snippets/_sl-install-metricflow.md';
+
import CreateModel from '/snippets/_sl-create-semanticmodel.md';
import DefineMetrics from '/snippets/_sl-define-metrics.md';
import ConfigMetric from '/snippets/_sl-configure-metricflow.md';
import TestQuery from '/snippets/_sl-test-and-query-metrics.md';
+import ConnectQueryAPI from '/snippets/_sl-connect-and-query-api.md';
+import RunProdJob from '/snippets/_sl-run-prod-job.md';
-
-
diff --git a/website/docs/reference/node-selection/putting-it-together.md b/website/docs/reference/node-selection/putting-it-together.md
index 8faf02e6cc9..48fc5188b32 100644
--- a/website/docs/reference/node-selection/putting-it-together.md
+++ b/website/docs/reference/node-selection/putting-it-together.md
@@ -4,16 +4,16 @@ title: "Putting it together"
```bash
- $ dbt run --select my_package.*+ # select all models in my_package and their children
- $ dbt run --select +some_model+ # select some_model and all parents and children
+dbt run --select "my_package.*+" # select all models in my_package and their children
+dbt run --select "+some_model+" # select some_model and all parents and children
- $ dbt run --select tag:nightly+ # select "nightly" models and all children
- $ dbt run --select +tag:nightly+ # select "nightly" models and all parents and children
+dbt run --select "tag:nightly+" # select "nightly" models and all children
+dbt run --select "+tag:nightly+" # select "nightly" models and all parents and children
- $ dbt run --select @source:snowplow # build all models that select from snowplow sources, plus their parents
+dbt run --select "@source:snowplow" # build all models that select from snowplow sources, plus their parents
- $ dbt test --select config.incremental_strategy:insert_overwrite,test_name:unique # execute all `unique` tests that select from models using the `insert_overwrite` incremental strategy
- ```
+dbt test --select "config.incremental_strategy:insert_overwrite,test_name:unique" # execute all `unique` tests that select from models using the `insert_overwrite` incremental strategy
+```
@@ -22,8 +22,8 @@ and feed exports, while _excluding_ the biggest incremental models (and one othe
```bash
- $ dbt run --select @source:snowplow,tag:nightly models/export --exclude package:snowplow,config.materialized:incremental export_performance_timing
- ```
+dbt run --select "@source:snowplow,tag:nightly models/export" --exclude "package:snowplow,config.materialized:incremental export_performance_timing"
+```
This command selects all models that:
diff --git a/website/docs/reference/node-selection/set-operators.md b/website/docs/reference/node-selection/set-operators.md
index 7d6b6c2411c..af399b9cad5 100644
--- a/website/docs/reference/node-selection/set-operators.md
+++ b/website/docs/reference/node-selection/set-operators.md
@@ -11,7 +11,7 @@ Run snowplow_sessions, all ancestors of snowplow_sessions, fct_orders, and all a
```bash
- $ dbt run --select +snowplow_sessions +fct_orders
+dbt run --select "+snowplow_sessions +fct_orders"
```
### Intersections
@@ -22,15 +22,15 @@ Run all the common ancestors of snowplow_sessions and fct_orders:
```bash
- $ dbt run --select +snowplow_sessions,+fct_orders
- ```
+dbt run --select "+snowplow_sessions,+fct_orders"
+```
Run all the common descendents of stg_invoices and stg_accounts:
```bash
- $ dbt run --select stg_invoices+,stg_accounts+
+dbt run --select "stg_invoices+,stg_accounts+"
```
@@ -38,5 +38,5 @@ Run models that are in the marts/finance subdirectory *and* tagged nightly:
```bash
- $ dbt run --select marts.finance,tag:nightly
- ```
+dbt run --select "marts.finance,tag:nightly"
+```
diff --git a/website/docs/reference/node-selection/state-comparison-caveats.md b/website/docs/reference/node-selection/state-comparison-caveats.md
index baeeb7e4c75..73947c80a66 100644
--- a/website/docs/reference/node-selection/state-comparison-caveats.md
+++ b/website/docs/reference/node-selection/state-comparison-caveats.md
@@ -27,8 +27,8 @@ The command `dbt test -s state:modified` will include both:
As long as you're adding or changing tests at the same time that you're adding or changing the resources (models, seeds, snapshots) they select from, all should work the way you expect with "simple" state selection:
```shell
-$ dbt run -s state:modified
-$ dbt test -s state:modified
+dbt run -s "state:modified"
+dbt test -s "state:modified"
```
This can get complicated, however. If you add a new test without modifying its underlying model, or add a test that selects from a new model and an old unmodified one, you may need to test a model without having first run it.
@@ -36,8 +36,8 @@ This can get complicated, however. If you add a new test without modifying its u
In v0.18.0, you needed to handle this by building the unmodified models needed for modified tests:
```shell
-$ dbt run -s state:modified @state:modified,1+test_type:data
-$ dbt test -s state:modified
+dbt run -s "state:modified @state:modified,1+test_type:data"
+dbt test -s "state:modified"
```
In v0.19.0, dbt added support for deferring upstream references when testing. If a test selects from a model that doesn't exist as a database object in your current environment, dbt will look to the other environment instead—the one defined in your state manifest. This enables you to use "simple" state selection without risk of query failure, but it may have some surprising consequences for tests with multiple parents. For instance, if you have a `relationships` test that depends on one modified model and one unmodified model, the test query will select from data "across" two different environments. If you limit or sample your data in development and CI, it may not make much sense to test for referential integrity, knowing there's a good chance of mismatch.
@@ -45,8 +45,8 @@ In v0.19.0, dbt added support for deferring upstream references when testing. If
If you're a frequent user of `relationships` tests or data tests, or frequently find yourself adding tests without modifying their underlying models, consider tweaking the selection criteria of your CI job. For instance:
```shell
-$ dbt run -s state:modified
-$ dbt test -s state:modified --exclude test_name:relationships
+dbt run -s "state:modified"
+dbt test -s "state:modified" --exclude "test_name:relationships"
```
### False positives
@@ -58,7 +58,7 @@ State comparison works by identifying discrepancies between two manifests. Thos
dbt will do its best to capture *only* changes that are the result of modifications made in development. In projects with intricate env-aware logic, dbt will err on the side of running too many models (i.e. false positives). Over the next several versions of dbt, we're working on:
- iterative improvements to dbt's built-in detective abilities
-- better options for more complex projects, in the form of more-specific subselectors (see [this issue](https://github.com/dbt-labs/dbt-core/issues/2704))
+- better options for more complex projects, in the form of more-specific sub-selectors (see [this issue](https://github.com/dbt-labs/dbt-core/issues/2704))
State comparison is now able to detect env-aware config in `dbt_project.yml`. For instance, this target-based config would register as a modification in v0.18.0, but in v0.19.0 it no longer will:
diff --git a/website/docs/reference/node-selection/syntax.md b/website/docs/reference/node-selection/syntax.md
index 10fdebd0f94..616bcfe6447 100644
--- a/website/docs/reference/node-selection/syntax.md
+++ b/website/docs/reference/node-selection/syntax.md
@@ -14,7 +14,7 @@ dbt's node selection syntax makes it possible to run only specific resources in
| [compile](/reference/commands/compile) | `--select`, `--exclude`, `--selector`, `--inline` |
| [freshness](/reference/commands/source) | `--select`, `--exclude`, `--selector` |
| [build](/reference/commands/build) | `--select`, `--exclude`, `--selector`, `--resource-type`, `--defer` |
-| [docs generate](/reference/commands/cmd-docs) | `--select` |
+| [docs generate](/reference/commands/cmd-docs) | `--select`, `--exclude`, `--selector`, `--defer` |
:::info Nodes and resources
@@ -25,6 +25,8 @@ We use the terms "
By default, `dbt run` executes _all_ of the models in the dependency graph; `dbt seed` creates all seeds, `dbt snapshot` performs every snapshot. The `--select` flag is used to specify a subset of nodes to execute.
+To follow [POSIX standards](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html) and make things easier to understand, we recommend CLI users use quotes when passing arguments to the `--select` or `--exclude` option (including single or multiple space-delimited, or comma-delimited arguments). Not using quotes might not work reliably on all operating systems, terminals, and user interfaces. For example, `dbt run --select "my_dbt_project_name"` runs all models in your project.
+
### How does selection work?
1. dbt gathers all the resources that are matched by one or more of the `--select` criteria, in the order of selection methods (e.g. `tag:`), then graph operators (e.g. `+`), then finally set operators ([unions](/reference/node-selection/set-operators#unions), [intersections](/reference/node-selection/set-operators#intersections), [exclusions](/reference/node-selection/exclude)).
@@ -52,28 +54,28 @@ Examples:
```bash
- $ dbt run --select my_dbt_project_name # runs all models in your project
- $ dbt run --select my_dbt_model # runs a specific model
- $ dbt run --select path.to.my.models # runs all models in a specific directory
- $ dbt run --select my_package.some_model # run a specific model in a specific package
- $ dbt run --select tag:nightly # run models with the "nightly" tag
- $ dbt run --select path/to/models # run models contained in path/to/models
- $ dbt run --select path/to/my_model.sql # run a specific model by its path
+dbt run --select "my_dbt_project_name" # runs all models in your project
+dbt run --select "my_dbt_model" # runs a specific model
+dbt run --select "path.to.my.models" # runs all models in a specific directory
+dbt run --select "my_package.some_model" # run a specific model in a specific package
+dbt run --select "tag:nightly" # run models with the "nightly" tag
+dbt run --select "path/to/models" # run models contained in path/to/models
+dbt run --select "path/to/my_model.sql" # run a specific model by its path
```
dbt supports a shorthand language for defining subsets of nodes. This language uses the characters `+`, `@`, `*`, and `,`.
```bash
- # multiple arguments can be provided to --select
- $ dbt run --select my_first_model my_second_model
+# multiple arguments can be provided to --select
+ dbt run --select "my_first_model my_second_model"
- # these arguments can be projects, models, directory paths, tags, or sources
- $ dbt run --select tag:nightly my_model finance.base.*
+# these arguments can be projects, models, directory paths, tags, or sources
+dbt run --select "tag:nightly my_model finance.base.*"
- # use methods and intersections for more complex selectors
- $ dbt run --select path:marts/finance,tag:nightly,config.materialized:table
- ```
+# use methods and intersections for more complex selectors
+dbt run --select "path:marts/finance,tag:nightly,config.materialized:table"
+```
As your selection logic gets more complex, and becomes unwieldly to type out as command-line arguments,
consider using a [yaml selector](/reference/node-selection/yaml-selectors). You can use a predefined definition with the `--selector` flag.
@@ -151,7 +153,7 @@ After issuing one of the above commands, you can reference the results by adding
```bash
# You can also set the DBT_ARTIFACT_STATE_PATH environment variable instead of the --state flag.
-$ dbt run --select result: --defer --state path/to/prod/artifacts
+dbt run --select "result: --defer --state path/to/prod/artifacts"
```
The available options depend on the resource (node) type:
@@ -170,7 +172,7 @@ The available options depend on the resource (node) type:
The state and result selectors can also be combined in a single invocation of dbt to capture errors from a previous run OR any new or modified models.
```bash
-$ dbt run --select result:+ state:modified+ --defer --state ./
+dbt run --select "result:+ state:modified+ --defer --state ./"
```
### Fresh rebuilds
@@ -184,7 +186,7 @@ As example:
```bash
# Command step order
dbt source freshness
-dbt build --select source_status:fresher+
+dbt build --select "source_status:fresher+"
```
@@ -203,6 +205,6 @@ After issuing one of the above commands, you can reference the source freshness
```bash
# You can also set the DBT_ARTIFACT_STATE_PATH environment variable instead of the --state flag.
-$ dbt source freshness # must be run again to compare current to previous state
-$ dbt build --select source_status:fresher+ --state path/to/prod/artifacts
+dbt source freshness # must be run again to compare current to previous state
+dbt build --select "source_status:fresher+" --state path/to/prod/artifacts
```
diff --git a/website/docs/reference/node-selection/test-selection-examples.md b/website/docs/reference/node-selection/test-selection-examples.md
index 478aac932c5..ace886df258 100644
--- a/website/docs/reference/node-selection/test-selection-examples.md
+++ b/website/docs/reference/node-selection/test-selection-examples.md
@@ -19,14 +19,14 @@ Run generic tests only:
```bash
- dbt test --select test_type:generic
+ dbt test --select "test_type:generic"
```
Run singular tests only:
```bash
- dbt test --select test_type:singular
+ dbt test --select "test_type:singular"
```
In both cases, `test_type` checks a property of the test itself. These are forms of "direct" test selection.
@@ -87,8 +87,8 @@ By default, a test will run when ANY parent is selected; we call this "eager" in
In this mode, any test that depends on unbuilt resources will raise an error.
```shell
- dbt test --select orders
- dbt build --select orders
+dbt test --select "orders"
+dbt build --select "orders"
```
@@ -102,8 +102,10 @@ It will only include tests whose references are each within the selected nodes.
Put another way, it will prevent tests from running if one or more of its parents is unselected.
```shell
- dbt test --select orders --indirect-selection=cautious
- dbt build --select orders --indirect-selection=cautious
+
+dbt test --select "orders" --indirect-selection=cautious
+dbt build --select "orders" --indirect-selection=cautious
+
```
@@ -122,8 +124,8 @@ By default, a test will run when ANY parent is selected; we call this "eager" in
In this mode, any test that depends on unbuilt resources will raise an error.
```shell
- dbt test --select orders
- dbt build --select orders
+dbt test --select "orders"
+dbt build --select "orders"
```
@@ -137,8 +139,10 @@ It will only include tests whose references are each within the selected nodes.
Put another way, it will prevent tests from running if one or more of its parents is unselected.
```shell
- dbt test --select orders --indirect-selection=cautious
- dbt build --select orders --indirect-selection=cautious
+
+dbt test --select "orders" --indirect-selection=cautious
+dbt build --select "orders" --indirect-selection=cautious
+
```
@@ -152,8 +156,9 @@ It will only include tests whose references are each within the selected nodes (
This is useful in the same scenarios as "cautious", but also includes when a test depends on a model **and** a direct ancestor of that model (like confirming an aggregation has the same totals as its input).
```shell
- dbt test --select orders --indirect-selection=buildable
- dbt build --select orders --indirect-selection=buildable
+dbt test --select "orders" --indirect-selection=buildable
+dbt build --select "orders" --indirect-selection=buildable
+
```
@@ -172,8 +177,8 @@ By default, a test will run when ANY parent is selected; we call this "eager" in
In this mode, any test that depends on unbuilt resources will raise an error.
```shell
- dbt test --select orders
- dbt build --select orders
+dbt test --select "orders"
+dbt build --select "orders"
```
@@ -187,8 +192,9 @@ It will only include tests whose references are each within the selected nodes.
Put another way, it will prevent tests from running if one or more of its parents is unselected.
```shell
- dbt test --select orders --indirect-selection=cautious
- dbt build --select orders --indirect-selection=cautious
+dbt test --select "orders" --indirect-selection=cautious
+dbt build --select "orders" --indirect-selection=cautious
+
```
@@ -202,8 +208,8 @@ It will only include tests whose references are each within the selected nodes (
This is useful in the same scenarios as "cautious", but also includes when a test depends on a model **and** a direct ancestor of that model (like confirming an aggregation has the same totals as its input).
```shell
-dbt test --select orders --indirect-selection=buildable
-dbt build --select orders --indirect-selection=buildable
+dbt test --select "orders" --indirect-selection=buildable
+dbt build --select "orders" --indirect-selection=buildable
```
@@ -213,8 +219,10 @@ dbt build --select orders --indirect-selection=buildable
This mode will only include tests whose references are each within the selected nodes and will ignore all tests from attached nodes.
```shell
- dbt test --select orders --indirect-selection=empty
- dbt build --select orders --indirect-selection=empty
+
+dbt test --select "orders" --indirect-selection=empty
+dbt build --select "orders" --indirect-selection=empty
+
```
@@ -234,25 +242,25 @@ The following examples should feel somewhat familiar if you're used to executing
```bash
# Run tests on a model (indirect selection)
- dbt test --select customers
-
+ dbt test --select "customers"
+
# Run tests on two or more specific models (indirect selection)
- dbt test --select customers orders
+ dbt test --select "customers orders"
# Run tests on all models in the models/staging/jaffle_shop directory (indirect selection)
- dbt test --select staging.jaffle_shop
+ dbt test --select "staging.jaffle_shop"
# Run tests downstream of a model (note this will select those tests directly!)
- dbt test --select stg_customers+
+ dbt test --select "stg_customers+"
# Run tests upstream of a model (indirect selection)
- dbt test --select +stg_customers
+ dbt test --select "+stg_customers"
# Run tests on all models with a particular tag (direct + indirect)
- dbt test --select tag:my_model_tag
+ dbt test --select "tag:my_model_tag"
# Run tests on all models with a particular materialization (indirect selection)
- dbt test --select config.materialized:table
+ dbt test --select "config.materialized:table"
```
@@ -261,19 +269,20 @@ The following examples should feel somewhat familiar if you're used to executing
```bash
# tests on all sources
- dbt test --select source:*
- # tests on one source
- dbt test --select source:jaffle_shop
+ dbt test --select "source:*"
+ # tests on one source
+ dbt test --select "source:jaffle_shop"
+
# tests on two or more specific sources
- dbt test --select source:jaffle_shop source:raffle_bakery
+ dbt test --select "source:jaffle_shop source:raffle_bakery"
# tests on one source table
- dbt test --select source:jaffle_shop.customers
+ dbt test --select "source:jaffle_shop.customers"
# tests on everything _except_ sources
- dbt test --exclude source:*
+ dbt test --exclude "source:*"
```
### More complex selection
@@ -282,10 +291,12 @@ Through the combination of direct and indirect selection, there are many ways to
```bash
- dbt test --select assert_total_payment_amount_is_positive # directly select the test by name
- dbt test --select payments,test_type:singular # indirect selection, v1.2
- dbt test --select payments,test_type:data # indirect selection, v0.18.0
- dbt test --select payments --data # indirect selection, earlier versions
+
+ dbt test --select "assert_total_payment_amount_is_positive" # directly select the test by name
+ dbt test --select "payments,test_type:singular" # indirect selection, v1.2
+ dbt test --select "payments,test_type:data" # indirect selection, v0.18.0
+ dbt test --select "payments" --data # indirect selection, earlier versions
+
```
@@ -294,13 +305,14 @@ Through the combination of direct and indirect selection, there are many ways to
```bash
# Run tests on all models with a particular materialization
- dbt test --select config.materialized:table
+ dbt test --select "config.materialized:table"
# Run tests on all seeds, which use the 'seed' materialization
- dbt test --select config.materialized:seed
+ dbt test --select "config.materialized:seed"
# Run tests on all snapshots, which use the 'snapshot' materialization
- dbt test --select config.materialized:snapshot
+ dbt test --select "config.materialized:snapshot"
+
```
Note that this functionality may change in future versions of dbt.
@@ -328,7 +340,8 @@ models:
```bash
- dbt test --select tag:my_column_tag
+ dbt test --select "tag:my_column_tag"
+
```
Currently, tests "inherit" tags applied to columns, sources, and source tables. They do _not_ inherit tags applied to models, seeds, or snapshots. In all likelihood, those tests would still be selected indirectly, because the tag selects its parent. This is a subtle distinction, and it may change in future versions of dbt.
@@ -356,5 +369,6 @@ models:
```bash
- dbt test --select tag:my_test_tag
+ dbt test --select "tag:my_test_tag"
+
```
diff --git a/website/docs/reference/node-selection/yaml-selectors.md b/website/docs/reference/node-selection/yaml-selectors.md
index 78342e32779..1e3f8d8d1e2 100644
--- a/website/docs/reference/node-selection/yaml-selectors.md
+++ b/website/docs/reference/node-selection/yaml-selectors.md
@@ -34,6 +34,7 @@ Each `definition` is comprised of one or more arguments, which can be one of the
Use the `union` and `intersection` operator-equivalent keywords to organize multiple arguments.
### CLI-style
+
```yml
definition:
'tag:nightly'
@@ -42,6 +43,7 @@ definition:
This simple syntax supports use of the `+`, `@`, and `*` [graph](/reference/node-selection/graph-operators) operators, but it does not support [set](/reference/node-selection/set-operators) operators or `exclude`.
### Key-value
+
```yml
definition:
tag: nightly
@@ -317,7 +319,7 @@ selectors:
Then in our job definition:
```bash
-$ dbt run --selector nightly_diet_snowplow
+dbt run --selector nightly_diet_snowplow
```
## Default
@@ -325,6 +327,7 @@ $ dbt run --selector nightly_diet_snowplow
Selectors may define a boolean `default` property. If a selector has `default: true`, dbt will use this selector's criteria when tasks do not define their own selection criteria.
Let's say we define a default selector that only selects resources defined in our root project:
+
```yml
selectors:
- name: root_project_only
@@ -338,16 +341,18 @@ selectors:
```
If I run an "unqualified" command, dbt will use the selection criteria defined in `root_project_only`—that is, dbt will only build / freshness check / generate compiled SQL for resources defined in my root project.
+
```
-$ dbt build
-$ dbt source freshness
-$ dbt docs generate
+dbt build
+dbt source freshness
+dbt docs generate
```
If I run a command that defines its own selection criteria (via `--select`, `--exclude`, or `--selector`), dbt will ignore the default selector and use the flag criteria instead. It will not try to combine the two.
-```
-$ dbt run --select model_a
-$ dbt run --exclude model_a
+
+```bash
+dbt run --select "model_a"
+dbt run --exclude model_a
```
Only one selector may set `default: true` for a given invocation; otherwise, dbt will return an error. You may use a Jinja expression to adjust the value of `default` depending on the environment, however:
diff --git a/website/docs/reference/references-overview.md b/website/docs/reference/references-overview.md
index 16afd01607c..85a374c5aa3 100644
--- a/website/docs/reference/references-overview.md
+++ b/website/docs/reference/references-overview.md
@@ -4,6 +4,8 @@ id: "references-overview"
sidebar_label: "About References"
description: "Connect dbt to any data platform in dbt Cloud or dbt Core, using a dedicated adapter plugin"
hide_table_of_contents: true
+pagination_next: null
+pagination_prev: null
---
The References section contains reference materials for developing with dbt, which includes dbt Cloud and dbt Core.
diff --git a/website/docs/reference/resource-configs/bigquery-configs.md b/website/docs/reference/resource-configs/bigquery-configs.md
index 89a750f47bd..ffbaa37c059 100644
--- a/website/docs/reference/resource-configs/bigquery-configs.md
+++ b/website/docs/reference/resource-configs/bigquery-configs.md
@@ -414,7 +414,7 @@ models:
columns:
- name: field
policy_tags:
- - 'projects//locations//taxonomies//policyTags/'
+ - 'projects//locations//taxonomies//policyTags/'
```
diff --git a/website/docs/reference/resource-configs/delimiter.md b/website/docs/reference/resource-configs/delimiter.md
new file mode 100644
index 00000000000..58d6ba8344a
--- /dev/null
+++ b/website/docs/reference/resource-configs/delimiter.md
@@ -0,0 +1,126 @@
+---
+resource_types: [seeds]
+datatype:
+default_value: ","
+---
+
+## Definition
+
+You can use this optional seed configuration to customize how you separate values in a [seed](/docs/build/seeds) by providing the one-character string.
+
+* The delimiter defaults to a comma when not specified.
+* Explicitly set the `delimiter` configuration value if you want seed files to use a different delimiter, such as "|" or ";".
+
+:::info New in 1.7!
+
+Delimiter is new functionality available beginning with dbt Core v1.7.
+
+:::
+
+
+## Usage
+
+Specify a delimiter in your `dbt_project.yml` file to customize the global separator for all seed values:
+
+
+
+```yml
+seeds:
+ :
+ +delimiter: "|" # default project delimiter for seeds will be "|"
+ :
+ +delimiter: "," # delimiter for seeds in seed_subdirectory will be ","
+```
+
+
+
+
+Or use a custom delimiter to override the values for a specific seed:
+
+
+
+```yml
+version: 2
+
+seeds:
+ - name:
+ config:
+ delimiter: "|"
+```
+
+
+
+## Examples
+For a project with:
+
+* `name: jaffle_shop` in the `dbt_project.yml` file
+* `seed-paths: ["seeds"]` in the `dbt_project.yml` file
+
+### Use a custom delimiter to override global values
+
+You can set a default behavior for all seeds with an exception for one seed, `seed_a`, which uses a comma:
+
+
+
+```yml
+seeds:
+ jaffle_shop:
+ +delimiter: "|" # default delimiter for seeds in jaffle_shop project will be "|"
+ seed_a:
+ +delimiter: "," # delimiter for seed_a will be ","
+```
+
+
+
+Your corresponding seed files would be formatted like this:
+
+
+
+```text
+col_a|col_b|col_c
+1|2|3
+4|5|6
+...
+```
+
+
+
+
+
+```text
+name,id
+luna,1
+doug,2
+...
+```
+
+
+
+Or you can configure custom behavior for one seed. The `country_codes` uses the ";" delimiter:
+
+
+
+```yml
+version: 2
+
+seeds:
+ - name: country_codes
+ config:
+ delimiter: ";"
+```
+
+
+
+The `country_codes` seed file would be formatted like this:
+
+
+
+```text
+country_code;country_name
+US;United States
+CA;Canada
+GB;United Kingdom
+...
+```
+
+
diff --git a/website/docs/reference/seed-configs.md b/website/docs/reference/seed-configs.md
index d74f414cbfe..429aa9444ae 100644
--- a/website/docs/reference/seed-configs.md
+++ b/website/docs/reference/seed-configs.md
@@ -23,6 +23,7 @@ seeds:
[](/reference/resource-configs/resource-path):
[+](/reference/resource-configs/plus-prefix)[quote_columns](/reference/resource-configs/quote_columns): true | false
[+](/reference/resource-configs/plus-prefix)[column_types](/reference/resource-configs/column_types): {column_name: datatype}
+ [+](/reference/resource-configs/plus-prefix)[delimiter](/reference/resource-configs/delimiter):
```
@@ -43,6 +44,7 @@ seeds:
config:
[quote_columns](/reference/resource-configs/quote_columns): true | false
[column_types](/reference/resource-configs/column_types): {column_name: datatype}
+ [delimiter](/reference/resource-configs/grants):
```
diff --git a/website/docs/terms/model.md b/website/docs/terms/model.md
new file mode 100644
index 00000000000..c589cc196a7
--- /dev/null
+++ b/website/docs/terms/model.md
@@ -0,0 +1,9 @@
+---
+id: model
+title: Model
+description: A model is an essential building block of the DAG
+displayText: model
+hoverSnippet: A model is an essential building block of the DAG
+---
+
+A model is an essential building block of the DAG that lives in a single file and contains logic that transforms data. This logic can be expressed as a SQL `select` statement or a Python dataframe operation. Models can be materialized in the warehouse in different ways — most of these materializations require models to be built in the warehouse.
\ No newline at end of file
diff --git a/website/sidebars.js b/website/sidebars.js
index 8fed122f444..fc87c1142c7 100644
--- a/website/sidebars.js
+++ b/website/sidebars.js
@@ -7,6 +7,7 @@ const sidebarSettings = {
collapsed: true,
link: { type: "doc", id: "docs/supported-data-platforms" },
items: [
+ "docs/supported-data-platforms",
"docs/connect-adapters",
"docs/verified-adapters",
"docs/trusted-adapters",
@@ -17,12 +18,12 @@ const sidebarSettings = {
{
type: "category",
label: "About dbt Cloud",
+ link: { type: "doc", id: "docs/cloud/about-cloud/dbt-cloud-features" },
items: [
"docs/cloud/about-cloud/dbt-cloud-features",
"docs/cloud/about-cloud/architecture",
"docs/cloud/about-cloud/tenancy",
"docs/cloud/about-cloud/regions-ip-addresses",
- "docs/cloud/about-cloud/about-cloud-ide",
"docs/cloud/about-cloud/browsers",
],
}, // About dbt Cloud directory
@@ -35,6 +36,7 @@ const sidebarSettings = {
type: "category",
label: "Set up dbt",
collapsed: true,
+ link: { type: "doc", id: "docs/about-setup" },
items: [
"docs/about-setup",
"docs/environments-in-dbt",
@@ -42,12 +44,14 @@ const sidebarSettings = {
type: "category",
label: "dbt Cloud",
collapsed: true,
+ link: { type: "doc", id: "docs/cloud/about-cloud-setup" },
items: [
"docs/cloud/about-cloud-setup",
"docs/dbt-cloud-environments",
{
type: "category",
label: "Connect data platform",
+ link: { type: "doc", id: "docs/cloud/connect-data-platform/about-connections" },
items: [
"docs/cloud/connect-data-platform/about-connections",
"docs/cloud/connect-data-platform/connect-starburst-trino",
@@ -61,13 +65,15 @@ const sidebarSettings = {
{
type: "category",
label: "Manage access",
+ link: { type: "doc", id: "docs/cloud/manage-access/about-user-access" },
items: [
"docs/cloud/manage-access/about-user-access",
- "docs/cloud/manage-access/seats-and-users",
{
type: "category",
- label: "Permissions",
+ label: "User permissions and licenses",
+ link: { type: "doc", id: "docs/cloud/manage-access/seats-and-users" },
items: [
+ "docs/cloud/manage-access/seats-and-users",
"docs/cloud/manage-access/self-service-permissions",
"docs/cloud/manage-access/enterprise-permissions",
],
@@ -75,7 +81,8 @@ const sidebarSettings = {
{
type: "category",
- label: "Single sign-on",
+ label: "Single sign-on and Oauth",
+ link: { type: "doc", id: "docs/cloud/manage-access/sso-overview" },
items: [
"docs/cloud/manage-access/sso-overview",
"docs/cloud/manage-access/auth0-migration",
@@ -83,16 +90,11 @@ const sidebarSettings = {
"docs/cloud/manage-access/set-up-sso-okta",
"docs/cloud/manage-access/set-up-sso-google-workspace",
"docs/cloud/manage-access/set-up-sso-azure-active-directory",
- ],
- }, // SSO
- {
- type: "category",
- label: "OAuth with data platforms",
- items: [
"docs/cloud/manage-access/set-up-snowflake-oauth",
+ "docs/cloud/manage-access/set-up-databricks-oauth",
"docs/cloud/manage-access/set-up-bigquery-oauth",
],
- }, // oauth
+ }, // SSO
"docs/cloud/manage-access/audit-log",
],
}, // Manage access
@@ -100,38 +102,60 @@ const sidebarSettings = {
{
type: "category",
label: "Configure Git",
+ link: { type: "doc", id: "docs/cloud/git/git-configuration-in-dbt-cloud" },
items: [
+ "docs/cloud/git/git-configuration-in-dbt-cloud",
+ "docs/cloud/git/import-a-project-by-git-url",
"docs/cloud/git/connect-github",
"docs/cloud/git/connect-gitlab",
{
type: "category",
label: "Azure DevOps",
+ link: { type: "doc", id: "docs/cloud/git/connect-azure-devops" },
items: [
"docs/cloud/git/connect-azure-devops",
"docs/cloud/git/setup-azure",
"docs/cloud/git/authenticate-azure",
],
},
- "docs/cloud/git/import-a-project-by-git-url",
],
}, // Supported Git providers
{
type: "category",
- label: "Develop in the IDE",
- link: {
- type: "doc",
- id: "docs/cloud/dbt-cloud-ide/develop-in-the-cloud",
- },
+ label: "Develop in dbt Cloud",
+ link: { type: "doc", id: "docs/cloud/about-cloud-develop" },
items: [
- "docs/cloud/dbt-cloud-ide/ide-user-interface",
- "docs/cloud/dbt-cloud-ide/lint-format",
- "docs/cloud/dbt-cloud-ide/dbt-cloud-tips",
+ "docs/cloud/about-cloud-develop",
+ "docs/cloud/about-cloud-develop-defer",
+ {
+ type: "category",
+ label: "dbt Cloud CLI",
+ link: { type: "doc", id: "docs/cloud/cloud-cli-installation" },
+ items: [
+ "docs/cloud/cloud-cli-installation",
+ "docs/cloud/configure-cloud-cli",
+ ],
+ },
+ {
+ type: "category",
+ label: "dbt Cloud IDE",
+ link: { type: "doc", id: "docs/cloud/dbt-cloud-ide/develop-in-the-cloud" },
+ items: [
+ "docs/cloud/dbt-cloud-ide/develop-in-the-cloud",
+ "docs/cloud/dbt-cloud-ide/ide-user-interface",
+ "docs/cloud/dbt-cloud-ide/lint-format",
+ "docs/cloud/dbt-cloud-ide/dbt-cloud-tips",
+ ],
+ },
],
- }, // dbt Cloud IDE directory
+ }, // dbt Cloud develop directory
{
type: "category",
label: "Secure your tenant",
+ link: { type: "doc", id: "docs/cloud/secure/secure-your-tenant" },
items: [
+ "docs/cloud/secure/secure-your-tenant",
+ "docs/cloud/secure/ip-restrictions",
"docs/cloud/secure/about-privatelink",
"docs/cloud/secure/snowflake-privatelink",
"docs/cloud/secure/databricks-privatelink",
@@ -149,13 +173,15 @@ const sidebarSettings = {
collapsed: true,
link: { type: "doc", id: "docs/core/about-core-setup" },
items: [
- "docs/core/about-the-cli",
+ "docs/core/about-core-setup",
+ "docs/core/about-dbt-core",
"docs/core/dbt-core-environments",
{
type: "category",
label: "Install dbt",
link: { type: "doc", id: "docs/core/installation" },
items: [
+ "docs/core/installation",
"docs/core/homebrew-install",
"docs/core/pip-install",
"docs/core/docker-install",
@@ -170,6 +196,7 @@ const sidebarSettings = {
id: "docs/core/connect-data-platform/about-core-connections",
},
items: [
+ "docs/core/connect-data-platform/about-core-connections",
"docs/core/connect-data-platform/profiles.yml",
"docs/core/connect-data-platform/connection-profiles",
"docs/core/connect-data-platform/bigquery-setup",
@@ -224,16 +251,19 @@ const sidebarSettings = {
type: "category",
label: "Build dbt projects",
collapsed: true,
+ link: { type: "doc", id: "docs/build/projects" },
items: [
"docs/build/projects",
{
type: "category",
label: "Build your DAG",
collapsed: true,
+ link: { type: "doc", id: "docs/build/models" },
items: [
{
type: "category",
label: "Models",
+ link: { type: "doc", id: "docs/build/models" },
items: [
"docs/build/models",
"docs/build/sql-models",
@@ -257,38 +287,43 @@ const sidebarSettings = {
link: { type: "doc", id: "docs/build/build-metrics-intro" },
collapsed: true,
items: [
+ "docs/build/build-metrics-intro",
"docs/build/sl-getting-started",
{
type: "category",
label: "About MetricFlow",
link: { type: "doc", id: "docs/build/about-metricflow" },
items: [
+ "docs/build/about-metricflow",
"docs/build/join-logic",
"docs/build/validation",
+ "docs/build/saved-queries",
"docs/build/metricflow-time-spine",
- "docs/build/metricflow-cli",
- ]
+ "docs/build/metricflow-commands",
+ ],
},
{
type: "category",
label: "Semantic models",
link: { type: "doc", id: "docs/build/semantic-models" },
items: [
+ "docs/build/semantic-models",
"docs/build/dimensions",
"docs/build/entities",
- "docs/build/measures"
- ]
+ "docs/build/measures",
+ ],
},
{
type: "category",
label: "Metrics",
link: { type: "doc", id: "docs/build/metrics-overview" },
items: [
+ "docs/build/metrics-overview",
"docs/build/cumulative",
"docs/build/derived",
"docs/build/ratio",
"docs/build/simple",
- ]
+ ],
},
],
},
@@ -296,7 +331,9 @@ const sidebarSettings = {
type: "category",
label: "Enhance your models",
collapsed: true,
+ link: { type: "doc", id: "docs/build/enhance-your-models" },
items: [
+ "docs/build/enhance-your-models",
"docs/build/materializations",
"docs/build/incremental-models",
],
@@ -305,7 +342,9 @@ const sidebarSettings = {
type: "category",
label: "Enhance your code",
collapsed: true,
+ link: { type: "doc", id: "docs/build/enhance-your-code" },
items: [
+ "docs/build/enhance-your-code",
"docs/build/project-variables",
"docs/build/environment-variables",
"docs/build/packages",
@@ -316,7 +355,9 @@ const sidebarSettings = {
type: "category",
label: "Organize your outputs",
collapsed: true,
+ link: { type: "doc", id: "docs/build/organize-your-outputs" },
items: [
+ "docs/build/organize-your-outputs",
"docs/build/custom-schemas",
"docs/build/custom-databases",
"docs/build/custom-aliases",
@@ -333,6 +374,7 @@ const sidebarSettings = {
collapsed: true,
link: { type: "doc", id: "docs/deploy/deployments" },
items: [
+ "docs/deploy/deployments",
"docs/deploy/job-scheduler",
"docs/deploy/deploy-environments",
"docs/deploy/continuous-integration",
@@ -341,6 +383,7 @@ const sidebarSettings = {
label: "Jobs",
link: { type: "doc", id: "docs/deploy/jobs" },
items: [
+ "docs/deploy/jobs",
"docs/deploy/deploy-jobs",
"docs/deploy/ci-jobs",
"docs/deploy/job-commands",
@@ -351,6 +394,7 @@ const sidebarSettings = {
label: "Monitor jobs and alerts",
link: { type: "doc", id: "docs/deploy/monitor-jobs" },
items: [
+ "docs/deploy/monitor-jobs",
"docs/deploy/run-visibility",
"docs/deploy/retry-jobs",
"docs/deploy/job-notifications",
@@ -366,11 +410,14 @@ const sidebarSettings = {
{
type: "category",
label: "Collaborate with others",
+ link: { type: "doc", id: "docs/collaborate/collaborate-with-others" },
items: [
+ "docs/collaborate/collaborate-with-others",
"docs/collaborate/explore-projects",
{
type: "category",
label: "Git version control",
+ link: { type: "doc", id: "docs/collaborate/git-version-control" },
items: [
"docs/collaborate/git-version-control",
"docs/collaborate/git/version-control-basics",
@@ -382,6 +429,7 @@ const sidebarSettings = {
{
type: "category",
label: "Document your dbt projects",
+ link: { type: "doc", id: "docs/collaborate/documentation" },
items: [
"docs/collaborate/documentation",
"docs/collaborate/build-and-view-your-docs",
@@ -396,6 +444,7 @@ const sidebarSettings = {
id: "docs/collaborate/govern/about-model-governance",
},
items: [
+ "docs/collaborate/govern/about-model-governance",
"docs/collaborate/govern/model-access",
"docs/collaborate/govern/model-contracts",
"docs/collaborate/govern/model-versions",
@@ -407,8 +456,10 @@ const sidebarSettings = {
{
type: "category",
label: "Use the dbt Semantic Layer",
+ collapsed: true,
link: { type: "doc", id: "docs/use-dbt-semantic-layer/dbt-sl" },
items: [
+ "docs/use-dbt-semantic-layer/dbt-sl",
"docs/use-dbt-semantic-layer/quickstart-sl",
"docs/use-dbt-semantic-layer/setup-sl",
"docs/use-dbt-semantic-layer/sl-architecture",
@@ -418,7 +469,8 @@ const sidebarSettings = {
link: { type: "doc", id: "docs/use-dbt-semantic-layer/avail-sl-integrations" },
items: [
"docs/use-dbt-semantic-layer/avail-sl-integrations",
- //"docs/use-dbt-semantic-layer/gsheets",
+ "docs/use-dbt-semantic-layer/gsheets",
+ "docs/use-dbt-semantic-layer/tableau",
],
},
],
@@ -427,12 +479,15 @@ const sidebarSettings = {
type: "category",
label: "dbt Cloud APIs",
collapsed: true,
+ link: { type: "doc", id: "docs/dbt-cloud-apis/overview" },
items: [
"docs/dbt-cloud-apis/overview",
{
type: "category",
label: "Authentication",
+ link: { type: "doc", id: "docs/dbt-cloud-apis/authentication" },
items: [
+ "docs/dbt-cloud-apis/authentication",
"docs/dbt-cloud-apis/user-tokens",
"docs/dbt-cloud-apis/service-tokens",
],
@@ -442,6 +497,7 @@ const sidebarSettings = {
label: "Administrative API",
link: { type: "doc", id: "docs/dbt-cloud-apis/admin-cloud-api" },
items: [
+ "docs/dbt-cloud-apis/admin-cloud-api",
{
type: "link",
label: "API v2 (legacy docs)",
@@ -464,18 +520,25 @@ const sidebarSettings = {
label: "Discovery API",
link: { type: "doc", id: "docs/dbt-cloud-apis/discovery-api" },
items: [
+ "docs/dbt-cloud-apis/discovery-api",
"docs/dbt-cloud-apis/discovery-use-cases-and-examples",
"docs/dbt-cloud-apis/project-state",
"docs/dbt-cloud-apis/discovery-querying",
{
type: "category",
label: "Schema",
+ link: { type: "doc", id: "docs/dbt-cloud-apis/discovery-schema-environment" },
items: [
+ "docs/dbt-cloud-apis/discovery-schema-environment",
{
type: "category",
label: "Job",
- link: { type: "doc", id: "docs/dbt-cloud-apis/discovery-schema-job" },
+ link: {
+ type: "doc",
+ id: "docs/dbt-cloud-apis/discovery-schema-job",
+ },
items: [
+ "docs/dbt-cloud-apis/discovery-schema-job",
"docs/dbt-cloud-apis/discovery-schema-job-model",
"docs/dbt-cloud-apis/discovery-schema-job-models",
"docs/dbt-cloud-apis/discovery-schema-job-metric",
@@ -495,11 +558,6 @@ const sidebarSettings = {
],
},
{
- type: "category",
- label: "Environment",
- link: { type: "doc", id: "docs/dbt-cloud-apis/discovery-schema-environment" },
- items: [
- {
type: "category",
label: "Applied",
items: [
@@ -513,9 +571,7 @@ const sidebarSettings = {
// items: [
// // insert pages here
// ],
- // },
- ],
- },
+ // },
],
},
],
@@ -525,6 +581,7 @@ const sidebarSettings = {
label: "Semantic Layer APIs",
link: { type: "doc", id: "docs/dbt-cloud-apis/sl-api-overview" },
items: [
+ "docs/dbt-cloud-apis/sl-api-overview",
"docs/dbt-cloud-apis/sl-jdbc",
"docs/dbt-cloud-apis/sl-graphql",
"docs/dbt-cloud-apis/sl-manifest",
@@ -535,6 +592,7 @@ const sidebarSettings = {
{
type: "category",
label: "Available dbt versions",
+ link: { type: "doc", id: "docs/dbt-versions/core" },
items: [
"docs/dbt-versions/core",
"docs/dbt-versions/upgrade-core-in-cloud",
@@ -543,6 +601,7 @@ const sidebarSettings = {
{
type: "category",
label: "dbt Cloud Release Notes",
+ link: { type: "doc", id: "docs/dbt-versions/dbt-cloud-release-notes" },
items: [
"docs/dbt-versions/dbt-cloud-release-notes",
{
@@ -693,6 +752,7 @@ const sidebarSettings = {
"reference/seed-properties",
"reference/seed-configs",
"reference/resource-configs/column_types",
+ "reference/resource-configs/delimiter",
"reference/resource-configs/quote_columns",
],
},
@@ -938,7 +998,19 @@ const sidebarSettings = {
},
{
type: "category",
- label: "Materializations best practices",
+ label: "How we build our dbt Mesh projects",
+ link: {
+ type: "doc",
+ id: "guides/best-practices/how-we-mesh/mesh-1-intro",
+ },
+ items: [
+ "guides/best-practices/how-we-mesh/mesh-2-structures",
+ "guides/best-practices/how-we-mesh/mesh-3-implementation",
+ ],
+ },
+ {
+ type: "category",
+ label: "Materialization best practices",
link: {
type: "doc",
id: "guides/best-practices/materializations/materializations-guide-1-guide-overview",
diff --git a/website/snippets/_adapters-trusted.md b/website/snippets/_adapters-trusted.md
index 96f62a925eb..7747ce16dec 100644
--- a/website/snippets/_adapters-trusted.md
+++ b/website/snippets/_adapters-trusted.md
@@ -2,19 +2,18 @@
Install with dbt Core
"
icon="materialize"/>
Install with dbt Core
"
icon="oracle"/>
diff --git a/website/snippets/_adapters-verified.md b/website/snippets/_adapters-verified.md
index 7caf099b7d1..3cc1e800448 100644
--- a/website/snippets/_adapters-verified.md
+++ b/website/snippets/_adapters-verified.md
@@ -2,61 +2,60 @@
-* Install these adapters using the CLI as they're not currently supported in dbt Cloud.
@@ -88,16 +81,7 @@ import SlSetUp from '/snippets/_new-sl-setup.md';
## Connect and query API
-You can query your metrics in a JDBC-enabled tool or use existing first-class integrations with the dbt Semantic Layer.
-
-You must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment, hosted in North America (Additional region support coming soon).
-
-- To learn how to use the JDBC or GraphQL API and what tools you can query it with, refer to the {frontMatter.meta.api_name}.
- - * To authenticate, you need to [generate a service token](/docs/dbt-cloud-apis/service-tokens) with Semantic Layer Only and Metadata Only permissions. - * Refer to the [SQL query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the APIs. - -- To learn more about the sophisticated integrations that connect to the dbt Semantic Layer, refer to [Available integrations](/docs/use-dbt-semantic-layer/avail-sl-integrations) for more info. +
-import NewSLChanges from '/snippets/_new-sl-changes.md';
-
-
@@ -27,7 +24,7 @@ The dbt Semantic Layer includes the following components:
| **[MetricFlow](/docs/build/about-metricflow)** | MetricFlow in dbt allows users to centrally define their semantic models and metrics with YAML specifications. | ✅ | ✅ | ✅ | BSL package (code is source available) |
| **MetricFlow Server**| A proprietary server that takes metric requests and generates optimized SQL for the specific data platform. | ❌ | ✅ | ✅ | Proprietary, Cloud (Team & Enterprise)|
| **Semantic Layer Gateway** | A service that passes queries to MetricFlow server and executes the SQL generated by MetricFlow against the data platform|
❌| ✅ | ✅ | Proprietary, Cloud (Team & Enterprise) | -| **Semantic Layer API** | The interfaces that allow users to submit metric queries include the MetricFlow CLI and JDBC API. They also serve as the foundation for building first-class integrations with various tools. | ❌ | ✅ | ✅ | Proprietary, Cloud (Team & Enterprise)| +| **Semantic Layer APIs** | The interfaces that allow users to submit metric queries using the GraphQL and JDBC APIs. They also serve as the foundation for building first-class integrations with various tools. | ❌ | ✅ | ✅ | Proprietary, Cloud (Team & Enterprise)| ## Related questions diff --git a/website/docs/docs/use-dbt-semantic-layer/tableau.md b/website/docs/docs/use-dbt-semantic-layer/tableau.md new file mode 100644 index 00000000000..c505ea323f4 --- /dev/null +++ b/website/docs/docs/use-dbt-semantic-layer/tableau.md @@ -0,0 +1,67 @@ +--- +title: "Tableau (beta)" +description: "Use Tableau worksheets to query the dbt Semantic Layer and produce dashboards with trusted date." +tags: [Semantic Layer] +sidebar_label: "Tableau (beta)" +--- + +:::info Beta functionality +The Tableau integration with the dbt Semantic Layer is a [beta feature](/docs/dbt-versions/product-lifecycles#dbt-cloud). +::: + + +The Tableau integration allows you to use worksheets to query the Semantic Layer directly and produce your dashboards with trusted data. This integration provides a live connection to the dbt Semantic Layer through Tableau Desktop. + +## Prerequisites + +1. You must have [Tableau Desktop](https://www.tableau.com/en-gb/products/desktop) installed +2. Authenticate with either Tableau Server or Tableau Cloud +3. You need your dbt Cloud host, [Environment ID](/docs/use-dbt-semantic-layer/setup-sl#set-up-dbt-semantic-layer) and [service token](/docs/dbt-cloud-apis/service-tokens) to log in. This account should be set up with the dbt Semantic Layer. +4. You must a dbt Cloud Team or Enterprise [account](https://www.getdbt.com/pricing) and multi-tenant [deployment](/docs/cloud/about-cloud/regions-ip-addresses). (Single-Tenant coming soon) + + +## Installing + +1. Download our [connector file](https://github.com/dbt-labs/semantic-layer-tableau-connector/releases/download/v1.0.0/dbt_semantic_layer.taco) locally and add it to your default folder: + - Windows: `C:\Users\\[Windows User]\Documents\My Tableau Repository\Connectors` + - Mac: `/Users/[user]/Documents/My Tableau Repository/Connectors` + - Linux: `/opt/tableau/connectors` +2. Install the [JDBC driver](/docs/dbt-cloud-apis/sl-jdbc) to the folder based on your operating system: + - Windows: `C:\Program Files\Tableau\Drivers` + - Mac: `~/Library/Tableau/Drivers` + - Linux: ` /opt/tableau/tableau_driver/jdbc` +3. Open Tableau Desktop and find the **dbt Semantic Layer by dbt Labs** connector on the left-hand side. +4. Connect with your Host, Environment ID, and service token information that's provided to you in your dbt Cloud Semantic Layer configuration. + + +## Using the integration + +Once you authenticate, the system will direct you to the data source page with all the metrics and dimensions configured in your Semantic Layer. + +- From there, go directly to a worksheet in the bottom left hand corner. +- Then, you'll find all the metrics and dimensions that are available to query on the left-hand side of your window. + +Visit the [Tableau documentation](https://help.tableau.com/current/pro/desktop/en-us/gettingstarted_overview.htm) to learn more about how to use Tableau worksheets and dashboards. + +## Things to note + +- All metrics use the "SUM" aggregation type, and this can't be altered. The dbt Semantic Layer controls the aggregation type and it is intentionally fixed. Keep in mind that the underlying aggregation in the dbt Semantic Layer might not be "SUM" (even though "SUM" is Tableau's default). +- Tableau surfaces all metrics and dimensions from the dbt Semantic Layer on the left-hand side. Note, that not all metrics and dimensions can be combined with one another. You will receive an error message if a particular dimension cannot be sliced with a metric (or vice versa). + - To display available metrics and dimensions, dbt Semantic Layer returns metadata for a fake table with the dimensions and metrics as 'columns' on this table. Because of this, you can't actually query this table for previews or extracts. + - Since this is treated as a table, dbt Semantic Layer can't dynamically change what is available. This means we display _all_ available metrics and dimensions even if a particular metric and dimension combination isn't available. + +- Certain Table calculations like "Totals" and "Percent Of" may not be accurate when using metrics aggregated in a non-additive way (such as count distinct) + +## Unsupported functionality + +The following Tableau features aren't supported at this time, however, the dbt Semantic Layer may support some of this functionality in a future release: + +- Updating the data source page +- Using "Extract" mode to view yur data +- Unioning Tables +- Writing Custom SQL +- Table Extensions +- Cross Database Joins +- All functions in Analysis --> Create Calculated Field +- Filtering on a Date Part time dimension for a Cumulative metric type +- Changing your date dimension to use "Week Number" diff --git a/website/docs/docs/verified-adapters.md b/website/docs/docs/verified-adapters.md index a2d28a612d6..170bc8f885b 100644 --- a/website/docs/docs/verified-adapters.md +++ b/website/docs/docs/verified-adapters.md @@ -13,6 +13,10 @@ The verification process serves as the on-ramp to integration with dbt Cloud. As To learn more, see [Verifying a new adapter](/guides/dbt-ecosystem/adapter-development/7-verifying-a-new-adapter). +import MSCallout from '/snippets/_microsoft-adapters-soon.md'; + +
+
+dbt generates the `package-lock.yml` file in the _project_root_ where `packages.yml` is recorded, which contains all the resolved packages, the first time you run `dbt deps`. Each subsequent run records the packages installed in this file. If the subsequent `dbt deps` runs contain no updated packages in `depenedencies.yml` or `packages.yml`, dbt-core installs from `package-lock.yml`.
+
+When you update the package spec and run `dbt deps` again, the package-lock and package files update accordingly. You can run `dbt deps --lock` to update the `package-lock.yml` with the most recent dependencies from `packages`.
+
+The `--add` flag allows you to add a package to the `packages.yml` with configurable `--version` and `--source` information. The `--dry-run` flag, when set to `False`(default), recompiles the `package-lock.yml` file after a new package is added to the `packages.yml` file. Set the flag to `True` for the changes to not persist.
+
+Examples of the `--add` flag:
+```shell
+# add package from hub (--source arg defaults to "hub")
+dbt deps add --package dbt-labs/dbt_utils --version 1.0.0
+
+# add package from hub with semantic version
+dbt deps add --package dbt-labs/snowplow --version ">=0.7.0,<0.8.0"
+
+# add package from git
+dbt deps add --package https://github.com/fivetran/dbt_amplitude --version v0.3.0 --source git
+
+# add package from local (--version not required for local)
+dbt deps add --package /opt/dbt/redshift --source local
+
+# add package to packages.yml WITHOUT updating package-lock.yml
+dbt deps add --package dbt-labs/dbt_utils --version 1.0.0 --dry-run True
+
+```
+
\ No newline at end of file
diff --git a/website/docs/reference/commands/list.md b/website/docs/reference/commands/list.md
index 93a0b87dd93..5caabdc2b2e 100644
--- a/website/docs/reference/commands/list.md
+++ b/website/docs/reference/commands/list.md
@@ -8,6 +8,7 @@ id: "list"
The `dbt ls` command lists resources in your dbt project. It accepts selector arguments that are similar to those provided in [dbt run](/reference/commands/run). `dbt list` is an alias for `dbt ls`. While `dbt ls` will read your [connection profile](/docs/core/connect-data-platform/connection-profiles) to resolve [`target`](/reference/dbt-jinja-functions/target)-specific logic, this command will not connect to your database or run any queries.
### Usage
+
```
dbt ls
[--resource-type {model,semantic_model,source,seed,snapshot,metric,test,exposure,analysis,default,all}]
@@ -85,7 +86,7 @@ $ dbt ls --select snowplow.* --output json --output-keys "name resource_type des
```
-$ dbt ls --select snowplow.* --output json --output-keys name resource_type description
+$ dbt ls --select snowplow.* --output json --output-keys "name resource_type description"
{"name": "snowplow_events", "description": "This is a pretty cool model", ...}
{"name": "snowplow_page_views", "description": "This model is even cooler", ...}
...
diff --git a/website/docs/reference/commands/seed.md b/website/docs/reference/commands/seed.md
index 8a410706842..d0cd199ea12 100644
--- a/website/docs/reference/commands/seed.md
+++ b/website/docs/reference/commands/seed.md
@@ -12,7 +12,7 @@ The `dbt seed` command will load `csv` files located in the `seed-paths` directo
Specific seeds can be run using the `--select` flag to `dbt seed`. Example:
```
-$ dbt seed --select country_codes
+$ dbt seed --select "country_codes"
Found 2 models, 3 tests, 0 archives, 0 analyses, 53 macros, 0 operations, 2 seed files
14:46:15 | Concurrency: 1 threads (target='dev')
diff --git a/website/docs/reference/commands/show.md b/website/docs/reference/commands/show.md
index 5bdcfacc1e8..a0e5d68c83f 100644
--- a/website/docs/reference/commands/show.md
+++ b/website/docs/reference/commands/show.md
@@ -16,7 +16,7 @@ The results of the preview query are not materialized in the data warehouse, or
Example:
```
-dbt show --select model_name.sql
+dbt show --select "model_name.sql"
```
or
```
@@ -26,7 +26,7 @@ dbt show --inline "select * from {{ ref('model_name') }}"
The following is an example of `dbt show` output for a model named `stg_orders`:
```bash
-dbt show --select stg_orders
+dbt show --select "stg_orders"
21:17:38 Running with dbt=1.5.0-b5
21:17:38 Found 5 models, 20 tests, 0 snapshots, 0 analyses, 425 macros, 0 operations, 3 seed files, 0 sources, 0 exposures, 0 metrics, 0 groups
21:17:38
@@ -46,7 +46,7 @@ dbt show --select stg_orders
For example, if you've just built a model that has a failing test, you can quickly preview the test failures right in the terminal, to find values of `id` that are duplicated:
```bash
-$ dbt build -s my_model_with_duplicates
+$ dbt build -s "my_model_with_duplicates"
13:22:47 Running with dbt=1.5.0
...
13:22:48 Completed with 1 error and 0 warnings:
@@ -58,7 +58,7 @@ $ dbt build -s my_model_with_duplicates
13:22:48
13:22:48 Done. PASS=1 WARN=0 ERROR=1 SKIP=0 TOTAL=2
-$ dbt show -s unique_my_model_with_duplicates_id
+$ dbt show -s "unique_my_model_with_duplicates_id"
13:22:53 Running with dbt=1.5.0
13:22:53 Found 4 models, 2 tests, 0 snapshots, 0 analyses, 309 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics, 0 groups
13:22:53
diff --git a/website/docs/reference/commands/source.md b/website/docs/reference/commands/source.md
index b29bf7dadc6..697ae2b5fcc 100644
--- a/website/docs/reference/commands/source.md
+++ b/website/docs/reference/commands/source.md
@@ -20,10 +20,10 @@ By default, `dbt source freshness` will calculate freshness information for all
```bash
# Snapshot freshness for all Snowplow tables:
-$ dbt source freshness --select source:snowplow
+$ dbt source freshness --select "source:snowplow"
# Snapshot freshness for a particular source table:
-$ dbt source freshness --select source:snowplow.event
+$ dbt source freshness --select "source:snowplow.event"
```
### Configuring source freshness output
diff --git a/website/docs/reference/commands/test.md b/website/docs/reference/commands/test.md
index a1a63729568..c050d82a0ab 100644
--- a/website/docs/reference/commands/test.md
+++ b/website/docs/reference/commands/test.md
@@ -10,22 +10,22 @@ The tests to run can be selected using the `--select` flag discussed [here](/ref
```bash
# run tests for one_specific_model
-dbt test --select one_specific_model
+dbt test --select "one_specific_model"
# run tests for all models in package
-dbt test --select some_package.*
+dbt test --select "some_package.*"
# run only tests defined singularly
-dbt test --select test_type:singular
+dbt test --select "test_type:singular"
# run only tests defined generically
-dbt test --select test_type:generic
+dbt test --select "test_type:generic"
# run singular tests limited to one_specific_model
-dbt test --select one_specific_model,test_type:singular
+dbt test --select "one_specific_model,test_type:singular"
# run generic tests limited to one_specific_model
-dbt test --select one_specific_model,test_type:generic
+dbt test --select "one_specific_model,test_type:generic"
```
For more information on writing tests, see the [Testing Documentation](/docs/build/tests).
diff --git a/website/docs/reference/dbt-commands.md b/website/docs/reference/dbt-commands.md
index 862829ef809..1448d9849d3 100644
--- a/website/docs/reference/dbt-commands.md
+++ b/website/docs/reference/dbt-commands.md
@@ -2,54 +2,58 @@
title: "dbt Command reference"
---
-dbt is typically run one of two ways:
+You can run dbt using the following tools:
-* In [dbt Cloud](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud)
-* On the [command line interface](/docs/core/about-the-cli) (CLI)
+- In your browser with the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud)
+- On the command line interface using the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation) or open-source [dbt Core](/docs/core/about-dbt-core), both of which enable you to execute dbt commands. The key distinction is the dbt Cloud CLI is tailored for dbt Cloud's infrastructure and integrates with all its [features](/docs/cloud/about-cloud/dbt-cloud-features).
The following sections outline the commands supported by dbt and their relevant flags. For information about selecting models on the command line, consult the docs on [Model selection syntax](/reference/node-selection/syntax).
### Available commands
-
-
-Use the following dbt commands in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [CLI](/docs/core/about-the-cli). Use the `dbt` prefix. For example, to run the `test` command, type `dbt test`.
-
-| Command | Description | Version |
-| ------- | ----------- | ------- |
-| [build](/reference/commands/build) | Build and test all selected resources (models, seeds, snapshots, tests) | All [supported versions](/docs/dbt-versions/core) |
-| [clean](/reference/commands/clean) | Deletes artifacts present in the dbt project | All [supported versions](/docs/dbt-versions/core) |
-| [clone](/reference/commands/clone) | Clone selected models from the specified state | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) |
-| [compile](/reference/commands/compile) | Compiles (but does not run) the models in a project | All [supported versions](/docs/dbt-versions/core) |
-| [debug](/reference/commands/debug) | Debugs dbt connections and projects | All [supported versions](/docs/dbt-versions/core) |
-| [deps](/reference/commands/deps) | Downloads dependencies for a project | All [supported versions](/docs/dbt-versions/core) |
-| [docs](/reference/commands/cmd-docs) | Generates documentation for a project | All [supported versions](/docs/dbt-versions/core) |
-| [list](/reference/commands/list) | Lists resources defined in a dbt project | All [supported versions](/docs/dbt-versions/core) |
-| [parse](/reference/commands/parse) | Parses a project and writes detailed timing info | All [supported versions](/docs/dbt-versions/core) |
-| [retry](/reference/commands/retry) | Retry the last run `dbt` command from the point of failure | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) |
-| [run](/reference/commands/run) | Runs the models in a project | All [supported versions](/docs/dbt-versions/core) |
-| [run-operation](/reference/commands/run-operation) | Invoke a macro, including running arbitrary maintenance SQL against
the database | All [supported versions](/docs/dbt-versions/core) | -| [seed](/reference/commands/seed) | Loads CSV files into the database | All [supported versions](/docs/dbt-versions/core) | -| [show](/reference/commands/show) | Preview table rows post-transformation | All [supported versions](/docs/dbt-versions/core) | -| [snapshot](/reference/commands/snapshot) | Executes "snapshot" jobs defined in a project | All [supported versions](/docs/dbt-versions/core) | -| [source](/reference/commands/source) | Provides tools for working with source data (including validating that
sources are "fresh") | All [supported versions](/docs/dbt-versions/core) | -| [test](/reference/commands/test) | Executes tests defined in a project | All [supported versions](/docs/dbt-versions/core) | -| [init](/reference/commands/init) | Initializes a new dbt project (CLI only) | All [supported versions](/docs/dbt-versions/core) | - +
+
+All commands in the table are compatible with either the dbt Cloud IDE, dbt Cloud CLI, or dbt Core.
+
+You can run dbt commands in your specific tool by prefixing them with `dbt`. For example, to run the `test` command, type `dbt test`.
+
+| Command | Description | Compatible tools | Version |
+| ------- | ----------- | ---------------- | ------- |
+| [build](/reference/commands/build) | Build and test all selected resources (models, seeds, snapshots, tests) | All | All [supported versions](/docs/dbt-versions/core) |
+| cancel | Cancels the most recent invocation.| dbt Cloud CLI | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) |
+| [clean](/reference/commands/clean) | Deletes artifacts present in the dbt project | All | All [supported versions](/docs/dbt-versions/core) |
+| [clone](/reference/commands/clone) | Clone selected models from the specified state | dbt Cloud CLI
dbt Core | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) | +| [compile](/reference/commands/compile) | Compiles (but does not run) the models in a project | All | All [supported versions](/docs/dbt-versions/core) | +| [debug](/reference/commands/debug) | Debugs dbt connections and projects | dbt Core | All [supported versions](/docs/dbt-versions/core) | +| [deps](/reference/commands/deps) | Downloads dependencies for a project | All | All [supported versions](/docs/dbt-versions/core) | +| [docs](/reference/commands/cmd-docs) | Generates documentation for a project | All | All [supported versions](/docs/dbt-versions/core) | +| help | Displays help information for any command | dbt Core
dbt Cloud CLI | All [supported versions](/docs/dbt-versions/core) | +| [list](/reference/commands/list) | Lists resources defined in a dbt project | All | All [supported versions](/docs/dbt-versions/core) | +| [parse](/reference/commands/parse) | Parses a project and writes detailed timing info | All | All [supported versions](/docs/dbt-versions/core) | +| reattach | Reattaches to the most recent invocation to retrieve logs and artifacts. | dbt Cloud CLI | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) | +| [retry](/reference/commands/retry) | Retry the last run `dbt` command from the point of failure | All | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) | +| [run](/reference/commands/run) | Runs the models in a project | All | All [supported versions](/docs/dbt-versions/core) | +| [run-operation](/reference/commands/run-operation) | Invoke a macro, including running arbitrary maintenance SQL against the database | All | All [supported versions](/docs/dbt-versions/core) | +| [seed](/reference/commands/seed) | Loads CSV files into the database | All | All [supported versions](/docs/dbt-versions/core) | +| [show](/reference/commands/show) | Preview table rows post-transformation | All | All [supported versions](/docs/dbt-versions/core) | +| [snapshot](/reference/commands/snapshot) | Executes "snapshot" jobs defined in a project | All | All [supported versions](/docs/dbt-versions/core) | +| [source](/reference/commands/source) | Provides tools for working with source data (including validating that sources are "fresh") | All | All [supported versions](/docs/dbt-versions/core) | +| [test](/reference/commands/test) | Executes tests defined in a project | All | All [supported versions](/docs/dbt-versions/core) | +| [init](/reference/commands/init) | Initializes a new dbt project | dbt Core | All [supported versions](/docs/dbt-versions/core) |
-
+
Select the tabs that are relevant to your development workflow. For example, if you develop in the dbt Cloud IDE, select **dbt Cloud**.
-
+
Use the following dbt commands in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) and use the `dbt` prefix. For example, to run the `test` command, type `dbt test`.
- [build](/reference/commands/build): build and test all selected resources (models, seeds, snapshots, tests)
-- [clone](/reference/commands/clone): clone selected nodes from specified state (requires dbt 1.6 or higher)
+- [clone](/reference/commands/clone): clone selected nodes from the specified state (requires dbt 1.6 or higher)
- [compile](/reference/commands/compile): compiles (but does not run) the models in a project
- [deps](/reference/commands/deps): downloads dependencies for a project
- [docs](/reference/commands/cmd-docs) : generates documentation for a project
@@ -64,13 +68,13 @@ Use the following dbt commands in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/
-
+
-Use the following dbt commands in the [CLI](/docs/core/about-the-cli) and use the `dbt` prefix. For example, to run the `test` command, type `dbt test`.
+Use the following dbt commands in [dbt Core](/docs/core/about-dbt-core) and use the `dbt` prefix. For example, to run the `test` command, type `dbt test`.
- [build](/reference/commands/build): build and test all selected resources (models, seeds, snapshots, tests)
- [clean](/reference/commands/clean): deletes artifacts present in the dbt project
-- [clone](/reference/commands/clone): clone selected models from specified state (requires dbt 1.6 or higher)
+- [clone](/reference/commands/clone): clone selected models from the specified state (requires dbt 1.6 or higher)
- [compile](/reference/commands/compile): compiles (but does not run) the models in a project
- [debug](/reference/commands/debug): debugs dbt connections and projects
- [deps](/reference/commands/deps): downloads dependencies for a project
diff --git a/website/docs/reference/dbt-jinja-functions/model.md b/website/docs/reference/dbt-jinja-functions/model.md
index e967debd01f..9ccf0759470 100644
--- a/website/docs/reference/dbt-jinja-functions/model.md
+++ b/website/docs/reference/dbt-jinja-functions/model.md
@@ -52,15 +52,9 @@ To view the structure of `models` and their definitions:
Use the following table to understand how the versioning pattern works and match the Manifest version with the dbt version:
-| dbt version | Manifest version |
-| ----------- | ---------------- |
-| `v1.5` | [Manifest v9](https://schemas.getdbt.com/dbt/manifest/v9/index.html)
-| `v1.4` | [Manifest v8](https://schemas.getdbt.com/dbt/manifest/v8/index.html)
-| `v1.3` | [Manifest v7](https://schemas.getdbt.com/dbt/manifest/v7/index.html)
-| `v1.2` | [Manifest v6](https://schemas.getdbt.com/dbt/manifest/v6/index.html)
-| `v1.1` | [Manifest v5](https://schemas.getdbt.com/dbt/manifest/v5/index.html)
-
+import ManifestVersions from '/snippets/_manifest-versions.md';
+
The following is a list of all available configurations in the `dbt_project.yml` file.
@@ -19,6 +21,81 @@ The following is a list of all available configurations in the `dbt_project.yml`
dbt uses YAML in a few different places. If you're new to YAML, it would be worth taking the time to learn how arrays, dictionaries and strings are represented.
:::
+
+
+
+
+
+```yml
+[name](/reference/project-configs/name): string
+
+[config-version](/reference/project-configs/config-version): 2
+[version](/reference/project-configs/version): version
+
+[profile](/reference/project-configs/profile): profilename
+
+[model-paths](/reference/project-configs/model-paths): [directorypath]
+[seed-paths](/reference/project-configs/seed-paths): [directorypath]
+[test-paths](/reference/project-configs/test-paths): [directorypath]
+[analysis-paths](/reference/project-configs/analysis-paths): [directorypath]
+[macro-paths](/reference/project-configs/macro-paths): [directorypath]
+[snapshot-paths](/reference/project-configs/snapshot-paths): [directorypath]
+[docs-paths](/reference/project-configs/docs-paths): [directorypath]
+[asset-paths](/reference/project-configs/asset-paths): [directorypath]
+
+[target-path](/reference/project-configs/target-path): directorypath
+[log-path](/reference/project-configs/log-path): directorypath
+[packages-install-path](/reference/project-configs/packages-install-path): directorypath
+
+[clean-targets](/reference/project-configs/clean-targets): [directorypath]
+
+[query-comment](/reference/project-configs/query-comment): string
+
+[require-dbt-version](/reference/project-configs/require-dbt-version): version-range | [version-range]
+
+[dbt-cloud](/docs/cloud/cloud-cli-installation):
+ project-id: project_id #Required
+ defer-env-id: 5678 #Optional
+
+[quoting](/reference/project-configs/quoting):
+ database: true | false
+ schema: true | false
+ identifier: true | false
+
+models:
+ [](/reference/model-configs)
+
+seeds:
+ [](/reference/seed-configs)
+
+snapshots:
+ [](/reference/snapshot-configs)
+
+sources:
+ [](source-configs)
+
+tests:
+ [](/reference/test-configs)
+
+vars:
+ [](/docs/build/project-variables)
+
+[on-run-start](/reference/project-configs/on-run-start-on-run-end): sql-statement | [sql-statement]
+[on-run-end](/reference/project-configs/on-run-start-on-run-end): sql-statement | [sql-statement]
+
+[dispatch](/reference/project-configs/dispatch-config):
+ - macro_namespace: packagename
+ search_order: [packagename]
+
+[restrict-access](/docs/collaborate/govern/model-access): true | false
+
+```
+
+
+
+
+
+
```yml
@@ -79,6 +156,9 @@ vars:
search_order: [packagename]
[restrict-access](/docs/collaborate/govern/model-access): true | false
+
```
+
+
diff --git a/website/docs/reference/global-configs/command-line-flags.md b/website/docs/reference/global-configs/command-line-flags.md
index 6496c92da6d..fbe89ce28f1 100644
--- a/website/docs/reference/global-configs/command-line-flags.md
+++ b/website/docs/reference/global-configs/command-line-flags.md
@@ -4,60 +4,95 @@ id: "command-line-flags"
sidebar: "Command line flags"
---
-Command line (CLI) flags immediately follow `dbt` and precede your subcommand. When set, CLI flags override environment variables and profile configs.
+For consistency, command-line interface (CLI) flags should come right after the `dbt` prefix and its subcommands. This includes "global" flags (supported for all commands). When set, CLI flags override environment variables and profile configs.
-Use this non-boolean config structure, replacing `` with the config you are enabling or disabling, `` with the new setting for the config, and `` with the command this config applies to:
+For example, instead of using:
+
+```bash
+dbt --no-populate-cache run
+```
+
+You should use:
+
+```bash
+dbt run --no-populate-cache
+```
+
+Historically, passing flags (such as "global flags") _before_ the subcommand is a legacy functionality that dbt Labs can remove at any time. We do not support using the same flag before and after the subcommand.
+
+## Using boolean and non-boolean flags
+
+You can construct your commands with boolean flags to enable or disable or with non-boolean flags that use specific values, such as strings.
+
+
+
+
+
+Use this non-boolean config structure:
+- Replacing `` with the command this config applies to.
+- `` with the config you are enabling or disabling, and
+- `` with the new setting for the config.
```text
-$ --=
+ --=
```
-Non-boolean config examples:
+### Example
```text
-dbt --printer-width=80 run
-dbt --indirect-selection=eager test
+dbt run --printer-width=80
+dbt test --indirect-selection=eager
```
-To turn on boolean configs, you would use the `--` CLI flag, and a `--no-` CLI flag to turn off boolean configs, replacing `` with the config you are enabling or disabling and `` with the command this config applies to.
+
+
+
+
+To enable or disable boolean configs:
+- Use `` this config applies to.
+- Followed by `--` to turn it on, or `--no-` to turn it off.
+- Replace `` with the config you are enabling or disabling
-Boolean config structure:
```text
-dbt --
-dbt --no-
+dbt --
+dbt --no-
```
-Boolean config example:
+### Example
```text
-dbt --version-check run
-dbt --no-version-check run
+dbt run --version-check
+dbt run --no-version-check
```
-
\ No newline at end of file
+
+
+
+
+
diff --git a/website/docs/reference/node-selection/defer.md b/website/docs/reference/node-selection/defer.md
index e13a4f6648a..03c3b2aac12 100644
--- a/website/docs/reference/node-selection/defer.md
+++ b/website/docs/reference/node-selection/defer.md
@@ -17,16 +17,16 @@ It is possible to use separate state for `state:modified` and `--defer`, by pass
### Usage
```shell
-$ dbt run --select [...] --defer --state path/to/artifacts
-$ dbt test --select [...] --defer --state path/to/artifacts
+dbt run --select [...] --defer --state path/to/artifacts
+dbt test --select [...] --defer --state path/to/artifacts
```
```shell
-$ dbt run --models [...] --defer --state path/to/artifacts
-$ dbt test --models [...] --defer --state path/to/artifacts
+dbt run --models [...] --defer --state path/to/artifacts
+dbt test --models [...] --defer --state path/to/artifacts
```
@@ -101,7 +101,7 @@ I want to test my changes. Nothing exists in my development schema, `dev_alice`.
```shell
-$ dbt run --select model_b
+dbt run --select "model_b"
```
@@ -128,7 +128,7 @@ Unless I had previously run `model_a` into this development environment, `dev_al
```shell
-$ dbt run --select model_b --defer --state prod-run-artifacts
+dbt run --select "model_b" --defer --state prod-run-artifacts
```
@@ -186,7 +186,7 @@ models:
```shell
-dbt test --select model_b
+dbt test --select "model_b"
```
@@ -211,7 +211,7 @@ The `relationships` test requires both `model_a` and `model_b`. Because I did no
```shell
-dbt test --select model_b --defer --state prod-run-artifacts
+dbt test --select "model_b" --defer --state prod-run-artifacts
```
diff --git a/website/docs/reference/node-selection/exclude.md b/website/docs/reference/node-selection/exclude.md
index 9ad4bd1cc0e..d2c140d1bb5 100644
--- a/website/docs/reference/node-selection/exclude.md
+++ b/website/docs/reference/node-selection/exclude.md
@@ -7,19 +7,19 @@ sidebar_label: "Exclude"
dbt provides an `--exclude` flag with the same semantics as `--select`. Models specified with the `--exclude` flag will be removed from the set of models selected with `--select`.
```bash
-$ dbt run --select my_package.*+ --exclude my_package.a_big_model+ # select all models in my_package and their children except a_big_model and its children
+dbt run --select "my_package".*+ --exclude "my_package.a_big_model+" # select all models in my_package and their children except a_big_model and its children
```
Exclude a specific resource by its name or lineage:
```bash
# test
-$ dbt test --exclude not_null_orders_order_id # test all models except the not_null_orders_order_id test
-$ dbt test --exclude orders # test all models except tests associated with the orders model
+dbt test --exclude "not_null_orders_order_id" # test all models except the not_null_orders_order_id test
+dbt test --exclude "orders" # test all models except tests associated with the orders model
# seed
-$ dbt seed --exclude account_parent_mappings # load all seeds except account_parent_mappings
+dbt seed --exclude "account_parent_mappings" # load all seeds except account_parent_mappings
# snapshot
-$ dbt snapshot --exclude snap_order_statuses # execute all snapshots except snap_order_statuses
+dbt snapshot --exclude "snap_order_statuses" # execute all snapshots except snap_order_statuses
```
diff --git a/website/docs/reference/node-selection/graph-operators.md b/website/docs/reference/node-selection/graph-operators.md
index 4fdc2f10628..8cba43e1b52 100644
--- a/website/docs/reference/node-selection/graph-operators.md
+++ b/website/docs/reference/node-selection/graph-operators.md
@@ -7,9 +7,9 @@ If placed at the front of the model selector, `+` will select all parents of the
```bash
- $ dbt run --select my_model+ # select my_model and all children
- $ dbt run --select +my_model # select my_model and all parents
- $ dbt run --select +my_model+ # select my_model, and all of its parents and children
+dbt run --select "my_model+" # select my_model and all children
+dbt run --select "+my_model" # select my_model and all parents
+dbt run --select "+my_model+" # select my_model, and all of its parents and children
```
@@ -20,9 +20,9 @@ to step through.
```bash
- $ dbt run --select my_model+1 # select my_model and its first-degree children
- $ dbt run --select 2+my_model # select my_model, its first-degree parents, and its second-degree parents ("grandparents")
- $ dbt run --select 3+my_model+4 # select my_model, its parents up to the 3rd degree, and its children down to the 4th degree
+dbt run --select "my_model+1" # select my_model and its first-degree children
+dbt run --select "2+my_model" # select my_model, its first-degree parents, and its second-degree parents ("grandparents")
+dbt run --select "3+my_model+4" # select my_model, its parents up to the 3rd degree, and its children down to the 4th degree
```
@@ -32,5 +32,5 @@ The `@` operator is similar to `+`, but will also include _the parents of the ch
@@ -96,10 +96,10 @@ dbt run --select some_model
The `fqn` method is used to select nodes based off their "fully qualified names" (FQN) within the dbt graph. The default output of [`dbt list`](/reference/commands/list) is a listing of FQN.
-```
-dbt run --select fqn:some_model
-dbt run --select fqn:your_project.some_model
-dbt run --select fqn:some_package.some_other_model
+```bash
+dbt run --select "fqn:some_model"
+dbt run --select "fqn:your_project.some_model"
+dbt run --select "fqn:some_package.some_other_model"
```
### The "package" method
@@ -111,10 +111,10 @@ selectors unambiguous.
```bash
# These three selectors are equivalent
- dbt run --select package:snowplow
- dbt run --select snowplow
- dbt run --select snowplow.*
- ```
+ dbt run --select "package:snowplow"
+ dbt run --select "snowplow"
+ dbt run --select "snowplow.*"
+```
### The "config" method
@@ -124,10 +124,10 @@ The `config` method is used to select models that match a specified [node config
```bash
- $ dbt run --select config.materialized:incremental # run all models that are materialized incrementally
- $ dbt run --select config.schema:audit # run all models that are created in the `audit` schema
- $ dbt run --select config.cluster_by:geo_country # run all models clustered by `geo_country`
- ```
+dbt run --select "config.materialized:incremental" # run all models that are materialized incrementally
+dbt run --select "config.schema:audit" # run all models that are created in the `audit` schema
+dbt run --select "config.cluster_by:geo_country" # run all models clustered by `geo_country`
+```
@@ -135,7 +135,8 @@ The `config` method is used to select models that match a specified [node config
While most config values are strings, you can also use the `config` method to match boolean configs, dictionary keys, and values in lists.
For example, given a model with the following configurations:
-```
+
+```bash
{{ config(
materialized = 'incremental',
unique_key = ['column_a', 'column_b'],
@@ -148,10 +149,10 @@ select ...
You can select using any of the following:
```bash
-$ dbt ls -s config.materialized:incremental
-$ dbt ls -s config.unique_key:column_a
-$ dbt ls -s config.grants.select:reporter
-$ dbt ls -s config.transient:true
+dbt ls -s config.materialized:incremental
+dbt ls -s config.unique_key:column_a
+dbt ls -s config.grants.select:reporter
+dbt ls -s config.transient:true
```
@@ -162,10 +163,10 @@ The `test_type` method is used to select tests based on their type, `singular` o
- ```bash
- $ dbt test --select test_type:generic # run all generic tests
- $ dbt test --select test_type:singular # run all singular tests
- ```
+```bash
+dbt test --select "test_type:generic" # run all generic tests
+dbt test --select "test_type:singular" # run all singular tests
+```
### The "test_name" method
@@ -176,10 +177,10 @@ that defines it. For more information about how generic tests are defined, read
```bash
- $ dbt test --select test_name:unique # run all instances of the `unique` test
- $ dbt test --select test_name:equality # run all instances of the `dbt_utils.equality` test
- $ dbt test --select test_name:range_min_max # run all instances of a custom schema test defined in the local project, `range_min_max`
- ```
+dbt test --select "test_name:unique" # run all instances of the `unique` test
+dbt test --select "test_name:equality" # run all instances of the `dbt_utils.equality` test
+dbt test --select "test_name:range_min_max" # run all instances of a custom schema test defined in the local project, `range_min_max`
+```
### The "state" method
@@ -204,9 +205,9 @@ The `state` method is used to select nodes by comparing them against a previous
```bash
- $ dbt test --select state:new # run all tests on new models + and new tests on old models
- $ dbt run --select state:modified # run all models that have been modified
- $ dbt ls --select state:modified # list all modified nodes (not just models)
+dbt test --select "state:new " # run all tests on new models + and new tests on old models
+dbt run --select "state:modified" # run all models that have been modified
+dbt ls --select "state:modified" # list all modified nodes (not just models)
```
@@ -236,18 +237,18 @@ The `exposure` method is used to select parent resources of a specified [exposur
```bash
- $ dbt run --select +exposure:weekly_kpis # run all models that feed into the weekly_kpis exposure
- $ dbt test --select +exposure:* # test all resources upstream of all exposures
- $ dbt ls --select +exposure:* --resource-type source # list all sources upstream of all exposures
- ```
+dbt run --select "+exposure:weekly_kpis" # run all models that feed into the weekly_kpis exposure
+dbt test --select "+exposure:*" # test all resources upstream of all exposures
+dbt ls --select "+exposure:*" --resource-type source # list all sources upstream of all exposures
+```
### The "metric" method
The `metric` method is used to select parent resources of a specified [metric](/docs/build/metrics). Use in conjunction with the `+` operator.
```bash
-$ dbt build --select +metric:weekly_active_users # build all resources upstream of weekly_active_users metric
-$ dbt ls --select +metric:* --resource-type source # list all source tables upstream of all metrics
+dbt build --select "+metric:weekly_active_users" # build all resources upstream of weekly_active_users metric
+dbt ls --select "+metric:*" --resource-type source # list all source tables upstream of all metrics
```
### The "result" method
@@ -255,10 +256,10 @@ $ dbt ls --select +metric:* --resource-type source # list all source tables
The `result` method is related to the `state` method described above and can be used to select resources based on their result status from a prior run. Note that one of the dbt commands [`run`, `test`, `build`, `seed`] must have been performed in order to create the result on which a result selector operates. You can use `result` selectors in conjunction with the `+` operator.
```bash
-$ dbt run --select result:error --state path/to/artifacts # run all models that generated errors on the prior invocation of dbt run
-$ dbt test --select result:fail --state path/to/artifacts # run all tests that failed on the prior invocation of dbt test
-$ dbt build --select 1+result:fail --state path/to/artifacts # run all the models associated with failed tests from the prior invocation of dbt build
-$ dbt seed --select result:error --state path/to/artifacts # run all seeds that generated errors on the prior invocation of dbt seed.
+dbt run --select "result:error" --state path/to/artifacts # run all models that generated errors on the prior invocation of dbt run
+dbt test --select "result:fail" --state path/to/artifacts # run all tests that failed on the prior invocation of dbt test
+dbt build --select "1+result:fail" --state path/to/artifacts # run all the models associated with failed tests from the prior invocation of dbt build
+dbt seed --select "result:error" --state path/to/artifacts # run all seeds that generated errors on the prior invocation of dbt seed.
```
### The "source_status" method
@@ -276,8 +277,8 @@ After issuing one of the above commands, you can reference the source freshness
```bash
# You can also set the DBT_ARTIFACT_STATE_PATH environment variable instead of the --state flag.
-$ dbt source freshness # must be run again to compare current to previous state
-$ dbt build --select source_status:fresher+ --state path/to/prod/artifacts
+dbt source freshness # must be run again to compare current to previous state
+dbt build --select "source_status:fresher+" --state path/to/prod/artifacts
```
@@ -286,8 +287,8 @@ $ dbt build --select source_status:fresher+ --state path/to/prod/artifacts
```bash
# You can also set the DBT_STATE environment variable instead of the --state flag.
-$ dbt source freshness # must be run again to compare current to previous state
-$ dbt build --select source_status:fresher+ --state path/to/prod/artifacts
+dbt source freshness # must be run again to compare current to previous state
+dbt build --select "source_status:fresher+" --state path/to/prod/artifacts
```
@@ -305,9 +306,9 @@ Supported in v1.5 or newer.
The `group` method is used to select models defined within a [group](/reference/resource-configs/group).
- ```bash
- dbt run --select group:finance # run all models that belong to the finance group.
- ```
+```bash
+dbt run --select "group:finance" # run all models that belong to the finance group.
+```
@@ -324,9 +325,9 @@ Supported in v1.5 or newer.
The `access` method selects models based on their [access](/reference/resource-configs/access) property.
```bash
-dbt list --select access:public # list all public models
-dbt list --select access:private # list all private models
-dbt list --select access:protected # list all protected models
+dbt list --select "access:public" # list all public models
+dbt list --select "access:private" # list all private models
+dbt list --select "access:protected" # list all protected models
```
@@ -344,11 +345,11 @@ Supported in v1.5 or newer.
The `version` method selects [versioned models](/docs/collaborate/govern/model-versions) based on their [version identifier](/reference/resource-properties/versions) and [latest version](/reference/resource-properties/latest_version).
```bash
-dbt list --select version:latest # only 'latest' versions
-dbt list --select version:prerelease # versions newer than the 'latest' version
+dbt list --select "version:latest" # only 'latest' versions
+dbt list --select "version:prerelease" # versions newer than the 'latest' version
dbt list --select version:old # versions older than the 'latest' version
-dbt list --select version:none # models that are *not* versioned
+dbt list --select "version:none" # models that are *not* versioned
```
- - * To authenticate, you need to [generate a service token](/docs/dbt-cloud-apis/service-tokens) with Semantic Layer Only and Metadata Only permissions. - * Refer to the [SQL query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the APIs. - -- To learn more about the sophisticated integrations that connect to the dbt Semantic Layer, refer to [Available integrations](/docs/use-dbt-semantic-layer/avail-sl-integrations) for more info. +
❌| ✅ | ✅ | Proprietary, Cloud (Team & Enterprise) | -| **Semantic Layer API** | The interfaces that allow users to submit metric queries include the MetricFlow CLI and JDBC API. They also serve as the foundation for building first-class integrations with various tools. | ❌ | ✅ | ✅ | Proprietary, Cloud (Team & Enterprise)| +| **Semantic Layer APIs** | The interfaces that allow users to submit metric queries using the GraphQL and JDBC APIs. They also serve as the foundation for building first-class integrations with various tools. | ❌ | ✅ | ✅ | Proprietary, Cloud (Team & Enterprise)| ## Related questions diff --git a/website/docs/docs/use-dbt-semantic-layer/tableau.md b/website/docs/docs/use-dbt-semantic-layer/tableau.md new file mode 100644 index 00000000000..c505ea323f4 --- /dev/null +++ b/website/docs/docs/use-dbt-semantic-layer/tableau.md @@ -0,0 +1,67 @@ +--- +title: "Tableau (beta)" +description: "Use Tableau worksheets to query the dbt Semantic Layer and produce dashboards with trusted date." +tags: [Semantic Layer] +sidebar_label: "Tableau (beta)" +--- + +:::info Beta functionality +The Tableau integration with the dbt Semantic Layer is a [beta feature](/docs/dbt-versions/product-lifecycles#dbt-cloud). +::: + + +The Tableau integration allows you to use worksheets to query the Semantic Layer directly and produce your dashboards with trusted data. This integration provides a live connection to the dbt Semantic Layer through Tableau Desktop. + +## Prerequisites + +1. You must have [Tableau Desktop](https://www.tableau.com/en-gb/products/desktop) installed +2. Authenticate with either Tableau Server or Tableau Cloud +3. You need your dbt Cloud host, [Environment ID](/docs/use-dbt-semantic-layer/setup-sl#set-up-dbt-semantic-layer) and [service token](/docs/dbt-cloud-apis/service-tokens) to log in. This account should be set up with the dbt Semantic Layer. +4. You must a dbt Cloud Team or Enterprise [account](https://www.getdbt.com/pricing) and multi-tenant [deployment](/docs/cloud/about-cloud/regions-ip-addresses). (Single-Tenant coming soon) + + +## Installing + +1. Download our [connector file](https://github.com/dbt-labs/semantic-layer-tableau-connector/releases/download/v1.0.0/dbt_semantic_layer.taco) locally and add it to your default folder: + - Windows: `C:\Users\\[Windows User]\Documents\My Tableau Repository\Connectors` + - Mac: `/Users/[user]/Documents/My Tableau Repository/Connectors` + - Linux: `/opt/tableau/connectors` +2. Install the [JDBC driver](/docs/dbt-cloud-apis/sl-jdbc) to the folder based on your operating system: + - Windows: `C:\Program Files\Tableau\Drivers` + - Mac: `~/Library/Tableau/Drivers` + - Linux: ` /opt/tableau/tableau_driver/jdbc` +3. Open Tableau Desktop and find the **dbt Semantic Layer by dbt Labs** connector on the left-hand side. +4. Connect with your Host, Environment ID, and service token information that's provided to you in your dbt Cloud Semantic Layer configuration. + + +## Using the integration + +Once you authenticate, the system will direct you to the data source page with all the metrics and dimensions configured in your Semantic Layer. + +- From there, go directly to a worksheet in the bottom left hand corner. +- Then, you'll find all the metrics and dimensions that are available to query on the left-hand side of your window. + +Visit the [Tableau documentation](https://help.tableau.com/current/pro/desktop/en-us/gettingstarted_overview.htm) to learn more about how to use Tableau worksheets and dashboards. + +## Things to note + +- All metrics use the "SUM" aggregation type, and this can't be altered. The dbt Semantic Layer controls the aggregation type and it is intentionally fixed. Keep in mind that the underlying aggregation in the dbt Semantic Layer might not be "SUM" (even though "SUM" is Tableau's default). +- Tableau surfaces all metrics and dimensions from the dbt Semantic Layer on the left-hand side. Note, that not all metrics and dimensions can be combined with one another. You will receive an error message if a particular dimension cannot be sliced with a metric (or vice versa). + - To display available metrics and dimensions, dbt Semantic Layer returns metadata for a fake table with the dimensions and metrics as 'columns' on this table. Because of this, you can't actually query this table for previews or extracts. + - Since this is treated as a table, dbt Semantic Layer can't dynamically change what is available. This means we display _all_ available metrics and dimensions even if a particular metric and dimension combination isn't available. + +- Certain Table calculations like "Totals" and "Percent Of" may not be accurate when using metrics aggregated in a non-additive way (such as count distinct) + +## Unsupported functionality + +The following Tableau features aren't supported at this time, however, the dbt Semantic Layer may support some of this functionality in a future release: + +- Updating the data source page +- Using "Extract" mode to view yur data +- Unioning Tables +- Writing Custom SQL +- Table Extensions +- Cross Database Joins +- All functions in Analysis --> Create Calculated Field +- Filtering on a Date Part time dimension for a Cumulative metric type +- Changing your date dimension to use "Week Number" diff --git a/website/docs/docs/verified-adapters.md b/website/docs/docs/verified-adapters.md index a2d28a612d6..170bc8f885b 100644 --- a/website/docs/docs/verified-adapters.md +++ b/website/docs/docs/verified-adapters.md @@ -13,6 +13,10 @@ The verification process serves as the on-ramp to integration with dbt Cloud. As To learn more, see [Verifying a new adapter](/guides/dbt-ecosystem/adapter-development/7-verifying-a-new-adapter). +import MSCallout from '/snippets/_microsoft-adapters-soon.md'; + +
the database | All [supported versions](/docs/dbt-versions/core) | -| [seed](/reference/commands/seed) | Loads CSV files into the database | All [supported versions](/docs/dbt-versions/core) | -| [show](/reference/commands/show) | Preview table rows post-transformation | All [supported versions](/docs/dbt-versions/core) | -| [snapshot](/reference/commands/snapshot) | Executes "snapshot" jobs defined in a project | All [supported versions](/docs/dbt-versions/core) | -| [source](/reference/commands/source) | Provides tools for working with source data (including validating that
sources are "fresh") | All [supported versions](/docs/dbt-versions/core) | -| [test](/reference/commands/test) | Executes tests defined in a project | All [supported versions](/docs/dbt-versions/core) | -| [init](/reference/commands/init) | Initializes a new dbt project (CLI only) | All [supported versions](/docs/dbt-versions/core) | - +
dbt Core | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) | +| [compile](/reference/commands/compile) | Compiles (but does not run) the models in a project | All | All [supported versions](/docs/dbt-versions/core) | +| [debug](/reference/commands/debug) | Debugs dbt connections and projects | dbt Core | All [supported versions](/docs/dbt-versions/core) | +| [deps](/reference/commands/deps) | Downloads dependencies for a project | All | All [supported versions](/docs/dbt-versions/core) | +| [docs](/reference/commands/cmd-docs) | Generates documentation for a project | All | All [supported versions](/docs/dbt-versions/core) | +| help | Displays help information for any command | dbt Core
dbt Cloud CLI | All [supported versions](/docs/dbt-versions/core) | +| [list](/reference/commands/list) | Lists resources defined in a dbt project | All | All [supported versions](/docs/dbt-versions/core) | +| [parse](/reference/commands/parse) | Parses a project and writes detailed timing info | All | All [supported versions](/docs/dbt-versions/core) | +| reattach | Reattaches to the most recent invocation to retrieve logs and artifacts. | dbt Cloud CLI | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) | +| [retry](/reference/commands/retry) | Retry the last run `dbt` command from the point of failure | All | Requires [dbt v1.6 or higher](/docs/dbt-versions/core) | +| [run](/reference/commands/run) | Runs the models in a project | All | All [supported versions](/docs/dbt-versions/core) | +| [run-operation](/reference/commands/run-operation) | Invoke a macro, including running arbitrary maintenance SQL against the database | All | All [supported versions](/docs/dbt-versions/core) | +| [seed](/reference/commands/seed) | Loads CSV files into the database | All | All [supported versions](/docs/dbt-versions/core) | +| [show](/reference/commands/show) | Preview table rows post-transformation | All | All [supported versions](/docs/dbt-versions/core) | +| [snapshot](/reference/commands/snapshot) | Executes "snapshot" jobs defined in a project | All | All [supported versions](/docs/dbt-versions/core) | +| [source](/reference/commands/source) | Provides tools for working with source data (including validating that sources are "fresh") | All | All [supported versions](/docs/dbt-versions/core) | +| [test](/reference/commands/test) | Executes tests defined in a project | All | All [supported versions](/docs/dbt-versions/core) | +| [init](/reference/commands/init) | Initializes a new dbt project | dbt Core | All [supported versions](/docs/dbt-versions/core) |
Install with dbt Core "
icon="materialize"/>
Install with dbt Core "
icon="oracle"/>
diff --git a/website/snippets/_adapters-verified.md b/website/snippets/_adapters-verified.md
index 7caf099b7d1..3cc1e800448 100644
--- a/website/snippets/_adapters-verified.md
+++ b/website/snippets/_adapters-verified.md
@@ -2,61 +2,60 @@
diff --git a/website/snippets/_cloud-cli-flag.md b/website/snippets/_cloud-cli-flag.md new file mode 100644 index 00000000000..523591a438c --- /dev/null +++ b/website/snippets/_cloud-cli-flag.md @@ -0,0 +1,5 @@ +:::info Public preview functionality + +The dbt Cloud CLI is currently in [public preview](/docs/dbt-versions/product-lifecycles#dbt-cloud). Share feedback or request features you'd like to see on the [dbt community Slack](https://getdbt.slack.com/archives/C05M77P54FL). + +::: diff --git a/website/snippets/_cloud-environments-info.md b/website/snippets/_cloud-environments-info.md index 5388379dc34..b0c8e86a2b5 100644 --- a/website/snippets/_cloud-environments-info.md +++ b/website/snippets/_cloud-environments-info.md @@ -3,17 +3,17 @@ In dbt Cloud, there are two types of environments: - Deployment environment — Determines the settings used when jobs created within that environment are executed. -- Development environment — Determines the settings used in the dbt Cloud IDE for that particular dbt Cloud project. +- Development environment — Determines the settings used in the dbt Cloud IDE or dbt Cloud CLI, for that particular project. Each dbt Cloud project can only have a single development environment but can have any number of deployment environments. | | Development Environments | Deployment Environments | | --- | --- | --- | -| Determines settings for | dbt Cloud IDE | dbt Cloud Job runs | +| Determines settings for | dbt Cloud IDE or dbt Cloud CLI | dbt Cloud Job runs | | How many can I have in my project? | 1 | Any number | :::note -For users familiar with development on the CLI, each environment is roughly analogous to an entry in your `profiles.yml` file, with some additional information about your repository to ensure the proper version of code is executed. More info on dbt core environments [here](/docs/core/dbt-core-environments). +For users familiar with development on dbt Core, each environment is roughly analogous to an entry in your `profiles.yml` file, with some additional information about your repository to ensure the proper version of code is executed. More info on dbt core environments [here](/docs/core/dbt-core-environments). ::: ## Common environment settings @@ -38,7 +38,7 @@ Both development and deployment environments have a section called **General Set By default, all environments will use the default branch in your repository (usually the `main` branch) when accessing your dbt code. This is overridable within each dbt Cloud Environment using the **Default to a custom branch** option. This setting have will have slightly different behavior depending on the environment type: -- **Development**: determines which branch in the dbt Cloud IDE developers create branches from and open PRs against +- **Development**: determines which branch in the dbt Cloud IDE or dbt Cloud CLI developers create branches from and open PRs against. - **Deployment:** determines the branch is cloned during job executions for each environment. For more info, check out this [FAQ page on this topic](/faqs/Environments/custom-branch-settings)! @@ -59,7 +59,7 @@ Something to note, Extended Attributes doesn't mask secret values. We recommend
-If you're developing in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud) or [orchestrating job runs](/docs/deploy/deployments), Extended Attributes parses through the provided YAML and extracts the `profiles.yml` attributes. For each individual attribute: +If you're developing in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud), [dbt Cloud CLI](/docs/cloud/cloud-cli-installation), or [orchestrating job runs](/docs/deploy/deployments), Extended Attributes parses through the provided YAML and extracts the `profiles.yml` attributes. For each individual attribute: - If the attribute exists in another source (such as your project settings), it will replace its value (like environment-level values) in the profile. It also overrides any custom environment variables. diff --git a/website/snippets/_enterprise-permissions-table.md b/website/snippets/_enterprise-permissions-table.md index 779c5bcb479..3eb313e0f5b 100644 --- a/website/snippets/_enterprise-permissions-table.md +++ b/website/snippets/_enterprise-permissions-table.md @@ -7,7 +7,7 @@ Key: Permissions: * Account-level permissions — Permissions related to management of the dbt Cloud account. For example, billing and account settings. -* Project-level permissions — Permissions related to the projects in dbt Cloud. For example, repos and access to the IDE. +* Project-level permissions — Permissions related to the projects in dbt Cloud. For example, repos and access to the IDE or dbt Cloud CLI. ### Account roles Account roles enable you to manage the dbt Cloud account and manage the account settings (for example, generating service tokens, inviting users, configuring SSO). They also provide project-level permissions. The **Account Admin** role is the highest level of access you can assign. @@ -20,8 +20,10 @@ Account roles enable you to manage the dbt Cloud account and manage the account | Audit logs | R | | | R | | | Auth provider | W | | | W | R | | Billing | W | W | | | R | +| Groups | W | | R | W | R | | Invitations | W | | W | W | R | | IP restrictions | W | | | W | R | +| Licenses | W | | W | W | R | | Members | W | | W | W | R | | Project (create) | W | | W | | | | Public models | R | R | R | R | R | @@ -34,25 +36,23 @@ Account roles enable you to manage the dbt Cloud account and manage the account |:-------------------------|:-------------:|:-------------:|:---------------:|:--------------:|:------:| | Connections | W | | W | | R | | Credentials | W | | W | | R | -| Custom env. variables | W | | W | | R | +| Custom env. variables | W | | W | | R | | dbt adapters | W | | W | | R | -| Develop (IDE) | W | | W | | | +| Develop (IDE or dbt Cloud CLI) | W | | W | | | | Environments | W | | W | | R | -| Groups | W | | R | W | R | | Jobs | W | | W | | R | -| Licenses | W | | W | W | R | | Metadata | R | | R | | R | | Permissions | W | | W | W | R | | Profile | W | | W | | R | | Projects | W | | W | R | R | | Repositories | W | | W | | R | | Runs | W | | W | | R | -| Semantic Layer Config | W | | W | | R | +| Semantic Layer Config | W | | W | | R | ### Project role permissions -The project roles enable you to work within the projects in various capacities. They primarily provide access to project-level permissions such as repos and the IDE, but may also provide some account-level permissions. +The project roles enable you to work within the projects in various capacities. They primarily provide access to project-level permissions such as repos and the IDE or dbt Cloud CLI, but may also provide some account-level permissions. #### Account permissions for project roles @@ -61,12 +61,14 @@ The project roles enable you to work within the projects in various capacities. | Account settings | R | | R | | R | | | | | | R | | | Auth provider | | | | | | | | | | | | | | Billing | | | | | | | | | | | | | -| Invitations | W | R | R | R | R | R | R | | | R | R | | -| Members | W | | R | R | R | | | | | R | R | | +| Groups | R | | R | R | R | | | | | R | R | | +| Invitations | W | R | R | R | R | R | R | | | R | R | | +| Licenses | W | R | R | R | R | R | R | | | | R | | +| Members | W | | R | R | R | | | | | R | R | | | Project (create) | | | | | | | | | | | | | -| Public models | R | R | R | R | R | R | R | R | R | R | R | R | +| Public models | R | R | R | R | R | R | R | R | R | R | R | R | | Service tokens | | | | | | | | | | | | | -| Webhooks | W | | | W | | | | | | | | W | +| Webhooks | W | | | W | | | | | | | | W | #### Project permissions for project roles @@ -74,13 +76,11 @@ The project roles enable you to work within the projects in various capacities. |--------------------------|:-----:|:-------:|:--------------:|:---------:|:---------:|:---------:|:-----------:|:--------:|:--------------:|:-----------:|:----------:|:------:| | Connections | W | R | W | R | R | R | | | | R | R | | | Credentials | W | W | W | W | R | W | | | | R | R | | -| Custom env. variables | W | W | W | W | W | W | R | | | R | W | | +| Custom env. variables | W | W | W | W | W | W | R | | | R | W | | | dbt adapters | W | W | W | W | R | W | | | | R | R | | -| Develop (IDE) | W | W | | W | | | | | | | | | +| Develop (IDE or dbt Cloud CLI) | W | W | | W | | | | | | | | | | Environments | W | R | R | R | R | W | R | | | R | R | | -| Groups | R | | R | R | R | | | | | R | R | | | Jobs | W | R | R | W | R | W | R | | | R | R | | -| Licenses | W | R | R | R | R | R | R | | | | R | | | Metadata | R | R | R | R | R | R | R | R | | R | R | | | Permissions | W | | R | R | R | | | | | | W | | | Profile | W | R | W | R | R | R | | | | R | R | | diff --git a/website/snippets/_manifest-versions.md b/website/snippets/_manifest-versions.md new file mode 100644 index 00000000000..c9b3e7af6ec --- /dev/null +++ b/website/snippets/_manifest-versions.md @@ -0,0 +1,11 @@ + +| dbt Core version | Manifest version | +|------------------|---------------------------------------------------------------| +| v1.7 | [v11](https://schemas.getdbt.com/dbt/manifest/v11/index.html) | +| v1.6 | [v10](https://schemas.getdbt.com/dbt/manifest/v10/index.html) | +| v1.5 | [v9](https://schemas.getdbt.com/dbt/manifest/v9/index.html) | +| v1.4 | [v8](https://schemas.getdbt.com/dbt/manifest/v8/index.html) | +| v1.3 | [v7](https://schemas.getdbt.com/dbt/manifest/v7/index.html) | +| v1.2 | [v6](https://schemas.getdbt.com/dbt/manifest/v6/index.html) | +| v1.1 | [v5](https://schemas.getdbt.com/dbt/manifest/v5/index.html) | +| v1.0 | [v4](https://schemas.getdbt.com/dbt/manifest/v4/index.html) | \ No newline at end of file diff --git a/website/snippets/_microsoft-adapters-soon.md b/website/snippets/_microsoft-adapters-soon.md new file mode 100644 index 00000000000..c3f30ef0939 --- /dev/null +++ b/website/snippets/_microsoft-adapters-soon.md @@ -0,0 +1,3 @@ +:::tip Coming soon +dbt Cloud support for the Microsoft Fabric and Azure Synapse Analytics adapters is coming soon! +::: \ No newline at end of file diff --git a/website/snippets/_new-sl-setup.md b/website/snippets/_new-sl-setup.md index b802db9c5ae..ad248bc3ca9 100644 --- a/website/snippets/_new-sl-setup.md +++ b/website/snippets/_new-sl-setup.md @@ -1,13 +1,13 @@ You can set up the dbt Semantic Layer in dbt Cloud at the environment and project level. Before you begin: -- You must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment, hosted in North America. +- You must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment. Single-tenant coming soon. - You must be part of the Owner group, and have the correct [license](/docs/cloud/manage-access/seats-and-users) and [permissions](/docs/cloud/manage-access/self-service-permissions) to configure the Semantic Layer: * Enterprise plan — Developer license with Account Admin permissions. Or Owner with a Developer license, assigned Project Creator, Database Admin, or Admin permissions. * Team plan — Owner with a Developer license. - You must have a successful run in your new environment. :::tip -If you're using the legacy Semantic Layer, we **highly** recommend you [upgrade your dbt version](/docs/dbt-versions/upgrade-core-in-cloud) to dbt v1.6 or higher to use the new dbt Semantic Layer. Refer to the dedicated [migration guide](/guides/migration/sl-migration) for more info. +If you're using the legacy Semantic Layer, dbt Labs strongly recommends that you [upgrade your dbt version](/docs/dbt-versions/upgrade-core-in-cloud) to dbt version 1.6 or newer to use the latest dbt Semantic Layer. Refer to the dedicated [migration guide](/guides/migration/sl-migration) for details. ::: 1. In dbt Cloud, create a new [deployment environment](/docs/deploy/deploy-environments#create-a-deployment-environment) or use an existing environment on dbt 1.6 or higher. diff --git a/website/snippets/_sl-connect-and-query-api.md b/website/snippets/_sl-connect-and-query-api.md new file mode 100644 index 00000000000..429f41c3bf6 --- /dev/null +++ b/website/snippets/_sl-connect-and-query-api.md @@ -0,0 +1,10 @@ +You can query your metrics in a JDBC-enabled tool or use existing first-class integrations with the dbt Semantic Layer. + +You must have a dbt Cloud Team or Enterprise [multi-tenant](/docs/cloud/about-cloud/regions-ip-addresses) deployment. Single-tenant coming soon. + +- To learn how to use the JDBC or GraphQL API and what tools you can query it with, refer to [dbt Semantic Layer APIs](/docs/dbt-cloud-apis/sl-api-overview). + + * To authenticate, you need to [generate a service token](/docs/dbt-cloud-apis/service-tokens) with Semantic Layer Only and Metadata Only permissions. + * Refer to the [SQL query syntax](/docs/dbt-cloud-apis/sl-jdbc#querying-the-api-for-metric-metadata) to query metrics using the API. + +- To learn more about the sophisticated integrations that connect to the dbt Semantic Layer, refer to [Available integrations](/docs/use-dbt-semantic-layer/avail-sl-integrations) for more info. diff --git a/website/snippets/_sl-create-semanticmodel.md b/website/snippets/_sl-create-semanticmodel.md index bc4276efcb6..6e0376ab10b 100644 --- a/website/snippets/_sl-create-semanticmodel.md +++ b/website/snippets/_sl-create-semanticmodel.md @@ -1,4 +1,4 @@ -The following steps will walk you through setting up semantic models, which you can do with the dbt Cloud IDE or the CLI. Semantic models consist of [entities](/docs/build/entities), [dimensions](/docs/build/dimensions), and [measures](/docs/build/measures). +The following steps describe how to set up semantic models. Semantic models consist of [entities](/docs/build/entities), [dimensions](/docs/build/dimensions), and [measures](/docs/build/measures). We highly recommend you read the overview of what a [semantic model](/docs/build/semantic-models) is before getting started. If you're working in the [Jaffle shop example](https://github.com/dbt-labs/jaffle-sl-template), delete the `orders.yml` config or delete the .yml extension so it's ignored during parsing. **We'll be rebuilding it step by step in this example.** diff --git a/website/snippets/_sl-define-metrics.md b/website/snippets/_sl-define-metrics.md index 29af3f5b7c3..3734e819c1b 100644 --- a/website/snippets/_sl-define-metrics.md +++ b/website/snippets/_sl-define-metrics.md @@ -1,4 +1,4 @@ -Now that you've created your first semantic model, it's time to define your first metric! You can define metrics with the dbt Cloud IDE or CLI. +Now that you've created your first semantic model, it's time to define your first metric! You can define metrics with the dbt Cloud IDE or command line. MetricFlow supports different metric types like [simple](/docs/build/simple), [ratio](/docs/build/ratio), [cumulative](/docs/build/cumulative), and [derived](/docs/build/derived). It's recommended that you read the [metrics overview docs](/docs/build/metrics-overview) before getting started. diff --git a/website/snippets/_sl-install-metricflow.md b/website/snippets/_sl-install-metricflow.md deleted file mode 100644 index 73e60d34e85..00000000000 --- a/website/snippets/_sl-install-metricflow.md +++ /dev/null @@ -1,8 +0,0 @@ -Install the [MetricFlow CLI](/docs/build/metricflow-cli) as an extension of a dbt adapter from PyPI. The MetricFlow CLI is compatible with Python versions 3.8, 3.9, 3.10 and 3.11 - -Use pip install `metricflow` and your [dbt adapter](/docs/supported-data-platforms): - -- Create or activate your virtual environment. `python -m venv venv` or `source your-venv/bin/activate` -- Run `pip install "dbt-metricflow[your_adapter_name]"` - - You must specify `[your_adapter_name]`. - - For example, run `pip install "dbt-metricflow[snowflake]"` if you use a Snowflake adapter. diff --git a/website/snippets/_sl-partner-links.md b/website/snippets/_sl-partner-links.md index fa8146c6418..59e5b54b7c0 100644 --- a/website/snippets/_sl-partner-links.md +++ b/website/snippets/_sl-partner-links.md @@ -2,26 +2,61 @@ The following tools integrate with the dbt Semantic Layer:
-Before you connect to these tools, you'll need to first [set up the dbt Semantic Layer](/docs/use-dbt-semantic-layer/setup-sl) and [generate a service token](/docs/dbt-cloud-apis/service-tokens) to create **Semantic Layer Only** and **Metadata Only** permissions. - +Before you connect to these tools, you'll need to first [set up the dbt Semantic Layer](/docs/use-dbt-semantic-layer/setup-sl) and [generate a service token](/docs/dbt-cloud-apis/service-tokens) to create **Semantic Layer Only** and **Metadata Only** permissions. diff --git a/website/snippets/_sl-plan-info.md b/website/snippets/_sl-plan-info.md index 5fba18de6bb..083ab2209bc 100644 --- a/website/snippets/_sl-plan-info.md +++ b/website/snippets/_sl-plan-info.md @@ -1,2 +1,2 @@ -To define and query metrics with the {props.product}, you must be on a {props.plan} multi-tenant plan, {props.instance} (Additional region support coming soon).
The re-released dbt Semantic Layer is available on dbt v1.6 or higher. dbt Core users can use the MetricFlow CLI to define metrics in their local project, but won't be able to dynamically query them with integrated tools.
+To define and query metrics with the {props.product}, you must be on a {props.plan} multi-tenant plan .
diff --git a/website/snippets/_sl-run-prod-job.md b/website/snippets/_sl-run-prod-job.md new file mode 100644 index 00000000000..a637b0b431e --- /dev/null +++ b/website/snippets/_sl-run-prod-job.md @@ -0,0 +1,7 @@ +Once you’ve defined metrics in your dbt project, you can perform a job run in your deployment environment in dbt Cloud to materialize your metrics. The deployment environment is only supported for the dbt Semantic Layer currently. + +1. Select **Deploy** from the top navigation bar. +2. Select **Jobs** to rerun the job with the most recent code in the deployment environment. +3. Your metric should appear as a red node in the dbt Cloud IDE and dbt directed acyclic graphs (DAG). + +
-- Understand [MetricFlow's](/docs/build/about-metricflow) key concepts, which powers the revamped dbt Semantic Layer. - + - To access the API and query metrics in downstream tools, you must have a dbt Cloud [Team or Enterprise](https://www.getdbt.com/pricing/) account. dbt Core or Developer accounts can define metrics but won't be able to dynamically query them.
+- Understand [MetricFlow's](/docs/build/about-metricflow) key concepts, which powers the latest dbt Semantic Layer.