clarify build on docs content and move doc to dag section (#5633)

this pr sets to update the following items to clarify dbt explorer and
dbt docs usage as there's been confusion per user feedback:

- updates artifacts page to mention dbt explorer, updates screenshots
- updates documentation page to mention explorer and clarifies which
configs are avaialble for egacy docs.
- moves documentation page to 'build your dag' so it follows dbt
workflows
- reworks 'build your docs in dbt cloud' and keeps it under
'collaborate.
- renames 'explore dbt projects' to 'navigate dbt explorer' so the
product/feature name is visible in docs.
- updates the links and adds redirects

Resolves #4882
Resolves #5304 
[docs
project](https://www.notion.so/dbtlabs/dbt-Explorer-and-legacy-docs-clarification-52b7a372082249e6a06b8cb5e7d1662c?pvs=4)

---------

Co-authored-by: dave-connors-3 <[email protected]>
Co-authored-by: Matt Shaver <[email protected]>

website/blog/2021-02-05-dbt-project-checklist.md

@@ -173,8 +173,8 @@ This post is the checklist I created to guide our internal work, and I’m shari
 
 Useful Links
 
-*   [FAQs for documentation](/docs/collaborate/documentation#faqs)
-*   [Doc blocks](/docs/collaborate/documentation#using-docs-blocks)
+*   [FAQs for documentation](/docs/build/documentation#faqs)
+*   [Doc blocks](/docs/build/documentation#using-docs-blocks)
 
 ## ✅ dbt Cloud specifics
 ----------------------------------------------------------------------------------------------------------------------------------------------------------

website/blog/2021-12-05-how-to-build-a-mature-dbt-project-from-scratch.md

@@ -87,7 +87,7 @@ The most important thing we’re introducing when your project is an infant is t
 
 * Introduce modularity with [{{ ref() }}](/reference/dbt-jinja-functions/ref) and [{{ source() }}](/reference/dbt-jinja-functions/source)
 
-* [Document](/docs/collaborate/documentation) and [test](/docs/build/data-tests) your first models
+* [Document](/docs/build/documentation) and [test](/docs/build/data-tests) your first models
 
 ![image alt text](/img/blog/building-a-mature-dbt-project-from-scratch/image_3.png)
 

website/blog/2022-09-28-analyst-to-ae.md

@@ -133,7 +133,7 @@ It’s much easier to keep to a naming guide when the writer has a deep understa
 
 If we want to know how certain logic was built technically, then we can reference the SQL code in dbt docs. If we want to know *why* a certain logic was built into that specific model, then that’s where we’d turn to the documentation.
 
-- Example of not-so-helpful documentation ([dbt docs can](https://docs.getdbt.com/docs/collaborate/documentation) build this dynamically):
+- Example of not-so-helpful documentation ([dbt docs can](https://docs.getdbt.com/docs/build/documentation) build this dynamically):
     - `Case when Zone = 1 and Level like 'A%' then 'True' else 'False' end as GroupB`
 - Example of better, more descriptive documentation (add to your dbt markdown file or column descriptions):
     - Group B is defined as Users in Zone 1 with a Level beginning with the letter 'A'. These users are accessing our new add-on product that began in Beta in August 2022. It's recommended to filter them out of the main Active Users metric.

website/blog/2023-02-14-passing-the-dbt-certification-exam.md

@@ -25,7 +25,7 @@ In this article, two Montreal Analytics consultants, Jade and Callie, discuss th
 
 **J:** To prepare for the exam, I built up a practice dbt project. All consultants do this as part of Montreal Analytics onboarding process, and this project allowed me to practice implementing sources and tests, refactoring SQL models, and debugging plenty of error messages. Additionally, I reviewed the [Certification Study Guide](https://www.getdbt.com/assets/uploads/dbt_certificate_study_guide.pdf) and attended group learning sessions.
 
-**C:** To prepare for the exam I reviewed the official dbt Certification Study Guide and the [official dbt docs](https://docs.getdbt.com/), and attended group study and learning sessions that were hosted by Montreal Analytics for all employees interested in taking the exam. As a group, we prioritized subjects that we felt less familiar with; for the first cohort of test takers this was mainly newer topics that haven’t yet become integral to a typical dbt project, such as [doc blocks](https://docs.getdbt.com/docs/collaborate/documentation#using-docs-blocks) and [configurations versus properties](https://docs.getdbt.com/reference/configs-and-properties). These sessions mainly covered the highlights and common “gotchas” that are experienced using these techniques. The sessions were moderated by a team member who had already successfully completed the dbt Certification, but operated in a very collaborative environment, so everyone could provide additional information, ask questions to the group, and provide feedback to other members of our certification taking group.
+**C:** To prepare for the exam I reviewed the official dbt Certification Study Guide and the [official dbt docs](https://docs.getdbt.com/), and attended group study and learning sessions that were hosted by Montreal Analytics for all employees interested in taking the exam. As a group, we prioritized subjects that we felt less familiar with; for the first cohort of test takers this was mainly newer topics that haven’t yet become integral to a typical dbt project, such as [doc blocks](https://docs.getdbt.com/docs/build/documentation#using-docs-blocks) and [configurations versus properties](https://docs.getdbt.com/reference/configs-and-properties). These sessions mainly covered the highlights and common “gotchas” that are experienced using these techniques. The sessions were moderated by a team member who had already successfully completed the dbt Certification, but operated in a very collaborative environment, so everyone could provide additional information, ask questions to the group, and provide feedback to other members of our certification taking group.
 
 I felt comfortable with the breadth of my dbt knowledge and had familiarity with most topics. However in my day-to-day implementation, I am often reliant on documentation or copying and pasting specific configurations in order to get the correct settings. Therefore, my focus was on memorizing important criteria for *how to use* certain features, particularly on the order/nesting of how the key YAML files are configured (dbt_project.yml, table.yml, source.yml).
 
@@ -75,4 +75,4 @@ Now, the first thing you must do when you’ve passed a test is to get yourself
 Standards and best practices are very important, but a test is a measure at a single point in time of a rapidly evolving industry. It’s also a measure of my test-taking abilities, my stress levels, and other things unrelated to my skill in data modeling; I wouldn’t be a good analyst if I didn’t recognize the faults of a measurement. I’m glad to have this check mark completed, but I will continue to stay up to date with changes, learn new data skills and techniques, and find ways to continue being a holistically helpful teammate to my colleagues and clients.
 
 
-You can learn more about the dbt Certification [here](https://www.getdbt.com/blog/dbt-certification-program/).
+You can learn more about the dbt Certification [here](https://www.getdbt.com/blog/dbt-certification-program/).

website/blog/2023-05-04-generating-dynamic-docs.md

@@ -215,7 +215,7 @@ Which in turn can be copy-pasted into a new `.yml` file. In our example, we writ
 
 ## Create docs blocks for the new columns
 
-[Docs blocks](https://docs.getdbt.com/docs/collaborate/documentation#using-docs-blocks) can be utilized to write more DRY and robust documentation. To use docs blocks, update your folder structure to contain a `.md` file. Your file structure should now look like this:
+[Docs blocks](https://docs.getdbt.com/docs/build/documentation#using-docs-blocks) can be utilized to write more DRY and robust documentation. To use docs blocks, update your folder structure to contain a `.md` file. Your file structure should now look like this:
 
 ```
 models/core/activity_based_interest

website/docs/best-practices/how-we-structure/6-the-rest-of-the-project.md

@@ -50,7 +50,7 @@ When structuring your YAML configuration files in a dbt project, you want to bal
   - The leading underscore ensures your YAML files will be sorted to the top of every folder to make them easy to separate from your models.
   - YAML files don’t need unique names in the way that SQL model files do, but including the directory (instead of simply `_sources.yml` in each folder), means you can fuzzy find the right file more quickly.
   - We’ve recommended several different naming conventions over the years, most recently calling these `schema.yml` files. We’ve simplified to recommend that these simply be labelled based on the YAML dictionary that they contain.
-  - If you utilize [doc blocks](https://docs.getdbt.com/docs/collaborate/documentation#using-docs-blocks) in your project, we recommend following the same pattern, and creating a `_[directory]__docs.md` markdown file per directory containing all your doc blocks for that folder of models.
+  - If you utilize [doc blocks](https://docs.getdbt.com/docs/build/documentation#using-docs-blocks) in your project, we recommend following the same pattern, and creating a `_[directory]__docs.md` markdown file per directory containing all your doc blocks for that folder of models.
 - ❌ **Config per project.** Some people put _all_ of their source and model YAML into one file. While you can technically do this, and while it certainly simplifies knowing what file the config you’re looking for will be in (as there is only one file), it makes it much harder to find specific configurations within that file. We recommend balancing those two concerns.
 - ⚠️ **Config per model.** On the other end of the spectrum, some people prefer to create one YAML file per model. This presents less of an issue than a single monolith file, as you can quickly search for files, know exactly where specific configurations exist, spot models without configs (and thus without tests) by looking at the file tree, and various other advantages. In our opinion, the extra files, tabs, and windows this requires creating, copying from, pasting to, closing, opening, and managing creates a somewhat slower development experience that outweighs the benefits. Defining config per directory is the most balanced approach for most projects, but if you have compelling reasons to use config per model, there are definitely some great projects that follow this paradigm.
 - ✅ **Cascade configs.** Leverage your `dbt_project.yml` to set default configurations at the directory level. Use the well-organized folder structure we’ve created thus far to define the baseline schemas and materializations, and use dbt’s cascading scope priority to define variations to this. For example, as below, define your marts to be materialized as tables by default, define separate schemas for our separate subfolders, and any models that need to use incremental materialization can be defined at the model level.

website/docs/docs/collaborate/documentation.md → website/docs/docs/build/documentation.md

@@ -1,11 +1,12 @@
 ---
-title: "About documentation"
+title: "Documentation"
 description: "Learn how good documentation for your dbt models helps stakeholders discover and understand your datasets."
 id: "documentation"
-pagination_next: "docs/collaborate/build-and-view-your-docs"
-pagination_prev: null
 ---
 
+Good documentation for your dbt models will help downstream consumers discover and understand the datasets you curate for them.
+dbt provides a way to generate documentation for your dbt project and render it as a website.
+
 ## Related documentation
 
 * [Declaring properties](/reference/configs-and-properties)
@@ -19,18 +20,12 @@ pagination_prev: null
 
 ## Overview
 
-Good documentation for your dbt models will help downstream consumers discover and understand the datasets which you curate for them.
-
-dbt provides a way to generate documentation for your dbt project and render it as a website. The documentation for your project includes:
+dbt provides a way to generate documentation for your dbt project. The documentation for your project includes:
 * **Information about your project**: including model code, a DAG of your project, any tests you've added to a column, and more.
 * **Information about your <Term id="data-warehouse" />**: including column data types, and <Term id="table" /> sizes. This information is generated by running queries against the information schema.
 
 Importantly, dbt also provides a way to add **descriptions** to models, columns, sources, and more, to further enhance your documentation.
 
-Here's an example docs site:
-
-<Lightbox src="/img/docs/building-a-dbt-project/dbt-docs-screenshot.png" title="Auto-generated dbt documentation website"/>
-
 ## Adding descriptions to your project
 To add descriptions to your project, use the `description:` key in the same files where you declare [tests](/docs/build/data-tests), like so:
 
@@ -60,13 +55,19 @@ models:
 
 </File>
 
-
 ## Generating project documentation
-You can generate a documentation site for your project (with or without descriptions) using the CLI.
 
-First, run `dbt docs generate` — this command tells dbt to compile relevant information about your dbt project and warehouse into `manifest.json` and `catalog.json` files respectively. To see the documentation for all columns and not just columns described in your project, ensure that you have created the models with `dbt run` beforehand.
+The default documentation experience in dbt Cloud is [dbt Explorer](/docs/collaborate/explore-projects), available on [Team or Enterprise plans](https://www.getdbt.com/pricing/). Use dbt Explorer to view your project's resources (such as models, tests, and metrics), its [metadata](/docs/collaborate/explore-projects#generate-metadata), and their lineage to gain a better understanding of its latest production state.
 
-Then, run `dbt docs serve` to use these `.json` files to populate a local website.
+dbt Cloud developer and dbt Core users can use [dbt Docs](/docs/collaborate/build-and-view-your-docs#dbt-docs), which generates basic documentation, but it doesn't offer the same speed, metadata, or visibility as dbt Explorer. 
+
+Generate documentation for you project by following these steps:
+
+1. Run `dbt docs generate` — this command tells dbt to compile relevant information about your dbt project and warehouse into `manifest.json` and `catalog.json` files, respectively. 
+2. Ensure that you have created the models with `dbt run` to view the documentation for all columns, not just those described in your project.
+3. Run `dbt docs serve` if you're developing locally to use these `.json` files to populate a local website.
+
+To view a resource, its metadata, and what commands are needed in dbt Explorer, refer to [generate metadata](/docs/collaborate/explore-projects#generate-metadata) for more details.
 
 ## FAQs
 <FAQ path="Project/example-projects" alt_header="Are there any example dbt documentation sites?"/>
@@ -75,8 +76,7 @@ Then, run `dbt docs serve` to use these `.json` files to populate a local websit
 <FAQ path="Docs/sharing-documentation" />
 <FAQ path="Docs/document-other-resources" />
 
-
-## Using Docs Blocks
+## Using docs blocks
 ### Syntax
 To declare a docs block, use the jinja `docs` tag. Docs blocks can contain arbitrary markdown, but they must be uniquely named. Their names may contain uppercase and lowercase letters (A-Z, a-z), digits (0-9), and underscores (_), but can't start with a digit.
 
@@ -128,9 +128,11 @@ models:
 
 In the resulting documentation, `'{{ doc("table_events") }}'` will be expanded to the markdown defined in the `table_events` docs block.
 
+
 ## Setting a custom overview
+*Currently available for dbt Docs only.*
 
-The "overview" shown in the documentation website can be overridden by supplying your own docs block called `__overview__`. By default, dbt supplies an overview with helpful information about the docs site itself. Depending on your needs, it may be a good idea to override this docs block with specific information about your company style guide, links to reports, or information about who to contact for help. To override the default overview, create a docs block that looks like this:
+The "overview" shown in the dbt Docs website can be overridden by supplying your own docs block called `__overview__`. By default, dbt supplies an overview with helpful information about the docs site itself. Depending on your needs, it may be a good idea to override this docs block with specific information about your company style guide, links to reports, or information about who to contact for help. To override the default overview, create a docs block that looks like this:
 
 <File name='models/overview.md'>
 
@@ -148,6 +150,7 @@ as well as the repo for this project \[here](https://github.com/dbt-labs/mrr-pla
 </File>
 
 ### Custom project-level overviews
+*Currently available for dbt Docs only.*
 
 You can set different overviews for each dbt project/package included in your documentation site
 by creating a docs block named `__[project_name]__`. For example, in order to define
@@ -174,31 +177,46 @@ up to page views and sessions.
 </File>
 
 ## Navigating the documentation site
-Using the docs interface, you can navigate to the documentation for a specific model. That might look something like this:
+
+Use [dbt Explorer](/docs/collaborate/explore-projects) for a richer documentation experience and more interactive experience for understanding your project's resources and lineage. Available on [Team or Enterprise plans](https://www.getdbt.com/pricing/).
+
+For additional details on how to explore your lineage and navigate your resources, refer to [dbt Explorer](/docs/collaborate/explore-projects).
+
+<Lightbox src="/img/docs/collaborate/dbt-explorer/example-model-details.png" width="100%" title="Example of resource details" />
+
+<Expandable alt_header="For dbt Docs">
+If you're using the dbt Docs interface, you can navigate to the documentation for a specific model. That might look something like this:
 
 <Lightbox src="/img/docs/building-a-dbt-project/testing-and-documentation/f2221dc-Screen_Shot_2018-08-14_at_6.29.55_PM.png" title="Auto-generated documentation for a dbt model"/>
 
 Here, you can see a representation of the project structure, a markdown description for a model, and a list of all of the columns (with documentation) in the model.
 
-From a docs page, you can click the green button in the bottom-right corner of the webpage to expand a "mini-map" of your DAG. This pane (shown below) will display the immediate parents and children of the model that you're exploring.
+From the dbt Docs page, you can click the green button in the bottom-right corner of the webpage to expand a "mini-map" of your DAG. This pane (shown below) will display the immediate parents and children of the model that you're exploring.
 
 <Lightbox src="/img/docs/building-a-dbt-project/testing-and-documentation/ec77c45-Screen_Shot_2018-08-14_at_6.31.56_PM.png" title="Opening the DAG mini-map"/>
 
 In this example, the `fct_subscription_transactions` model only has one direct parent. By clicking the "Expand" button in the top-right corner of the window, we can pivot the graph horizontally and view the full <Term id="data-lineage">lineage</Term> for our model. This lineage is filterable using the `--select` and `--exclude` flags, which are consistent with the semantics of [model selection syntax](/reference/node-selection/syntax). Further, you can right-click to interact with the DAG, jump to documentation, or share links to your graph visualization with your coworkers.
 
 <Lightbox src="/img/docs/building-a-dbt-project/testing-and-documentation/ac97fba-Screen_Shot_2018-08-14_at_6.35.14_PM.png" title="The full lineage for a dbt model"/>
 
+</Expandable>
+
 ## Deploying the documentation site
 
+With dbt Cloud, use [dbt Explorer](/docs/collaborate/explore-projects) automatically retrieves the metadata updates after each job run in the production or staging deployment environment so it always has the latest results for your project &mdash; meaning it's always automatically updated after each job run.
+
 :::caution Security
 
 The `dbt docs serve` command is only intended for local/development hosting of the documentation site. Please use one of the methods listed below (or similar) to ensure that your documentation site is hosted securely!
 
 :::
 
+#### For dbt Docs users
+
 dbt's documentation website was built to make it easy to host on the web. The site is "static,” meaning you don't need any "dynamic" servers to serve the docs. You can host your documentation in several ways:
 
-* Use [dbt Cloud](/docs/collaborate/documentation)
+* Use [dbt Cloud's](/docs/collaborate/build-and-view-your-docs) default documentation experience with [dbt Explorer](/docs/collaborate/explore-projects).
 * Host on [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteHosting.html) (optionally [with IP access restrictions](https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-3))
 * Publish with [Netlify](https://discourse.getdbt.com/t/publishing-dbt-docs-to-netlify/121)
 * Use your own web server like Apache/Nginx
+