Reorganize beta docs (pt 1) (#26019)

**NOTE:** Merge #26067 before merging this PR. ## Summary & Motivation I'm a fair way through auditing old and new docs content, and have started working on figuring out how to plug the remaining gaps in new docs. To that end, I reorganized the /guides section of new docs to make it easier to move old content to the new site. Big changes I made in this PR: * Moved docs from the `concepts` directory into `guides` subdirectories (`automate`, `build`, `deploy`, `monitor`, `test`). The `guides` docs already contain a lot of conceptual content; the distinction between `concepts` and `guides` seemed fairly arbitrary and not helpful for readers. * Added a top-level glossary page that can serve as a quick reference to documentation about key concepts, so we don't lose that aspect of having a separate concepts section. (This is currently an unlisted draft page that will be finished after more content has been moved over.) * Updated information architecture: * Moved docs into a directory structure that mirrors the information architecture of the website/URL structure to make it easier for readers to orient themselves in the docs and for contributors to reason about where to put stuff. * Added index pages to docs subsections so support engineerins, SEs, and others can send users and customers links to logical chunks of information. * Updated `sidebars.ts` to use autogenerated navigation whenever possible. This will hopefully make contributing to docs a little easier -- if someone adds a new page to a section, they don't need to go into `sidebars.ts` and manually add the page name to its section to make it visible. (Entire new sections do need to be added manually, though.) * Used `sidebar_position` in the front matter to organize the position of docs in the sidebar. Index pages use multiples of 10, while pages use multiples of 100. This give us a bit of space between sections and pages to add content as needed. ## How I Tested These Changes Local build. ## Changelog Updated beta docs directory structure, added index pages to subsections, updated page positioning via sidebars.ts and sidebar_position in page front matter, removed references section and moved YAML reference docs to guides/deploy. --------- Signed-off-by: nikki everett <[email protected]> Co-authored-by: colton <[email protected]>
dagster-io · Dec 5, 2024 · f42dc5f · f42dc5f · github-actions · Dec 5, 2024
1 parent b3df320
commit f42dc5f
Show file tree

Hide file tree

Showing 175 changed files with 624 additions and 833 deletions.
diff --git a/docs/docs-beta/.gitignore b/docs/docs-beta/.gitignore
@@ -3,6 +3,7 @@
 
 # Production
 /build
+!/docs/guides/build
 
 # Generated files
 .docusaurus

diff --git a/docs/docs-beta/README.md b/docs/docs-beta/README.md
@@ -15,8 +15,6 @@ To install dependencies:
 yarn install
 ```
 
-**Note**: The yarn binary is checked in, so you do not need to install yarn yourself.
-
 It also uses [Vale](https://vale.sh/) to check for issues in the documentation.
 
 Install Vale with:
@@ -44,7 +42,6 @@ The docs are broken down into the following sections:
 
 - [Tutorials](./docs/tutorials/)
 - [Guides](./docs/guides/)
-- [Concepts](./docs/concepts/)
 
 `sidebar.ts` and `docusaurus.config.ts` are the main configuration files for the documentation.
 
@@ -116,7 +113,7 @@ To build the site for production:
 yarn build
 ```
 
-This command generates static content into the `build` directory and can be served using any static contents hosting service. This also checks for any broken links in the documentation.
+This command generates static content into the `build` directory and can be served using any static contents hosting service. This also checks for any broken links in the documentation. Note that you will need to store Algolia credentials in local environment variables to build the site for production.
 
 ## Deployment
 

diff --git a/docs/docs-beta/docs/api/index.mdx b/docs/docs-beta/docs/api/index.mdx
@@ -1,6 +1,6 @@
 ---
 sidebar_class_name: hidden
-title: API Docs
+title: API reference
 ---
 
 import DocCardList from '@theme/DocCardList';

diff --git a/docs/docs-beta/docs/concepts/assets.md b/docs/docs-beta/docs/concepts/assets.md
diff --git a/docs/docs-beta/docs/concepts/assets/asset-checks.md b/docs/docs-beta/docs/concepts/assets/asset-checks.md
diff --git a/docs/docs-beta/docs/concepts/assets/asset-dependencies.md b/docs/docs-beta/docs/concepts/assets/asset-dependencies.md
diff --git a/docs/docs-beta/docs/concepts/assets/thinking-in-assets.md b/docs/docs-beta/docs/concepts/assets/thinking-in-assets.md
diff --git a/docs/docs-beta/docs/concepts/automation/declarative-automation.md b/docs/docs-beta/docs/concepts/automation/declarative-automation.md
diff --git a/docs/docs-beta/docs/concepts/automation/schedules.md b/docs/docs-beta/docs/concepts/automation/schedules.md
diff --git a/docs/docs-beta/docs/concepts/automation/sensors.md b/docs/docs-beta/docs/concepts/automation/sensors.md
diff --git a/docs/docs-beta/docs/concepts/execution.md b/docs/docs-beta/docs/concepts/execution.md
diff --git a/docs/docs-beta/docs/concepts/io-managers.md b/docs/docs-beta/docs/concepts/io-managers.md
diff --git a/docs/docs-beta/docs/concepts/ops-jobs.md b/docs/docs-beta/docs/concepts/ops-jobs.md
diff --git a/docs/docs-beta/docs/concepts/partitions.md b/docs/docs-beta/docs/concepts/partitions.md
diff --git a/docs/docs-beta/docs/concepts/resources.md b/docs/docs-beta/docs/concepts/resources.md
diff --git a/docs/docs-beta/docs/concepts/schedules.mdx b/docs/docs-beta/docs/concepts/schedules.mdx
diff --git a/docs/docs-beta/docs/concepts/sensors.md b/docs/docs-beta/docs/concepts/sensors.md
diff --git a/docs/docs-beta/docs/concepts/understanding-assets.md b/docs/docs-beta/docs/concepts/understanding-assets.md
diff --git a/docs/docs-beta/docs/dagster-plus.md b/docs/docs-beta/docs/dagster-plus.md
diff --git a/docs/docs-beta/docs/dagster-plus/access/authentication/scim-provisioning.md b/docs/docs-beta/docs/dagster-plus/access/authentication/scim-provisioning.md
diff --git a/docs/docs-beta/docs/dagster-plus/access/rbac.md b/docs/docs-beta/docs/dagster-plus/access/rbac.md
diff --git a/docs/docs-beta/docs/dagster-plus/deployment/branch-deployments.md b/docs/docs-beta/docs/dagster-plus/deployment/branch-deployments.md
diff --git a/docs/docs-beta/docs/dagster-plus/deployment/branch-deployments/change-tracking.md b/docs/docs-beta/docs/dagster-plus/deployment/branch-deployments/change-tracking.md
diff --git a/.../docs-beta/docs/dagster-plus/deployment/branch-deployments/dagster-cloud-cli.md b/.../docs-beta/docs/dagster-plus/deployment/branch-deployments/dagster-cloud-cli.md
diff --git a/docs/docs-beta/docs/dagster-plus/deployment/branch-deployments/testing.md b/docs/docs-beta/docs/dagster-plus/deployment/branch-deployments/testing.md
diff --git a/.../docs-beta/docs/dagster-plus/deployment/code-locations/code-location-history.md b/.../docs-beta/docs/dagster-plus/deployment/code-locations/code-location-history.md
@@ -1,7 +1,7 @@
 ---
 title: "Code location history and rollbacks"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 4
+sidebar_position: 100
 sidebar_label: "Code location history and rollbacks"
 ---
 
@@ -54,4 +54,4 @@ If you notice an issue with newly deployed code, or your code fails to deploy su
 ## Next steps
 
 - Learn more about [Code Locations](/dagster-plus/deployment/code-locations)
-- Learn how to [Alert when a code location fails to load](/dagster-plus/deployment/alerts#alerting-when-a-code-location-fails-to-load)
+- Learn how to [Alert when a code location fails to load](/dagster-plus/features/alerts#alerting-when-a-code-location-fails-to-load)
diff --git a/...dagster-plus/deployment/code-locations.md → ...r-plus/deployment/code-locations/index.md b/...dagster-plus/deployment/code-locations.md → ...r-plus/deployment/code-locations/index.md
@@ -1,6 +1,7 @@
 ---
 title: "Code locations"
 displayed_sidebar: "dagsterPlus"
+sidebar_position: 50
 ---
 
 # Code locations

diff --git a/docs/docs-beta/docs/dagster-plus/deployment/code-requirements.md b/docs/docs-beta/docs/dagster-plus/deployment/code-requirements.md
@@ -2,6 +2,7 @@
 title: 'Dagster+ code requirements'
 displayed_sidebar: 'dagsterPlus'
 sidebar_label: "Code requirements"
+sidebar_position: 100
 ---
 
 Your Dagster project must meet a few requirements to run in Dagster+.

diff --git a/docs/docs-beta/docs/dagster-plus/settings.md → ...-plus/deployment/dagster-plus-settings.md b/docs/docs-beta/docs/dagster-plus/settings.md → ...-plus/deployment/dagster-plus-settings.md
@@ -1,6 +1,7 @@
 ---
 title: "Dagster+ settings"
 unlisted: true
+sidebar_position: 200
 ---
 
 # Dagster+ settings
diff --git a/.../docs/dagster-plus/deployment/settings.md → ...er-plus/deployment/deployment-settings.md b/.../docs/dagster-plus/deployment/settings.md → ...er-plus/deployment/deployment-settings.md
@@ -1,8 +1,9 @@
 ---
 title: "Deployment settings"
 displayed_sidebar: "dagsterPlus"
-sidebar_label: "Settings"
+sidebar_label: "Deployment settings"
 unlisted: true
+sidebar_position: 300
 ---
 
 # Deployment settings
diff --git a/docs/docs-beta/docs/dagster-plus/deployment/deployment-types.md b/docs/docs-beta/docs/dagster-plus/deployment/deployment-types.md
@@ -1,5 +1,7 @@
 ---
+title: Deployment types
 unlisted: true
+sidebar_position: 400
 ---
 
-## Placeholder
+# Deployment types
diff --git a/docs/docs-beta/docs/dagster-plus/deployment/environment-variables/agent-config.md b/docs/docs-beta/docs/dagster-plus/deployment/environment-variables/agent-config.md
@@ -1,7 +1,7 @@
 ---
 title: "Set environment variables using agent config"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 2
+sidebar_position: 300
 sidebar_label: "Set with agent config"
 unlisted: true
 ---

diff --git a/docs/docs-beta/docs/dagster-plus/deployment/environment-variables/built-in.md b/docs/docs-beta/docs/dagster-plus/deployment/environment-variables/built-in.md
@@ -1,7 +1,7 @@
 ---
 title: "Built-in environment variables"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 3
+sidebar_position: 100
 sidebar_label: "Built-in variables"
 ---
 

diff --git a/docs/docs-beta/docs/dagster-plus/deployment/environment-variables/dagster-ui.md b/docs/docs-beta/docs/dagster-plus/deployment/environment-variables/dagster-ui.md
@@ -1,7 +1,7 @@
 ---
 title: "Setting environment variables with the Dagster+ UI"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 1
+sidebar_position: 200
 sidebar_label: "Set with Dagster+ UI"
 ---
 

diff --git a/...-plus/deployment/environment-variables.md → ...eployment/environment-variables/index.mdx b/...-plus/deployment/environment-variables.md → ...eployment/environment-variables/index.mdx
@@ -2,6 +2,11 @@
 title: "Environment variables"
 displayed_sidebar: "dagsterPlus"
 unlisted: true
+sidebar_position: 40
 ---
 
 # Environment variables
+
+import DocCardList from '@theme/DocCardList';
+
+<DocCardList />
diff --git a/...docs-beta/docs/dagster-plus/deployment/hybrid/agents/amazon-ecs-existing-vpc.md b/...docs-beta/docs/dagster-plus/deployment/hybrid/agents/amazon-ecs-existing-vpc.md
@@ -1,7 +1,7 @@
 ---
 title: "Amazon ECS agents (Existing VPC)"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 11
-sidebar_label: "Amazon ECS (existing)"
+sidebar_position: 110
+sidebar_label: "Amazon ECS (Existing VPC)"
 unlisted: true
 ---
diff --git a/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/amazon-ecs-new-vpc.md b/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/amazon-ecs-new-vpc.md
@@ -1,7 +1,7 @@
 ---
 title: "Amazon ECS agents (New VPC)"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 10
-sidebar_label: "Amazon ECS (new)"
+sidebar_position: 100
+sidebar_label: "Amazon ECS (New VPC)"
 unlisted: true
 ---
diff --git a/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/docker.md b/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/docker.md
@@ -1,7 +1,7 @@
 ---
 title: "Docker agents"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 20
+sidebar_position: 200
 sidebar_label: "Docker"
 unlisted: true
 ---

diff --git a/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/index.mdx b/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/index.mdx
@@ -0,0 +1,11 @@
+---
+title: "Agents"
+displayed_sidebar: "dagsterPlus"
+sidebar_class_name: hidden
+---
+
+# Agents
+
+import DocCardList from '@theme/DocCardList';
+
+<DocCardList />
diff --git a/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/kubernetes.md b/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/kubernetes.md
@@ -1,7 +1,7 @@
 ---
 title: "Running Dagster+ agents on Kubernetes"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 30
+sidebar_position: 300
 sidebar_label: "Kubernetes"
 ---
 
@@ -95,7 +95,7 @@ An exhaustive list of settings is available [here](/dagster-plus/deployment/hybr
 
 ### Configure your agents to serve branch deployments
 
-[Branch deployments](/dagster-plus/deployment/branch-deployments) are lightweight staging environments created for each code change. To configure your Dagster+ agent to manage them:
+[Branch deployments](/dagster-plus/features/branch-deployments/index.mdx) are lightweight staging environments created for each code change. To configure your Dagster+ agent to manage them:
 
 ```yaml
 # values.yaml

diff --git a/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/local.md b/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/local.md
@@ -1,7 +1,7 @@
 ---
 title: "Running a Dagster+ agent locally"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 40
+sidebar_position: 600
 sidebar_label: "Local"
 ---
 

diff --git a/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/multiple.md b/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/multiple.md
@@ -1,7 +1,7 @@
 ---
 title: "Using multiple agents"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 50
+sidebar_position: 400
 unlisted: true
 ---
 

diff --git a/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/settings.md b/docs/docs-beta/docs/dagster-plus/deployment/hybrid/agents/settings.md
@@ -1,7 +1,7 @@
 ---
 title: "Hybrid agent settings"
 displayed_sidebar: "dagsterPlus"
-sidebar_position: 60
+sidebar_position: 700
 sidebar_label: "Settings"
 unlisted: true
 ---

diff --git a/docs/docs-beta/docs/dagster-plus/deployment/hybrid/architecture.md b/docs/docs-beta/docs/dagster-plus/deployment/hybrid/architecture.md
@@ -1,7 +1,7 @@
 ---
 title: 'Dagster+ Hybrid architecture'
 displayed_sidebar: 'dagsterPlus'
-sidebar_position: 10
+sidebar_position: 100
 ---
 
 # Dagster+ Hybrid architecture