more broken links. ty unit tests

docs/content/concepts/dagster-pipes.mdx

@@ -74,12 +74,12 @@ While Dagster Pipes is lightweight and flexible, there are a few limitations to
 
 Ready to get started with Dagster Pipes? Depending on what your goal is, how you approach using Dagster Pipes can differ.
 
-- **If you’re writing scripts that are being orchestrated**, all you’ll need to do is lightly modify your existing scripts. Check out the [Dagster Pipes tutorial](/guides/dagster-pipes/subprocess) to get up and running.
+- **If you’re writing scripts that are being orchestrated**, all you’ll need to do is lightly modify your existing scripts. Check out the [Dagster Pipes tutorial](/concepts/dagster-pipes/subprocess) to get up and running.
 
 - **If you need to orchestrate scripts others have written**, you can use our ready-to-go integrations to execute scripts in your chosen technology:
 
-  - [Amazon Web Services Lambda](/guides/dagster-pipes/aws-lambda)
-  - [Databricks](/guides/dagster-pipes/databricks)
-  - [Kubernetes](/guides/dagster-pipes/kubernetes)
+  - [Amazon Web Services Lambda](/concepts/dagster-pipes/aws-lambda)
+  - [Databricks](/concepts/dagster-pipes/databricks)
+  - [Kubernetes](/concepts/dagster-pipes/kubernetes)
 
-- **If you don’t see your integration or you want to fully customize your Pipes experience**, check out the [Dagster Pipes details and customization guide](/guides/dagster-pipes/dagster-pipes-details-and-customization) to learn how to create a custom experience.
+- **If you don’t see your integration or you want to fully customize your Pipes experience**, check out the [Dagster Pipes details and customization guide](/concepts/dagster-pipes/dagster-pipes-details-and-customization) to learn how to create a custom experience.

docs/content/concepts/dagster-pipes/aws-lambda.mdx

@@ -14,7 +14,7 @@ description: "Learn to integrate Dagster Pipes with AWS Lambda to launch externa
   instead.
 </Note>
 
-In this guide, we’ll show you how to use [Dagster Pipes](/guides/dagster-pipes) with Dagster’s AWS Lambda integration to invoke a Lambda function and execute external code.
+In this guide, we’ll show you how to use [Dagster Pipes](/concepts/dagster-pipes) with Dagster’s AWS Lambda integration to invoke a Lambda function and execute external code.
 
 Pipes allows your code to interact with Dagster outside of a full Dagster environment. Instead, the environment only needs to contain `dagster-pipes`, a single-file Python package with no dependencies that can be installed from PyPI or easily vendored. `dagster-pipes` handles streaming `stdout`/`stderr` and Dagster events back to the orchestration process.
 
@@ -64,10 +64,10 @@ For simplicity we're demonstrating this step using the UI, but you can also do t
 After the function is created, you'll be redirected to a page like the following:
 
 <Image
-alt="The Lambda function details page in the AWS UI"
-src="/images/guides/dagster-pipes/aws-lambda/aws-lambda-function-details.png"
-width={1920}
-height={1153}
+  alt="The Lambda function details page in the AWS UI"
+  src="/images/guides/dagster-pipes/aws-lambda/aws-lambda-function-details.png"
+  width={1920}
+  height={1153}
 />
 
 ### Step 1.2: Add a dagster-pipes file
@@ -86,10 +86,10 @@ Next, you'll add `dagster-pipes` to the function.
 1. In the **Code source** section of the page, add a new file. This can be accomplished with **File > New file** or by clicking the green **+** icon next to the open `lambda_function` tab:
 
    <Image
-   alt="Highlighted New file option in a menu of the Code source section on the Lambda function details page"
-   src="/images/guides/dagster-pipes/aws-lambda/aws-lambda-create-new-file.png"
-   width={2386}
-   height={1196}
+     alt="Highlighted New file option in a menu of the Code source section on the Lambda function details page"
+     src="/images/guides/dagster-pipes/aws-lambda/aws-lambda-create-new-file.png"
+     width={2386}
+     height={1196}
    />
 
 2. **In a new browser tab**, navigate to the following URL:
@@ -101,10 +101,10 @@ Next, you'll add `dagster-pipes` to the function.
 3. Copy the contents of `__init__.py` into the new file you created in AWS. **Note**: We recommend adding the source link and the date you copied the contents to the top of the file as comments:
 
    <Image
-   alt="The copied contents of the Dagster Pipes file into a file in the AWS UI"
-   src="/images/guides/dagster-pipes/aws-lambda/aws-lambda-add-dagster-pipes.png"
-   width={2390}
-   height={1090}
+     alt="The copied contents of the Dagster Pipes file into a file in the AWS UI"
+     src="/images/guides/dagster-pipes/aws-lambda/aws-lambda-add-dagster-pipes.png"
+     width={2390}
+     height={1090}
    />
 
 4. Save the file as `dagster_pipes.py`.
@@ -154,7 +154,7 @@ Let's review what this code does:
 
   We're using the default context loader (<PyObject object="PipesDefaultContextLoader" module="dagster_pipes" />) and message writer (<PyObject object="PipesDefaultMessageWriter" module="dagster_pipes" />) in this example. These objects establish communication between the orchestration and external process. On the orchestration end, these match a corresponding `PipesLambdaEventContextInjector` and `PipesLambdaLogsMessageReader`, which are instantiated inside the <PyObject module="dagster_aws.pipes" object="PipesLambdaClient" />.
 
-- **Inside the body of the context manager (<PyObject object="open_dagster_pipes" module="dagster_pipes" />), retrieve a log and report an asset materialization.** These calls use the temporary communications channels established by <PyObject object="PipesDefaultContextLoader" module="dagster_pipes" /> and <PyObject object="PipesDefaultMessageWriter" module="dagster_pipes" />. To see the full range of what you can do with the <PyObject object="PipesContext" module="dagster_pipes" />, see the API docs or the general [Pipes documentation](/guides/dagster-pipes).
+- **Inside the body of the context manager (<PyObject object="open_dagster_pipes" module="dagster_pipes" />), retrieve a log and report an asset materialization.** These calls use the temporary communications channels established by <PyObject object="PipesDefaultContextLoader" module="dagster_pipes" /> and <PyObject object="PipesDefaultMessageWriter" module="dagster_pipes" />. To see the full range of what you can do with the <PyObject object="PipesContext" module="dagster_pipes" />, see the API docs or the general [Pipes documentation](/concepts/dagster-pipes).
 
 At this point you can execute the rest of your AWS Lambda code as normal, invoking various <PyObject object="PipesContext" module="dagster_pipes" /> APIs as needed.
 
@@ -244,10 +244,10 @@ In this step, you’ll invoke the AWS Lambda function you defined in [Step 1](#s
    <!-- ![Event log for AWS Lambda run](/images/guides/dagster-pipes/aws-lambda/aws-lambda-dagster-ui.png) -->
 
    <Image
-   alt="Event log for AWS Lambda run"
-   src="/images/guides/dagster-pipes/aws-lambda/aws-lambda-dagster-ui.png"
-   width={1501}
-   height={1416}
+     alt="Event log for AWS Lambda run"
+     src="/images/guides/dagster-pipes/aws-lambda/aws-lambda-dagster-ui.png"
+     width={1501}
+     height={1416}
    />
 
 ---

docs/content/concepts/dagster-pipes/dagster-pipes-details-and-customization.mdx

@@ -5,7 +5,7 @@ description: "Learn about Dagster Pipes APIs and how to compose them to create a
 
 # Dagster Pipes details and customization
 
-[Dagster Pipes](/guides/dagster-pipes) is a toolkit for integrating Dagster with an arbitrary external compute environment. While many users will be well-served by the simplified interface offered by Pipes client objects (e.g. <PyObject object="PipesSubprocessClient" />, <PyObject object="PipesDatabricksClient" module="dagster_databricks"/>), others will need a greater level of control over Pipes. This is particularly the case for users seeking to connect large existing codebases to Dagster.
+[Dagster Pipes](/concepts/dagster-pipes) is a toolkit for integrating Dagster with an arbitrary external compute environment. While many users will be well-served by the simplified interface offered by Pipes client objects (e.g. <PyObject object="PipesSubprocessClient" />, <PyObject object="PipesDatabricksClient" module="dagster_databricks"/>), others will need a greater level of control over Pipes. This is particularly the case for users seeking to connect large existing codebases to Dagster.
 
 This guide will cover the lower level Pipes APIs and how you can compose them to provide a custom solution for your data platform.
 
@@ -16,10 +16,10 @@ This guide will cover the lower level Pipes APIs and how you can compose them to
 <!-- ![Detailed overview of a Dagster Pipes session](/images/guides/dagster-pipes/pipes-overview.png) -->
 
 <Image
-alt="Detailed overview of a Dagster Pipes session"
-src="/images/guides/dagster-pipes/pipes-overview.png"
-width={1300}
-height={510}
+  alt="Detailed overview of a Dagster Pipes session"
+  src="/images/guides/dagster-pipes/pipes-overview.png"
+  width={1300}
+  height={510}
 />
 
 <table
@@ -177,7 +177,7 @@ A **Pipes session** is the time spanning:
 
 There are separate APIs for interacting with a Pipes session in the orchestration and external processes. The orchestration process API is defined in `dagster`. The external process API is defined by a Pipes integration library that is loaded by user code in the external process. This library knows how to interpret the bootstrap payload and spin up a context loader and message writer.
 
-At present the only official Dagster Pipes integration library is Python’s [`dagster-pipes`](/\_apidocs/libraries/dagster-pipes), available on [PyPI](https://pypi.org/project/dagster-pipes/). The library has no dependencies and fits in a [single file](https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-pipes/dagster_pipes/\__init\_\_.py), so it may also be trivially vendored.
+At present the only official Dagster Pipes integration library is Python’s [`dagster-pipes`](/_apidocs/libraries/dagster-pipes), available on [PyPI](https://pypi.org/project/dagster-pipes/). The library has no dependencies and fits in a [single file](https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-pipes/dagster_pipes/__init__.py), so it may also be trivially vendored.
 
 ### Session lifecycle (orchestration process)
 
@@ -251,7 +251,7 @@ For <PyObject object="open_pipes_session" />, these three parts perform the foll
 
 ### Session lifecycle (external process)
 
-As noted above, currently the only existing Pipes integration library is Python’s [`dagster-pipes`](/\_apidocs/libraries/dagster-pipes). The below example therefore uses Python and `dagster-pipes`. In the future we will be releasing `dagster-pipes` equivalents for selected other languages. and the concepts illustrated here should map straightforwardly to these other integration libraries.
+As noted above, currently the only existing Pipes integration library is Python’s [`dagster-pipes`](/_apidocs/libraries/dagster-pipes). The below example therefore uses Python and `dagster-pipes`. In the future we will be releasing `dagster-pipes` equivalents for selected other languages. and the concepts illustrated here should map straightforwardly to these other integration libraries.
 
 A Pipes session is represented in the external process by a <PyObject object="PipesContext" module="dagster_pipes" /> object. A session created by the launching orchestration process can be connected to with <PyObject object="open_dagster_pipes" module="dagster_pipes" /> from `dagster-pipes`:
 

docs/content/concepts/dagster-pipes/databricks.mdx

@@ -5,7 +5,7 @@ description: "Learn to integrate Dagster Pipes with Databricks to launch externa
 
 # Integrating Databricks with Dagster Pipes
 
-In this guide, we’ll show you how to use [Dagster Pipes](/guides/dagster-pipes) with Dagster’s Databricks integration to launch Databricks jobs.
+In this guide, we’ll show you how to use [Dagster Pipes](/concepts/dagster-pipes) with Dagster’s Databricks integration to launch Databricks jobs.
 
 Pipes allows your Databricks jobs to stream logs (including `stdout` and `stderr` of the driver process) and events back to Dagster. This does not require a full Dagster environment on Databricks; instead:
 
@@ -132,7 +132,7 @@ Let's review what's happening in this code:
 
 ### Step 1.2: Define the Databricks Pipes client and definitions
 
-The [`dagster-databricks`](/\_apidocs/libraries/dagster-databricks) library provides a <PyObject module="dagster_databricks" object="PipesDatabricksClient" />, which is a pre-built Dagster resource that allows you to quickly get Pipes working with your Databricks workspace.
+The [`dagster-databricks`](/_apidocs/libraries/dagster-databricks) library provides a <PyObject module="dagster_databricks" object="PipesDatabricksClient" />, which is a pre-built Dagster resource that allows you to quickly get Pipes working with your Databricks workspace.
 
 Add the following to the bottom of `dagster_databricks_pipes.py` to define the resource and a <PyObject object="Definitions" /> object that binds it to the `databricks_asset`:
 
@@ -210,13 +210,13 @@ with open_dagster_pipes(
 
 Before we go any futher, let's review what this script does:
 
-- **Imports `PipesDbfsContextLoader`, `PipesDbfsMessageWriter`, and `open_dagster_pipes` from `dagster_pipes`.** The <PyObject object="PipesDbfsContextLoader" module="dagster_pipes" /> and <PyObject object="PipesDbfsMessageWriter" module="dagster_pipes" /> are DBFS-specific implementations of the <PyObject object="PipesContextLoader" /> and <PyObject object="PipesMessageWriter" />. Refer to the [Dagster Pipes details and customization Guide](/guides/dagster-pipes/dagster-pipes-details-and-customization) for protocol details.
+- **Imports `PipesDbfsContextLoader`, `PipesDbfsMessageWriter`, and `open_dagster_pipes` from `dagster_pipes`.** The <PyObject object="PipesDbfsContextLoader" module="dagster_pipes" /> and <PyObject object="PipesDbfsMessageWriter" module="dagster_pipes" /> are DBFS-specific implementations of the <PyObject object="PipesContextLoader" /> and <PyObject object="PipesMessageWriter" />. Refer to the [Dagster Pipes details and customization Guide](/concepts/dagster-pipes/dagster-pipes-details-and-customization) for protocol details.
 
   Both objects write temporary files on DBFS for communication between the orchestration and external process. The <PyObject object="PipesDbfsContextLoader" module="dagster_pipes" /> and <PyObject object="PipesDbfsMessageWriter" module="dagster_pipes" /> match a corresponding `PipesDbfsContextInjector` and `PipesDbfsMessageReader` on the orchestration end, which are instantiated inside the <PyObject object="PipesDatabricksClient" module="dagster_databricks" />.
 
 - **Passes the context loader and message writer to the <PyObject object="open_dagster_pipes" module="dagster_pipes" /> context manager**, which yields an instance of <PyObject object="PipesContext" module="dagster_pipes" /> called `pipes`.
 
-  Inside the body of the context manager are various calls against `pipes` to retrieve an extra, log, and report an asset materialization. All of these calls will use the DBFS temporary file-based communications channels established by <PyObject object="PipesDbfsContextLoader" module="dagster_pipes" /> and <PyObject object="PipesDbfsMessageWriter" module="dagster_pipes" />. To see the full range of what you can do with the <PyObject object="PipesContext" module="dagster_pipes" />, see the API docs or the general [Pipes guide](/guides/dagster-pipes).
+  Inside the body of the context manager are various calls against `pipes` to retrieve an extra, log, and report an asset materialization. All of these calls will use the DBFS temporary file-based communications channels established by <PyObject object="PipesDbfsContextLoader" module="dagster_pipes" /> and <PyObject object="PipesDbfsMessageWriter" module="dagster_pipes" />. To see the full range of what you can do with the <PyObject object="PipesContext" module="dagster_pipes" />, see the API docs or the general [Pipes guide](/concepts/dagster-pipes).
 
 At this point you can execute the rest of your Databricks code as normal, invoking various <PyObject object="PipesContext" module="dagster_pipes" /> APIs as needed.
 
@@ -263,19 +263,19 @@ In this step, you’ll run the Databricks job you created in [Step 1.2](#step-12
 2. Navigate to [localhost:3000](http://localhost:3000/), where you should see the UI:
 
    <Image
-   alt="Databricks asset"
-   src="/images/guides/dagster-pipes/databricks/asset.png"
-   width={2554}
-   height={1335}
+     alt="Databricks asset"
+     src="/images/guides/dagster-pipes/databricks/asset.png"
+     width={2554}
+     height={1335}
    />
 
 3. Click **Materialize** near the top right corner of the page, then click **View** on the **Launched Run** popup. Wait for the run to complete, and the event log should look like this:
 
    <Image
-   alt="Event log for Databricks run"
-   src="/images/guides/dagster-pipes/databricks/run.png"
-   width={2554}
-   height={1335}
+     alt="Event log for Databricks run"
+     src="/images/guides/dagster-pipes/databricks/run.png"
+     width={2554}
+     height={1335}
    />
 
 ---

docs/content/concepts/dagster-pipes/kubernetes.mdx

@@ -14,7 +14,7 @@ description: "Learn to integrate Dagster Pipes with Kubernetes to launch externa
   instead.
 </Note>
 
-In this guide, we’ll show you how to use [Dagster Pipes](/guides/dagster-pipes) with Dagster’s Kubernetes integration to launch Kubernetes pods and execute external code.
+In this guide, we’ll show you how to use [Dagster Pipes](/concepts/dagster-pipes) with Dagster’s Kubernetes integration to launch Kubernetes pods and execute external code.
 
 Pipes allows your code to interact with Dagster outside of a full Dagster environment. Instead, the environment only needs to contain `dagster-pipes`, a single-file Python package with no dependencies that can be installed from PyPI or easily vendored. `dagster-pipes` handles streaming `stdout`/`stderr` and Dagster events back to the orchestration process.
 
@@ -77,7 +77,7 @@ Let's review what this code does:
 
   We're using the default context loader (<PyObject object="PipesDefaultContextLoader" module="dagster_pipes" />) and message writer (<PyObject object="PipesDefaultMessageWriter" module="dagster_pipes" />) in this example. These objects establish communication between the orchestration and external process. On the orchestration end, these match a corresponding `PipesContextInjector` and `PipesMessageReader`, which are instantiated inside the <PyObject module="dagster_k8s" object="PipesK8sClient" />.
 
-- **Inside the body of the context manager (<PyObject object="open_dagster_pipes" module="dagster_pipes" />), retrieve a log and report an asset materialization.** These calls use the temporary communications channels established by <PyObject object="PipesDefaultContextLoader" module="dagster_pipes" /> and <PyObject object="PipesDefaultMessageWriter" module="dagster_pipes" />. To see the full range of what you can do with the <PyObject object="PipesContext" module="dagster_pipes" />, see the API docs or the general [Pipes documentation](/guides/dagster-pipes).
+- **Inside the body of the context manager (<PyObject object="open_dagster_pipes" module="dagster_pipes" />), retrieve a log and report an asset materialization.** These calls use the temporary communications channels established by <PyObject object="PipesDefaultContextLoader" module="dagster_pipes" /> and <PyObject object="PipesDefaultMessageWriter" module="dagster_pipes" />. To see the full range of what you can do with the <PyObject object="PipesContext" module="dagster_pipes" />, see the API docs or the general [Pipes documentation](/concepts/dagster-pipes).
 
 At this point you can execute the rest of your Kubernetes code as normal, invoking various <PyObject object="PipesContext" module="dagster_pipes" /> APIs as needed.
 
@@ -224,10 +224,10 @@ In this step, you’ll run the Kubernetes container you defined in [Step 1](#ste
 3. Click **Materialize** near the top right corner of the page, then click **View** on the **Launched Run** popup. Wait for the run to complete, and the event log should look like this:
 
    <Image
-   alt="Event log for Kubernetes run"
-   src="/images/guides/dagster-pipes/kubernetes/run.png"
-   width={1784}
-   height={1495}
+     alt="Event log for Kubernetes run"
+     src="/images/guides/dagster-pipes/kubernetes/run.png"
+     width={1784}
+     height={1495}
    />
 
 ---