diff --git a/.hugo-version b/.hugo-version index e4f97977a..de52889fc 100644 --- a/.hugo-version +++ b/.hugo-version @@ -1 +1 @@ -0.136.3 \ No newline at end of file +0.140.2 \ No newline at end of file diff --git a/content/_index.md b/content/_index.md index add291a8b..ebc55f976 100644 --- a/content/_index.md +++ b/content/_index.md @@ -28,10 +28,10 @@ Use [W&B Weave](https://weave-docs.wandb.ai/) to manage all aspects of integrati ##### Build AI models -Use [W&B Models](/guides) to manage all aspects of building your own AI models, including training, fine-tuning, reporting, automating hyperparameter sweeps, and using our model registry to assist with versioning and reproducibility. +Use [W&B Models]({{< relref "/guides/" >}}) to manage all aspects of building your own AI models, including training, fine-tuning, reporting, automating hyperparameter sweeps, and using our model registry to assist with versioning and reproducibility. -- [Introduction](/guides) -- [Quickstart](/quickstart) +- [Introduction]({{< relref "/guides/" >}}) +- [Quickstart]({{< relref "/guides/quickstart/" >}}) - [YouTube Tutorial](https://www.youtube.com/watch?v=tHAFujRhZLA) - [Online Course](https://www.wandb.courses/courses/wandb-101) diff --git a/content/guides/_index.md b/content/guides/_index.md index a5971cf0f..7557af4ed 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -20,34 +20,34 @@ Weights & Biases (W&B) is the AI developer platform, with tools for training mod {{< img src="/images/general/architecture.png" alt="" >}} -W&B consists of three major components: [Models](/guides/models.md), [Weave](https://wandb.github.io/weave/), and [Core](/guides/core.md): +W&B consists of three major components: [Models]({{< relref "/guides/models.md" >}}), [Weave](https://wandb.github.io/weave/), and [Core]({{< relref "/guides/core/" >}}): -**[W&B Models](/guides/models.md)** is a set of lightweight, interoperable tools for machine learning practitioners training and fine-tuning models. -- [Experiments](/guides/track/intro.md): Machine learning experiment tracking -- [Sweeps](/guides/sweeps/intro.md): Hyperparameter tuning and model optimization -- [Registry](/guides/registry/intro.md): Publish and share your ML models and datasets +**[W&B Models]({{< relref "/guides/models/" >}})** is a set of lightweight, interoperable tools for machine learning practitioners training and fine-tuning models. +- [Experiments]({{< relref "/guides/models/track/" >}}): Machine learning experiment tracking +- [Sweeps]({{< relref "/guides/models/sweeps/" >}}): Hyperparameter tuning and model optimization +- [Registry]({{< relref "/guides/models/registry/" >}}): Publish and share your ML models and datasets -**[W&B Weave](https://wandb.github.io/weave/)** is a lightweight toolkit for tracking and evaluating LLM applications. +**[W&B Weave](/guides/weave/)** is a lightweight toolkit for tracking and evaluating LLM applications. -**[W&B Core](/guides/core.md)** is set of powerful building blocks for tracking and visualizing data and models, and communicating results. -- [Artifacts](/guides/artifacts/intro.md): Version assets and track lineage -- [Tables](/guides/tables/intro.md): Visualize and query tabular data -- [Reports](/guides/reports/intro.md): Document and collaborate on your discoveries - +**[W&B Core]({{< relref "/guides/core/" >}})** is set of powerful building blocks for tracking and visualizing data and models, and communicating results. +- [Artifacts]({{< relref "/guides/core/artifacts/" >}}): Version assets and track lineage +- [Tables]({{< relref "/guides/core/tables/" >}}): Visualize and query tabular data +- [Reports]({{< relref "/guides/core/reports/" >}}): Document and collaborate on your discoveries + ## How does W&B work? Read the following sections in this order if you are a first-time user of W&B and you are interested in training, tracking, and visualizing machine learning models and experiments: -1. Learn about [runs](./runs/intro.md), W&B's basic unit of computation. -2. Create and track machine learning experiments with [Experiments](./track/intro.md). -3. Discover W&B's flexible and lightweight building block for dataset and model versioning with [Artifacts](./artifacts/intro.md). -4. Automate hyperparameter search and explore the space of possible models with [Sweeps](./sweeps/intro.md). -5. Manage the model lifecycle from training to production with [Model Registry](./model_registry/intro.md). -6. Visualize predictions across model versions with our [Data Visualization](./tables/intro.md) guide. -7. Organize runs, embed and automate visualizations, describe your findings, and share updates with collaborators with [Reports](./reports/intro.md). +1. Learn about [runs]({{< relref "/guides/models/track/runs/" >}}), W&B's basic unit of computation. +2. Create and track machine learning experiments with [Experiments]({{< relref "/guides/models/track/" >}}). +3. Discover W&B's flexible and lightweight building block for dataset and model versioning with [Artifacts]({{< relref "/guides/core/artifacts/" >}}). +4. Automate hyperparameter search and explore the space of possible models with [Sweeps]({{< relref "/guides/models/sweeps/" >}}). +5. Manage the model lifecycle from training to production with [Registry]({{< relref "/guides/models/registry/" >}}). +6. Visualize predictions across model versions with our [Data Visualization]({{< relref "/guides/core/tables/" >}}) guide. +7. Organize runs, embed and automate visualizations, describe your findings, and share updates with collaborators with [Reports]({{< relref "/guides/core/reports/" >}}). ## Are you a first-time user of W&B? -Try the [quickstart](../quickstart.md) to learn how to install W&B and how to add W&B to your code. \ No newline at end of file +Try the [quickstart]({{< relref "/guides/quickstart/" >}}) to learn how to install W&B and how to add W&B to your code. \ No newline at end of file diff --git a/content/guides/core/_index.md b/content/guides/core/_index.md index 97b731459..afb139162 100644 --- a/content/guides/core/_index.md +++ b/content/guides/core/_index.md @@ -7,13 +7,13 @@ weight: 5 no_list: true --- -W&B Core is the foundational framework supporting [W&B Models](./models.md) and [W&B Weave](./weave_platform.md), and is itself supported by the [W&B Platform](./hosting/_index.md). +W&B Core is the foundational framework supporting [W&B Models]({{< relref "/guides/models/" >}}) and [W&B Weave]({{< relref "/guides/weave/" >}}), and is itself supported by the [W&B Platform]({{< relref "/guides/hosting/" >}}). {{< img src="/images/general/core.png" alt="" >}} W&B Core provides capabilities across the entire ML lifecycle. With W&B Core, you can: -- [Version and manage ML](./artifacts/_index.md) pipelines with full lineage tracing for easy auditing and reproducibility. -- Explore and evaluate data and metrics using [interactive, configurable visualizations](./tables/_index.md). -- [Document and share](./reports/_index_.md) insights across the entire organization by generating live reports in digestible, visual formats that are easily understood by non-technical stakeholders. -- [Query and create visualizations of your data](../guides/app/features/panels/query-panel/_index.md) that serve your custom needs. +- [Version and manage ML]({{< relref "./artifacts/" >}}) pipelines with full lineage tracing for easy auditing and reproducibility. +- Explore and evaluate data and metrics using [interactive, configurable visualizations]({{< relref "./tables/" >}}). +- [Document and share]({{< relref "./reports/" >}}) insights across the entire organization by generating live reports in digestible, visual formats that are easily understood by non-technical stakeholders. +- [Query and create visualizations of your data]({{< relref "/guides/models/app/features/panels/query-panels/" >}}) that serve your custom needs. diff --git a/content/guides/core/artifacts/_index.md b/content/guides/core/artifacts/_index.md index 3ce6e140e..d97173de0 100644 --- a/content/guides/core/artifacts/_index.md +++ b/content/guides/core/artifacts/_index.md @@ -15,10 +15,10 @@ weight: 1 {{< cta-button productLink="https://wandb.ai/wandb/arttest/artifacts/model/iv3_trained/5334ab69740f9dda4fed/lineage" colabLink="https://colab.research.google.com/github/wandb/examples/blob/master/colabs/wandb-artifacts/Artifact_fundamentals.ipynb" >}} -Use W&B Artifacts to track and version data as the inputs and outputs of your [W&B Runs](../runs/intro.md). For example, a model training run might take in a dataset as input and produce a trained model as output. You can log hyperparameters, metadatra, and metrics to a run, and you can use an artifact to log, track, and version the dataset used to train the model as input and another artifact for the resulting model checkpoints as output. +Use W&B Artifacts to track and version data as the inputs and outputs of your [W&B Runs]({{< relref "/guides/models/track/runs/" >}}). For example, a model training run might take in a dataset as input and produce a trained model as output. You can log hyperparameters, metadatra, and metrics to a run, and you can use an artifact to log, track, and version the dataset used to train the model as input and another artifact for the resulting model checkpoints as output. ## Use cases -You can use artifacts throughout your entire ML workflow as inputs and outputs of [runs](../runs/intro.md). You can use datasets, models, or even other artifacts as inputs for processing. +You can use artifacts throughout your entire ML workflow as inputs and outputs of [runs]({{< relref "/guides/models/track/runs/" >}}). You can use datasets, models, or even other artifacts as inputs for processing. {{< img src="/images/artifacts/artifacts_landing_page2.png" >}} @@ -26,7 +26,7 @@ You can use artifacts throughout your entire ML workflow as inputs and outputs o |------------------------|-----------------------------|------------------------------| | Model Training | Dataset (training and validation data) | Trained Model | | Dataset Pre-Processing | Dataset (raw data) | Dataset (pre-processed data) | -| Model Evaluation | Model + Dataset (test data) | [W&B Table](../tables/intro.md) | +| Model Evaluation | Model + Dataset (test data) | [W&B Table]({{< relref "/guides/core/tables/" >}}) | | Model Optimization | Model | Optimized Model | @@ -37,8 +37,8 @@ The proceeding code snippets are meant to be run in order. ## Create an artifact Create an artifact with four lines of code: -1. Create a [W&B run](../runs/intro.md). -2. Create an artifact object with the [`wandb.Artifact`](../../ref/python/artifact.md) API. +1. Create a [W&B run]({{< relref "/guides/models/track/runs/" >}}). +2. Create an artifact object with the [`wandb.Artifact`]({{< relref "/ref/python/artifact.md" >}}) API. 3. Add one or more files, such as a model file or dataset, to your artifact object. 4. Log your artifact to W&B. @@ -56,11 +56,11 @@ artifact.save() ``` {{% alert %}} -See the [track external files](./track-external-files.md) page for information on how to add references to files or directories stored in external object storage, like an Amazon S3 bucket. +See the [track external files]({{< relref "./track-external-files.md" >}}) page for information on how to add references to files or directories stored in external object storage, like an Amazon S3 bucket. {{% /alert %}} ## Download an artifact -Indicate the artifact you want to mark as input to your run with the [`use_artifact`](../../ref/python/run.md#use_artifact) method. +Indicate the artifact you want to mark as input to your run with the [`use_artifact`]({{< relref "/ref/python/run.md#use_artifact" >}}) method. Following the preceding code snippet, this next code block shows how to use the `training_dataset` artifact: @@ -76,12 +76,12 @@ datadir = artifact.download() #downloads the full "my_data" artifact to the defa ``` {{% alert %}} -You can pass a custom path into the `root` [parameter](../../ref/python/artifact.md) to download an artifact to a specific directory. For alternate ways to download artifacts and to see additional parameters, see the guide on [downloading and using artifacts](./download-and-use-an-artifact.md). +You can pass a custom path into the `root` [parameter]({{< relref "/ref/python/artifact.md" >}}) to download an artifact to a specific directory. For alternate ways to download artifacts and to see additional parameters, see the guide on [downloading and using artifacts]({{< relref "./download-and-use-an-artifact.md" >}}). {{% /alert %}} ## Next steps -* Learn how to [version](./create-a-new-artifact-version.md), [update](./update-an-artifact.md), or [delete](./delete-artifacts.md) artifacts. -* Learn how to trigger downstream workflows in response to changes to your artifacts with [artifact automation](./project-scoped-automations.md). -* Learn about the [model registry](../model_registry/intro.md), a space that houses trained models. -* Explore the [Python SDK](../../ref/python/artifact.md) and [CLI](../../ref/cli/wandb-artifact/README.md) reference guides. \ No newline at end of file +* Learn how to [version]({{< relref "./create-a-new-artifact-version.md" >}}) and [update]({{< relref "./update-an-artifact.md" >}}) artifacts. +* Learn how to trigger downstream workflows in response to changes to your artifacts with [artifact automation]({{< relref "/guides/models/automations/project-scoped-automations/" >}}). +* Learn about the [registry]({{< relref "/guides/models/registry/" >}}), a space that houses trained models. +* Explore the [Python SDK]({{< relref "/ref/python/artifact.md" >}}) and [CLI]({{< relref "/ref/cli/wandb-artifact/" >}}) reference guides. \ No newline at end of file diff --git a/content/guides/core/artifacts/artifacts-walkthrough.md b/content/guides/core/artifacts/artifacts-walkthrough.md index 79dd261e7..3b64f271b 100644 --- a/content/guides/core/artifacts/artifacts-walkthrough.md +++ b/content/guides/core/artifacts/artifacts-walkthrough.md @@ -5,7 +5,7 @@ description: >- displayed_sidebar: default title: "Tutorial: Create, track, and use a dataset artifact" --- -This walkthrough demonstrates how to create, track, and use a dataset artifact from [W&B Runs](../runs/intro.md). +This walkthrough demonstrates how to create, track, and use a dataset artifact from [W&B Runs]({{< relref "/guides/models/track/runs/" >}}). ## 1. Log into W&B @@ -19,7 +19,7 @@ wandb.login() ## 2. Initialize a run -Use the [`wandb.init()`](../../ref/python/init.md) API to generate a background process to sync and log data as a W&B Run. Provide a project name and a job type: +Use the [`wandb.init()`]({{< relref "/ref/python/init.md" >}}) API to generate a background process to sync and log data as a W&B Run. Provide a project name and a job type: ```python # Create a W&B Run. Here we specify 'dataset' as the job type since this example @@ -29,7 +29,7 @@ run = wandb.init(project="artifacts-example", job_type="upload-dataset") ## 3. Create an artifact object -Create an artifact object with the [`wandb.Artifact()`](../../ref/python/artifact.md) API. Provide a name for the artifact and a description of the file type for the `name` and `type` parameters, respectively. +Create an artifact object with the [`wandb.Artifact()`]({{< relref "/ref/python/artifact.md" >}}) API. Provide a name for the artifact and a description of the file type for the `name` and `type` parameters, respectively. For example, the following code snippet demonstrates how to create an artifact called `‘bicycle-dataset’` with a `‘dataset’` label: @@ -37,7 +37,7 @@ For example, the following code snippet demonstrates how to create an artifact c artifact = wandb.Artifact(name="bicycle-dataset", type="dataset") ``` -For more information about how to construct an artifact, see [Construct artifacts](./construct-an-artifact.md). +For more information about how to construct an artifact, see [Construct artifacts]({{< relref "./construct-an-artifact.md" >}}). ## Add the dataset to the artifact @@ -60,15 +60,15 @@ Use the W&B run objects `log_artifact()` method to both save your artifact versi run.log_artifact(artifact) ``` -A `'latest'` alias is created by default when you log an artifact. For more information about artifact aliases and versions, see [Create a custom alias](./create-a-custom-alias.md) and [Create new artifact versions](./create-a-new-artifact-version.md), respectively. +A `'latest'` alias is created by default when you log an artifact. For more information about artifact aliases and versions, see [Create a custom alias]({{< relref "./create-a-custom-alias.md" >}}) and [Create new artifact versions]({{< relref "./create-a-new-artifact-version.md" >}}), respectively. ## 5. Download and use the artifact The following code example demonstrates the steps you can take to use an artifact you have logged and saved to the W&B servers. 1. First, initialize a new run object with **`wandb.init()`.** -2. Second, use the run objects [`use_artifact()`](../../ref/python/run.md#use_artifact) method to tell W&B what artifact to use. This returns an artifact object. -3. Third, use the artifacts [`download()`](../../ref/python/artifact.md#download) method to download the contents of the artifact. +2. Second, use the run objects [`use_artifact()`]({{< relref "/ref/python/run.md#use_artifact" >}}) method to tell W&B what artifact to use. This returns an artifact object. +3. Third, use the artifacts [`download()`]({{< relref "/ref/python/artifact.md#download" >}}) method to download the contents of the artifact. ```python # Create a W&B Run. Here we specify 'training' for 'type' @@ -82,4 +82,4 @@ artifact = run.use_artifact("bicycle-dataset:latest") artifact_dir = artifact.download() ``` -Alternatively, you can use the Public API (`wandb.Api`) to export (or update data) data already saved in a W&B outside of a Run. See [Track external files](./track-external-files.md) for more information. +Alternatively, you can use the Public API (`wandb.Api`) to export (or update data) data already saved in a W&B outside of a Run. See [Track external files]({{< relref "./track-external-files.md" >}}) for more information. diff --git a/content/guides/core/artifacts/construct-an-artifact.md b/content/guides/core/artifacts/construct-an-artifact.md index 21213f29b..151d6e057 100644 --- a/content/guides/core/artifacts/construct-an-artifact.md +++ b/content/guides/core/artifacts/construct-an-artifact.md @@ -9,17 +9,17 @@ title: Create an artifact weight: 2 --- -Use the W&B Python SDK to construct artifacts from [W&B Runs](../../ref/python/run.md). You can add [files, directories, URIs, and files from parallel runs to artifacts](#add-files-to-an-artifact). After you add a file to an artifact, save the artifact to the W&B Server or [your own private server](../hosting/hosting-options/self-managed.md). +Use the W&B Python SDK to construct artifacts from [W&B Runs]({{< relref "/ref/python/run.md" >}}). You can add [files, directories, URIs, and files from parallel runs to artifacts]({{< relref "#add-files-to-an-artifact" >}}). After you add a file to an artifact, save the artifact to the W&B Server or [your own private server]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}). -For information on how to track external files, such as files stored in Amazon S3, see the [Track external files](./track-external-files.md) page. +For information on how to track external files, such as files stored in Amazon S3, see the [Track external files]({{< relref "./track-external-files.md" >}}) page. ## How to construct an artifact -Construct a [W&B Artifact](../../ref/python/artifact.md) in three steps: +Construct a [W&B Artifact]({{< relref "/ref/python/artifact.md" >}}) in three steps: ### 1. Create an artifact Python object with `wandb.Artifact()` -Initialize the [`wandb.Artifact()`](../../ref/python/artifact.md) class to create an artifact object. Specify the following parameters: +Initialize the [`wandb.Artifact()`]({{< relref "/ref/python/artifact.md" >}}) class to create an artifact object. Specify the following parameters: * **Name**: Specify a name for your artifact. The name should be unique, descriptive, and easy to remember. Use an artifacts name to both: identify the artifact in the W&B App UI and when you want to use that artifact. * **Type**: Provide a type. The type should be simple, descriptive and correspond to a single step of your machine learning pipeline. Common artifact types include `'dataset'` or `'model'`. @@ -28,7 +28,7 @@ Initialize the [`wandb.Artifact()`](../../ref/python/artifact.md) class to creat {{% alert %}} The "name" and "type" you provide is used to create a directed acyclic graph. This means you can view the lineage of an artifact on the W&B App. -See the [Explore and traverse artifact graphs](./explore-and-traverse-an-artifact-graph.md) for more information. +See the [Explore and traverse artifact graphs]({{< relref "./explore-and-traverse-an-artifact-graph.md" >}}) for more information. {{% /alert %}} @@ -36,7 +36,7 @@ See the [Explore and traverse artifact graphs](./explore-and-traverse-an-artifac Artifacts can not have the same name, even if you specify a different type for the types parameter. In other words, you can not create an artifact named `cats` of type `dataset` and another artifact with the same name of type `model`. {{% /alert %}} -You can optionally provide a description and metadata when you initialize an artifact object. For more information on available attributes and parameters, see [`wandb.Artifact`](../../ref/python/artifact.md) Class definition in the Python SDK Reference Guide. +You can optionally provide a description and metadata when you initialize an artifact object. For more information on available attributes and parameters, see [`wandb.Artifact`]({{< relref "/ref/python/artifact.md" >}}) Class definition in the Python SDK Reference Guide. The proceeding example demonstrates how to create a dataset artifact: @@ -50,17 +50,17 @@ Replace the string arguments in the preceding code snippet with your own name an ### 2. Add one more files to the artifact -Add files, directories, external URI references (such as Amazon S3) and more with artifact methods. For example, to add a single text file, use the [`add_file`](../../ref/python/artifact.md#add_file) method: +Add files, directories, external URI references (such as Amazon S3) and more with artifact methods. For example, to add a single text file, use the [`add_file`]({{< relref "/ref/python/artifact.md#add_file" >}}) method: ```python artifact.add_file(local_path="hello_world.txt", name="optional-name") ``` -You can also add multiple files with the [`add_dir`](../../ref/python/artifact.md#add_dir) method. For more information on how to add files, see [Update an artifact](./update-an-artifact.md). +You can also add multiple files with the [`add_dir`]({{< relref "/ref/python/artifact.md#add_dir" >}}) method. For more information on how to add files, see [Update an artifact]({{< relref "./update-an-artifact.md" >}}). ### 3. Save your artifact to the W&B server -Finally, save your artifact to the W&B server. Artifacts are associated with a run. Therefore, use a run objects [`log_artifact()`](../../ref/python/run.md#log_artifact) method to save the artifact. +Finally, save your artifact to the W&B server. Artifacts are associated with a run. Therefore, use a run objects [`log_artifact()`]({{< relref "/ref/python/run.md#log_artifact" >}}) method to save the artifact. ```python # Create a W&B Run. Replace 'job-type'. @@ -69,7 +69,7 @@ run = wandb.init(project="artifacts-example", job_type="job-type") run.log_artifact(artifact) ``` -You can optionally construct an artifact outside of a W&B run. For more information, see [Track external files](./track-external-files.md). +You can optionally construct an artifact outside of a W&B run. For more information, see [Track external files]({{< relref "./track-external-files.md" >}}). {{% alert color="secondary" %}} Calls to `log_artifact` are performed asynchronously for performant uploads. This can cause surprising behavior when logging artifacts in a loop. For example: @@ -166,7 +166,7 @@ The proceeding API calls produce the proceeding artifact content: Artifacts track checksums and other information for reproducibility if the URI has a scheme that W&B library knows how to handle. -Add an external URI reference to an artifact with the [`add_reference`](../../ref/python/artifact.md#add_reference) method. Replace the `'uri'` string with your own URI. Optionally pass the desired path within the artifact for the name parameter. +Add an external URI reference to an artifact with the [`add_reference`]({{< relref "/ref/python/artifact.md#add_reference" >}}) method. Replace the `'uri'` string with your own URI. Optionally pass the desired path within the artifact for the name parameter. ```python # Add a URI reference diff --git a/content/guides/core/artifacts/create-a-new-artifact-version.md b/content/guides/core/artifacts/create-a-new-artifact-version.md index 241577805..126f681ee 100644 --- a/content/guides/core/artifacts/create-a-new-artifact-version.md +++ b/content/guides/core/artifacts/create-a-new-artifact-version.md @@ -9,7 +9,7 @@ title: Create an artifact version weight: 6 --- -Create a new artifact version with a single [run](../runs/intro.md) or collaboratively with distributed runs. You can optionally create a new artifact version from a previous version known as an [incremental artifact](#create-a-new-artifact-version-from-an-existing-version). +Create a new artifact version with a single [run]({{< relref "/guides/models/track/runs/" >}}) or collaboratively with distributed runs. You can optionally create a new artifact version from a previous version known as an [incremental artifact]({{< relref "#create-a-new-artifact-version-from-an-existing-version" >}}). {{% alert %}} We recommend that you create an incremental artifact when you need to apply changes to a subset of files in an artifact, where the size of the original artifact is significantly larger. diff --git a/content/guides/core/artifacts/download-and-use-an-artifact.md b/content/guides/core/artifacts/download-and-use-an-artifact.md index 435ea6089..8559005df 100644 --- a/content/guides/core/artifacts/download-and-use-an-artifact.md +++ b/content/guides/core/artifacts/download-and-use-an-artifact.md @@ -17,11 +17,11 @@ Team members with view-only seats cannot download artifacts. ### Download and use an artifact stored on W&B -Download and use an artifact stored in W&B either inside or outside of a W&B Run. Use the Public API ([`wandb.Api`](../../ref/python/public-api/api.md)) to export (or update data) already saved in W&B. For more information, see the W&B [Public API Reference guide](../../ref/python/public-api/README.md). +Download and use an artifact stored in W&B either inside or outside of a W&B Run. Use the Public API ([`wandb.Api`]({{< relref "/ref/python/public-api/api.md" >}})) to export (or update data) already saved in W&B. For more information, see the W&B [Public API Reference guide]({{< relref "/ref/python/public-api/" >}}). {{< tabpane text=true >}} {{% tab header="During a run" %}} -First, import the W&B Python SDK. Next, create a W&B [Run](../../ref/python/run.md): +First, import the W&B Python SDK. Next, create a W&B [Run]({{< relref "/ref/python/run.md" >}}): ```python import wandb @@ -29,7 +29,7 @@ import wandb run = wandb.init(project="", job_type="") ``` -Indicate the artifact you want to use with the [`use_artifact`](../../ref/python/run.md#use_artifact) method. This returns a run object. In the proceeding code snippet specifies an artifact called `'bike-dataset'` with the alias `'latest'`: +Indicate the artifact you want to use with the [`use_artifact`]({{< relref "/ref/python/run.md#use_artifact" >}}) method. This returns a run object. In the proceeding code snippet specifies an artifact called `'bike-dataset'` with the alias `'latest'`: ```python artifact = run.use_artifact("bike-dataset:latest") @@ -41,9 +41,9 @@ Use the object returned to download all the contents of the artifact: datadir = artifact.download() ``` -You can optionally pass a path to the root parameter to download the contents of the artifact to a specific directory. For more information, see the [Python SDK Reference Guide](../../ref/python/artifact.md#download). +You can optionally pass a path to the root parameter to download the contents of the artifact to a specific directory. For more information, see the [Python SDK Reference Guide]({{< relref "/ref/python/artifact.md#download" >}}). -Use the [`get_path`](../../ref/python/artifact.md#get_path) method to download only subset of files: +Use the [`get_path`]({{< relref "/ref/python/artifact.md#get_path" >}}) method to download only subset of files: ```python path = artifact.get_path(name) @@ -54,7 +54,7 @@ This fetches only the file at the path `name`. It returns an `Entry` object with * `Entry.download`: Downloads file from the artifact at path `name` * `Entry.ref`: If `add_reference` stored the entry as a reference, returns the URI -References that have schemes that W&B knows how to handle get downloaded just like artifact files. For more information, see [Track external files](../../guides/artifacts/track-external-files.md). +References that have schemes that W&B knows how to handle get downloaded just like artifact files. For more information, see [Track external files]({{< relref "/guides/core/artifacts/track-external-files.md" >}}). {{% /tab %}} {{% tab header="Outside of a run" %}} First, import the W&B SDK. Next, create an artifact from the Public API Class. Provide the entity, project, artifact, and alias associated with that artifact: @@ -72,7 +72,7 @@ Use the object returned to download the contents of the artifact: artifact.download() ``` -You can optionally pass a path the `root` parameter to download the contents of the artifact to a specific directory. For more information, see the [API Reference Guide](../../ref/python/artifact.md#download). +You can optionally pass a path the `root` parameter to download the contents of the artifact to a specific directory. For more information, see the [API Reference Guide]({{< relref "/ref/python/artifact.md#download" >}}). {{% /tab %}} {{% tab header="W&B CLI" %}} Use the `wandb artifact get` command to download an artifact from the W&B server. @@ -120,7 +120,7 @@ artifact = run.use_artifact("my-entity/my-project/artifact:alias") ### Construct and use an artifact simultaneously -Simultaneously construct and use an artifact. Create an artifact object and pass it to use_artifact. This creates an artifact in W&B if it does not exist yet. The [`use_artifact`](../../ref/python/run.md#use_artifact) API is idempotent, so you can call it as many times as you like. +Simultaneously construct and use an artifact. Create an artifact object and pass it to use_artifact. This creates an artifact in W&B if it does not exist yet. The [`use_artifact`]({{< relref "/ref/python/run.md#use_artifact" >}}) API is idempotent, so you can call it as many times as you like. ```python import wandb @@ -130,4 +130,4 @@ artifact.add_file("model.h5") run.use_artifact(artifact) ``` -For more information about constructing an artifact, see [Construct an artifact](../../guides/artifacts/construct-an-artifact.md). \ No newline at end of file +For more information about constructing an artifact, see [Construct an artifact]({{< relref "/guides/core/artifacts/construct-an-artifact.md" >}}). \ No newline at end of file diff --git a/content/guides/core/artifacts/explore-and-traverse-an-artifact-graph.md b/content/guides/core/artifacts/explore-and-traverse-an-artifact-graph.md index 65c6f9a37..0898f7ee9 100644 --- a/content/guides/core/artifacts/explore-and-traverse-an-artifact-graph.md +++ b/content/guides/core/artifacts/explore-and-traverse-an-artifact-graph.md @@ -57,7 +57,7 @@ Clicking on a node opens a preview with an overview of the node. Clicking on the {{< img src="/images/artifacts/lineage3b.gif" alt="Searching a run cluster" >}} ## Use the API to track lineage -You can also navigate a graph using the [W&B API](../../ref/python/public-api/api.md). +You can also navigate a graph using the [W&B API]({{< relref "/ref/python/public-api/api.md" >}}). Create an artifact. First, create a run with `wandb.init`. Then,create a new artifact or retrieve an existing one with `wandb.Artifact`. Next, add files to the artifact with `.add_file`. Finally, log the artifact to the run with `.log_artifact`. The finished code looks something like this: @@ -71,7 +71,7 @@ with wandb.init() as run: run.log_artifact(artifact) ``` -Use the artifact object's [`logged_by`](../../ref/python/artifact.md#logged_by) and [`used_by`](../../ref/python/artifact.md#used_by) methods to walk the graph from the artifact: +Use the artifact object's [`logged_by`]({{< relref "/ref/python/artifact.md#logged_by" >}}) and [`used_by`]({{< relref "/ref/python/artifact.md#used_by" >}}) methods to walk the graph from the artifact: ```python # Walk up and down the graph from an artifact: @@ -79,6 +79,6 @@ producer_run = artifact.logged_by() consumer_runs = artifact.used_by() ``` ## Next steps -- [Explore artifacts in more detail](../artifacts/artifacts-walkthrough.md) -- [Manage artifact storage](../artifacts/delete-artifacts.md) +- [Explore artifacts in more detail]({{< relref "/guides/core/artifacts/artifacts-walkthrough.md" >}}) +- [Manage artifact storage]({{< relref "/guides/core/artifacts/manage-data/delete-artifacts.md" >}}) - [Explore an artifacts project](https://wandb.ai/wandb-smle/artifact_workflow/artifacts/raw_dataset/raw_data/v0/lineage) \ No newline at end of file diff --git a/content/guides/core/artifacts/manage-data/delete-artifacts.md b/content/guides/core/artifacts/manage-data/delete-artifacts.md index 7bb1071f4..db41e908e 100644 --- a/content/guides/core/artifacts/manage-data/delete-artifacts.md +++ b/content/guides/core/artifacts/manage-data/delete-artifacts.md @@ -12,7 +12,7 @@ Delete artifacts interactively with the App UI or programmatically with the W&B The contents of the artifact remain as a soft-delete, or pending deletion state, until a regularly run garbage collection process reviews all artifacts marked for deletion. The garbage collection process deletes associated files from storage if the artifact and its associated files are not used by a previous or subsequent artifact versions. -The sections in this page describe how to delete specific artifact versions, how to delete an artifact collection, how to delete artifacts with and without aliases, and more. You can schedule when artifacts are deleted from W&B with TTL policies. For more information, see [Manage data retention with Artifact TTL policy](./ttl.md). +The sections in this page describe how to delete specific artifact versions, how to delete an artifact collection, how to delete artifacts with and without aliases, and more. You can schedule when artifacts are deleted from W&B with TTL policies. For more information, see [Manage data retention with Artifact TTL policy]({{< relref "./ttl.md" >}}). {{% alert %}} Artifacts that are scheduled for deletion with a TTL policy, deleted with the W&B SDK, or deleted with the W&B App UI are first soft-deleted. Artifacts that are soft deleted undergo garbage collection before they are hard-deleted. @@ -27,7 +27,7 @@ To delete an artifact version: 3. On the right hand side of the workspace, select the kebab dropdown. 4. Choose Delete. -An artifact version can also be deleted programatically via the [delete()](/ref/python/artifact#delete) method. See the examples below. +An artifact version can also be deleted programatically via the [delete()]({{< relref "/ref/python/artifact#delete" >}}) method. See the examples below. ### Delete multiple artifact versions with aliases @@ -98,7 +98,7 @@ To delete an artifact collection: 3. Select the kebab dropdown next to the artifact collection name. 4. Choose Delete. -You can also delete artifact collection programmatically with the [delete()](../../ref/python/artifact.md#delete) method. Provide the name of the project and entity for the `project` and `entity` keys in `wandb.Api`, respectively: +You can also delete artifact collection programmatically with the [delete()]({{< relref "/ref/python/artifact.md#delete" >}}) method. Provide the name of the project and entity for the `project` and `entity` keys in `wandb.Api`, respectively: ```python import wandb @@ -129,9 +129,9 @@ The `X` indicates you must satisfy the requirement: | | Environment variable | Enable versioning | | -----------------------------------------------| ------------------------| ----------------- | | Shared cloud | | | -| Shared cloud with [secure storage connector](../hosting/data-security/secure-storage-connector.md)| | X | +| Shared cloud with [secure storage connector]({{< relref "/guides/hosting/data-security/secure-storage-connector.md" >}})| | X | | Dedicated cloud | | | -| Dedicated cloud with [secure storage connector](../hosting/data-security/secure-storage-connector.md)| | X | +| Dedicated cloud with [secure storage connector]({{< relref "/guides/hosting/data-security/secure-storage-connector.md" >}})| | X | | Customer-managed cloud | X | X | | Customer managed on-prem | X | X | diff --git a/content/guides/core/artifacts/manage-data/storage.md b/content/guides/core/artifacts/manage-data/storage.md index 107c27b01..630f9a5c8 100644 --- a/content/guides/core/artifacts/manage-data/storage.md +++ b/content/guides/core/artifacts/manage-data/storage.md @@ -9,7 +9,7 @@ title: Manage artifact storage and memory allocation W&B stores artifact files in a private Google Cloud Storage bucket located in the United States by default. All files are encrypted at rest and in transit. -For sensitive files, we recommend you set up [Private Hosting](../hosting/intro.md) or use [reference artifacts](./track-external-files.md). +For sensitive files, we recommend you set up [Private Hosting]({{< relref "/guides/hosting/" >}}) or use [reference artifacts]({{< relref "../track-external-files.md" >}}). During training, W&B locally saves logs, artifacts, and configuration files in the following local directories: @@ -26,7 +26,7 @@ Depending on the machine on `wandb` is initialized on, these default folders may ### Clean up local artifact cache -W&B caches artifact files to speed up downloads across versions that share files in common. Over time this cache directory can become large. Run the [`wandb artifact cache cleanup`](../../ref/cli/wandb-artifact/wandb-artifact-cache/README.md) command to prune the cache and to remove any files that have not been used recently. +W&B caches artifact files to speed up downloads across versions that share files in common. Over time this cache directory can become large. Run the [`wandb artifact cache cleanup`]({{< relref "/ref/cli/wandb-artifact/wandb-artifact-cache/" >}}) command to prune the cache and to remove any files that have not been used recently. The proceeding code snippet demonstrates how to limit the size of the cache to 1GB. Copy and paste the code snippet into your terminal: diff --git a/content/guides/core/artifacts/manage-data/ttl.md b/content/guides/core/artifacts/manage-data/ttl.md index d06f4f568..f66981aca 100644 --- a/content/guides/core/artifacts/manage-data/ttl.md +++ b/content/guides/core/artifacts/manage-data/ttl.md @@ -10,7 +10,7 @@ title: Manage artifact data retention {{< cta-button colabLink="https://colab.research.google.com/github/wandb/examples/blob/kas-artifacts-ttl-colab/colabs/wandb-artifacts/WandB_Artifacts_Time_to_live_TTL_Walkthrough.ipynb" >}} -Schedule when artifacts are deleted from W&B with W&B Artifact time-to-live (TTL) policy. When you delete an artifact, W&B marks that artifact as a *soft-delete*. In other words, the artifact is marked for deletion but files are not immediately deleted from storage. For more information on how W&B deletes artifacts, see the [Delete artifacts](./delete-artifacts.md) page. +Schedule when artifacts are deleted from W&B with W&B Artifact time-to-live (TTL) policy. When you delete an artifact, W&B marks that artifact as a *soft-delete*. In other words, the artifact is marked for deletion but files are not immediately deleted from storage. For more information on how W&B deletes artifacts, see the [Delete artifacts]({{< relref "./delete-artifacts.md" >}}) page. Check out [this](https://www.youtube.com/watch?v=hQ9J6BoVmnc) video tutorial to learn how to manage data retention with Artifacts TTL in the W&B App. @@ -18,7 +18,7 @@ Check out [this](https://www.youtube.com/watch?v=hQ9J6BoVmnc) video tutorial to W&B deactivates the option to set a TTL policy for model artifacts linked to the Model Registry. This is to help ensure that linked models do not accidentally expire if used in production workflows. {{% /alert %}} {{% alert %}} -* Only team admins can view a [team's settings](../app/settings-page/team-settings.md) and access team level TTL settings such as (1) permitting who can set or edit a TTL policy or (2) setting a team default TTL. +* Only team admins can view a [team's settings]({{< relref "/guides/models/app/settings-page/team-settings.md" >}}) and access team level TTL settings such as (1) permitting who can set or edit a TTL policy or (2) setting a team default TTL. * If you do not see the option to set or edit a TTL policy in an artifact's details in the W&B App UI or if setting a TTL programmatically does not successfully change an artifact's TTL property, your team admin has not given you permissions to do so. {{% /alert %}} @@ -31,7 +31,7 @@ The following Artifact types indicate an auto-generated Artifact: - `job` - Any Artifact type starting with: `wandb-*` -You can check an Artifact's type on the [W&B platform](../artifacts/explore-and-traverse-an-artifact-graph.md) or programmatically: +You can check an Artifact's type on the [W&B platform]({{< relref "/guides/core/artifacts/explore-and-traverse-an-artifact-graph.md" >}}) or programmatically: ```python import wandb @@ -68,15 +68,15 @@ For all the code snippets below, replace the content wrapped in `<>` with your i Use the W&B Python SDK to define a TTL policy when you create an artifact. TTL policies are typically defined in days. {{% alert %}} -Defining a TTL policy when you create an artifact is similar to how you normally [create an artifact](./construct-an-artifact.md). With the exception that you pass in a time delta to the artifact's `ttl` attribute. +Defining a TTL policy when you create an artifact is similar to how you normally [create an artifact]({{< relref "../construct-an-artifact.md" >}}). With the exception that you pass in a time delta to the artifact's `ttl` attribute. {{% /alert %}} The steps are as follows: -1. [Create an artifact](./construct-an-artifact.md). -2. [Add content to the artifact](./construct-an-artifact.md#add-files-to-an-artifact) such as files, a directory, or a reference. +1. [Create an artifact]({{< relref "../construct-an-artifact.md" >}}). +2. [Add content to the artifact]({{< relref "../construct-an-artifact.md#add-files-to-an-artifact" >}}) such as files, a directory, or a reference. 3. Define a TTL time limit with the [`datetime.timedelta`](https://docs.python.org/3/library/datetime.html) data type that is part of Python's standard library. -4. [Log the artifact](./construct-an-artifact.md#3-save-your-artifact-to-the-wb-server). +4. [Log the artifact]({{< relref "../construct-an-artifact.md#3-save-your-artifact-to-the-wb-server" >}}). The following code snippet demonstrates how to create an artifact and set a TTL policy. @@ -103,9 +103,9 @@ When you modify an artifact's TTL, the time the artifact takes to expire is stil {{< tabpane text=true >}} {{% tab header="Python SDK" %}} -1. [Fetch your artifact](./download-and-use-an-artifact.md). +1. [Fetch your artifact]({{< relref "../download-and-use-an-artifact.md" >}}). 2. Pass in a time delta to the artifact's `ttl` attribute. -3. Update the artifact with the [`save`](../../ref/python/run.md#save) method. +3. Update the artifact with the [`save`]({{< relref "/ref/python/run.md#save" >}}) method. The following code snippet shows how to set a TTL policy for an artifact: @@ -180,9 +180,9 @@ Artifacts with TTL turned off will not inherit an artifact collection's TTL. Ref {{< tabpane text=true >}} {{% tab header="Python SDK" %}} -1. [Fetch your artifact](./download-and-use-an-artifact.md). +1. [Fetch your artifact]({{< relref "../download-and-use-an-artifact.md" >}}). 2. Set the artifact's `ttl` attribute to `None`. -3. Update the artifact with the [`save`](../../ref/python/run.md#save) method. +3. Update the artifact with the [`save`]({{< relref "/ref/python/run.md#save" >}}) method. The following code snippet shows how to turn off a TTL policy for an artifact: diff --git a/content/guides/core/artifacts/track-external-files.md b/content/guides/core/artifacts/track-external-files.md index cbd035128..e3032782f 100644 --- a/content/guides/core/artifacts/track-external-files.md +++ b/content/guides/core/artifacts/track-external-files.md @@ -9,23 +9,23 @@ title: Track external files weight: 7 --- -Use **reference artifacts** to track files saved outside the W&B system, for example in an Amazon S3 bucket, GCS bucket, Azure blob, HTTP file server, or even an NFS share. Log artifacts outside of a [W&B Run](/ref/python/run) with the W&B [CLI](/ref/cli). +Use **reference artifacts** to track files saved outside the W&B system, for example in an Amazon S3 bucket, GCS bucket, Azure blob, HTTP file server, or even an NFS share. Log artifacts outside of a [W&B Run]({{< relref "/ref/python/run" >}}) with the W&B [CLI]({{< relref "/ref/cli" >}}). ### Log artifacts outside of runs W&B creates a run when you log an artifact outside of a run. Each artifact belongs to a run, which in turn belongs to a project. An artifact (version) also belongs to a collection, and has a type. -Use the [`wandb artifact put`](/ref/cli/wandb-artifact/wandb-artifact-put) command to upload an artifact to the W&B server outside of a W&B run. Provide the name of the project you want the artifact to belong to along with the name of the artifact (`project/artifact_name`).Optionally provide the type (`TYPE`). Replace `PATH` in the code snippet below with the file path of the artifact you want to upload. +Use the [`wandb artifact put`]({{< relref "/ref/cli/wandb-artifact/wandb-artifact-put" >}}) command to upload an artifact to the W&B server outside of a W&B run. Provide the name of the project you want the artifact to belong to along with the name of the artifact (`project/artifact_name`).Optionally provide the type (`TYPE`). Replace `PATH` in the code snippet below with the file path of the artifact you want to upload. ```bash $ wandb artifact put --name project/artifact_name --type TYPE PATH ``` -W&B will create a new project if a the project you specify does not exist. For information on how to download an artifact, see [Download and use artifacts](/guides/artifacts/download-and-use-an-artifact). +W&B will create a new project if a the project you specify does not exist. For information on how to download an artifact, see [Download and use artifacts]({{< relref "/guides/core/artifacts/download-and-use-an-artifact" >}}). ## Track artifacts outside of W&B -Use W&B Artifacts for dataset versioning and model lineage, and use **reference artifacts** to track files saved outside the W&B server. In this mode an artifact only stores metadata about the files, such as URLs, size, and checksums. The underlying data never leaves your system. See the [Quick start](/guides/artifacts/artifacts-walkthrough) for information on how to save files and directories to W&B servers instead. +Use W&B Artifacts for dataset versioning and model lineage, and use **reference artifacts** to track files saved outside the W&B server. In this mode an artifact only stores metadata about the files, such as URLs, size, and checksums. The underlying data never leaves your system. See the [Quick start]({{< relref "/guides/core/artifacts/artifacts-walkthrough" >}}) for information on how to save files and directories to W&B servers instead. The following describes how to construct reference artifacts and how to best incorporate them into your workflows. diff --git a/content/guides/core/artifacts/update-an-artifact.md b/content/guides/core/artifacts/update-an-artifact.md index fb5e6240a..67c624c99 100644 --- a/content/guides/core/artifacts/update-an-artifact.md +++ b/content/guides/core/artifacts/update-an-artifact.md @@ -10,7 +10,7 @@ weight: 4 Pass desired values to update the `description`, `metadata`, and `alias` of an artifact. Call the `save()` method to update the artifact on the W&B servers. You can update an artifact during a W&B Run or outside of a Run. -Use the W&B Public API ([`wandb.Api`](../../ref/python/public-api/api.md)) to update an artifact outside of a run. Use the Artifact API ([`wandb.Artifact`](../../ref/python/artifact.md)) to update an artifact during a run. +Use the W&B Public API ([`wandb.Api`]({{< relref "/ref/python/public-api/api.md" >}})) to update an artifact outside of a run. Use the Artifact API ([`wandb.Artifact`]({{< relref "/ref/python/artifact.md" >}})) to update an artifact during a run. {{% alert color="secondary" %}} You can not update the alias of artifact linked to a model in Model Registry. @@ -19,7 +19,7 @@ You can not update the alias of artifact linked to a model in Model Registry. {{< tabpane text=true >}} {{% tab header="During a run" %}} -The proceeding code example demonstrates how to update the description of an artifact using the [`wandb.Artifact`](../../ref/python/artifact.md) API: +The proceeding code example demonstrates how to update the description of an artifact using the [`wandb.Artifact`]({{< relref "/ref/python/artifact.md" >}}) API: ```python import wandb @@ -62,7 +62,7 @@ artifact.aliases = ["replaced"] artifact.save() ``` -For more information, see the Weights and Biases [Artifact API](../../ref/python/artifact.md). +For more information, see the Weights and Biases [Artifact API]({{< relref "/ref/python/artifact.md" >}}). {{% /tab %}} {{% tab header="With collections" %}} You can also update an Artifact collection in the same way as a singular artifact: @@ -76,7 +76,7 @@ artifact.name = "" artifact.description = "" artifact.save() ``` -For more information, see the [Artifacts Collection](../../ref/python/public-api/api.md) reference. +For more information, see the [Artifacts Collection]({{< relref "/ref/python/public-api/api.md" >}}) reference. {{% /tab %}} {{% /tabpane %}} diff --git a/content/guides/core/reports/_index.md b/content/guides/core/reports/_index.md index b1fe9d798..340c7e550 100644 --- a/content/guides/core/reports/_index.md +++ b/content/guides/core/reports/_index.md @@ -42,12 +42,12 @@ Create a collaborative report with a few clicks. 6. Click **Publish to project**. 7. Click the **Share** button to share your report with collaborators. -See the [Create a report](./create-a-report.md) page for more information on how to create reports interactively an programmatically with the W&B Python SDK. +See the [Create a report]({{< relref "./create-a-report.md" >}}) page for more information on how to create reports interactively an programmatically with the W&B Python SDK. ## How to get started Depending on your use case, explore the following resources to get started with W&B Reports: * Check out our [video demonstration](https://www.youtube.com/watch?v=2xeJIv_K_eI) to get an overview of W&B Reports. -* Explore the [Reports gallery](./reports-gallery.md) for examples of live reports. -* Try the [Programmatic Workspaces](../../tutorials/workspaces.md) tutorial to learn how to create and customize your workspace. +* Explore the [Reports gallery]({{< relref "./reports-gallery.md" >}}) for examples of live reports. +* Try the [Programmatic Workspaces]({{< relref "/tutorials/workspaces.md" >}}) tutorial to learn how to create and customize your workspace. * Read curated Reports in [W&B Fully Connected](http://wandb.me/fc). \ No newline at end of file diff --git a/content/guides/core/reports/create-a-report.md b/content/guides/core/reports/create-a-report.md index 21bcd69cd..36ec14962 100644 --- a/content/guides/core/reports/create-a-report.md +++ b/content/guides/core/reports/create-a-report.md @@ -48,7 +48,7 @@ Create a report programmatically with the `wandb` library. import wandb import wandb_workspaces.reports.v2 as wr ``` -3. Create a report with `wandb_workspaces.reports.v2.Report`. Create a report instance with the Report Class Public API ([`wandb.apis.reports`](/ref/python/public-api/api#reports)). Specify a name for the project. +3. Create a report with `wandb_workspaces.reports.v2.Report`. Create a report instance with the Report Class Public API ([`wandb.apis.reports`]({{< relref "/ref/python/public-api/api#reports" >}})). Specify a name for the project. ```python report = wr.Report(project="report_standard") ``` @@ -58,6 +58,6 @@ Create a report programmatically with the `wandb` library. report.save() ``` -For information on how to edit a report interactively with the App UI or programmatically, see [Edit a report](/guides/reports/edit-a-report). +For information on how to edit a report interactively with the App UI or programmatically, see [Edit a report]({{< relref "/guides/core/reports/edit-a-report" >}}). {{% /tab %}} {{< /tabpane >}} diff --git a/content/guides/core/reports/cross-project-reports.md b/content/guides/core/reports/cross-project-reports.md index 6d866ca35..8a284b979 100644 --- a/content/guides/core/reports/cross-project-reports.md +++ b/content/guides/core/reports/cross-project-reports.md @@ -24,7 +24,7 @@ Share a view-only link to a report that is in a private project or team project. {{< img src="/images/reports/magic-links.gif" alt="" >}} -View-only report links add a secret access token to the URL, so anyone who opens the link can view the page. Anyone can use the magic link to view the report without logging in first. For customers on [W&B Local](../hosting/intro.md) private cloud installations, these links remain behind your firewall, so only members of your team with access to your private instance _and_ access to the view-only link can view the report. +View-only report links add a secret access token to the URL, so anyone who opens the link can view the page. Anyone can use the magic link to view the report without logging in first. For customers on [W&B Local]({{< relref "/guides/hosting/" >}}) private cloud installations, these links remain behind your firewall, so only members of your team with access to your private instance _and_ access to the view-only link can view the report. In **view-only mode**, someone who is not logged in can see the charts and mouse over to see tooltips of values, zoom in and out on charts, and scroll through columns in the table. When in view mode, they cannot create new charts or new table queries to explore the data. View-only visitors to the report link won't be able to click a run to get to the run page. Also, the view-only visitors would not be able to see the share modal but instead would see a tooltip on hover which says: `Sharing not available for view only access`. diff --git a/content/guides/core/reports/edit-a-report.md b/content/guides/core/reports/edit-a-report.md index 1bad1e8f4..28af0c51c 100644 --- a/content/guides/core/reports/edit-a-report.md +++ b/content/guides/core/reports/edit-a-report.md @@ -17,7 +17,7 @@ _Panel grids_ are a specific type of block that hold panels and _run sets_. Run {{% alert %}} -Check out the [Programmatic workspaces tutorial](../../tutorials/workspaces.md) for a step by step example on how create and customize a saved workspace view. +Check out the [Programmatic workspaces tutorial]({{< relref "/tutorials/workspaces.md" >}}) for a step by step example on how create and customize a saved workspace view. {{% /alert %}} {{% alert %}} diff --git a/content/guides/core/tables/_index.md b/content/guides/core/tables/_index.md index 06e701ffe..2fc796a4b 100644 --- a/content/guides/core/tables/_index.md +++ b/content/guides/core/tables/_index.md @@ -32,11 +32,11 @@ A Table is a two-dimensional grid of data where each column has a single type of Log a table with a few lines of code: -- [`wandb.init()`](../../ref/python/init.md): Create a [run](../runs/intro.md) to track results. -- [`wandb.Table()`](../../ref/python/data-types/table.md): Create a new table object. +- [`wandb.init()`]({{< relref "/ref/python/init.md" >}}): Create a [run]({{< relref "/guides/models/track/runs/" >}}) to track results. +- [`wandb.Table()`]({{< relref "/ref/python/data-types/table.md" >}}): Create a new table object. - `columns`: Set the column names. - `data`: Set the contents of the table. -- [`run.log()`](../../ref/python/log.md): Log the table to save it to W&B. +- [`run.log()`]({{< relref "/ref/python/log.md" >}}): Log the table to save it to W&B. ```python showLineNumbers import wandb @@ -47,5 +47,5 @@ run.log({"Table Name": my_table}) ``` ## How to get started -* [Quickstart](./tables-walkthrough.md): Learn to log data tables, visualize data, and query data. -* [Tables Gallery](./tables-gallery.md): See example use cases for Tables. \ No newline at end of file +* [Quickstart]({{< relref "./tables-walkthrough.md" >}}): Learn to log data tables, visualize data, and query data. +* [Tables Gallery]({{< relref "./tables-gallery.md" >}}): See example use cases for Tables. \ No newline at end of file diff --git a/content/guides/core/tables/tables-download.md b/content/guides/core/tables/tables-download.md index 1096f9720..54c1e5a99 100644 --- a/content/guides/core/tables/tables-download.md +++ b/content/guides/core/tables/tables-download.md @@ -45,6 +45,6 @@ df.to_csv("example.csv", encoding="utf-8") ``` # Next Steps -- Check out the [reference documentation](../artifacts/construct-an-artifact.md) on `artifacts`. -- Go through our [Tables Walktrough](../tables/tables-walkthrough.md) guide. +- Check out the [reference documentation]({{< relref "/guides/core/artifacts/construct-an-artifact.md" >}}) on `artifacts`. +- Go through our [Tables Walktrough]({{< relref "/guides/core/tables/tables-walkthrough.md" >}}) guide. - Check out the [Dataframe](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) reference docs. \ No newline at end of file diff --git a/content/guides/core/tables/tables-gallery.md b/content/guides/core/tables/tables-gallery.md index 94e56bde1..87c11af01 100644 --- a/content/guides/core/tables/tables-gallery.md +++ b/content/guides/core/tables/tables-gallery.md @@ -11,7 +11,7 @@ The following sections highlight some of the ways you can use tables: ### View your data -Log metrics and rich media during model training or evaluation, then visualize results in a persistent database synced to the cloud, or to your [hosting instance](/guides/hosting). +Log metrics and rich media during model training or evaluation, then visualize results in a persistent database synced to the cloud, or to your [hosting instance]({{< relref "/guides/hosting" >}}). {{< img src="/images/data_vis/tables_see_data.png" alt="Browse examples and verify the counts and distribution of your data" max-width="90%" >}} diff --git a/content/guides/core/tables/tables-walkthrough.md b/content/guides/core/tables/tables-walkthrough.md index 59483e54b..9d312c18b 100644 --- a/content/guides/core/tables/tables-walkthrough.md +++ b/content/guides/core/tables/tables-walkthrough.md @@ -18,11 +18,11 @@ Log a table with W&B. You can either construct a new table or pass a Pandas Data {{< tabpane text=true >}} {{% tab header="Construct a table" value="construct" %}} To construct and log a new Table, you will use: -- [`wandb.init()`](../../ref/python/init.md): Create a [run](../runs/intro.md) to track results. -- [`wandb.Table()`](../../ref/python/data-types/table.md): Create a new table object. +- [`wandb.init()`]({{< relref "/ref/python/init.md" >}}): Create a [run]({{< relref "/guides/models/track/runs/" >}}) to track results. +- [`wandb.Table()`]({{< relref "/ref/python/data-types/table.md" >}}): Create a new table object. - `columns`: Set the column names. - `data`: Set the contents of each row. -- [`run.log()`](../../ref/python/log.md): Log the table to save it to W&B. +- [`run.log()`]({{< relref "/ref/python/log.md" >}}): Log the table to save it to W&B. Here's an example: ```python @@ -49,7 +49,7 @@ my_table = wandb.Table(dataframe=df) wandb.log({"Table Name": my_table}) ``` -For more information on supported data types, see the [`wandb.Table`](../../ref/python/data-types/table.md) in the W&B API Reference Guide. +For more information on supported data types, see the [`wandb.Table`]({{< relref "/ref/python/data-types/table.md" >}}) in the W&B API Reference Guide. {{% /tab %}} {{< /tabpane >}} diff --git a/content/guides/core/tables/visualize-tables.md b/content/guides/core/tables/visualize-tables.md index 2ffeb44c5..35da74a35 100644 --- a/content/guides/core/tables/visualize-tables.md +++ b/content/guides/core/tables/visualize-tables.md @@ -23,11 +23,11 @@ W&B Tables posses the following behaviors: 1. **Stateless in an artifact context**: any table logged alongside an artifact version resets to its default state after you close the browser window 2. **Stateful in a workspace or report context**: any changes you make to a table in a single run workspace, multi-run project workspace, or Report persists. -For information on how to save your current W&B Table view, see [Save your view](#save-your-view). +For information on how to save your current W&B Table view, see [Save your view]({{< relref "#save-your-view" >}}). {{% /alert %}} ## How to view two tables -Compare two tables with a [merged view](#merged-view) or a [side-by-side view](#side-by-side-view). For example, the image below demonstrates a table comparison of MNIST data. +Compare two tables with a [merged view]({{< relref "#merged-view" >}}) or a [side-by-side view]({{< relref "#side-by-side-view" >}}). For example, the image below demonstrates a table comparison of MNIST data. {{< img src="/images/data_vis/table_comparison.png" alt="Left: mistakes after 1 training epochs, Right: mistakes after 5 epochs" max-width="90%" >}} @@ -72,7 +72,7 @@ To view the two tables side-by-side, change the first dropdown from "Merge Table ## Compare artifacts -You can also [compare tables across time](#compare-tables-across-time) or [model variants](#compare-tables-across-model-variants). +You can also [compare tables across time]({{< relref "#compare-tables-across-time" >}}) or [model variants]({{< relref "#compare-tables-across-model-variants" >}}). ### Compare tables across time diff --git a/content/guides/hosting/_index.md b/content/guides/hosting/_index.md index 09b197db1..7817dd6a9 100644 --- a/content/guides/hosting/_index.md +++ b/content/guides/hosting/_index.md @@ -6,13 +6,13 @@ title: W&B Platform weight: 6 no_list: true --- -W&B Platform is the foundational infrastructure, tooling and governance scaffolding which supports the W&B products like [Core](../core.md), [Models](../models.md) and [Weave](../weave_platform.md). +W&B Platform is the foundational infrastructure, tooling and governance scaffolding which supports the W&B products like [Core]({{< relref "../core.md" >}}), [Models]({{< relref "../models.md" >}}) and [Weave]({{< relref "/guides/weave/" >}}). W&B Platform is available in three different deployment options: -* [W&B Multi-tenant Cloud](#wb-multi-tenant-cloud) -* [W&B Dedicated Cloud](#wb-dedicated-cloud) -* [W&B Customer-managed](#wb-customer-managed) +* [W&B Multi-tenant Cloud]({{< relref "#wb-multi-tenant-cloud" >}}) +* [W&B Dedicated Cloud]({{< relref "#wb-dedicated-cloud" >}}) +* [W&B Customer-managed]({{< relref "#wb-customer-managed" >}}) The following responsibility matrix outlines some of the key differences between the different options: @@ -24,17 +24,17 @@ The following sections provide an overview of each deployment type. ### W&B Multi-tenant Cloud W&B Multi-tenant Cloud is a fully managed service deployed in W&B's cloud infrastructure, where you can seamlessly access the W&B products at the desired scale, with cost-efficient options for pricing, and with continuous updates for the latest features and functionalities. W&B recommends to use the Multi-tenant Cloud for your product trial, or to manage your production AI workflows if you do not need the security of a private deployment, self-service onboarding is important, and cost efficiency is critical. -See [W&B Multi-tenant Cloud](./hosting-options/saas_cloud.md) for more information. +See [W&B Multi-tenant Cloud]({{< relref "./hosting-options/saas_cloud.md" >}}) for more information. ### W&B Dedicated Cloud W&B Dedicated Cloud is a single-tenant, fully managed service deployed in W&B's cloud infrastructure. It is the best place to onboard W&B if your organization requires conformance to strict governance controls including data residency, have need of advanced security capabilities, and are looking to optimize their AI operating costs by not having to build & manage the required infrastructure with security, scale & performance characteristics. -See [W&B Dedicated Cloud](./hosting-options/dedicated_cloud.md) for more information. +See [W&B Dedicated Cloud]({{< relref "./hosting-options/dedicated_cloud.md" >}}) for more information. ### W&B Customer-Managed With this option, you can deploy and manage W&B Server on your own managed infrastructure. W&B Server is a self-contained packaged mechanism to run the W&B Platform & its supported W&B products. W&B recommends this option if all your existing infrastructure is on-prem, or your organization has strict regulatory needs that are not satisfied by W&B Dedicated Cloud. With this option, you are fully responsible to manage the provisioning, and continuous maintenance & upgrades of the infrastructure required to support W&B Server. -See [W&B Self Managed](./hosting-options/self-managed.md) for more information. +See [W&B Self Managed]({{< relref "./hosting-options/self-managed.md" >}}) for more information. ## Next steps diff --git a/content/guides/hosting/data-security/data-encryption.md b/content/guides/hosting/data-security/data-encryption.md index a165f228a..27dbe1fa0 100644 --- a/content/guides/hosting/data-security/data-encryption.md +++ b/content/guides/hosting/data-security/data-encryption.md @@ -6,7 +6,7 @@ menu: title: Data encryption in Dedicated cloud --- -W&B uses a W&B-managed cloud-native key to encrypt the W&B-managed database and object storage in every [Dedicated cloud](../hosting-options/dedicated_cloud.md), by using the customer-managed encryption key (CMEK) capability in each cloud. In this case, W&B acts as a `customer` of the cloud provider, while providing the W&B platform as a service to you. Using a W&B-managed key means that W&B has control over the keys that it uses to encrypt the data in each cloud, thus doubling down on its promise to provide a highly safe and secure platform to all of its customers. +W&B uses a W&B-managed cloud-native key to encrypt the W&B-managed database and object storage in every [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}), by using the customer-managed encryption key (CMEK) capability in each cloud. In this case, W&B acts as a `customer` of the cloud provider, while providing the W&B platform as a service to you. Using a W&B-managed key means that W&B has control over the keys that it uses to encrypt the data in each cloud, thus doubling down on its promise to provide a highly safe and secure platform to all of its customers. W&B uses a `unique key` to encrypt the data in each customer instance, providing another layer of isolation between Dedicated cloud tenants. The capability is available on AWS, Azure and GCP. diff --git a/content/guides/hosting/data-security/ip-allowlisting.md b/content/guides/hosting/data-security/ip-allowlisting.md index 67b2faf3b..c5447dab1 100644 --- a/content/guides/hosting/data-security/ip-allowlisting.md +++ b/content/guides/hosting/data-security/ip-allowlisting.md @@ -7,11 +7,11 @@ title: Configure IP allowlisting for Dedicated Cloud weight: 3 --- -You can restrict access to your [Dedicated Cloud](../hosting-options/dedicated_cloud.md) instance from only an authorized list of IP addresses. This applies to the access from your AI workloads to the W&B APIs and from your user browsers to the W&B app UI as well. Once IP allowlisting has been set up for your Dedicated Cloud instance, W&B denies any requests from other unauthorized locations. Reach out to your W&B team to configure IP allowlisting for your Dedicated Cloud instance. +You can restrict access to your [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) instance from only an authorized list of IP addresses. This applies to the access from your AI workloads to the W&B APIs and from your user browsers to the W&B app UI as well. Once IP allowlisting has been set up for your Dedicated Cloud instance, W&B denies any requests from other unauthorized locations. Reach out to your W&B team to configure IP allowlisting for your Dedicated Cloud instance. IP allowlisting is available on Dedicated Cloud instances on AWS, GCP and Azure. -You can use IP allowlisting with [secure private connectivity](./private-connectivity.md). If you use IP allowlisting with secure private connectivity, W&B recommends using secure private connectivity for all traffic from your AI workloads and majority of the traffic from your user browsers if possible, while using IP allowlisting for instance administration from privileged locations. +You can use IP allowlisting with [secure private connectivity]({{< relref "./private-connectivity.md" >}}). If you use IP allowlisting with secure private connectivity, W&B recommends using secure private connectivity for all traffic from your AI workloads and majority of the traffic from your user browsers if possible, while using IP allowlisting for instance administration from privileged locations. {{% alert color="secondary" %}} W&B strongly recommends to use [CIDR blocks](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) assigned to your corporate or business egress gateways rather than individual `/32` IP addresses. Using individual IP addresses is not scalable and has strict limits per cloud. diff --git a/content/guides/hosting/data-security/presigned-urls.md b/content/guides/hosting/data-security/presigned-urls.md index f37b366fc..bad5297ad 100644 --- a/content/guides/hosting/data-security/presigned-urls.md +++ b/content/guides/hosting/data-security/presigned-urls.md @@ -13,7 +13,7 @@ When needed, AI workloads or user browser clients within your network request pr ## Team-level access control -Each pre-signed URL is restricted to specific buckets based on [team level access control](../iam/manage-organization.md#add-and-manage-teams) in the W&B platform. If a user is part of a team which is mapped to a blob storage bucket using [secure storage connector](./secure-storage-connector.md), and if that user is part of only that team, then the pre-signed URLs generated for their requests would not have permissions to access blob storage buckets mapped to other teams. +Each pre-signed URL is restricted to specific buckets based on [team level access control]({{< relref "/guides/hosting/iam/access-management/manage-organization.md#add-and-manage-teams" >}}) in the W&B platform. If a user is part of a team which is mapped to a blob storage bucket using [secure storage connector]({{< relref "./secure-storage-connector.md" >}}), and if that user is part of only that team, then the pre-signed URLs generated for their requests would not have permissions to access blob storage buckets mapped to other teams. {{% alert %}} W&B recommends adding users to only the teams that they are supposed to be a part of. diff --git a/content/guides/hosting/data-security/private-connectivity.md b/content/guides/hosting/data-security/private-connectivity.md index 0e1ec0110..f1d18bd9b 100644 --- a/content/guides/hosting/data-security/private-connectivity.md +++ b/content/guides/hosting/data-security/private-connectivity.md @@ -25,4 +25,4 @@ Once enabled, W&B creates a private endpoint service for your instance and provi If you would like to use this feature, contact your W&B team. {{% /alert %}} -You can use secure private connectivity with [IP allowlisting](./ip-allowlisting.md). If you use secure private connectivity for IP allowlisting, W&B recommends that you secure private connectivity for all traffic from your AI workloads and majority of the traffic from your user browsers if possible, while using IP allowlisting for instance administration from privileged locations. \ No newline at end of file +You can use secure private connectivity with [IP allowlisting]({{< relref "./ip-allowlisting.md" >}}). If you use secure private connectivity for IP allowlisting, W&B recommends that you secure private connectivity for all traffic from your AI workloads and majority of the traffic from your user browsers if possible, while using IP allowlisting for instance administration from privileged locations. \ No newline at end of file diff --git a/content/guides/hosting/data-security/secure-storage-connector.md b/content/guides/hosting/data-security/secure-storage-connector.md index e8c1148cf..be6180abf 100644 --- a/content/guides/hosting/data-security/secure-storage-connector.md +++ b/content/guides/hosting/data-security/secure-storage-connector.md @@ -7,11 +7,11 @@ title: Bring your own bucket (BYOB) weight: 1 --- -Bring your own bucket (BYOB) allows you to store W&B artifacts and other related sensitive data in your own cloud or on-prem infrastructure. In case of [Dedicated cloud](../hosting-options/dedicated_cloud.md) or [SaaS Cloud](../hosting-options/saas_cloud.md), data that you store in your bucket is not copied to the W&B managed infrastructure. +Bring your own bucket (BYOB) allows you to store W&B artifacts and other related sensitive data in your own cloud or on-prem infrastructure. In case of [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}), data that you store in your bucket is not copied to the W&B managed infrastructure. {{% alert %}} -* Communication between W&B SDK / CLI / UI and your buckets occurs using [pre-signed URLs](./presigned-urls.md). -* W&B uses a garbage collection process to delete W&B Artifacts. For more information, see [Deleting Artifacts](../../artifacts/delete-artifacts.md). +* Communication between W&B SDK / CLI / UI and your buckets occurs using [pre-signed URLs]({{< relref "./presigned-urls.md" >}}). +* W&B uses a garbage collection process to delete W&B Artifacts. For more information, see [Deleting Artifacts]({{< relref "/guides/core/artifacts/manage-data/delete-artifacts.md" >}}). * You can specify a sub-path when configuring a bucket, to ensure that W&B does not store any files in a folder at the root of the bucket. It can help you better conform to your organzation's bucket governance policy. {{% /alert %}} @@ -26,7 +26,7 @@ You can configure your bucket at both the instance level and separately for one For example, suppose you have a team called Kappa in your organization. Your organization (and Team Kappa) use the Instance level storage bucket by default. Next, you create a team called Omega. When you create Team Omega, you configure a Team level storage bucket for that team. Files generated by Team Omega are not accessible by Team Kappa. However, files created by Team Kappa are accessible by Team Omega. If you want to isolate data for Team Kappa, you must configure a Team level storage bucket for them as well. {{% alert %}} -Team level storage bucket provides the same benefits for [Self-managed](../hosting-options/self-managed.md) instances, especially when different business units and departments share an instance to efficiently utilize the infrastructure and administrative resources. This also applies to firms that have separate project teams managing AI workflows for separate customer engagements. +Team level storage bucket provides the same benefits for [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instances, especially when different business units and departments share an instance to efficiently utilize the infrastructure and administrative resources. This also applies to firms that have separate project teams managing AI workflows for separate customer engagements. {{% /alert %}} ## Availability matrix @@ -44,7 +44,7 @@ Once you configure a instance or team level storage bucket for your Dedicated cl ## Cross-cloud or S3-compatible storage for team-level BYOB -You can connect to a cloud-native storage bucket in another cloud or to an S3-compatible storage bucket like [MinIO](https://github.com/minio/minio) for team-level BYOB in your [Dedicated cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instance. +You can connect to a cloud-native storage bucket in another cloud or to an S3-compatible storage bucket like [MinIO](https://github.com/minio/minio) for team-level BYOB in your [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instance. To enable the use of cross-cloud or S3-compatible storage, specify the storage bucket including the relevant access key in one of the following formats, using the `GORILLA_SUPPORTED_FILE_STORES` environment variable for your W&B instance. @@ -81,7 +81,7 @@ gs://:@ {{% alert %}} -Connectivity to S3-compatible storage for team-level BYOB is not available in [SaaS Cloud](../hosting-options/saas_cloud.md). Also, connectivity to an AWS bucket for team-level BYOB is cross-cloud in [SaaS Cloud](../hosting-options/saas_cloud.md), as that instance resides in GCP. That cross-cloud connectivity doesn't use the access key and environment variable based mechanism as outlined previously for [Dedicated cloud](../hosting-options/dedicated_cloud.md) and [Self-managed](../hosting-options/self-managed.md) instances. +Connectivity to S3-compatible storage for team-level BYOB is not available in [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}). Also, connectivity to an AWS bucket for team-level BYOB is cross-cloud in [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}), as that instance resides in GCP. That cross-cloud connectivity doesn't use the access key and environment variable based mechanism as outlined previously for [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) and [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instances. {{% /alert %}} Reach out to W&B Support at support@wandb.com for more information. @@ -134,10 +134,10 @@ W&B recommends that you use a Terraform module managed by W&B to provision a sto Replace `` and `` accordingly. - If you are using [SaaS Cloud](../hosting-options/saas_cloud.md) or [Dedicated cloud](../hosting-options/dedicated_cloud.md), replace `` with the corresponding value: + If you are using [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) or [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}), replace `` with the corresponding value: - * For [SaaS Cloud](../hosting-options/saas_cloud.md): `arn:aws:iam::725579432336:role/WandbIntegration` - * For [Dedicated cloud](../hosting-options/dedicated_cloud.md): `arn:aws:iam::830241207209:root` + * For [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}): `arn:aws:iam::725579432336:role/WandbIntegration` + * For [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}): `arn:aws:iam::830241207209:root` This policy grants your AWS account full access to the key and also assigns the required permissions to the AWS account hosting the W&B Platform. Keep a record of the KMS Key ARN. @@ -172,7 +172,7 @@ W&B recommends that you use a Terraform module managed by W&B to provision a sto ] ``` - 5. Grant the required S3 permissions to the AWS account hosting the W&B Platform, which requires these permissions to generate [pre-signed URLs](./presigned-urls.md) that AI workloads in your cloud infrastructure or user browsers utilize to access the bucket. + 5. Grant the required S3 permissions to the AWS account hosting the W&B Platform, which requires these permissions to generate [pre-signed URLs]({{< relref "./presigned-urls.md" >}}) that AI workloads in your cloud infrastructure or user browsers utilize to access the bucket. ```json { @@ -205,14 +205,14 @@ W&B recommends that you use a Terraform module managed by W&B to provision a sto } ``` - Replace `` accordingly and keep a record of the bucket name. If you are using [Dedicated cloud](../hosting-options/dedicated_cloud.md), share the bucket name with your W&B team in case of instance level BYOB. In case of team level BYOB on any deployment type, [configure the bucket while creating the team](#configure-byob-in-wb). + Replace `` accordingly and keep a record of the bucket name. If you are using [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}), share the bucket name with your W&B team in case of instance level BYOB. In case of team level BYOB on any deployment type, [configure the bucket while creating the team]({{< relref "#configure-byob-in-wb" >}}). - If you are using [SaaS Cloud](../hosting-options/saas_cloud.md) or [Dedicated cloud](../hosting-options/dedicated_cloud.md), replace `` with the corresponding value. + If you are using [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) or [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}), replace `` with the corresponding value. - * For [SaaS Cloud](../hosting-options/saas_cloud.md): `arn:aws:iam::725579432336:role/WandbIntegration` - * For [Dedicated cloud](../hosting-options/dedicated_cloud.md): `arn:aws:iam::830241207209:root` + * For [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}): `arn:aws:iam::725579432336:role/WandbIntegration` + * For [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}): `arn:aws:iam::830241207209:root` - For more details, see the [AWS self-managed hosting guide](../self-managed/aws-tf.md). + For more details, see the [AWS self-managed hosting guide]({{< relref "/guides/hosting/hosting-options/self-managed/install-on-public-cloud/aws-tf.md" >}}). {{% /tab %}} {{% tab header="GCP" value="gcp"%}} @@ -253,12 +253,12 @@ W&B recommends that you use a Terraform module managed by W&B to provision a sto gsutil cors get gs:// ``` -2. If you are using [SaaS Cloud](../hosting-options/saas_cloud.md) or [Dedicated cloud](../hosting-options/dedicated_cloud.md), grant the `Storage Admin` role to the GCP service account linked to the W&B Platform: +2. If you are using [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) or [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}), grant the `Storage Admin` role to the GCP service account linked to the W&B Platform: - * For [SaaS Cloud](../hosting-options/saas_cloud.md), the account is: `wandb-integration@wandb-production.iam.gserviceaccount.com` - * For [Dedicated cloud](../hosting-options/dedicated_cloud.md) the account is: `deploy@wandb-production.iam.gserviceaccount.com` + * For [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}), the account is: `wandb-integration@wandb-production.iam.gserviceaccount.com` + * For [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) the account is: `deploy@wandb-production.iam.gserviceaccount.com` - Keep a record of the bucket name. If you are using [Dedicated cloud](../hosting-options/dedicated_cloud.md), share the bucket name with your W&B team in case of instance level BYOB. In case of team level BYOB on any deployment type, [configure the bucket while creating the team](#configure-byob-in-wb). + Keep a record of the bucket name. If you are using [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}), share the bucket name with your W&B team in case of instance level BYOB. In case of team level BYOB on any deployment type, [configure the bucket while creating the team]({{< relref "#configure-byob-in-wb" >}}). {{% /tab %}} {{% tab header="Azure" value="azure"%}} @@ -281,9 +281,9 @@ W&B recommends that you use a Terraform module managed by W&B to provision a sto | Exposed Headers | `*` | | Max Age | `3600` | -2. Generate a storage account access key, and keep a record of that along with the storage account name. If you are using [Dedicated cloud](../hosting-options/dedicated_cloud.md), share the storage account name and access key with your W&B team using a secure sharing mechanism. +2. Generate a storage account access key, and keep a record of that along with the storage account name. If you are using [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}), share the storage account name and access key with your W&B team using a secure sharing mechanism. - For the team level BYOB, W&B recommends that you use [Terraform](https://github.com/wandb/terraform-azurerm-wandb/tree/main/modules/secure_storage_connector) to provision the Azure Blob Storage bucket along with the necessary access mechanism and permissions. If you use [Dedicated cloud](../hosting-options/dedicated_cloud.md), provide the OIDC issuer URL for your instance. Make a note of details that you need to [configure the bucket while creating the team](#configure-byob-in-wb): + For the team level BYOB, W&B recommends that you use [Terraform](https://github.com/wandb/terraform-azurerm-wandb/tree/main/modules/secure_storage_connector) to provision the Azure Blob Storage bucket along with the necessary access mechanism and permissions. If you use [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}), provide the OIDC issuer URL for your instance. Make a note of details that you need to [configure the bucket while creating the team]({{< relref "#configure-byob-in-wb" >}}): * Storage account name * Storage container name @@ -298,7 +298,7 @@ W&B recommends that you use a Terraform module managed by W&B to provision a sto {{% tab header="Team level" value="team" %}} {{% alert %}} -If you're connecting to a cloud-native storage bucket in another cloud or to an S3-compatible storage bucket like [MinIO](https://github.com/minio/minio) for team-level BYOB in your [Dedicated cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instance, refer to [Cross-cloud or S3-compatible storage for team-level BYOB](#cross-cloud-or-s3-compatible-storage-for-team-level-byob). In such cases, you must specify the storage bucket using the `GORILLA_SUPPORTED_FILE_STORES` environment variable for your W&B instance, before you configure it for a team using the instructions below. +If you're connecting to a cloud-native storage bucket in another cloud or to an S3-compatible storage bucket like [MinIO](https://github.com/minio/minio) for team-level BYOB in your [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instance, refer to [Cross-cloud or S3-compatible storage for team-level BYOB]({{< relref "#cross-cloud-or-s3-compatible-storage-for-team-level-byob" >}}). In such cases, you must specify the storage bucket using the `GORILLA_SUPPORTED_FILE_STORES` environment variable for your W&B instance, before you configure it for a team using the instructions below. {{% /alert %}} To configure a storage bucket at the team level when you create a W&B Team: @@ -310,11 +310,11 @@ To configure a storage bucket at the team level when you create a W&B Team: Multiple W&B Teams can use the same cloud storage bucket. To enable this, select an existing cloud storage bucket from the dropdown. 4. From the **Cloud provider** dropdown, select your cloud provider. -5. Provide the name of your storage bucket for the **Name** field. If you have a [Dedicated cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instance on Azure, provide the values for **Account name** and **Container name** fields. +5. Provide the name of your storage bucket for the **Name** field. If you have a [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instance on Azure, provide the values for **Account name** and **Container name** fields. 6. (Optional) Provide the bucket sub-path in the optional **Path** field. Do this if you would not like W&B to store any files in a folder at the root of the bucket. 7. (Optional if using AWS bucket) Provide the ARN of your KMS encryption key for the **KMS key ARN** field. 8. (Optional if using Azure bucket) Provide the values for the **Tenant ID** and the **Managed Identity Client ID** fields. -9. (Optional on [SaaS Cloud](../hosting-options/saas_cloud.md)) Optionally invite team members when creating the team. +9. (Optional on [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}})) Optionally invite team members when creating the team. 10. Press the **Create Team** button. {{< img src="/images/hosting/prod_setup_secure_storage.png" alt="" >}} diff --git a/content/guides/hosting/env-vars.md b/content/guides/hosting/env-vars.md index a3cb475c5..ce6cc3748 100644 --- a/content/guides/hosting/env-vars.md +++ b/content/guides/hosting/env-vars.md @@ -8,7 +8,7 @@ title: Configure environment variables weight: 7 --- -In addition to configuring instance level settings via the System Settings admin UI, W&B also provides a way to configure these values via code using Environment Variables. Also, refer to [advanced configuration for IAM](./iam/advanced_env_vars.md). +In addition to configuring instance level settings via the System Settings admin UI, W&B also provides a way to configure these values via code using Environment Variables. Also, refer to [advanced configuration for IAM]({{< relref "./iam/advanced_env_vars.md" >}}). ## Environment variable reference diff --git a/content/guides/hosting/hosting-options/dedicated_cloud/_index.md b/content/guides/hosting/hosting-options/dedicated_cloud/_index.md index 1f8785aac..98e728634 100644 --- a/content/guides/hosting/hosting-options/dedicated_cloud/_index.md +++ b/content/guides/hosting/hosting-options/dedicated_cloud/_index.md @@ -11,38 +11,38 @@ url: guides/hosting/hosting-options/dedicated_cloud W&B Dedicated Cloud is a single-tenant, fully managed platform deployed in W&B's AWS, GCP or Azure cloud accounts. Each Dedicated Cloud instance has its own isolated network, compute and storage from other W&B Dedicated Cloud instances. Your W&B specific metadata and data is stored in an isolated cloud storage and is processed using isolated cloud compute services. -W&B Dedicated Cloud is available in [multiple global regions for each cloud provider](./dedicated_regions.md) +W&B Dedicated Cloud is available in [multiple global regions for each cloud provider]({{< relref "./dedicated_regions.md" >}}) ## Data security -You can bring your own bucket (BYOB) using the [secure storage connector](../data-security/secure-storage-connector.md) at the [instance and team levels](../data-security/secure-storage-connector.md#configuration-options) to store your files such as models, datasets, and more. +You can bring your own bucket (BYOB) using the [secure storage connector]({{< relref "/guides/hosting/data-security/secure-storage-connector.md" >}}) at the [instance and team levels]({{< relref "/guides/hosting/data-security/secure-storage-connector.md#configuration-options" >}}) to store your files such as models, datasets, and more. Similar to W&B Multi-tenant Cloud, you can configure a single bucket for multiple teams or you can use separate buckets for different teams. If you do not configure secure storage connector for a team, that data is stored in the instance level bucket. {{< img src="/images/hosting/dedicated_cloud_arch.png" alt="" >}} -In addition to BYOB with secure storage connector, you can utilize [IP allowlisting](../data-security/ip-allowlisting.md) to restrict access to your Dedicated Cloud instance from only trusted network locations. +In addition to BYOB with secure storage connector, you can utilize [IP allowlisting]({{< relref "/guides/hosting/data-security/ip-allowlisting.md" >}}) to restrict access to your Dedicated Cloud instance from only trusted network locations. -You can also privately connect to your Dedicated Cloud instance using [cloud provider's secure connectivity solution](../data-security/private-connectivity.md). +You can also privately connect to your Dedicated Cloud instance using [cloud provider's secure connectivity solution]({{< relref "/guides/hosting/data-security/private-connectivity.md" >}}). ## Identity and access management (IAM) Use the identity and access management capabilities for secure authentication and effective authorization in your W&B Organization. The following features are available for IAM in Dedicated Cloud instances: -* Authenticate with [SSO using OpenID Connect (OIDC)](../iam/sso.md) or with [LDAP](../iam/ldap.md). -* [Configure appropriate user roles](../iam/manage-organization.md#assign-or-update-a-users-role) at the scope of the organization and within a team. -* Define the scope of a W&B project to limit who can view, edit, and submit W&B runs to it with [restricted projects](../iam/restricted-projects.md). -* Leverage JSON Web Tokens with [identity federation](../iam/identity_federation.md) to access W&B APIs. +* Authenticate with [SSO using OpenID Connect (OIDC)]({{< relref "/guides/hosting/iam/authentication/sso.md" >}}) or with [LDAP]({{< relref "/guides/hosting/iam/authentication/ldap.md" >}}). +* [Configure appropriate user roles]({{< relref "/guides/hosting/iam/access-management/manage-organization.md#assign-or-update-a-users-role" >}}) at the scope of the organization and within a team. +* Define the scope of a W&B project to limit who can view, edit, and submit W&B runs to it with [restricted projects]({{< relref "/guides/hosting/iam/access-management/restricted-projects.md" >}}). +* Leverage JSON Web Tokens with [identity federation]({{< relref "/guides/hosting/iam/authentication/identity_federation.md" >}}) to access W&B APIs. ## Monitor -Use [Audit logs](../monitoring-usage/audit-logging.md) to track user activity within your teams and to conform to your enterprise governance requirements. Also, you can view organization usage in our Dedicated Cloud instance with [W&B Organization Dashboard](../monitoring-usage/org_dashboard.md). +Use [Audit logs]({{< relref "/guides/hosting/monitoring-usage/audit-logging.md" >}}) to track user activity within your teams and to conform to your enterprise governance requirements. Also, you can view organization usage in our Dedicated Cloud instance with [W&B Organization Dashboard]({{< relref "/guides/hosting/monitoring-usage/org_dashboard.md" >}}). ## Maintenance Similar to W&B Multi-tenant Cloud, you do not incur the overhead and costs of provisioning and maintaining the W&B platform with Dedicated Cloud. -To understand how W&B manages updates on Dedicated Cloud, refer to the [server release process](../server-release-process.md). +To understand how W&B manages updates on Dedicated Cloud, refer to the [server release process]({{< relref "/guides/hosting/hosting-options/self-managed/server-upgrade-process.md" >}}). ## Compliance @@ -50,7 +50,7 @@ Security controls for W&B Dedicated Cloud are periodically audited internally an ## Migration options -Migration to Dedicated Cloud from a [Self-managed instance](./self-managed.md) or [Multi-tenant Cloud](./saas_cloud.md) is supported. +Migration to Dedicated Cloud from a [Self-managed instance]({{< relref "../self-managed/" >}}) or [Multi-tenant Cloud]({{< relref "../saas_cloud.md" >}}) is supported. ## Next steps diff --git a/content/guides/hosting/hosting-options/dedicated_cloud/export-data-from-dedicated-cloud.md b/content/guides/hosting/hosting-options/dedicated_cloud/export-data-from-dedicated-cloud.md index 58806946d..32f98c458 100644 --- a/content/guides/hosting/hosting-options/dedicated_cloud/export-data-from-dedicated-cloud.md +++ b/content/guides/hosting/hosting-options/dedicated_cloud/export-data-from-dedicated-cloud.md @@ -8,16 +8,16 @@ title: Export data from Dedicated cloud url: guides/hosting/export-data-from-dedicated-cloud --- -If you would like to export all the data managed in your Dedicated cloud instance, you can use the W&B SDK API to extract the runs, metrics, artifacts, and more with the [Import and Export API](/ref/python/public-api/). The following table has covers some of the key exporting use cases. +If you would like to export all the data managed in your Dedicated cloud instance, you can use the W&B SDK API to extract the runs, metrics, artifacts, and more with the [Import and Export API]({{< relref "/ref/python/public-api/" >}}). The following table has covers some of the key exporting use cases. | Purpose | Documentation | |---------|---------------| -| Export project metadata | [Projects API](/ref/python/public-api/projects/) | -| Export runs in a project | [Runs API](/ref/python/public-api/runs/) | -| Export reports | [Reports API](/guides/reports/clone-and-export-reports/) | -| Export artifacts | [Explore artifact graphs](/guides/artifacts/explore-and-traverse-an-artifact-graph), [Download and use artifacts](/guides/artifacts/download-and-use-an-artifact/#download-and-use-an-artifact-stored-on-wb) | +| Export project metadata | [Projects API]({{< relref "/ref/python/public-api/projects/" >}}) | +| Export runs in a project | [Runs API]({{< relref "/ref/python/public-api/runs/" >}}) | +| Export reports | [Reports API]({{< relref "/guides/core/reports/clone-and-export-reports/" >}}) | +| Export artifacts | [Explore artifact graphs]({{< relref "/guides/core/artifacts/explore-and-traverse-an-artifact-graph" >}}), [Download and use artifacts]({{< relref "/guides/core/artifacts/download-and-use-an-artifact/#download-and-use-an-artifact-stored-on-wb" >}}) | -If you manage artifacts stored in the Dedicated cloud with [Secure Storage Connector](/guides/app/features/teams/#secure-storage-connector), you may not need to export the artifacts using the W&B SDK API. +If you manage artifacts stored in the Dedicated cloud with [Secure Storage Connector]({{< relref "/guides/models/app/settings-page/teams/#secure-storage-connector" >}}), you may not need to export the artifacts using the W&B SDK API. {{% alert %}} Using W&B SDK API to export all of your data can be slow if you have a large number of runs, artifacts etc. W&B recommends running the export process in appropriately sized batches so as not to overwhelm your Dedicated cloud instance. diff --git a/content/guides/hosting/hosting-options/saas_cloud.md b/content/guides/hosting/hosting-options/saas_cloud.md index 0cbf42158..fff09a39f 100644 --- a/content/guides/hosting/hosting-options/saas_cloud.md +++ b/content/guides/hosting/hosting-options/saas_cloud.md @@ -15,14 +15,14 @@ W&B Multi-tenant Cloud is a fully managed platform deployed in W&B's Google Clou For non enterprise plan users, all data is only stored in the shared cloud storage and is processed with shared cloud compute services. Depending on your pricing plan, you may be subject to storage limits. -Enterprise plan users can [bring their own bucket (BYOB) using the secure storage connector](../data-security/secure-storage-connector.md) at the [team level](../data-security/secure-storage-connector.md#configuration-options) to store their files such as models, datasets, and more. You can configure a single bucket for multiple teams or you can use separate buckets for different W&B Teams. If you do not configure secure storage connector for a team, that data is stored in the shared cloud storage. +Enterprise plan users can [bring their own bucket (BYOB) using the secure storage connector]({{< relref "/guides/hosting/data-security/secure-storage-connector.md" >}}) at the [team level]({{< relref "/guides/hosting/data-security/secure-storage-connector.md#configuration-options" >}}) to store their files such as models, datasets, and more. You can configure a single bucket for multiple teams or you can use separate buckets for different W&B Teams. If you do not configure secure storage connector for a team, that data is stored in the shared cloud storage. ## Identity and access management (IAM) If you are on enterprise plan, you can use the identity and access managements capabilities for secure authentication and effective authorization in your W&B Organization. The following features are available for IAM in Multi-tenant Cloud: * SSO authentication with OIDC or SAML. Reach out to your W&B team or support if you would like to configure SSO for your organization. -* [Configure appropriate user roles](../iam/manage-organization.md#assign-or-update-a-users-role) at the scope of the organization and within a team. -* Define the scope of a W&B project to limit who can view, edit, and submit W&B runs to it with [restricted projects](../iam/restricted-projects.md). +* [Configure appropriate user roles]({{< relref "/guides/hosting/iam/access-management/manage-organization.md#assign-or-update-a-users-role" >}}) at the scope of the organization and within a team. +* Define the scope of a W&B project to limit who can view, edit, and submit W&B runs to it with [restricted projects]({{< relref "/guides/hosting/iam/access-management/restricted-projects.md" >}}). ## Monitor Organization admins can manage usage and billing for their account from the `Billing` tab in their account view. If using the shared cloud storage on Multi-tenant Cloud, an admin can optimize storage usage across different teams in their organization. diff --git a/content/guides/hosting/hosting-options/self-managed/_index.md b/content/guides/hosting/hosting-options/self-managed/_index.md index 7c8f7bc37..7938749f5 100644 --- a/content/guides/hosting/hosting-options/self-managed/_index.md +++ b/content/guides/hosting/hosting-options/self-managed/_index.md @@ -13,20 +13,20 @@ cascade: ## Use self-managed cloud or on-prem infrastructure {{% alert %}} -W&B recommends fully managed deployment options such as [W&B Multi-tenant Cloud](./saas_cloud.md) or [W&B Dedicated Cloud](./dedicated_cloud.md) deployment types. W&B fully managed services are simple and secure to use, with minimum to no configuration required. +W&B recommends fully managed deployment options such as [W&B Multi-tenant Cloud]({{< relref "../saas_cloud.md" >}}) or [W&B Dedicated Cloud]({{< relref "../dedicated_cloud/" >}}) deployment types. W&B fully managed services are simple and secure to use, with minimum to no configuration required. {{% /alert %}} -Deploy W&B Server on your [AWS, GCP, or Azure cloud account](#deploy-wb-server-within-self-managed-cloud-accounts) or within your [on-premises infrastructure](#deploy-wb-server-in-on-prem-infrastructure). +Deploy W&B Server on your [AWS, GCP, or Azure cloud account]({{< relref "#deploy-wb-server-within-self-managed-cloud-accounts" >}}) or within your [on-premises infrastructure]({{< relref "#deploy-wb-server-in-on-prem-infrastructure" >}}). Your IT/DevOps/MLOps team is responsible for provisioning your deployment, managing upgrades, and continuously maintaining your self managed W&B Server instance. - + ## Deploy W&B Server within self managed cloud accounts W&B recommends that you use official W&B Terraform scripts to deploy W&B Server into your AWS, GCP, or Azure cloud account. -See specific cloud provider documentation for more information on how to set up W&B Server in [AWS](../self-managed/aws-tf.md), [GCP](../self-managed/gcp-tf.md) or [Azure](../self-managed/azure-tf.md). +See specific cloud provider documentation for more information on how to set up W&B Server in [AWS]({{< relref "/guides/hosting/hosting-options/self-managed/install-on-public-cloud/aws-tf.md" >}}), [GCP]({{< relref "/guides/hosting/hosting-options/self-managed/install-on-public-cloud/gcp-tf.md" >}}) or [Azure]({{< relref "/guides/hosting/hosting-options/self-managed/install-on-public-cloud/azure-tf.md" >}}). ## Deploy W&B Server in on-prem infrastructure @@ -37,11 +37,11 @@ You need to configure several infrastructure components in order to set up W&B S - Amazon S3-compatible object storage - Redis cache cluster -See [Install on on-prem infrastructure](../self-managed/bare-metal.md) for more information on how to install W&B Server on your on-prem infrastructure. W&B can provide recommendations for the different components and provide guidance through the installation process. +See [Install on on-prem infrastructure]({{< relref "/guides/hosting/hosting-options/self-managed/bare-metal.md" >}}) for more information on how to install W&B Server on your on-prem infrastructure. W&B can provide recommendations for the different components and provide guidance through the installation process. ## Deploy W&B Server on a custom cloud platform -You can deploy W&B Server to a cloud platform that is not AWS, GCP, or Azure. Requirements for that are similar to that for deploying in [on-prem infrastructure](#deploy-wb-server-in-on-prem-infrastructure). +You can deploy W&B Server to a cloud platform that is not AWS, GCP, or Azure. Requirements for that are similar to that for deploying in [on-prem infrastructure]({{< relref "#deploy-wb-server-in-on-prem-infrastructure" >}}). ## Obtain your W&B Server license diff --git a/content/guides/hosting/hosting-options/self-managed/bare-metal.md b/content/guides/hosting/hosting-options/self-managed/bare-metal.md index aa6f45064..ace2a1f42 100644 --- a/content/guides/hosting/hosting-options/self-managed/bare-metal.md +++ b/content/guides/hosting/hosting-options/self-managed/bare-metal.md @@ -8,7 +8,7 @@ title: Deploy W&B Platform On-premises --- {{% alert %}} -W&B recommends fully managed deployment options such as [W&B Multi-tenant Cloud](../hosting-options/saas_cloud.md) or [W&B Dedicated Cloud](../hosting-options//dedicated_cloud.md) deployment types. W&B fully managed services are simple and secure to use, with minimum to no configuration required. +W&B recommends fully managed deployment options such as [W&B Multi-tenant Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) or [W&B Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud/" >}}) deployment types. W&B fully managed services are simple and secure to use, with minimum to no configuration required. {{% /alert %}} @@ -16,7 +16,7 @@ Reach out to the W&B Sales Team for related question: [contact@wandb.com](mailto ## Infrastructure guidelines -Before you start deploying W&B, refer to the [reference architecture](./ref-arch.md#infrastructure-requirements), especially the infrastructure requirements. +Before you start deploying W&B, refer to the [reference architecture]({{< relref "./install-on-public-cloud/ref-arch.md#infrastructure-requirements" >}}), especially the infrastructure requirements. {{% alert %}} W&B strongly recommends to deploy W&B Server into a Kubernetes cluster using the W&B Kubernetes Operator. Deploying to a Kubernetes cluster with the operator ensures that you can use all the existing and latest W&B features. @@ -48,7 +48,7 @@ It is important to keep in mind that these specifications are minimum requiremen ### Database server -W&B recommends a [MySQL 8](#mysql-database) database as a metadata store. The shape of the model parameters and related metadata impact the performance of the database. The database size grows as the ML practitioners track more training runs, and incurs read heavy load when queries are executed in run tables, users workspaces, and reports. +W&B recommends a [MySQL 8]({{< relref "#mysql-database" >}}) database as a metadata store. The shape of the model parameters and related metadata impact the performance of the database. The database size grows as the ML practitioners track more training runs, and incurs read heavy load when queries are executed in run tables, users workspaces, and reports. To ensure optimal performance W&B recommends deploying the W&B database on to a server with the following starting specs: @@ -62,13 +62,13 @@ To ensure optimal performance W&B recommends deploying the W&B database on to a Again, W&B recommends monitoring the resource usage and performance of the database to ensure that it operates optimally and to make adjustments as necessary. -Additionally, W&B recommends the following [parameter overrides](#mysql-database) to tune the DB for MySQL 8. +Additionally, W&B recommends the following [parameter overrides]({{< relref "#mysql-database" >}}) to tune the DB for MySQL 8. ### Object storage W&B is compatible with an object storage that supports S3 API interface, Signed URLs and CORS. W&B recommends specifying the storage array to the current needs of your practitioners and to capacity plan on a regular cadence. -More details on object store configuration can be found in the [how-to section](../self-managed/bare-metal.md#object-store). +More details on object store configuration can be found in the [how-to section]({{< relref "/guides/hosting/hosting-options/self-managed/bare-metal.md#object-store" >}}). Some tested and working providers: - [MinIO](https://min.io/) @@ -78,7 +78,7 @@ Some tested and working providers: #### Secure Storage Connector -The [Secure Storage Connector](../data-security/secure-storage-connector.md) is not available for teams at this time for bare metal deployments. +The [Secure Storage Connector]({{< relref "/guides/hosting/data-security/secure-storage-connector.md" >}}) is not available for teams at this time for bare metal deployments. ## MySQL database @@ -186,7 +186,7 @@ mc mb --region=us-east1 local/local-files ## Deploy W&B Server application to Kubernetes -The recommended installation method is with the official W&B Helm chart. Follow [this section](../operator.md#deploy-wb-with-helm-cli) to deploy the W&B Server application. +The recommended installation method is with the official W&B Helm chart. Follow [this section]({{< relref "./kubernetes-operator/#deploy-wb-with-helm-cli" >}}) to deploy the W&B Server application. ### OpenShift diff --git a/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/aws-tf.md b/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/aws-tf.md index dca0b61fe..4c380b45b 100644 --- a/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/aws-tf.md +++ b/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/aws-tf.md @@ -9,7 +9,7 @@ weight: 10 --- {{% alert %}} -W&B recommends fully managed deployment options such as [W&B Multi-tenant Cloud](../hosting-options/saas_cloud.md) or [W&B Dedicated Cloud](../hosting-options//dedicated_cloud.md) deployment types. W&B fully managed services are simple and secure to use, with minimum to no configuration required. +W&B recommends fully managed deployment options such as [W&B Multi-tenant Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) or [W&B Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud/" >}}) deployment types. W&B fully managed services are simple and secure to use, with minimum to no configuration required. {{% /alert %}} W&B recommends using the [W&B Server AWS Terraform Module](https://registry.terraform.io/modules/wandb/wandb/aws/latest) to deploy the platform on AWS. @@ -213,7 +213,7 @@ This is the most straightforward deployment option configuration that creates al Another deployment option uses `Redis` to cache the SQL queries and speed up the application response when loading the metrics for the experiments. -You need to add the option `create_elasticache_subnet = true` to the same `main.tf` file described in the [Recommended deployment](#recommended-deployment-option) section to enable the cache. +You need to add the option `create_elasticache_subnet = true` to the same `main.tf` file described in the [Recommended deployment]({{< relref "#recommended-deployment-option" >}}) section to enable the cache. ``` module "wandb_infra" { @@ -233,7 +233,7 @@ module "wandb_infra" { Deployment option 3 consists of enabling the external `message broker`. This is optional because the W&B brings embedded a broker. This option doesn't bring a performance improvement. -The AWS resource that provides the message broker is the `SQS`, and to enable it, you will need to add the option `use_internal_queue = false` to the same `main.tf` described in the [Recommended deployment](#recommended-deployment-option) section. +The AWS resource that provides the message broker is the `SQS`, and to enable it, you will need to add the option `use_internal_queue = false` to the same `main.tf` described in the [Recommended deployment]({{< relref "#recommended-deployment-option" >}}) section. ``` module "wandb_infra" { @@ -259,9 +259,9 @@ The [Terraform Module](https://github.com/wandb/terraform-aws-wandb) provides se To use an Amazon S3 bucket as a file storage backend for W&B, you will need to: -* [Create an Amazon S3 Bucket and Bucket Notifications](#create-an-s3-bucket-and-bucket-notifications) -* [Create SQS Queue](#create-an-sqs-queue) -* [Grant Permissions to Node Running W&B](#grant-permissions-to-node-that-runs-wb) +* [Create an Amazon S3 Bucket and Bucket Notifications]({{< relref "#create-an-s3-bucket-and-bucket-notifications" >}}) +* [Create SQS Queue]({{< relref "#create-an-sqs-queue" >}}) +* [Grant Permissions to Node Running W&B]({{< relref "#grant-permissions-to-node-that-runs-wb" >}}) you'll need to create a bucket, along with an SQS queue configured to receive object creation notifications from that bucket. Your instance will need permissions to read from this queue. @@ -382,14 +382,14 @@ Follow the steps outlined here to update W&B: Alternatively, you can add the `wandb_version` to the `terraform.tfvars` and create a variable with the same name and instead of using the literal value, use the `var.wandb_version` {{% /alert %}} -2. After you update your configuration, complete the steps described in the [Recommended deployment section](#recommended-deployment-option). +2. After you update your configuration, complete the steps described in the [Recommended deployment section]({{< relref "#recommended-deployment-option" >}}). ## Migrate to operator-based AWS Terraform modules This section details the steps required to upgrade from _pre-operator_ to _post-operator_ environments using the [terraform-aws-wandb](https://registry.terraform.io/modules/wandb/wandb/aws/latest) module. {{% alert %}} -The transition to a Kubernetes [operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) pattern is necessary for the W&B architecture. See [this section](../operator.md#reasons-for-the-architecture-shift) for a detailed explanation for the architecture shift. +The transition to a Kubernetes [operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) pattern is necessary for the W&B architecture. See [this section]({{< relref "../kubernetes-operator/#reasons-for-the-architecture-shift" >}}) for a detailed explanation for the architecture shift. {{% /alert %}} diff --git a/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/azure-tf.md b/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/azure-tf.md index 3664cbeb3..85cf78c21 100644 --- a/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/azure-tf.md +++ b/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/azure-tf.md @@ -9,7 +9,7 @@ weight: 30 --- {{% alert %}} -W&B recommends fully managed deployment options such as [W&B Multi-tenant Cloud](../hosting-options/saas_cloud.md) or [W&B Dedicated Cloud](../hosting-options//dedicated_cloud.md) deployment types. W&B fully managed services are simple and secure to use, with minimum to no configuration required. +W&B recommends fully managed deployment options such as [W&B Multi-tenant Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) or [W&B Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud/" >}}) deployment types. W&B fully managed services are simple and secure to use, with minimum to no configuration required. {{% /alert %}} @@ -174,7 +174,7 @@ This is the most straightforward deployment option configuration that will creat Another deployment option uses `Redis` to cache the SQL queries and speed up the application response when loading the metrics for the experiments. -You must add the option `create_redis = true` to the same `main.tf` file that you used in [recommended deployment](#recommended-deployment) to enable the cache. +You must add the option `create_redis = true` to the same `main.tf` file that you used in [recommended deployment]({{< relref "#recommended-deployment" >}}) to enable the cache. ```bash # Spin up all required services @@ -197,7 +197,7 @@ module "wandb" { Deployment option 3 consists of enabling the external `message broker`. This is optional because the W&B brings embedded a broker. This option doesn't bring a performance improvement. -The Azure resource that provides the message broker is the `Azure Event Grid`, and to enable it, you must add the option `use_internal_queue = false` to the same `main.tf` that you used in the [recommended deployment](#recommended-deployment) +The Azure resource that provides the message broker is the `Azure Event Grid`, and to enable it, you must add the option `use_internal_queue = false` to the same `main.tf` that you used in the [recommended deployment]({{< relref "#recommended-deployment" >}}) ```bash # Spin up all required services module "wandb" { @@ -219,4 +219,4 @@ module "wandb" { ## Other deployment options You can combine all three deployment options adding all configurations to the same file. -The [Terraform Module](https://github.com/wandb/terraform-azure-wandb) provides several options that you can combine along with the standard options and the minimal configuration found in [recommended deployment](#recommended-deployment) \ No newline at end of file +The [Terraform Module](https://github.com/wandb/terraform-azure-wandb) provides several options that you can combine along with the standard options and the minimal configuration found in [recommended deployment]({{< relref "#recommended-deployment" >}}) \ No newline at end of file diff --git a/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/gcp-tf.md b/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/gcp-tf.md index e592d98d7..1819b583d 100644 --- a/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/gcp-tf.md +++ b/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/gcp-tf.md @@ -9,7 +9,7 @@ weight: 20 --- {{% alert %}} -W&B recommends fully managed deployment options such as [W&B Multi-tenant Cloud](../hosting-options/saas_cloud.md) or [W&B Dedicated Cloud](../hosting-options//dedicated_cloud.md) deployment types. W&B fully managed services are simple and secure to use, with minimum to no configuration required. +W&B recommends fully managed deployment options such as [W&B Multi-tenant Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) or [W&B Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud/" >}}) deployment types. W&B fully managed services are simple and secure to use, with minimum to no configuration required. {{% /alert %}} @@ -115,7 +115,7 @@ This is the most straightforward deployment option configuration that will creat 1. Create the `main.tf` - In the same directory where you created the files in the [General Steps](#general-steps), create a file `main.tf` with the following content: + In the same directory where you created the files in the [General Steps]({{< relref "#general-steps" >}}), create a file `main.tf` with the following content: ``` provider "google" { @@ -176,7 +176,7 @@ This is the most straightforward deployment option configuration that will creat Another deployment option uses `Redis` to cache the SQL queries and speedup the application response when loading the metrics for the experiments. -You need to add the option `create_redis = true` to the same `main.tf` file specified in the recommended [Deployment option section](#deployment---recommended-20-mins) to enable the cache. +You need to add the option `create_redis = true` to the same `main.tf` file specified in the recommended [Deployment option section]({{< relref "#deployment---recommended-20-mins" >}}) to enable the cache. ``` [...] @@ -201,7 +201,7 @@ module "wandb" { Deployment option 3 consists of enabling the external `message broker`. This is optional because the W&B brings embedded a broker. This option doesn't bring a performance improvement. -The GCP resource that provides the message broker is the `Pub/Sub`, and to enable it, you will need to add the option `use_internal_queue = false` to the same `main.tf` specified in the recommended [Deployment option section](#deployment---recommended-20-mins) +The GCP resource that provides the message broker is the `Pub/Sub`, and to enable it, you will need to add the option `use_internal_queue = false` to the same `main.tf` specified in the recommended [Deployment option section]({{< relref "#deployment---recommended-20-mins" >}}) ``` [...] @@ -235,9 +235,9 @@ The [Terraform Module](https://github.com/wandb/terraform-google-wandb) provides To use a GCP Storage bucket as a file storage backend for W&B, you will need to create a: -* [PubSub Topic and Subscription](#create-pubsub-topic-and-subscription) -* [Storage Bucket](#create-storage-bucket) -* [PubSub Notification](#create-pubsub-notification) +* [PubSub Topic and Subscription]({{< relref "#create-pubsub-topic-and-subscription" >}}) +* [Storage Bucket]({{< relref "#create-storage-bucket" >}}) +* [PubSub Notification]({{< relref "#create-pubsub-notification" >}}) ### Create PubSub Topic and Subscription @@ -336,4 +336,4 @@ Follow the steps outlined here to update W&B: Alternatively, you can add the `wandb_version` to the `terraform.tfvars` and create a variable with the same name and instead of using the literal value, use the `var.wandb_version` {{% /alert %}} -2. After you update your configuration, complete the steps described in the [Deployment option section](#deployment---recommended-20-mins). \ No newline at end of file +2. After you update your configuration, complete the steps described in the [Deployment option section]({{< relref "#deployment---recommended-20-mins" >}}). \ No newline at end of file diff --git a/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/ref-arch.md b/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/ref-arch.md index 4b70e9aa9..3332a2971 100644 --- a/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/ref-arch.md +++ b/content/guides/hosting/hosting-options/self-managed/install-on-public-cloud/ref-arch.md @@ -21,11 +21,11 @@ Consider carefully whether a self-managed approach with W&B is suitable for your A strong understanding of how to run and maintain production-grade application is an important prerequisite before you deploy self-managed W&B. If your team needs assistance, our Professional Services team and partners offer support for implementation and optimization. -To learn more about managed solutions for running W&B instead of managing it yourself, refer to [W&B Multi-tenant Cloud](../hosting-options/saas_cloud.md) and [W&B Dedicated Cloud](../hosting-options/dedicated_cloud.md). +To learn more about managed solutions for running W&B instead of managing it yourself, refer to [W&B Multi-tenant Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) and [W&B Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}). ## Infrastructure -![W&B infrastructure diagram](/images/hosting/reference_architecture.png) +{{< img src="/images/hosting/reference_architecture.png" alt="W&B infrastructure diagram" >}} ### Application layer @@ -38,7 +38,7 @@ The storage layer consists of a MySQL database and object storage. The MySQL dat ## Infrastructure requirements ### Kubernetes -The W&B Server application is deployed as a [Kubernetes Operator](../operator.md) that deploys multiple Pods. For this reason, W&B requires a Kubernetes cluster with: +The W&B Server application is deployed as a [Kubernetes Operator]({{< relref "../kubernetes-operator/" >}}) that deploys multiple Pods. For this reason, W&B requires a Kubernetes cluster with: - A fully configured and functioning Ingress controller - The capability to provision Persistent Volumes. @@ -84,7 +84,7 @@ Please note: W&B does not recommend the use self-signed certificates and custom W&B runs on the Intel (x86) CPU architecture. ARM is not supported. ## Infrastructure provisioning -Terraform is the recommended way to deploy W&B for production. Using Terraform, you define the required resources, their references to other resources, and their dependencies. W&B provides Terraform modules for the major cloud providers. For details, refer to [Deploy W&B Server within self managed cloud accounts](../hosting-options/self-managed.md#deploy-wb-server-within-self-managed-cloud-accounts). +Terraform is the recommended way to deploy W&B for production. Using Terraform, you define the required resources, their references to other resources, and their dependencies. W&B provides Terraform modules for the major cloud providers. For details, refer to [Deploy W&B Server within self managed cloud accounts]({{< relref "/guides/hosting/hosting-options/self-managed.md#deploy-wb-server-within-self-managed-cloud-accounts" >}}). ## Sizing diff --git a/content/guides/hosting/hosting-options/self-managed/kubernetes-operator/_index.md b/content/guides/hosting/hosting-options/self-managed/kubernetes-operator/_index.md index 511cb12cb..7ef30fc46 100644 --- a/content/guides/hosting/hosting-options/self-managed/kubernetes-operator/_index.md +++ b/content/guides/hosting/hosting-options/self-managed/kubernetes-operator/_index.md @@ -46,30 +46,30 @@ metadata: The `controller-manager` installs [charts/operator-wandb](https://github.com/wandb/helm-charts/tree/main/charts/operator-wandb) based on the spec of the custom resource, release channel, and a user defined config. The configuration specification hierarchy enables maximum configuration flexibility at the user end and enables W&B to release new images, configurations, features, and Helm updates automatically. -Refer to the [configuration specification hierarchy](#configuration-specification-hierarchy) and [configuration reference](#configuration-reference-for-wb-operator) for configuration options. +Refer to the [configuration specification hierarchy]({{< relref "#configuration-specification-hierarchy" >}}) and [configuration reference]({{< relref "#configuration-reference-for-wb-operator" >}}) for configuration options. ### Configuration specification hierarchy Configuration specifications follow a hierarchical model where higher-level specifications override lower-level ones. Here’s how it works: - **Release Channel Values**: This base level configuration sets default values and configurations based on the release channel set by W&B for the deployment. - **User Input Values**: Users can override the default settings provided by the Release Channel Spec through the System Console. -- **Custom Resource Values**: The highest level of specification, which comes from the user. Any values specified here override both the User Input and Release Channel specifications. For a detailed description of the configuration options, see [Configuration Reference](#configuration-reference-for-wb-operator). +- **Custom Resource Values**: The highest level of specification, which comes from the user. Any values specified here override both the User Input and Release Channel specifications. For a detailed description of the configuration options, see [Configuration Reference]({{< relref "#configuration-reference-for-wb-operator" >}}). This hierarchical model ensures that configurations are flexible and customizable to meet varying needs while maintaining a manageable and systematic approach to upgrades and changes. ### Requirements to use the W&B Kubernetes Operator Satisfy the following requirements to deploy W&B with the W&B Kubernetes operator: -Refer to the [reference architecture](./self-managed/ref-arch.md#infrastructure-requirements). In addition, [obtain a valid W&B Server license](./hosting-options/self-managed.md#obtain-your-wb-server-license). +Refer to the [reference architecture]({{< relref "../install-on-public-cloud/ref-arch.md#infrastructure-requirements" >}}). In addition, [obtain a valid W&B Server license]({{< relref "../#obtain-your-wb-server-license" >}}). -See [this](./self-managed/bare-metal.md) guide for a detailed explanation on how to set up and configure a self-managed installation. +See [this]({{< relref "../bare-metal.md" >}}) guide for a detailed explanation on how to set up and configure a self-managed installation. Depending on the installation method, you might need to meet the following requirements: * Kubectl installed and configured with the correct Kubernetes cluster context. * Helm is installed. ### Air-gapped installations -See the [Deploy W&B in airgapped environment with Kubernetes](./operator-airgapped) tutorial on how to install the W&B Kubernetes Operator in an airgapped environment. +See the [Deploy W&B in airgapped environment with Kubernetes]({{< relref "./operator-airgapped" >}}) tutorial on how to install the W&B Kubernetes Operator in an airgapped environment. ## Deploy W&B Server application This section describes different ways to deploy the W&B Kubernetes operator. @@ -78,9 +78,9 @@ The W&B Operator is the default and recommended installation method for W&B Serv {{% /alert %}} **Choose one of the following:** -- If you have provisioned all required external services and want to deploy W&B onto Kubernetes with Helm CLI, continue [here](#deploy-wb-with-helm-cli). -- If you prefer managing infrastructure and the W&B Server with Terraform, continue [here](#deploy-wb-with-helm-terraform-module). -- If you want to utilize the W&B Cloud Terraform Modules, continue [here](#deploy-wb-with-wb-cloud-terraform-modules). +- If you have provisioned all required external services and want to deploy W&B onto Kubernetes with Helm CLI, continue [here]({{< relref "#deploy-wb-with-helm-cli" >}}). +- If you prefer managing infrastructure and the W&B Server with Terraform, continue [here]({{< relref "#deploy-wb-with-helm-terraform-module" >}}). +- If you want to utilize the W&B Cloud Terraform Modules, continue [here]({{< relref "#deploy-wb-with-wb-cloud-terraform-modules" >}}). ### Deploy W&B with Helm CLI W&B provides a Helm Chart to deploy the W&B Kubernetes operator to a Kubernetes cluster. This approach allows you to deploy W&B Server with Helm CLI or a continuous delivery tool like ArgoCD. Make sure that the above mentioned requirements are in place. @@ -96,7 +96,7 @@ helm repo update ```shell helm upgrade --install operator wandb/operator -n wandb-cr --create-namespace ``` -3. Configure the W&B operator custom resource to trigger the W&B Server installation. Create an operator.yaml file to customize the W&B Operator deployment, specifying your custom configuration. See [Configuration Reference](#configuration-reference-for-wb-operator) for details. +3. Configure the W&B operator custom resource to trigger the W&B Server installation. Create an operator.yaml file to customize the W&B Operator deployment, specifying your custom configuration. See [Configuration Reference]({{< relref "#configuration-reference-for-wb-operator" >}}) for details. Once you have the specification YAML created and filled with your values, run the following and the operator applies the configuration and install the W&B Server application based on your configuration. @@ -106,7 +106,7 @@ helm upgrade --install operator wandb/operator -n wandb-cr --create-namespace Wait until the deployment completes. This takes a few minutes. -4. To verify the installation using the web UI, create the first admin user account, then follow the verification steps outlined in [Verify the installation](#verify-the-installation). +4. To verify the installation using the web UI, create the first admin user account, then follow the verification steps outlined in [Verify the installation]({{< relref "#verify-the-installation" >}}). ### Deploy W&B with Helm Terraform Module @@ -145,7 +145,7 @@ module "wandb" { } ``` -Note that the configuration options are the same as described in [Configuration Reference](#configuration-reference-for-wb-operator), but that the syntax has to follow the HashiCorp Configuration Language (HCL). The Terraform module creates the W&B custom resource definition (CRD). +Note that the configuration options are the same as described in [Configuration Reference]({{< relref "#configuration-reference-for-wb-operator" >}}), but that the syntax has to follow the HashiCorp Configuration Language (HCL). The Terraform module creates the W&B custom resource definition (CRD). To see how W&B&Biases themselves use the Helm Terraform module to deploy “Dedicated cloud” installations for customers, follow those links: - [AWS](https://github.com/wandb/terraform-aws-wandb/blob/45e1d746f53e78e73e68f911a1f8cad5408e74b6/main.tf#L225) @@ -164,11 +164,11 @@ W&B provides a set of Terraform Modules for AWS, GCP and Azure. Those modules de This integration ensures that W&B Kubernetes Operator is ready to use for your instance with minimal setup, providing a streamlined path to deploying and managing W&B Server in your cloud environment. -For a detailed description on how to use these modules, refer to this [section](./hosting-options/self-managed.md#deploy-wb-server-within-self-managed-cloud-accounts) to self-managed installations section in the docs. +For a detailed description on how to use these modules, refer to this [section]({{< relref "../#deploy-wb-server-within-self-managed-cloud-accounts" >}}) to self-managed installations section in the docs. ### Verify the installation -To verify the installation, W&B recommends using the [W&B CLI](../../ref/cli/README.md). The verify command executes several tests that verify all components and configurations. +To verify the installation, W&B recommends using the [W&B CLI]({{< relref "/ref/cli/" >}}). The verify command executes several tests that verify all components and configurations. {{% alert %}} This step assumes that the first admin user account is created with the browser. @@ -245,7 +245,7 @@ This section describes how to update the W&B Kubernetes operator. {{% alert %}} * Updating the W&B Kubernetes operator does not update the W&B server application. -* See the instructions [here](#migrate-self-managed-instances-to-wb-operator) if you use a Helm chart that does not user the W&B Kubernetes operator before you follow the proceeding instructions to update the W&B operator. +* See the instructions [here]({{< relref "#migrate-self-managed-instances-to-wb-operator" >}}) if you use a Helm chart that does not user the W&B Kubernetes operator before you follow the proceeding instructions to update the W&B operator. {{% /alert %}} Copy and paste the code snippets below into your terminal. @@ -274,17 +274,17 @@ The W&B Operator is the default and recommended installation method for W&B Serv {{% /alert %}} - If you used the official W&B Cloud Terraform Modules, navigate to the appropriate documentation and follow the steps there: - - [AWS](#migrate-to-operator-based-aws-terraform-modules) - - [GCP](#migrate-to-operator-based-gcp-terraform-modules) - - [Azure](#migrate-to-operator-based-azure-terraform-modules) -- If you used the [W&B Non-Operator Helm chart](https://github.com/wandb/helm-charts/tree/main/charts/wandb), continue [here](#migrate-to-operator-based-helm-chart). -- If you used the [W&B Non-Operator Helm chart with Terraform](https://registry.terraform.io/modules/wandb/wandb/kubernetes/latest), continue [here](#migrate-to-operator-based-terraform-helm-chart). -- If you created the Kubernetes resources with manifests, continue [here](#migrate-to-operator-based-helm-chart). + - [AWS]({{< relref "#migrate-to-operator-based-aws-terraform-modules" >}}) + - [GCP]({{< relref "#migrate-to-operator-based-gcp-terraform-modules" >}}) + - [Azure]({{< relref "#migrate-to-operator-based-azure-terraform-modules" >}}) +- If you used the [W&B Non-Operator Helm chart](https://github.com/wandb/helm-charts/tree/main/charts/wandb), continue [here]({{< relref "#migrate-to-operator-based-helm-chart" >}}). +- If you used the [W&B Non-Operator Helm chart with Terraform](https://registry.terraform.io/modules/wandb/wandb/kubernetes/latest), continue [here]({{< relref "#migrate-to-operator-based-terraform-helm-chart" >}}). +- If you created the Kubernetes resources with manifests, continue [here]({{< relref "#migrate-to-operator-based-helm-chart" >}}). ### Migrate to Operator-based AWS Terraform Modules -For a detailed description of the migration process, continue [here](self-managed/aws-tf.md#migrate-to-operator-based-aws-terraform-modules). +For a detailed description of the migration process, continue [here]({{< relref "../install-on-public-cloud/aws-tf.md#migrate-to-operator-based-aws-terraform-modules" >}}). ### Migrate to Operator-based GCP Terraform Modules @@ -309,7 +309,7 @@ Follow these steps to migrate to the Operator-based Helm chart: ``` You now have all the configuration values you need for the next step. -2. Create a file called `operator.yaml`. Follow the format described in the [Configuration Reference](#configuration-reference-for-wb-operator). Use the values from step 1. +2. Create a file called `operator.yaml`. Follow the format described in the [Configuration Reference]({{< relref "#configuration-reference-for-wb-operator" >}}). Use the values from step 1. 3. Scale the current deployment to 0 pods. This step is stops the current deployment. ```shell @@ -329,7 +329,7 @@ Follow these steps to migrate to the Operator-based Helm chart: ``` The deployment takes a few minutes to complete. -7. Verify the installation. Make sure that everything works by following the steps in [Verify the installation](#verify-the-installation). +7. Verify the installation. Make sure that everything works by following the steps in [Verify the installation]({{< relref "#verify-the-installation" >}}). 8. Remove to old installation. Uninstall the old helm chart or delete the resources that were created with manifests. @@ -338,18 +338,18 @@ Follow these steps to migrate to the Operator-based Helm chart: Follow these steps to migrate to the Operator-based Helm chart: -1. Prepare Terraform config. Replace the Terraform code from the old deployment in your Terraform config with the one that is described [here](#deploy-wb-with-helm-terraform-module). Set the same variables as before. Do not change .tfvars file if you have one. +1. Prepare Terraform config. Replace the Terraform code from the old deployment in your Terraform config with the one that is described [here]({{< relref "#deploy-wb-with-helm-terraform-module" >}}). Set the same variables as before. Do not change .tfvars file if you have one. 2. Execute Terraform run. Execute terraform init, plan and apply -3. Verify the installation. Make sure that everything works by following the steps in [Verify the installation](#verify-the-installation). +3. Verify the installation. Make sure that everything works by following the steps in [Verify the installation]({{< relref "#verify-the-installation" >}}). 4. Remove to old installation. Uninstall the old helm chart or delete the resources that were created with manifests. ## Configuration Reference for W&B Server -This section describes the configuration options for W&B Server application. The application receives its configuration as custom resource definition named [WeightsAndBiases](#how-it-works). Some configuration options are exposed with the below configuration, some need to be set as environment variables. +This section describes the configuration options for W&B Server application. The application receives its configuration as custom resource definition named [WeightsAndBiases]({{< relref "#how-it-works" >}}). Some configuration options are exposed with the below configuration, some need to be set as environment variables. -The documentation has two lists of environment variables: [basic](./env-vars.md) and [advanced](./iam/advanced_env_vars.md). Only use environment variables if the configuration option that you need are not exposed using Helm Chart. +The documentation has two lists of environment variables: [basic]({{< relref "/guides/hosting/env-vars/" >}}) and [advanced]({{< relref "/guides/hosting/iam/advanced_env_vars/" >}}). Only use environment variables if the configuration option that you need are not exposed using Helm Chart. The W&B Server application configuration file for a production deployment requires the following contents. This YAML file defines the desired state of your W&B deployment, including the version, environment variables, external resources like databases, and other necessary settings. @@ -497,7 +497,7 @@ global: ### Ingress -To identify the ingress class, see this FAQ [entry](#how-to-identify-the-kubernetes-ingress-class). +To identify the ingress class, see this FAQ [entry]({{< relref "#how-to-identify-the-kubernetes-ingress-class" >}}). **Without TLS** @@ -798,7 +798,7 @@ customCACerts: ## FAQ ### How to get the W&B Operator Console password -See [Accessing the W&B Kubernetes Operator Management Console](#access-the-wb-management-console). +See [Accessing the W&B Kubernetes Operator Management Console]({{< relref "#access-the-wb-management-console" >}}). ### How to access the W&B Operator Console if Ingress doesn’t work @@ -811,7 +811,7 @@ kubectl port-forward svc/wandb-console 8082 Access the console in the browser with `https://localhost:8082/` console. -See [Accessing the W&B Kubernetes Operator Management Console](#access-the-wb-management-console) on how to get the password (Option 2). +See [Accessing the W&B Kubernetes Operator Management Console]({{< relref "#access-the-wb-management-console" >}}) on how to get the password (Option 2). ### How to view W&B Server logs diff --git a/content/guides/hosting/hosting-options/self-managed/server-upgrade-process.md b/content/guides/hosting/hosting-options/self-managed/server-upgrade-process.md index c0de6bccc..336e41128 100644 --- a/content/guides/hosting/hosting-options/self-managed/server-upgrade-process.md +++ b/content/guides/hosting/hosting-options/self-managed/server-upgrade-process.md @@ -14,8 +14,8 @@ Update your W&B Server Version and License with the same method you installed W& | Release Type | Description | | ---------------- | ------------------ | -| [Terraform](#update-with-terraform) | W&B supports three public Terraform modules for cloud deployment: [AWS](https://registry.terraform.io/modules/wandb/wandb/aws/latest), [GCP](https://registry.terraform.io/modules/wandb/wandb/google/latest), and [Azure](https://registry.terraform.io/modules/wandb/wandb/azurerm/latest). | -| [Helm](#update-with-helm) | You can use the [Helm Chart](https://github.com/wandb/helm-charts) to install W&B into an existing Kubernetes cluster. | +| [Terraform]({{< relref "#update-with-terraform" >}}) | W&B supports three public Terraform modules for cloud deployment: [AWS](https://registry.terraform.io/modules/wandb/wandb/aws/latest), [GCP](https://registry.terraform.io/modules/wandb/wandb/google/latest), and [Azure](https://registry.terraform.io/modules/wandb/wandb/azurerm/latest). | +| [Helm]({{< relref "#update-with-helm" >}}) | You can use the [Helm Chart](https://github.com/wandb/helm-charts) to install W&B into an existing Kubernetes cluster. | ## Update with Terraform diff --git a/content/guides/hosting/iam/_index.md b/content/guides/hosting/iam/_index.md index ed444bb4c..59743b9d0 100644 --- a/content/guides/hosting/iam/_index.md +++ b/content/guides/hosting/iam/_index.md @@ -10,17 +10,17 @@ cascade: weight: 2 --- -W&B Platform has three IAM scopes within W&B: [Organizations](#organization), [Teams](#team), and [Projects](#project). +W&B Platform has three IAM scopes within W&B: [Organizations]({{< relref "#organization" >}}), [Teams]({{< relref "#team" >}}), and [Projects]({{< relref "#project" >}}). ## Organization An *Organization* is the root scope in your W&B account or instance. All actions in your account or instance take place within the context of that root scope, including managing users, managing teams, managing projects within teams, tracking usage and more. -If you are using [Multi-tenant Cloud](../hosting-options/saas_cloud.md), you may have more than one organization where each may correspond to a business unit, a personal user, a joint partnership with another business and more. +If you are using [Multi-tenant Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}), you may have more than one organization where each may correspond to a business unit, a personal user, a joint partnership with another business and more. -If you are using [Dedicated Cloud](../hosting-options/dedicated_cloud.md) or a [Self-managed instance](../hosting-options/self-managed.md), it corresponds to one organization. Your company may have more than one of Dedicated Cloud or Self-managed instances to map to different business units or departments, though that is strictly an optional way to manage AI practioners across your businesses or departments. +If you are using [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or a [Self-managed instance]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}), it corresponds to one organization. Your company may have more than one of Dedicated Cloud or Self-managed instances to map to different business units or departments, though that is strictly an optional way to manage AI practioners across your businesses or departments. -For more information, see [Manage orrganizations](./manage-organization.md). +For more information, see [Manage orrganizations]({{< relref "./access-management/manage-organization.md" >}}). ## Team @@ -28,11 +28,11 @@ A *Team* is a subscope within a organization, that may map to a business unit / AI projects are organized within the context of a team. The access control within a team is governed by team admins, who may or may not be admins at the parent organization level. -For more information, see [Add and manage teams](./manage-organization.md#add-and-manage-teams). +For more information, see [Add and manage teams]({{< relref "./access-management/manage-organization.md#add-and-manage-teams" >}}). ## Project A *Project* is a subscope within a team, that maps to an actual AI project with specific intended outcomes. You may have more than one project within a team. Each project has a visibility mode which determines who can access it. -Every project is comprised of [Workspaces](../../track/workspaces.md) and [Reports](../../reports/intro.md), and is linked to relevant [Artifacts](../../artifacts/intro.md), [Sweeps](../../sweeps/intro.md), [Launch Jobs](../../launch/intro.md) and [Automations](../../artifacts/project-scoped-automations.md). \ No newline at end of file +Every project is comprised of [Workspaces]({{< relref "/guides/models/track/workspaces.md" >}}) and [Reports]({{< relref "/guides/core/reports/" >}}), and is linked to relevant [Artifacts]({{< relref "/guides/core/artifacts/" >}}), [Sweeps]({{< relref "/guides/models/sweeps/" >}}), [Launch Jobs]({{< relref "/launch/" >}}) and [Automations]({{< relref "/guides/models/automations/project-scoped-automations.md" >}}). \ No newline at end of file diff --git a/content/guides/hosting/iam/access-management/_index.md b/content/guides/hosting/iam/access-management/_index.md index bd94ec5c5..912848e36 100644 --- a/content/guides/hosting/iam/access-management/_index.md +++ b/content/guides/hosting/iam/access-management/_index.md @@ -20,7 +20,7 @@ W&B recommends to have more than one instance admin in an organization. It is a A *team administrator* is a user in organization that has administrative permissions within a team. -The organization administrator can access and use an organization's account settings at `https://wandb.ai/account-settings/` to invite users, assign or update a user's role, create teams, remove users from your organization, assign the billing administrator, and more. See [Add and manage users](./manage-organization.md#add-and-manage-users) for more information. +The organization administrator can access and use an organization's account settings at `https://wandb.ai/account-settings/` to invite users, assign or update a user's role, create teams, remove users from your organization, assign the billing administrator, and more. See [Add and manage users]({{< relref "./manage-organization.md#add-and-manage-users" >}}) for more information. Once an organization administrator creates a team, the instance administrator or ateam administrator can: @@ -28,7 +28,7 @@ Once an organization administrator creates a team, the instance administrator or - Assign or update a team member's role. - Automatically add new users to a team when they join your organization. -Both the organization administrator and the team administrator use team dashboards at `https://wandb.ai/` to manage teams. For more information on what organization administrators and team administrators can do, see [Add and manage teams](./manage-organization.md#add-and-manage-teams). +Both the organization administrator and the team administrator use team dashboards at `https://wandb.ai/` to manage teams. For more information on what organization administrators and team administrators can do, see [Add and manage teams]({{< relref "./manage-organization.md#add-and-manage-teams" >}}). ## Limit visibility to specific projects @@ -37,4 +37,4 @@ Define the scope of a W&B project to limit who can view, edit, and submit W&B ru An organization admin, team admin, or the owner of a project can both set and edit a project's visibility. -For more information, see [Project visibility](./restricted-projects.md). \ No newline at end of file +For more information, see [Project visibility]({{< relref "./restricted-projects.md" >}}). \ No newline at end of file diff --git a/content/guides/hosting/iam/access-management/manage-organization.md b/content/guides/hosting/iam/access-management/manage-organization.md index b27b08c8d..fed814e95 100644 --- a/content/guides/hosting/iam/access-management/manage-organization.md +++ b/content/guides/hosting/iam/access-management/manage-organization.md @@ -6,15 +6,15 @@ menu: title: Manage your organization weight: 1 --- -As an administrator of an organization you can [manage individual users](#add-and-manage-users) within your organization and [manage teams](#add-and-manage-teams). +As an administrator of an organization you can [manage individual users]({{< relref "#add-and-manage-users" >}}) within your organization and [manage teams]({{< relref "#add-and-manage-teams" >}}). -As a team administrator you can [manage teams](#add-and-manage-teams). +As a team administrator you can [manage teams]({{< relref "#add-and-manage-teams" >}}). {{% alert %}} The following workflow applies to users with instance administrator roles. Reach out to an administrator in your organization if you believe you should have instance administrator permissions. {{% /alert %}} -If you are looking to simplify user management in your organization, refer to [Automate user and team management](./automate_iam.md). +If you are looking to simplify user management in your organization, refer to [Automate user and team management]({{< relref "../automate_iam.md" >}}). @@ -62,7 +62,7 @@ Administrators can invite users to their organization, as well as specific teams 3. Select **Invite new user**. 4. In the modal that appears, provide the email or username of the user in the **Email or username** field. 5. (Recommended) Add the user to a team from the **Choose teams** dropdown menu. -6. From the **Select role** dropdown, select the role to assign to the user. You can change the user's role at a later time. See the table listed in [Assign a role](#assign-or-update-a-team-members-role) for more information about possible roles. +6. From the **Select role** dropdown, select the role to assign to the user. You can change the user's role at a later time. See the table listed in [Assign a role]({{< relref "#assign-or-update-a-team-members-role" >}}) for more information about possible roles. 7. Choose the **Send invite** button. W&B sends an invite link using a third-party email server to the user's email after you select the **Send invite** button. A user can access your organization once they accept the invite. @@ -72,7 +72,7 @@ W&B sends an invite link using a third-party email server to the user's email af 1. Navigate to `https://.io/console/settings/`. Replace `` with your organization name. 2. Select the **Add user** button 3. Within the modal that appears, provide the email of the new user in the **Email** field. -4. Select a role to assign to the user from the **Role** dropdown. You can change the user's role at a later time. See the table listed in [Assign a role](#assign-or-update-a-team-members-role) for more information about possible roles. +4. Select a role to assign to the user from the **Role** dropdown. You can change the user's role at a later time. See the table listed in [Assign a role]({{< relref "#assign-or-update-a-team-members-role" >}}) for more information about possible roles. 5. Check the **Send invite email to user** box if you want W&B to send an invite link using a third-party email server to the user's email. 6. Select the **Add new user** button. {{% /tab %}} @@ -85,7 +85,7 @@ A W&B user with matching email domain can sign in to your W&B Organization with {{% alert title="Enable SSO for authentication" %}} W&B strongly recommends and encourages that users authenticate using Single Sign-On (SSO). Reach out to your W&B team to enable SSO for your organization. -To learn more about how to setup SSO with Dedicated cloud or Self-managed instances, refer to [SSO with OIDC](./sso.md) or [SSO with LDAP](./ldap.md).{{% /alert %}} +To learn more about how to setup SSO with Dedicated cloud or Self-managed instances, refer to [SSO with OIDC]({{< relref "../authentication/sso.md" >}}) or [SSO with LDAP]({{< relref "../authentication/ldap.md" >}}).{{% /alert %}} W&B assigned auto-provisioning users "Member" roles by default. You can change the role of auto-provisioned users at any time. @@ -140,7 +140,7 @@ To automatically assign non-invited new users to a default team when they join y 2. In the upper right corner of the page, select the **User menu** dropdown. From the dropdown, choose **Settings**. 3. Within the **Settings** tab, select **General**. 4. Choose the **Claim domain** button within **Domain capture**. -5. Select the team that you want new users to automatically join from the **Default team** dropdown. If no teams are available, you'll need to update team settings. See the instructions in [Add and manage teams](#add-and-manage-teams). +5. Select the team that you want new users to automatically join from the **Default team** dropdown. If no teams are available, you'll need to update team settings. See the instructions in [Add and manage teams]({{< relref "#add-and-manage-teams" >}}). 6. Click the **Claim email domain** button. You must enable domain matching within a team's settings before you can automatically assign non-invited new users to that team. @@ -225,10 +225,6 @@ Use your organization's dashboard to create and manage teams within your organi - Automate the addition of users to a team when they join your organization. - Manage team storage with the team's dashboard at `https://wandb.ai/`. - - - - ### Create a team Use your organization's dashboard to create a team: @@ -253,16 +249,16 @@ Invite users to a team in your organization. Use the team's dashboard to invite {{< img src="/images/hosting/team_settings.png" alt="" >}} 3. Select the **Users** tab. 4. Choose on **Invite a new user**. -5. Within the modal that appears, provide the email of the user in the **Email or username** field and select the role to assign to that user from the **Select a team** role dropdown. For more information about roles a user can have in a team, see [Team roles](#assign-or-update-a-team-members-role). +5. Within the modal that appears, provide the email of the user in the **Email or username** field and select the role to assign to that user from the **Select a team** role dropdown. For more information about roles a user can have in a team, see [Team roles]({{< relref "#assign-or-update-a-team-members-role" >}}). 6. Choose on the **Send invite** button. -In addition to inviting users manually with email invites, you can automatically add new users to a team if the new user's [email matches the domain of your organization](#domain-capture). +In addition to inviting users manually with email invites, you can automatically add new users to a team if the new user's [email matches the domain of your organization]({{< relref "#domain-capture" >}}). ### Match members to a team organization during sign up Allow new users within your organization discover Teams within your organization when they sign-up. New users must have a verified email domain that matches your organization's verified email domain. Verified new users can view a list of verified teams that belong to an organization when they sign up for a W&B account. -An organization administrator must enable domain claiming. To enable domain capture, see the steps described in [Domain capture](#domain-capture). +An organization administrator must enable domain claiming. To enable domain capture, see the steps described in [Domain capture]({{< relref "#domain-capture" >}}). ### Assign or update a team member's role @@ -281,13 +277,6 @@ This table lists the roles you can assign to a member of a team: | Service (Enterprise-only feature) | A service worker or service account is an API key that is useful for utilizing W&B with your run automation tools. If you use an API key from a service account for your team, ensure to set the environment variable `WANDB_USERNAME` to correctly attribute runs to the appropriate user. | | Custom Roles (Enterprise-only feature) | Custom roles allow organization administrators to compose new roles by inheriting from the preceding View-Only or Member roles, and adding additional permissions to achieve fine-grained access control. Team administrators can then assign any of those custom roles to users in their respective teams. Refer to [this article](https://wandb.ai/wandb_fc/announcements/reports/Introducing-Custom-Roles-for-W-B-Teams--Vmlldzo2MTMxMjQ3) for details. | - - - {{% alert %}} Only enterprise licenses on Dedicated cloud or Self-managed deployment can assign custom roles to members in a team. {{% /alert %}} diff --git a/content/guides/hosting/iam/advanced_env_vars.md b/content/guides/hosting/iam/advanced_env_vars.md index 865d82855..8f2dc746c 100644 --- a/content/guides/hosting/iam/advanced_env_vars.md +++ b/content/guides/hosting/iam/advanced_env_vars.md @@ -6,7 +6,7 @@ menu: title: Advanced IAM configuration --- -In addition to basic [environment variables](../env-vars.md), you can use environment variables to configure IAM options for your [Dedicated Cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instance. +In addition to basic [environment variables]({{< relref "../env-vars.md" >}}), you can use environment variables to configure IAM options for your [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instance. Choose any of the following environment variables for your instance depending on your IAM needs. diff --git a/content/guides/hosting/iam/authentication/identity_federation.md b/content/guides/hosting/iam/authentication/identity_federation.md index f22fe5097..78361ed4a 100644 --- a/content/guides/hosting/iam/authentication/identity_federation.md +++ b/content/guides/hosting/iam/authentication/identity_federation.md @@ -49,7 +49,7 @@ As part of the workflow to exchange the JWT for a W&B access token and then acce * The JWT signature is verified using the JWKS at the W&B organization level. This is the first line of defense, and if this fails, that means there's a problem with your JWKS or how your JWT is signed. * The `iss` claim in the JWT should be equal to the issuer URL configured at the organization level. * The `sub` claim in the JWT should be equal to the user's email address as configured in the W&B organization. -* The `aud` claim in the JWT should be equal to the name of the W&B organization which houses the project that you are accessing as part of your AI workflow. In case of [Dedicated Cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instances, you could configure an instance-level environment variable `SKIP_AUDIENCE_VALIDATION` to `true` to skip validation of the audience claim, or use `wandb` as the audience. +* The `aud` claim in the JWT should be equal to the name of the W&B organization which houses the project that you are accessing as part of your AI workflow. In case of [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instances, you could configure an instance-level environment variable `SKIP_AUDIENCE_VALIDATION` to `true` to skip validation of the audience claim, or use `wandb` as the audience. * The `exp` claim in the JWT is checked to see if the token is valid or has expired and needs to be refreshed. ## External service accounts @@ -62,9 +62,9 @@ To configure an external service account: * Press `New service account` * Provide a name for the service account, select `Federated Identity` as the `Authentication Method`, provide a `Subject`, and press `Create` -The `sub` claim in the external service account's JWT should be same as what the team admin configures as its subject in the team-level **Service Accounts** tab. That claim is verified as part of [JWT validation](#jwt-validation). The `aud` claim requirement is similar to that for human user JWTs. +The `sub` claim in the external service account's JWT should be same as what the team admin configures as its subject in the team-level **Service Accounts** tab. That claim is verified as part of [JWT validation]({{< relref "#jwt-validation" >}}). The `aud` claim requirement is similar to that for human user JWTs. -When [using an external service account's JWT to access W&B](#using-the-jwt-to-access-wb), it's typically easier to automate the workflow to generate the initial JWT and continuously refresh it. If you would like to attribute the runs logged using an external service account to a human user, you can configure the environment variables `WANDB_USERNAME` or `WANDB_USER_EMAIL` for your AI workflow, similar to how it's done for the built-in service accounts. +When [using an external service account's JWT to access W&B]({{< relref "#using-the-jwt-to-access-wb" >}}), it's typically easier to automate the workflow to generate the initial JWT and continuously refresh it. If you would like to attribute the runs logged using an external service account to a human user, you can configure the environment variables `WANDB_USERNAME` or `WANDB_USER_EMAIL` for your AI workflow, similar to how it's done for the built-in service accounts. {{% alert %}} W&B recommends to use a mix of built-in and external service accounts across your AI workloads with different levels of data sensitivity, in order to strike a balance between flexibility and simplicity. diff --git a/content/guides/hosting/iam/authentication/ldap.md b/content/guides/hosting/iam/authentication/ldap.md index 5133d530a..1d0bc1fae 100644 --- a/content/guides/hosting/iam/authentication/ldap.md +++ b/content/guides/hosting/iam/authentication/ldap.md @@ -42,7 +42,7 @@ Set LDAP an connection with the following environment variables: | `LOCAL_LDAP_GROUP_ALLOW_LIST` | No | | | `LOCAL_LDAP_LOGIN` | No | | -See the [Configuration parameters](#configuration-parameters) section for definitions of each environment variable. Note that the environment variable prefix `LOCAL_LDAP` was omitted from the definition names for clarity. +See the [Configuration parameters]({{< relref "#configuration-parameters" >}}) section for definitions of each environment variable. Note that the environment variable prefix `LOCAL_LDAP` was omitted from the definition names for clarity. {{% /tab %}} {{< /tabpane >}} diff --git a/content/guides/hosting/iam/authentication/service-accounts.md b/content/guides/hosting/iam/authentication/service-accounts.md index 32ce5d354..3bee5a7d0 100644 --- a/content/guides/hosting/iam/authentication/service-accounts.md +++ b/content/guides/hosting/iam/authentication/service-accounts.md @@ -11,15 +11,15 @@ A service account represents a non-human or machine user that can automatically A service account's API key allows the caller to read from or write to projects within the service account's scope. -Service accounts allow for centralized management of workflows by multiple users or teams, to automate experiment tracking for W&B Models or to log traces for W&B Weave. You have the option to associate a human user's identity with a workflow managed by a service account, by using either of the [environment variables](../../track/environment-variables.md) `WANDB_USERNAME` or `WANDB_USER_EMAIL`. +Service accounts allow for centralized management of workflows by multiple users or teams, to automate experiment tracking for W&B Models or to log traces for W&B Weave. You have the option to associate a human user's identity with a workflow managed by a service account, by using either of the [environment variables]({{< relref "/guides/models/track/environment-variables.md" >}}) `WANDB_USERNAME` or `WANDB_USER_EMAIL`. {{% alert %}} -Service accounts are available on [Dedicated Cloud](../hosting-options/dedicated_cloud.md), [Self-managed instances](../hosting-options/self-managed.md) with an enterprise license, and enterprise accounts in [SaaS Cloud](../hosting-options/saas_cloud.md). +Service accounts are available on [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}), [Self-managed instances]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) with an enterprise license, and enterprise accounts in [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}). {{% /alert %}} ## Organization-scoped service accounts -Service accounts scoped to an organization have permissions to read and write in all projects in the organization, regardless of the team, with the exception of [restricted projects](./restricted-projects.md#visibility-scopes). Before an organization-scoped service account can access a restricted project, an admin of that project must explicitly add the service account to the project. +Service accounts scoped to an organization have permissions to read and write in all projects in the organization, regardless of the team, with the exception of [restricted projects]({{< relref "../access-management/restricted-projects.md#visibility-scopes" >}}). Before an organization-scoped service account can access a restricted project, an admin of that project must explicitly add the service account to the project. An organization admin can obtain the API key for an organization-scoped service account from the **Service Accounts** tab of the organization or account dashboard. @@ -38,7 +38,7 @@ An organization-scoped service account requires a default team, even though it h ## Team-scoped service accounts -A team-scoped service account can read and write in all projects within its team, except to [restricted projects](./restricted-projects.md#visibility-scopes) in that team. Before a team-scoped service account can access a restricted project, an admin of that project must explicitly add the service account to the project. +A team-scoped service account can read and write in all projects within its team, except to [restricted projects]({{< relref "../access-management/restricted-projects.md#visibility-scopes" >}}) in that team. Before a team-scoped service account can access a restricted project, an admin of that project must explicitly add the service account to the project. As a team admin, you can get the API key for a team-scoped service account in your team at `//service-accounts`. Alternatively you can go to the **Team settings** for your team and then refer to the **Service Accounts** tab. @@ -54,9 +54,9 @@ To create a new team scoped service account for your team: If you do not configure a team in your model training or generative AI app environment that uses a team-scoped service account, the model runs or weave traces log to the named project within the service account's parent team. In such a scenario, user attribution using the `WANDB_USERNAME` or `WANDB_USER_EMAIL` variables _do not work_ unless the referenced user is part of the service account's parent team. {{% alert color="warning" %}} -A team-scoped service account cannot log runs to a [team or restricted-scoped project](./restricted-projects.md#visibility-scopes) in a team different from its parent team, but it can log runs to an open visibility project within another team. +A team-scoped service account cannot log runs to a [team or restricted-scoped project]({{< relref "../access-management/restricted-projects.md#visibility-scopes" >}}) in a team different from its parent team, but it can log runs to an open visibility project within another team. {{% /alert %}} ### External service accounts -In addition to **Built-in** service accounts, W&B also supports team-scoped **External service accounts** with the W&B SDK and CLI using [Identity federation](./identity_federation.md#external-service-accounts) with identity providers (IdPs) that can issue JSON Web Tokens (JWTs). \ No newline at end of file +In addition to **Built-in** service accounts, W&B also supports team-scoped **External service accounts** with the W&B SDK and CLI using [Identity federation]({{< relref "./identity_federation.md#external-service-accounts" >}}) with identity providers (IdPs) that can issue JSON Web Tokens (JWTs). \ No newline at end of file diff --git a/content/guides/hosting/iam/authentication/sso.md b/content/guides/hosting/iam/authentication/sso.md index 2e6b20d93..616ebf7e8 100644 --- a/content/guides/hosting/iam/authentication/sso.md +++ b/content/guides/hosting/iam/authentication/sso.md @@ -20,9 +20,9 @@ The ID token is a JWT that contains the user's identity information, such as the In the context of W&B Server, access tokens authorize requests to APIs on behalf of the user, but since W&B Server’s primary concern is user authentication and identity, it only requires the ID token. -You can use environment variables to [configure IAM options](advanced_env_vars.md) for your [Dedicated cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instance. +You can use environment variables to [configure IAM options]({{< relref "advanced_env_vars.md" >}}) for your [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instance. -To assist with configuring Identity Providers for [Dedicated cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) W&B Server installations, follow these guidelines to follow for various IdPs. If you’re using the SaaS version of W&B, reach out to [support@wandb.com](mailto:support@wandb.com) for assistance in configuring an Auth0 tenant for your organization. +To assist with configuring Identity Providers for [Dedicated cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) W&B Server installations, follow these guidelines to follow for various IdPs. If you’re using the SaaS version of W&B, reach out to [support@wandb.com](mailto:support@wandb.com) for assistance in configuring an Auth0 tenant for your organization. {{< tabpane text=true >}} {{% tab header="Cognito" value="cognito" %}} @@ -176,7 +176,7 @@ To set up SSO, you need administrator privileges and the following information: Should your IdP require a OIDC Client Secret, specify it with the environment variable OIDC_SECRET. {{% /alert %}} -You can configure SSO using either the W&B Server UI or by passing [environment variables](../env-vars.md) to the `wandb/local` pod. The environment variables take precedence over UI. +You can configure SSO using either the W&B Server UI or by passing [environment variables]({{< relref "/guides/hosting/env-vars.md" >}}) to the `wandb/local` pod. The environment variables take precedence over UI. {{% alert %}} If you're unable to log in to your instance after configuring SSO, you can restart the instance with the `LOCAL_RESTORE=true` environment variable set. This outputs a temporary password to the containers logs and disables SSO. Once you've resolved any issues with SSO, you must remove that environment variable to enable SSO again. @@ -184,9 +184,9 @@ If you're unable to log in to your instance after configuring SSO, you can resta {{< tabpane text=true >}} {{% tab header="System Console" value="console" %}} -The System Console is the successor to the System Settings page. It is available with the [W&B Kubernetes Operator](../operator.md) based deployment. +The System Console is the successor to the System Settings page. It is available with the [W&B Kubernetes Operator]({{< relref "/guides/hosting/hosting-options/self-managed/kubernetes-operator.md" >}}) based deployment. -1. Refer to [Access the W&B Management Console](../operator.md#access-the-wb-management-console). +1. Refer to [Access the W&B Management Console]({{< relref "/guides/hosting/hosting-options/self-managed/kubernetes-operator.md#access-the-wb-management-console" >}}). 2. Navigate to **Settings**, then **Authentication**. Select **OIDC** in the **Type** dropdown. {{< img src="/images/hosting/sso_configure_via_console.png" alt="" >}} diff --git a/content/guides/hosting/iam/automate_iam.md b/content/guides/hosting/iam/automate_iam.md index f4d57fcaf..ea23220c1 100644 --- a/content/guides/hosting/iam/automate_iam.md +++ b/content/guides/hosting/iam/automate_iam.md @@ -20,19 +20,19 @@ There are broadly three categories of SCIM API - **User**, **Group**, and **Role ### User SCIM API -[User SCIM API](./scim.md#user-resource) allows for creating, deactivating, getting the details of a user, or listing all users in a W&B organization. This API also supports assigning predefined or custom roles to users in an organization. +[User SCIM API]({{< relref "./scim.md#user-resource" >}}) allows for creating, deactivating, getting the details of a user, or listing all users in a W&B organization. This API also supports assigning predefined or custom roles to users in an organization. {{% alert %}} Deactivate a user within a W&B organization with the `DELETE User` endpoint. Deactivated users can no longer sign in. However, deactivated users still appears in the organization's user list. -To fully remove a deactivated user from the user list, you must [remove the user from the organization](./manage-organization.md#remove-a-user). +To fully remove a deactivated user from the user list, you must [remove the user from the organization]({{< relref "access-management/manage-organization.md#remove-a-user" >}}). It is possible to re-enable a deactivated user, if needed. {{% /alert %}} ### Group SCIM API -[Group SCIM API](./scim.md#group-resource) allows for managing W&B teams, including creating or removing teams in an organization. Use the `PATCH Group` to add or remove users in an existing team. +[Group SCIM API]({{< relref "./scim.md#group-resource" >}}) allows for managing W&B teams, including creating or removing teams in an organization. Use the `PATCH Group` to add or remove users in an existing team. {{% alert %}} There is no notion of a `group of users having the same role` within W&B. A W&B team closely resembles a group, and allows diverse personas with different roles to work collaboratively on a set of related projects. Teams can consist of different groups of users. Assign each user in a team a role: team admin, member, viewer, or a custom role. @@ -42,7 +42,7 @@ W&B maps Group SCIM API endpoints to W&B teams because of the similarity between ### Custom role API -[Custom role SCIM API](./scim.md#role-resource) allows for managing custom roles, including creating, listing, or updating custom roles in an organization. +[Custom role SCIM API]({{< relref "./scim.md#role-resource" >}}) allows for managing custom roles, including creating, listing, or updating custom roles in an organization. {{% alert color="secondary" %}} Delete a custom role with caution. @@ -54,7 +54,7 @@ Update the inherited role for a custom role with the `PUT Role` endpoint. This o ## W&B Python SDK API -Just like how SCIM API allows you to automate user and team management, you can also use some of the methods available in the [W&B Python SDK API](../../../ref/python/public-api/api.md) for that purpose. Keep a note of the following methods: +Just like how SCIM API allows you to automate user and team management, you can also use some of the methods available in the [W&B Python SDK API]({{< relref "/ref/python/public-api/api.md" >}}) for that purpose. Keep a note of the following methods: | Method name | Purpose | |-------------|---------| diff --git a/content/guides/hosting/iam/scim.md b/content/guides/hosting/iam/scim.md index 5c2f474d3..cc50e555a 100644 --- a/content/guides/hosting/iam/scim.md +++ b/content/guides/hosting/iam/scim.md @@ -12,7 +12,7 @@ The System for Cross-domain Identity Management (SCIM) API allows instance or or The SCIM API is accessible at `/scim/` and supports the `/Users` and `/Groups` endpoints with a subset of the fields found in the [RC7643 protocol](https://www.rfc-editor.org/rfc/rfc7643). It additionally includes the `/Roles` endpoints which are not part of the official SCIM schema. W&B adds the `/Roles` endpoints to support automated management of custom roles in W&B organizations. {{% alert %}} -SCIM API applies to all hosting options including [Dedicated Cloud](../hosting-options/dedicated_cloud.md), [Self-managed instances](../hosting-options/self-managed.md), and [SaaS Cloud](../hosting-options/saas_cloud.md). In SaaS Cloud, the organization admin must configure the default organization in user settings to ensure that the SCIM API requests go to the right organization. The setting is available in the section `SCIM API Organization` within user settings. +SCIM API applies to all hosting options including [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}), [Self-managed instances]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}), and [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}). In SaaS Cloud, the organization admin must configure the default organization in user settings to ensure that the SCIM API requests go to the right organization. The setting is available in the section `SCIM API Organization` within user settings. {{% /alert %}} ## Authentication @@ -27,7 +27,7 @@ The SCIM user resource maps to W&B users. - **Endpoint:** **`/scim/Users/{id}`** - **Method**: GET -- **Description**: Retrieve the information for a specific user in your [SaaS Cloud](../hosting-options/saas_cloud.md) organization or your [Dedicated Cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instance by providing the user's unique ID. +- **Description**: Retrieve the information for a specific user in your [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) organization or your [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instance by providing the user's unique ID. - **Request Example**: ```bash @@ -68,7 +68,7 @@ GET /scim/Users/abc - **Endpoint:** **`/scim/Users`** - **Method**: GET -- **Description**: Retrieve the list of all users in your [SaaS Cloud](../hosting-options/saas_cloud.md) organization or your [Dedicated Cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instance. +- **Description**: Retrieve the list of all users in your [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) organization or your [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instance. - **Request Example**: ```bash @@ -180,11 +180,11 @@ POST /scim/Users - **Endpoint**: **`/scim/Users/{id}`** - **Method**: DELETE -- **Description**: Fully delete a user from your [SaaS Cloud](../hosting-options/saas_cloud.md) organization or your [Dedicated Cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instance by providing the user's unique ID. Use the [Create user](#create-user) API to add the user again to the organization or instance if needed. +- **Description**: Fully delete a user from your [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}) organization or your [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instance by providing the user's unique ID. Use the [Create user]({{< relref "#create-user" >}}) API to add the user again to the organization or instance if needed. - **Request Example**: {{% alert %}} -To temporarily deactivate the user, refer to [Deactivate user](#deactivate-user) API which uses the `PATCH` endpoint. +To temporarily deactivate the user, refer to [Deactivate user]({{< relref "#deactivate-user" >}}) API which uses the `PATCH` endpoint. {{% /alert %}} ```bash @@ -201,7 +201,7 @@ DELETE /scim/Users/abc - **Endpoint**: **`/scim/Users/{id}`** - **Method**: PATCH -- **Description**: Temporarily deactivate a user in your [Dedicated Cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instance by providing the user's unique ID. Use the [Reactivate user](#reactivate-user) API to reactivate the user when needed. +- **Description**: Temporarily deactivate a user in your [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instance by providing the user's unique ID. Use the [Reactivate user]({{< relref "#reactivate-user" >}}) API to reactivate the user when needed. - **Supported Fields**: | Field | Type | Required | @@ -210,7 +210,7 @@ DELETE /scim/Users/abc | value | Object | Object `{"active": false}` indicating that the user should be deactivated. | {{% alert %}} -User deactivation and reactivation operations are not supported in [SaaS Cloud](../hosting-options/saas_cloud.md). +User deactivation and reactivation operations are not supported in [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}). {{% /alert %}} - **Request Example**: @@ -266,7 +266,7 @@ This returns the User object. - **Endpoint**: **`/scim/Users/{id}`** - **Method**: PATCH -- **Description**: Reactivate a deactivated user in your [Dedicated Cloud](../hosting-options/dedicated_cloud.md) or [Self-managed](../hosting-options/self-managed.md) instance by providing the user's unique ID. +- **Description**: Reactivate a deactivated user in your [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) instance by providing the user's unique ID. - **Supported Fields**: | Field | Type | Required | @@ -275,7 +275,7 @@ This returns the User object. | value | Object | Object `{"active": true}` indicating that the user should be reactivated. | {{% alert %}} -User deactivation and reactivation operations are not supported in [SaaS Cloud](../hosting-options/saas_cloud.md). +User deactivation and reactivation operations are not supported in [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}). {{% /alert %}} - **Request Example**: @@ -331,7 +331,7 @@ This returns the User object. - **Endpoint**: **`/scim/Users/{id}`** - **Method**: PATCH -- **Description**: Assign an organization-level role to a user. The role can be one of `admin`, `viewer` or `member` as described [here](./manage-organization.md#invite-a-user). For [SaaS Cloud](../hosting-options/saas_cloud.md), ensure that you have configured the correct organization for SCIM API in user settings. +- **Description**: Assign an organization-level role to a user. The role can be one of `admin`, `viewer` or `member` as described [here]({{< relref "access-management/manage-organization.md#invite-a-user" >}}). For [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}), ensure that you have configured the correct organization for SCIM API in user settings. - **Supported Fields**: | Field | Type | Required | @@ -400,7 +400,7 @@ This returns the User object. - **Endpoint**: **`/scim/Users/{id}`** - **Method**: PATCH -- **Description**: Assign a team-level role to a user. The role can be one of `admin`, `viewer`, `member` or a custom role as described [here](./manage-organization.md#assign-or-update-a-team-members-role). For [SaaS Cloud](../hosting-options/saas_cloud.md), ensure that you have configured the correct organization for SCIM API in user settings. +- **Description**: Assign a team-level role to a user. The role can be one of `admin`, `viewer`, `member` or a custom role as described [here]({{< relref "access-management/manage-organization.md#assign-or-update-a-team-members-role" >}}). For [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}), ensure that you have configured the correct organization for SCIM API in user settings. - **Supported Fields**: | Field | Type | Required | diff --git a/content/guides/hosting/monitoring-usage/audit-logging.md b/content/guides/hosting/monitoring-usage/audit-logging.md index 3a03cd712..82d8c5d9b 100644 --- a/content/guides/hosting/monitoring-usage/audit-logging.md +++ b/content/guides/hosting/monitoring-usage/audit-logging.md @@ -11,12 +11,12 @@ Use W&B audit logs to track user activity within your organization and to confor | W&B Platform Deployment type | Audit logs access mechanism | |----------------------------|--------------------------------| -| [Self-managed](../hosting-options/self-managed.md) | Synced to instance-level bucket every 10 minutes. Also available using [the API](#fetch-audit-logs-using-api). | -| [Dedicated Cloud](../hosting-options/dedicated_cloud.md) with [secure storage connector (BYOB)](../data-security/secure-storage-connector.md) | Synced to instance-level bucket (BYOB) every 10 minutes. Also available using [the API](#fetch-audit-logs-using-api). | -| [Dedicated Cloud](../hosting-options/dedicated_cloud.md) with W&B managed storage (without BYOB) | Only available using [the API](#fetch-audit-logs-using-api). | +| [Self-managed]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}) | Synced to instance-level bucket every 10 minutes. Also available using [the API]({{< relref "#fetch-audit-logs-using-api" >}}). | +| [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) with [secure storage connector (BYOB)]({{< relref "/guides/hosting/data-security/secure-storage-connector.md" >}}) | Synced to instance-level bucket (BYOB) every 10 minutes. Also available using [the API]({{< relref "#fetch-audit-logs-using-api" >}}). | +| [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) with W&B managed storage (without BYOB) | Only available using [the API]({{< relref "#fetch-audit-logs-using-api" >}}). | {{% alert %}} -Audit logs are not available for [SaaS Cloud](../hosting-options/saas_cloud.md). +Audit logs are not available for [SaaS Cloud]({{< relref "/guides/hosting/hosting-options/saas_cloud.md" >}}). {{% /alert %}} Once you've access to your audit logs, analyze those using your preferred tools, such as [Pandas](https://pandas.pydata.org/docs/index.html), [Amazon Redshift](https://aws.amazon.com/redshift/), [Google BigQuery](https://cloud.google.com/bigquery), [Microsoft Fabric](https://www.microsoft.com/en-us/microsoft-fabric), and more. You may need to transform the JSON-formatted audit logs into a format relevant to the tool before analysis. Information on how to transform your audit logs for specific tools is outside the scope of W&B documentation. @@ -25,7 +25,7 @@ Once you've access to your audit logs, analyze those using your preferred tools, **Audit Log Retention:** If a compliance, security or risk team in your organization requires audit logs to be retained for a specific period of time, W&B recommends to periodically transfer the logs from your instance-level bucket to a long-term retention storage. If you're instead using the API to access the audit logs, you can implement a simple script that runs periodically (like daily or every few days) to fetch any logs that may have been generated since the time of the last script run, and store those in a short-term storage for analysis or directly transfer to a long-term retention storage. {{% /alert %}} -HIPAA compliance requires that you retain audit logs for a minimum of 6 years. For HIPAA-compliant [Dedicated Cloud](../hosting-options/dedicated_cloud.md) instances with [BYOB](../data-security/secure-storage-connector.md), you must configure guardrails for your managed storage including any long-term retention storage, to ensure that no internal or external user can delete audit logs before the end of the mandatory retention period. +HIPAA compliance requires that you retain audit logs for a minimum of 6 years. For HIPAA-compliant [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) instances with [BYOB]({{< relref "/guides/hosting/data-security/secure-storage-connector.md" >}}), you must configure guardrails for your managed storage including any long-term retention storage, to ensure that no internal or external user can delete audit logs before the end of the mandatory retention period. ## Audit log schema The following table lists all the different keys that might be present in your audit logs. Each log contains only the assets relevant to the corresponding action, and others are omitted from the log. @@ -33,7 +33,7 @@ The following table lists all the different keys that might be present in your a | Key | Definition | |---------| -------| |timestamp | Time stamp in [RFC3339 format](https://www.rfc-editor.org/rfc/rfc3339). For example: `2023-01-23T12:34:56Z`, represents `12:34:56 UTC` time on Jan 23, 2023. -|action | What [action](#actions) did the user take. +|action | What [action]({{< relref "#actions" >}}) did the user take. |actor_user_id | If present, ID of the logged-in user who performed the action. |response_code | Http response code for the action. |artifact_asset | If present, action was taken on this artifact id @@ -52,7 +52,7 @@ The following table lists all the different keys that might be present in your a |report_name | if present, action was taken on this report name. |user_email | if present, action was taken on this user email. -Personally identifiable information (PII), such as email ids and the names of projects, teams, and reports, is available only using the API endpoint option, and can be turned off as [described below](#fetch-audit-logs-using-api). +Personally identifiable information (PII), such as email ids and the names of projects, teams, and reports, is available only using the API endpoint option, and can be turned off as [described below]({{< relref "#fetch-audit-logs-using-api" >}}). ## Fetch audit logs using API An instance admin can fetch the audit logs for your W&B instance using the following API: @@ -64,7 +64,7 @@ An instance admin can fetch the audit logs for your W&B instance using the follo If your W&B instance URL is `https://mycompany.wandb.io` and you would like to get audit logs without PII for user activity within the last week, you must use the API endpoint `https://mycompany.wandb.io?numDays=7&anonymize=true`. {{% alert %}} -Only W&B [instance admins](../iam/access-management-intro) can fetch audit logs using the API. If you are not an instance admin or not logged into your organization, you get a `HTTP 403 Forbidden` error. +Only W&B [instance admins]({{< relref "/guides/hosting/iam/access-management/" >}}) can fetch audit logs using the API. If you are not an instance admin or not logged into your organization, you get a `HTTP 403 Forbidden` error. {{% /alert %}} The API response contains new-line separated JSON objects. Objects will include the fields described in the schema. It's the same format which is used when syncing audit log files to an instance-level bucket (wherever applicable as mentioned earlier). In those cases, the audit logs are located at the `/wandb-audit-logs` directory in your bucket. diff --git a/content/guides/hosting/monitoring-usage/org_dashboard.md b/content/guides/hosting/monitoring-usage/org_dashboard.md index 289001408..caddf2914 100644 --- a/content/guides/hosting/monitoring-usage/org_dashboard.md +++ b/content/guides/hosting/monitoring-usage/org_dashboard.md @@ -7,7 +7,7 @@ title: View organization dashboard --- {{% alert color="secondary" %}} -Organization dashboard is only available with [Dedicated Cloud](../hosting-options/dedicated_cloud.md) and [Self-managed instances](../hosting-options/self-managed.md). +Organization dashboard is only available with [Dedicated Cloud]({{< relref "/guides/hosting/hosting-options/dedicated_cloud.md" >}}) and [Self-managed instances]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}). {{% /alert %}} diff --git a/content/guides/hosting/monitoring-usage/prometheus-logging.md b/content/guides/hosting/monitoring-usage/prometheus-logging.md index 8d799bbc0..4ce4a564f 100644 --- a/content/guides/hosting/monitoring-usage/prometheus-logging.md +++ b/content/guides/hosting/monitoring-usage/prometheus-logging.md @@ -10,7 +10,7 @@ weight: 2 Use [Prometheus](https://prometheus.io/docs/introduction/overview/) with W&B Server. Prometheus installs are exposed as a [kubernetes ClusterIP service](https://github.com/wandb/terraform-kubernetes-wandb/blob/main/main.tf#L225). {{% alert color="secondary" %}} -Prometheus monitoring is only available with [Self-managed instances](../hosting-options/self-managed.md). +Prometheus monitoring is only available with [Self-managed instances]({{< relref "/guides/hosting/hosting-options/self-managed.md" >}}). {{% /alert %}} diff --git a/content/guides/hosting/privacy-settings.md b/content/guides/hosting/privacy-settings.md index d57cc6acc..4ccdfa10f 100644 --- a/content/guides/hosting/privacy-settings.md +++ b/content/guides/hosting/privacy-settings.md @@ -22,7 +22,7 @@ Team admins can configure privacy settings for their respective teams from withi * Allow any team member to invite other members (not just admins) * Turn off public sharing to outside of team for reports in private projects. This turns off existing magic links. * Allow users with matching organization email domain to join this team. - * This setting is applicable only to [SaaS Cloud](./hosting-options/saas_cloud.md). It's not available in [Dedicated Cloud](./hosting-options/dedicated_cloud.md) or [Self-managed](./hosting-options/self-managed.md) instances. + * This setting is applicable only to [SaaS Cloud]({{< relref "./hosting-options/saas_cloud.md" >}}). It's not available in [Dedicated Cloud]({{< relref "./hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "./hosting-options/self-managed.md" >}}) instances. * Enable code saving by default. ## Enforce privacy settings for all teams @@ -32,13 +32,13 @@ Organization admins can enforce privacy settings for all teams in their organiza * Enforce team visibility restrictions * Enable this option to hide all teams from non-members * Enforce privacy for future projects - * Enable this option to enforce all future projects in all teams to be private or [restricted](./iam/restricted-projects.md) + * Enable this option to enforce all future projects in all teams to be private or [restricted]({{< relref "./iam/access-management/restricted-projects.md" >}}) * Enforce invitation control * Enable this option to prevent non-admins from inviting members to any team * Enforce report sharing control * Enable this option to turn off public sharing of reports in private projects and deactivate existing magic links * Enforce team self joining restrictions * Enable this option to restrict users with matching organization email domain from self-joining any team - * This setting is applicable only to [SaaS Cloud](./hosting-options/saas_cloud.md). It's not available in [Dedicated Cloud](./hosting-options/dedicated_cloud.md) or [Self-managed](./hosting-options/self-managed.md) instances. + * This setting is applicable only to [SaaS Cloud]({{< relref "./hosting-options/saas_cloud.md" >}}). It's not available in [Dedicated Cloud]({{< relref "./hosting-options/dedicated_cloud.md" >}}) or [Self-managed]({{< relref "./hosting-options/self-managed.md" >}}) instances. * Enforce default code saving restrictions * Enable this option to turn off code saving by default for all teams \ No newline at end of file diff --git a/content/guides/hosting/server-release-process.md b/content/guides/hosting/server-release-process.md index cc8617af7..825a6738f 100644 --- a/content/guides/hosting/server-release-process.md +++ b/content/guides/hosting/server-release-process.md @@ -16,7 +16,7 @@ W&B Server releases apply to the **Dedicated Cloud** and **Self-managed** deploy | Patch | Patch releases include critical and high severity bug fixes. Patches are only rarely released, as needed. | | Feature | The feature release targets a specific release date for a new product feature, which occasionally happens before the standard monthly release. | -All releases are immediately deployed to all **Dedicated Cloud** instances once the acceptance testing phase is complete. It keeps those managed instances fully updated, making the latest features and fixes available to relevant customers. Customers with **Self-managed** instances are responsible for the [update process](./server-upgrade-process.md) on their own schedule, where they can use [the latest Docker image](https://hub.docker.com/r/wandb/local). Refer to [release support and end of life](#release-support-and-end-of-life-policy). +All releases are immediately deployed to all **Dedicated Cloud** instances once the acceptance testing phase is complete. It keeps those managed instances fully updated, making the latest features and fixes available to relevant customers. Customers with **Self-managed** instances are responsible for the [update process]({{< relref "/guides/hosting/hosting-options/self-managed/server-upgrade-process.md" >}}) on their own schedule, where they can use [the latest Docker image](https://hub.docker.com/r/wandb/local). Refer to [release support and end of life]({{< relref "#release-support-and-end-of-life-policy" >}}). {{% alert %}} - Some advanced features are available only with the enterprise license. So even if you get the latest Docker image but don't have an enterprise license, you would not be able to take advantage of the relevant advanced capabilities. @@ -34,7 +34,7 @@ Downtime might occur for the following scenarios: * A new feature or enhancement requires changes to the underlying infrastructure such as compute, storage or network. W&B tries to send relevant advance notifications to **Dedicated Cloud** customers. * An infrastructure change due to a security patch or to avoid `support end-of-life` for a particular version. For urgent changes, **Dedicated Cloud** customers may not receive advance notifications. The priority here is to keep the fleet secure and fully supported. -For both cases, updates roll out to all **Dedicated Cloud** instances without exception. Customers with **Self-managed** instances are responsible to manage such updates on their own schedule. Refer to [release support and end of life](#release-support-and-end-of-life-policy). +For both cases, updates roll out to all **Dedicated Cloud** instances without exception. Customers with **Self-managed** instances are responsible to manage such updates on their own schedule. Refer to [release support and end of life]({{< relref "#release-support-and-end-of-life-policy" >}}). ## Release support and end of life policy W&B supports every server release for six months from the release date. **Dedicated Cloud** instances are automatically updated. Customers with **Self-managed** instances are responsible to update their deployments in time to comply with the support policy. Avoid staying on a version older than six months as it would significantly limit support from W&B. diff --git a/content/guides/integrations/_index.md b/content/guides/integrations/_index.md index 57a7ab5b5..c19c8d30e 100644 --- a/content/guides/integrations/_index.md +++ b/content/guides/integrations/_index.md @@ -9,7 +9,7 @@ cascade: - url: guides/integrations/:filename --- -W&B integrations make it fast and easy to set up experiment tracking and data versioning inside existing projects. Check out integrations for ML frameworks such as [PyTorch](pytorch.md), ML libraries such as [Hugging Face](huggingface.md), or cloud services such as [Amazon SageMaker](other/sagemaker.md). +W&B integrations make it fast and easy to set up experiment tracking and data versioning inside existing projects. Check out integrations for ML frameworks such as [PyTorch]({{< relref "pytorch.md" >}}), ML libraries such as [Hugging Face]({{< relref "huggingface.md" >}}), or cloud services such as [Amazon SageMaker]({{< relref "sagemaker.md" >}}). diff --git a/content/guides/integrations/accelerate.md b/content/guides/integrations/accelerate.md index cd3eb9a80..7b9c5f59c 100644 --- a/content/guides/integrations/accelerate.md +++ b/content/guides/integrations/accelerate.md @@ -43,7 +43,7 @@ Explaining more, you need to: 1. Pass `log_with="wandb"` when initialising the Accelerator class 2. Call the [`init_trackers`](https://huggingface.co/docs/accelerate/main/en/package_reference/accelerator#accelerate.Accelerator.init_trackers) method and pass it: - a project name via `project_name` -- any parameters you want to pass to [`wandb.init`](/ref/python/init) via a nested dict to `init_kwargs` +- any parameters you want to pass to [`wandb.init`]({{< relref "/ref/python/init" >}}) via a nested dict to `init_kwargs` - any other experiment config information you want to log to your wandb run, via `config` 3. Use the `.log` method to log to Weigths & Biases; the `step` argument is optional 4. Call `.end_training` when finished training diff --git a/content/guides/integrations/add-wandb-to-any-library.md b/content/guides/integrations/add-wandb-to-any-library.md index a605df273..3fcd377b5 100644 --- a/content/guides/integrations/add-wandb-to-any-library.md +++ b/content/guides/integrations/add-wandb-to-any-library.md @@ -12,7 +12,7 @@ weight: 10 This guide provides best practices on how to integrate W&B into your Python library to get powerful Experiment Tracking, GPU and System Monitoring, Model Checkpointing, and more for your own library. {{% alert %}} -If you are still learning how to use W&B, we recommend exploring the other W&B Guides in these docs, such as [Experiment Tracking](/guides/track), before reading further. +If you are still learning how to use W&B, we recommend exploring the other W&B Guides in these docs, such as [Experiment Tracking]({{< relref "/guides/models/track" >}}), before reading further. {{% /alert %}} Below we cover best tips and best practices when the codebase you are working on is more complicated than a single Python training script or Jupyter notebook. The topics covered are: @@ -98,7 +98,7 @@ wandb.login() {{% /tab %}} {{% tab header="Environment Variable" value="environment" %}} -Set a [W&B environment variable](../track/environment-variables.md) for the API key: +Set a [W&B environment variable]({{< relref "/guides/models/track/environment-variables.md" >}}) for the API key: ```bash export WANDB_API_KEY=$YOUR_API_KEY @@ -238,13 +238,13 @@ Use `wandb.config.update` to update the config. Updating your configuration dict wandb.config.update({"model_parameters": 3500}) ``` -For more information on how to define a config file, see [Configure Experiments with wandb.config](/guides/track/config). +For more information on how to define a config file, see [Configure Experiments with wandb.config]({{< relref "/guides/models/track/config" >}}). ### Logging To W&B #### Log Metrics -Create a dictionary where the key value is the name of the metric. Pass this dictionary object to [`wandb.log`](/guides/track/log): +Create a dictionary where the key value is the name of the metric. Pass this dictionary object to [`wandb.log`]({{< relref "/guides/models/track/log" >}}): ```python for epoch in range(NUM_EPOCHS): @@ -269,7 +269,7 @@ wandb.log(metrics) {{< img src="/images/integrations/integrations_add_any_lib_log.png" alt="A W&B Workspace with 2 separate sections" >}} -For more on `wandb.log`, see [Log Data with wandb.log](/guides/track/log). +For more on `wandb.log`, see [Log Data with wandb.log]({{< relref "/guides/models/track/log" >}}). #### Preventing x-axis Misalignments @@ -310,7 +310,7 @@ Some considerations when logging data include: * For images, you can log sample predictions, segmentation masks, etc., to see the evolution over time. * For text, you can log tables of sample predictions for later exploration. -Refer to [Log Data with wandb.log](/guides/track/log) for a full guide on logging media, objects, plots, and more. +Refer to [Log Data with wandb.log]({{< relref "/guides/models/track/log" >}}) for a full guide on logging media, objects, plots, and more. ### Distributed Training @@ -319,7 +319,7 @@ For frameworks supporting distributed environments, you can adapt any of the fol * Detect which is the "main" process and only use `wandb` there. Any required data coming from other processes must be routed to the main process first. (This workflow is encouraged). * Call `wandb` in every process and auto-group them by giving them all the same unique `group` name. -See [Log Distributed Training Experiments](../track/log/distributed-training.md) for more details. +See [Log Distributed Training Experiments]({{< relref "/guides/models/track/log/distributed-training.md" >}}) for more details. ### Logging Model Checkpoints And More @@ -351,7 +351,7 @@ aliases = ["best", "epoch_10"] wandb.log_artifact(artifact, aliases=aliases) ``` -For information on how to create a custom alias, see [Create a Custom Alias](/guides/artifacts/create-a-custom-alias). +For information on how to create a custom alias, see [Create a Custom Alias]({{< relref "/guides/core/artifacts/create-a-custom-alias/" >}}). You can log output Artifacts at any frequency (for example, every epoch, every 500 steps, and so on) and they are automatically versioned. @@ -376,18 +376,18 @@ local_path = artifact.download("./tmp") Artifacts can be found in the Artifacts section of W&B and can be referenced with aliases generated automatically (`latest`, `v2`, `v3`) or manually when logging (`best_accuracy`, etc.). -To download an Artifact without creating a `wandb` run (through `wandb.init`), for example in distributed environments or for simple inference, you can instead reference the artifact with the [wandb API](/ref/python/public-api): +To download an Artifact without creating a `wandb` run (through `wandb.init`), for example in distributed environments or for simple inference, you can instead reference the artifact with the [wandb API]({{< relref "/ref/python/public-api" >}}): ```python artifact = wandb.Api().artifact("user/project/artifact:latest") local_path = artifact.download() ``` -For more information, see [Download and Use Artifacts](/guides/artifacts/download-and-use-an-artifact). +For more information, see [Download and Use Artifacts]({{< relref "/guides/core/artifacts/download-and-use-an-artifact" >}}). ### Hyper-parameter Tuning -If your library would like to leverage W&B hyper-parameter tuning, [W&B Sweeps](/guides/sweeps) can also be added to your library. +If your library would like to leverage W&B hyper-parameter tuning, [W&B Sweeps]({{< relref "/guides/models/sweeps/" >}}) can also be added to your library. ### Advanced Integrations diff --git a/content/guides/integrations/autotrain.md b/content/guides/integrations/autotrain.md index 8b7034739..19ef94fb1 100644 --- a/content/guides/integrations/autotrain.md +++ b/content/guides/integrations/autotrain.md @@ -48,7 +48,7 @@ Hugging Face AutoTrain expects your CSV custom dataset to have a specific format ## Train using `autotrain` -You can start training using the `autotrain` advanced from the command line or a notebook. Use the `--log` argument, or use `--log wandb` to log your results to a [W&B run](/guides/runs). +You can start training using the `autotrain` advanced from the command line or a notebook. Use the `--log` argument, or use `--log wandb` to log your results to a [W&B run]({{< relref "/guides/models/track/runs/" >}}). {{< tabpane text=true >}} diff --git a/content/guides/integrations/composer.md b/content/guides/integrations/composer.md index f26c5d246..342000bd0 100644 --- a/content/guides/integrations/composer.md +++ b/content/guides/integrations/composer.md @@ -46,7 +46,7 @@ Below the parameters for WandbLogger, see the [Composer documentation](https://d | `tags` | W&B tags (List[str], optional) | `log_artifacts` | Whether to log checkpoints to wandb, default: `false` (bool, optional)| | `rank_zero_only` | Whether to log only on the rank-zero process. When logging artifacts, it is highly recommended to log on all ranks. Artifacts from ranks ≥1 are not stored, which may discard pertinent information. For example, when using Deepspeed ZeRO, it would be impossible to restore from checkpoints without artifacts from all ranks, default: `True` (bool, optional) -| `init_kwargs` | Params to pass to `wandb.init` such as your wandb `config` etc [See here](/ref/python/init) for the full list `wandb.init` accepts +| `init_kwargs` | Params to pass to `wandb.init` such as your wandb `config` etc [See here]({{< relref "/ref/python/init" >}}) for the full list `wandb.init` accepts A typical usage would be: diff --git a/content/guides/integrations/diffusers.md b/content/guides/integrations/diffusers.md index 859853af4..d4b6a6c35 100644 --- a/content/guides/integrations/diffusers.md +++ b/content/guides/integrations/diffusers.md @@ -45,14 +45,14 @@ autolog(init=dict(project="diffusers_logging")) 2. Use `autolog` to initialize a Weights & Biases run and automatically track the inputs and the outputs from [all supported pipeline calls](https://github.com/wandb/wandb/blob/main/wandb/integration/diffusers/autologger.py#L12-L72). - You can call the `autolog()` function with the `init` parameter, which accepts a dictionary of parameters required by [`wandb.init()`](/ref/python/init). + You can call the `autolog()` function with the `init` parameter, which accepts a dictionary of parameters required by [`wandb.init()`]({{< relref "/ref/python/init" >}}). When you call `autolog()`, it initializes a Weights & Biases run and automatically tracks the inputs and the outputs from [all supported pipeline calls](https://github.com/wandb/wandb/blob/main/wandb/integration/diffusers/autologger.py#L12-L72). - - Each pipeline call is tracked into its own [table](/guides/tables) in the workspace, and the configs associated with the pipeline call is appended to the list of workflows in the configs for that run. - - The prompts, negative prompts, and the generated media are logged in a [`wandb.Table`](/guides/tables). + - Each pipeline call is tracked into its own [table]({{< relref "/guides/core/tables/" >}}) in the workspace, and the configs associated with the pipeline call is appended to the list of workflows in the configs for that run. + - The prompts, negative prompts, and the generated media are logged in a [`wandb.Table`]({{< relref "/guides/core/tables/" >}}). - All other configs associated with the experiment including seed and the pipeline architecture are stored in the config section for the run. - - The generated media for each pipeline call are also logged in [media panels](/guides/track/log/media) in the run. + - The generated media for each pipeline call are also logged in [media panels]({{< relref "/guides/models/track/log/media" >}}) in the run. {{% alert %}} You can find a list of supported pipeline calls [here](https://github.com/wandb/wandb/blob/main/wandb/integration/diffusers/autologger.py#L12-L72). In case, you want to request a new feature of this integration or report a bug associated with it, please open an issue on [https://github.com/wandb/wandb/issues](https://github.com/wandb/wandb/issues). @@ -147,7 +147,7 @@ wandb.finish() {{< img src="/images/integrations/diffusers-autolog-3.gif" alt="An example of how the autolog logs the configs of your experiment" >}} {{% alert %}} -You need to explicitly call [`wandb.finish()`](/ref/python/finish) when executing the code in IPython notebook environments after calling the pipeline. This is not necessary when executing python scripts. +You need to explicitly call [`wandb.finish()`]({{< relref "/ref/python/finish" >}}) when executing the code in IPython notebook environments after calling the pipeline. This is not necessary when executing python scripts. {{% /alert %}} ### Tracking multi-pipeline workflows diff --git a/content/guides/integrations/farama-gymnasium.md b/content/guides/integrations/farama-gymnasium.md index 7f6a7c2ec..a5efa0463 100644 --- a/content/guides/integrations/farama-gymnasium.md +++ b/content/guides/integrations/farama-gymnasium.md @@ -8,9 +8,9 @@ title: Farama Gymnasium weight: 90 --- -If you're using [Farama Gymnasium](https://gymnasium.farama.org/#) we will automatically log videos of your environment generated by `gymnasium.wrappers.Monitor`. Just set the `monitor_gym` keyword argument to [`wandb.init`](../../../ref/python/init.md) to `True`. +If you're using [Farama Gymnasium](https://gymnasium.farama.org/#) we will automatically log videos of your environment generated by `gymnasium.wrappers.Monitor`. Just set the `monitor_gym` keyword argument to [`wandb.init`]({{< relref "/ref/python/init.md" >}}) to `True`. -Our gymnasium integration is very light. We simply [look at the name of the video file](https://github.com/wandb/wandb/blob/c5fe3d56b155655980611d32ef09df35cd336872/wandb/integration/gym/__init__.py#LL69C67-L69C67) being logged from `gymnasium` and name it after that or fall back to `"videos"` if we don't find a match. If you want more control, you can always just manually [log a video](../../track/log/media.md). +Our gymnasium integration is very light. We simply [look at the name of the video file](https://github.com/wandb/wandb/blob/c5fe3d56b155655980611d32ef09df35cd336872/wandb/integration/gym/__init__.py#LL69C67-L69C67) being logged from `gymnasium` and name it after that or fall back to `"videos"` if we don't find a match. If you want more control, you can always just manually [log a video]({{< relref "/guides/models/track/log/media.md" >}}). Check out this [report](https://wandb.ai/raph-test/cleanrltest/reports/Mario-Bros-but-with-AI-Gymnasium-and-CleanRL---Vmlldzo0NTcxNTcw) to learn more on how to use Gymnasium with the CleanRL library. diff --git a/content/guides/integrations/fastai/_index.md b/content/guides/integrations/fastai/_index.md index 96715946c..87c0b5c5f 100644 --- a/content/guides/integrations/fastai/_index.md +++ b/content/guides/integrations/fastai/_index.md @@ -51,7 +51,7 @@ If you're using **fastai** to train your models, W&B has an easy integration usi ``` {{% alert %}} -If you use version 1 of Fastai, refer to the [Fastai v1 docs](v1.md). +If you use version 1 of Fastai, refer to the [Fastai v1 docs]({{< relref "v1.md" >}}). {{% /alert %}} ## WandbCallback Arguments diff --git a/content/guides/integrations/fastai/v1.md b/content/guides/integrations/fastai/v1.md index adf76f77a..540d41d47 100644 --- a/content/guides/integrations/fastai/v1.md +++ b/content/guides/integrations/fastai/v1.md @@ -8,7 +8,7 @@ title: fastai v1 {{% alert %}} This documentation is for fastai v1. -If you use the current version of fastai, you should refer to [fastai page](../intro.md). +If you use the current version of fastai, you should refer to [fastai page]({{< relref "../" >}}). {{% /alert %}} For scripts using fastai v1, we have a callback that can automatically log model topology, losses, metrics, weights, gradients, sample predictions and best trained model. diff --git a/content/guides/integrations/huggingface.md b/content/guides/integrations/huggingface.md index 63075ecca..be7db5c0d 100644 --- a/content/guides/integrations/huggingface.md +++ b/content/guides/integrations/huggingface.md @@ -37,7 +37,7 @@ If you'd rather dive straight into working code, check out this [Google Colab](h 3. To log in with your training script, you'll need to sign in to you account at www.wandb.ai, then **you will find your API key on the** [**Authorize page**](https://wandb.ai/authorize)**.** -If you are using Weights and Biases for the first time you might want to check out our [**quickstart**](../../quickstart.md) +If you are using Weights and Biases for the first time you might want to check out our [**quickstart**]({{< relref "/guides/quickstart.md" >}}) {{< tabpane text=true >}} @@ -151,7 +151,7 @@ Using TensorFlow? Just swap the PyTorch `Trainer` for the TensorFlow `TFTrainer` ### 4. Turn on model checkpointing -Using Weights & Biases' [Artifacts](../artifacts/intro.md), you can store up to 100GB of models and datasets for free and then use the Weights & Biases [Model Registry](../model_registry/intro.md) to register models to prepare them for staging or deployment in your production environment. +Using Weights & Biases' [Artifacts]({{< relref "/guides/core/artifacts/" >}}), you can store up to 100GB of models and datasets for free and then use the Weights & Biases [Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}) to register models to prepare them for staging or deployment in your production environment. Logging your Hugging Face model checkpoints to Artifacts can be done by setting the `WANDB_LOG_MODEL` environment variable to one of `end` or `checkpoint` or `false`: @@ -191,7 +191,7 @@ WANDB_LOG_MODEL="checkpoint" {{< /tabpane >}} -Any Transformers `Trainer` you initialize from now on will upload models to your W&B project. The model checkpoints you log will be viewable through the [Artifacts](../artifacts) UI, and include the full model lineage (see an example model checkpoint in the UI [here](https://wandb.ai/wandb/arttest/artifacts/model/iv3_trained/5334ab69740f9dda4fed/lineage?_gl=1*yyql5q*_ga*MTQxOTYyNzExOS4xNjg0NDYyNzk1*_ga_JH1SJHJQXJ*MTY5MjMwNzI2Mi4yNjkuMS4xNjkyMzA5NjM2LjM3LjAuMA..)). +Any Transformers `Trainer` you initialize from now on will upload models to your W&B project. The model checkpoints you log will be viewable through the [Artifacts]({{< relref "/guides/core/artifacts/" >}}) UI, and include the full model lineage (see an example model checkpoint in the UI [here](https://wandb.ai/wandb/arttest/artifacts/model/iv3_trained/5334ab69740f9dda4fed/lineage?_gl=1*yyql5q*_ga*MTQxOTYyNzExOS4xNjg0NDYyNzk1*_ga_JH1SJHJQXJ*MTY5MjMwNzI2Mi4yNjkuMS4xNjkyMzA5NjM2LjM3LjAuMA..)). {{% alert %}} @@ -200,9 +200,9 @@ However, If you pass a [`run_name`](https://huggingface.co/docs/transformers/mai {{% /alert %}} #### W&B Model Registry -Once you have logged your checkpoints to Artifacts, you can then register your best model checkpoints and centralize them across your team using the Weights & Biases **[Model Registry](../model_registry/intro.md)**. Here you can organize your best models by task, manage model lifecycle, facilitate easy tracking and auditing throughout the ML lifecyle, and [automate](/guides/artifacts/project-scoped-automations/#create-a-webhook-automation) downstream actions with webhooks or jobs. +Once you have logged your checkpoints to Artifacts, you can then register your best model checkpoints and centralize them across your team using the Weights & Biases **[Model Registry]({{< relref "/guides/models/registry/model_registry/" >}})**. Here you can organize your best models by task, manage model lifecycle, facilitate easy tracking and auditing throughout the ML lifecyle, and [automate]({{< relref "/guides/models/automations/project-scoped-automations/#create-a-webhook-automation" >}}) downstream actions with webhooks or jobs. -See the [Model Registry](../model_registry/intro.md) documentation for how to link a model Artifact to the Model Registry. +See the [Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}) documentation for how to link a model Artifact to the Model Registry. ### 5. Visualise evaluation outputs during training @@ -210,7 +210,7 @@ Visualing your model outputs during training or evaluation is often essential to By using the callbacks system in the Transformers Trainer, you can log additional helpful data to W&B such as your models' text generation outputs or other predictions to W&B Tables. -See the **[Custom logging section](#custom-logging-log-and-view-evaluation-samples-during-training)** below for a full guide on how to log evaluation outupts while training to log to a W&B Table like this: +See the **[Custom logging section]({{< relref "#custom-logging-log-and-view-evaluation-samples-during-training" >}})** below for a full guide on how to log evaluation outupts while training to log to a W&B Table like this: {{< img src="/images/integrations/huggingface_eval_tables.png" alt="Shows a W&B Table with evaluation outputs" >}} @@ -231,14 +231,14 @@ wandb.finish() ### 7. Visualize your results -Once you have logged your training results you can explore your results dynamically in the [W&B Dashboard](../track/workspaces.md). It's easy to compare across dozens of runs at once, zoom in on interesting findings, and coax insights out of complex data with flexible, interactive visualizations. +Once you have logged your training results you can explore your results dynamically in the [W&B Dashboard]({{< relref "/guides/models/track/workspaces.md" >}}). It's easy to compare across dozens of runs at once, zoom in on interesting findings, and coax insights out of complex data with flexible, interactive visualizations. ## Advanced features and FAQs ### How do I save the best model? If `load_best_model_at_end=True` is set in the `TrainingArguments` that are passed to the `Trainer`, then W&B will save the best performing model checkpoint to Artifacts. -If you'd like to centralize all your best model versions across your team to organize them by ML task, stage them for production, bookmark them for further evaluation, or kick off downstream Model CI/CD processes then ensure you're saving your model checkpoints to Artifacts. Once logged to Artifacts, these checkpoints can then be promoted to the [Model Registry](../model_registry/intro.md). +If you'd like to centralize all your best model versions across your team to organize them by ML task, stage them for production, bookmark them for further evaluation, or kick off downstream Model CI/CD processes then ensure you're saving your model checkpoints to Artifacts. Once logged to Artifacts, these checkpoints can then be promoted to the [Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}). ### How do I load a saved model? @@ -421,7 +421,7 @@ For a more detailed example please refer to this [colab](https://colab.research. ### What additional W&B settings are available? -Further configuration of what is logged with `Trainer` is possible by setting environment variables. A full list of W&B environment variables [can be found here](/guides/hosting/env-vars/). +Further configuration of what is logged with `Trainer` is possible by setting environment variables. A full list of W&B environment variables [can be found here]({{< relref "/guides/hosting/env-vars/" >}}). | Environment Variable | Usage | | -------------------- |----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| @@ -458,7 +458,7 @@ WANDB_SILENT=true The `WandbCallback` that `Trainer` uses will call `wandb.init` under the hood when `Trainer` is initialized. You can alternatively set up your runs manually by calling `wandb.init` before the`Trainer` is initialized. This gives you full control over your W&B run configuration. -An example of what you might want to pass to `init` is below. For more details on how to use `wandb.init`, [check out the reference documentation](../../ref/python/init.md). +An example of what you might want to pass to `init` is below. For more details on how to use `wandb.init`, [check out the reference documentation]({{< relref "/ref/python/init.md" >}}). ```python wandb.init( diff --git a/content/guides/integrations/hydra.md b/content/guides/integrations/hydra.md index a9bd826e5..2b4974b3b 100644 --- a/content/guides/integrations/hydra.md +++ b/content/guides/integrations/hydra.md @@ -43,7 +43,7 @@ def run_experiment(cfg): ## Troubleshoot multiprocessing -If your process hangs when started, this may be caused by [this known issue](../../track/log/distributed-training.md). To solve this, try to changing wandb's multiprocessing protocol either by adding an extra settings parameter to \`wandb.init\` as: +If your process hangs when started, this may be caused by [this known issue]({{< relref "/guides/models/track/log/distributed-training.md" >}}). To solve this, try to changing wandb's multiprocessing protocol either by adding an extra settings parameter to \`wandb.init\` as: ```python wandb.init(settings=wandb.Settings(start_method="thread")) @@ -57,7 +57,7 @@ $ export WANDB_START_METHOD=thread ## Optimize Hyperparameters -[W&B Sweeps](../../sweeps/intro.md) is a highly scalable hyperparameter search platform, which provides interesting insights and visualization about W&B experiments with minimal requirements code real-estate. Sweeps integrates seamlessly with Hydra projects with no-coding requirements. The only thing needed is a configuration file describing the various parameters to sweep over as normal. +[W&B Sweeps]({{< relref "/guides/models/sweeps/" >}}) is a highly scalable hyperparameter search platform, which provides interesting insights and visualization about W&B experiments with minimal requirements code real-estate. Sweeps integrates seamlessly with Hydra projects with no-coding requirements. The only thing needed is a configuration file describing the various parameters to sweep over as normal. A simple example `sweep.yaml` file would be: diff --git a/content/guides/integrations/keras.md b/content/guides/integrations/keras.md index 33039ede7..ca3c3c86a 100644 --- a/content/guides/integrations/keras.md +++ b/content/guides/integrations/keras.md @@ -13,11 +13,11 @@ weight: 160 W&B has three callbacks for Keras, available from `wandb` v0.13.4. For the legacy `WandbCallback` scroll down. -- **`WandbMetricsLogger`** : Use this callback for [Experiment Tracking](/guides/track). It logs your training and validation metrics along with system metrics to Weights and Biases. +- **`WandbMetricsLogger`** : Use this callback for [Experiment Tracking]({{< relref "/guides/models/track" >}}). It logs your training and validation metrics along with system metrics to Weights and Biases. -- **`WandbModelCheckpoint`** : Use this callback to log your model checkpoints to Weight and Biases [Artifacts](/guides/artifacts). +- **`WandbModelCheckpoint`** : Use this callback to log your model checkpoints to Weight and Biases [Artifacts]({{< relref "/guides/core/artifacts/" >}}). -- **`WandbEvalCallback`**: This base callback logs model predictions to Weights and Biases [Tables](/guides/tables) for interactive visualization. +- **`WandbEvalCallback`**: This base callback logs model predictions to Weights and Biases [Tables]({{< relref "/guides/core/tables/" >}}) for interactive visualization. These new callbacks: @@ -208,7 +208,7 @@ model.fit( ``` {{% alert %}} -The W&B [Artifact page](/guides/artifacts/explore-and-traverse-an-artifact-graph) includes Table logs by default, rather than the **Workspace** page. +The W&B [Artifact page]({{< relref "/guides/core/artifacts/explore-and-traverse-an-artifact-graph" >}}) includes Table logs by default, rather than the **Workspace** page. {{% /alert %}} ### `WandbEvalCallback` reference @@ -232,7 +232,7 @@ You can override the `on_train_begin` or `on_epoch_end` methods to have more fin ## `WandbCallback` [legacy] -Use the W&B library [`WandbCallback`](/ref/python/integrations/keras/wandbcallback) Class to automatically save all the metrics and the loss values tracked in `model.fit`. +Use the W&B library [`WandbCallback`]({{< relref "/ref/python/integrations/keras/wandbcallback" >}}) Class to automatically save all the metrics and the loss values tracked in `model.fit`. ```python import wandb @@ -258,7 +258,7 @@ See our [example repo](https://github.com/wandb/examples) for scripts, including The `WandbCallback` class supports a wide variety of logging configuration options: specifying a metric to monitor, tracking of weights and gradients, logging of predictions on training_data and validation_data, and more. -Check out [the reference documentation for the `keras.WandbCallback`](../../ref/python/integrations/keras/wandbcallback.md) for full details. +Check out [the reference documentation for the `keras.WandbCallback`]({{< relref "/ref/python/integrations/keras/wandbcallback.md" >}}) for full details. The `WandbCallback` diff --git a/content/guides/integrations/kubeflow-pipelines-kfp.md b/content/guides/integrations/kubeflow-pipelines-kfp.md index f5eb9f713..7b3932bf1 100644 --- a/content/guides/integrations/kubeflow-pipelines-kfp.md +++ b/content/guides/integrations/kubeflow-pipelines-kfp.md @@ -63,7 +63,7 @@ add = components.create_component_from_func(add) ### Pass environment variables to containers -You may need to explicitly pass [environment variables](../../track/environment-variables.md) to your containers. For two-way linking, you should also set the environment variables `WANDB_KUBEFLOW_URL` to the base URL of your Kubeflow Pipelines instance. For example, `https://kubeflow.mysite.com`. +You may need to explicitly pass [environment variables]({{< relref "/guides/models/track/environment-variables.md" >}}) to your containers. For two-way linking, you should also set the environment variables `WANDB_KUBEFLOW_URL` to the base URL of your Kubeflow Pipelines instance. For example, `https://kubeflow.mysite.com`. ```python import os @@ -100,7 +100,7 @@ Click on any Run in the Kubeflow Pipelines UI that has been logged with W&B. ### Via the web app UI -The web app UI has the same content as the `Visualizations` tab in Kubeflow Pipelines, but with more space. Learn [more about the web app UI here](/guides/app). +The web app UI has the same content as the `Visualizations` tab in Kubeflow Pipelines, but with more space. Learn [more about the web app UI here]({{< relref "/guides/models/app" >}}). {{< img src="/images/integrations/kubeflow_pipelines.png" alt="View details about a particular run (and link back to the Kubeflow UI)" >}} @@ -108,7 +108,7 @@ The web app UI has the same content as the `Visualizations` tab in Kubeflow Pipe ### Via the Public API (for programmatic access) -* For programmatic access, [see our Public API](/ref/python/public-api). +* For programmatic access, [see our Public API]({{< relref "/ref/python/public-api" >}}). ### Concept mapping from Kubeflow Pipelines to W&B @@ -116,10 +116,10 @@ Here's a mapping of Kubeflow Pipelines concepts to W&B | Kubeflow Pipelines | W&B | Location in W&B | | ------------------ | --- | --------------- | -| Input Scalar | [`config`](/guides/track/config) | [Overview tab](../../runs/intro.md#overview-tab) | -| Output Scalar | [`summary`](/guides/track/log) | [Overview tab](../../runs/intro.md#overview-tab) | -| Input Artifact | Input Artifact | [Artifacts tab](../../runs/intro.md#artifacts-tab) | -| Output Artifact | Output Artifact | [Artifacts tab](../../runs/intro.md#artifacts-tab) | +| Input Scalar | [`config`]({{< relref "/guides/models/track/config" >}}) | [Overview tab]({{< relref "/guides/models/track/runs/#overview-tab" >}}) | +| Output Scalar | [`summary`]({{< relref "/guides/models/track/log" >}}) | [Overview tab]({{< relref "/guides/models/track/runs/#overview-tab" >}}) | +| Input Artifact | Input Artifact | [Artifacts tab]({{< relref "/guides/models/track/runs/#artifacts-tab" >}}) | +| Output Artifact | Output Artifact | [Artifacts tab]({{< relref "/guides/models/track/runs/#artifacts-tab" >}}) | ## Fine-grain logging @@ -150,7 +150,7 @@ def train_model( ### With implicit wandb integrations -If you're using a [framework integration we support](/guides/integrations), you can also pass in the callback directly: +If you're using a [framework integration we support]({{< relref "/guides/integrations/" >}}), you can also pass in the callback directly: ```python @wandb_log diff --git a/content/guides/integrations/langchain.md b/content/guides/integrations/langchain.md deleted file mode 100644 index 5550261d8..000000000 --- a/content/guides/integrations/langchain.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -menu: - default: - identifier: langchain - parent: integrations -title: LangChain -weight: 180 ---- -[LangChain](https://python.langchain.com/) is a framework for developing applications powered by language models. - -## Using LangChain in Weights & Biases - -To use the Weights & Biases LangChain integration please see our [W&B Prompts Quickstart](./prompts/quickstart.md) - -Or try our [Google Colab Jupyter notebook](http://wandb.me/prompts-quickstart) \ No newline at end of file diff --git a/content/guides/integrations/lightgbm.md b/content/guides/integrations/lightgbm.md index 159e29a2e..f691e211c 100644 --- a/content/guides/integrations/lightgbm.md +++ b/content/guides/integrations/lightgbm.md @@ -29,7 +29,7 @@ Looking for working code examples? Check out [our repository of examples on GitH ## Tuning your hyperparameters with Sweeps -Attaining the maximum performance out of models requires tuning hyperparameters, like tree depth and learning rate. Weights & Biases includes [Sweeps](../sweeps/intro.md), a powerful toolkit for configuring, orchestrating, and analyzing large hyperparameter testing experiments. +Attaining the maximum performance out of models requires tuning hyperparameters, like tree depth and learning rate. Weights & Biases includes [Sweeps]({{< relref "/guides/models/sweeps/" >}}), a powerful toolkit for configuring, orchestrating, and analyzing large hyperparameter testing experiments. To learn more about these tools and see an example of how to use Sweeps with XGBoost, check out this interactive Colab notebook. diff --git a/content/guides/integrations/lightning.md b/content/guides/integrations/lightning.md index 4c7ea1c9d..b3c16e643 100644 --- a/content/guides/integrations/lightning.md +++ b/content/guides/integrations/lightning.md @@ -300,7 +300,7 @@ for epoch in range(num_epochs): ## Log the min/max of a metric -Using wandb's [`define_metric`](/ref/python/run#define_metric) function you can define whether you'd like your W&B summary metric to display the min, max, mean or best value for that metric. If `define`_`metric` _ isn't used, then the last value logged with appear in your summary metrics. See the `define_metric` [reference docs here](/ref/python/run#define_metric) and the [guide here](/guides/track/log/customize-logging-axes) for more. +Using wandb's [`define_metric`](/ref/python/run#define_metric) function you can define whether you'd like your W&B summary metric to display the min, max, mean or best value for that metric. If `define`_`metric` _ isn't used, then the last value logged with appear in your summary metrics. See the `define_metric` [reference docs here](/ref/python/run#define_metric) and the [guide here](/guides/models/track/log/customize-logging-axes) for more. To tell W&B to keep track of the max validation accuracy in the W&B summary metric, call `wandb.define_metric` only once, at the beginning of training: @@ -338,7 +338,7 @@ fabric.log_dict({"val_accuracy": val_accuracy}) ## Checkpoint a model -To save model checkpoints as W&B [Artifacts](/guides/artifacts/), +To save model checkpoints as W&B [Artifacts](/guides/core/artifacts/), use the Lightning [`ModelCheckpoint`](https://pytorch-lightning.readthedocs.io/en/stable/api/pytorch_lightning.callbacks.ModelCheckpoint.html#pytorch_lightning.callbacks.ModelCheckpoint) callback and set the `log_model` argument in the `WandbLogger`. {{< tabpane text=true >}} @@ -361,7 +361,7 @@ fabric = L.Fabric(loggers=[wandb_logger], callbacks=[checkpoint_callback]) {{< /tabpane >}} -The _latest_ and _best_ aliases are automatically set to easily retrieve a model checkpoint from a W&B [Artifact](/guides/artifacts/): +The _latest_ and _best_ aliases are automatically set to easily retrieve a model checkpoint from a W&B [Artifact](/guides/core/artifacts/): ```python # reference can be retrieved in artifacts panel @@ -414,11 +414,11 @@ optimizer.load_state_dict(full_checkpoint["optimizer"]) {{% /tab %}} {{< /tabpane >}} -The model checkpoints you log are viewable through the [W&B Artifacts](/guides/artifacts) UI, and include the full model lineage (see an example model checkpoint in the UI [here](https://wandb.ai/wandb/arttest/artifacts/model/iv3_trained/5334ab69740f9dda4fed/lineage?_gl=1*yyql5q*_ga*MTQxOTYyNzExOS4xNjg0NDYyNzk1*_ga_JH1SJHJQXJ*MTY5MjMwNzI2Mi4yNjkuMS4xNjkyMzA5NjM2LjM3LjAuMA..)). +The model checkpoints you log are viewable through the [W&B Artifacts](/guides/core/artifacts) UI, and include the full model lineage (see an example model checkpoint in the UI [here](https://wandb.ai/wandb/arttest/artifacts/model/iv3_trained/5334ab69740f9dda4fed/lineage?_gl=1*yyql5q*_ga*MTQxOTYyNzExOS4xNjg0NDYyNzk1*_ga_JH1SJHJQXJ*MTY5MjMwNzI2Mi4yNjkuMS4xNjkyMzA5NjM2LjM3LjAuMA..)). To bookmark your best model checkpoints and centralize them across your team, you can link them to the [W&B Model Registry](/guides/models). -Here you can organize your best models by task, manage model lifecycle, facilitate easy tracking and auditing throughout the ML lifecyle, and [automate](/guides/artifacts/project-scoped-automations/#create-a-webhook-automation) downstream actions with webhooks or jobs. +Here you can organize your best models by task, manage model lifecycle, facilitate easy tracking and auditing throughout the ML lifecyle, and [automate](/guides/core/artifacts/project-scoped-automations/#create-a-webhook-automation) downstream actions with webhooks or jobs. ## Log images, text, and more diff --git a/content/guides/integrations/metaflow.md b/content/guides/integrations/metaflow.md index 33baed5e6..9df53a69d 100644 --- a/content/guides/integrations/metaflow.md +++ b/content/guides/integrations/metaflow.md @@ -125,7 +125,7 @@ class WandbExampleFlow(FlowSpec): ## Access your data programmatically -You can access the information we've captured in three ways: inside the original Python process being logged using the [`wandb` client library](../../../ref/python/README.md), with the [web app UI](../../track/workspaces.md), or programmatically using [our Public API](../../../ref/python/public-api/README.md). `Parameter`s are saved to W&B's [`config`](../../track/config.md) and can be found in the [Overview tab](../../runs/intro.md#overview-tab). `datasets`, `models`, and `others` are saved to [W&B Artifacts](../../artifacts/intro.md) and can be found in the [Artifacts tab](../../runs/intro.md#artifacts-tab). Base python types are saved to W&B's [`summary`](../../track/log/intro.md) dict and can be found in the Overview tab. See our [guide to the Public API](../../track/public-api-guide.md) for details on using the API to get this information programmatically from outside . +You can access the information we've captured in three ways: inside the original Python process being logged using the [`wandb` client library]({{< relref "/ref/python/" >}}), with the [web app UI]({{< relref "/guides/models/track/workspaces.md" >}}), or programmatically using [our Public API]({{< relref "/ref/python/public-api/" >}}). `Parameter`s are saved to W&B's [`config`]({{< relref "/guides/models/track/config.md" >}}) and can be found in the [Overview tab]({{< relref "/guides/models/track/runs/#overview-tab" >}}). `datasets`, `models`, and `others` are saved to [W&B Artifacts]({{< relref "/guides/core/artifacts/" >}}) and can be found in the [Artifacts tab]({{< relref "/guides/models/track/runs/#artifacts-tab" >}}). Base python types are saved to W&B's [`summary`]({{< relref "/guides/models/track/log/" >}}) dict and can be found in the Overview tab. See our [guide to the Public API]({{< relref "/guides/models/track/public-api-guide.md" >}}) for details on using the API to get this information programmatically from outside . ### Cheat sheet diff --git a/content/guides/integrations/mmengine.md b/content/guides/integrations/mmengine.md index 8ed6ed584..8021c7a67 100644 --- a/content/guides/integrations/mmengine.md +++ b/content/guides/integrations/mmengine.md @@ -81,7 +81,7 @@ This section demonstrates a typical workflow using `WandbVisBackend` using [`mme ``` {{% alert %}} - You pass a dictionary of arguments for [W&B run initialization](/ref/python/init) input parameters to `init_kwargs`. + You pass a dictionary of arguments for [W&B run initialization]({{< relref "/ref/python/init" >}}) input parameters to `init_kwargs`. {{% /alert %}} 2. Initialize a `runner` with the `visualizer`, and call `runner.train()`. diff --git a/content/guides/integrations/openai-api.md b/content/guides/integrations/openai-api.md index 222c6d05d..af3154a4b 100644 --- a/content/guides/integrations/openai-api.md +++ b/content/guides/integrations/openai-api.md @@ -13,7 +13,7 @@ Use the W&B OpenAI API integration to log requests, responses, token counts and {{% alert %}} -See the [OpenAI fine-tuning integration](./openai-fine-tuning.md) to learn how to use W&B to track your fine-tuning experiments, models, and datasets and share your results with your colleagues. +See the [OpenAI fine-tuning integration]({{< relref "./openai-fine-tuning.md" >}}) to learn how to use W&B to track your fine-tuning experiments, models, and datasets and share your results with your colleagues. {{% /alert %}} Log your API inputs and outputs you can quickly evaluate the performance of difference prompts, compare different model settings (such as temperature), and track other usage metrics such as token usage. @@ -46,7 +46,7 @@ from wandb.integration.openai import autolog autolog({"project": "gpt5"}) ``` -You can optionally pass a dictionary with argument that `wandb.init()` accepts to `autolog`. This includes a project name, team name, entity, and more. For more information about [`wandb.init`](../../../ref/python/init.md), see the API Reference Guide. +You can optionally pass a dictionary with argument that `wandb.init()` accepts to `autolog`. This includes a project name, team name, entity, and more. For more information about [`wandb.init`]({{< relref "/ref/python/init.md" >}}), see the API Reference Guide. ### 2. Call the OpenAI API Each call you make to the OpenAI API is now logged to W&B automatically. @@ -68,7 +68,7 @@ response = openai.ChatCompletion.create(**chat_request_kwargs) ### 3. View your OpenAI API inputs and responses -Click on the W&B [run](../../runs/intro.md) link generated by `autolog` in **step 1**. This redirects you to your project workspace in the W&B App. +Click on the W&B [run]({{< relref "/guides/models/track/runs/" >}}) link generated by `autolog` in **step 1**. This redirects you to your project workspace in the W&B App. Select a run you created to view the trace table, trace timeline and the model architecture of the OpenAI LLM used. diff --git a/content/guides/integrations/openai-fine-tuning.md b/content/guides/integrations/openai-fine-tuning.md index 3cee15324..ac87f5879 100644 --- a/content/guides/integrations/openai-fine-tuning.md +++ b/content/guides/integrations/openai-fine-tuning.md @@ -86,7 +86,7 @@ WandbLogger.sync( | wait_for_job_success | Once an OpenAI fine-tuning job is started it usually takes a bit of time. To ensure that your metrics are logged to W&B as soon as the fine-tune job is finished, this setting will check every 60 seconds for the status of the fine-tune job to change to `succeeded`. Once the fine-tune job is detected as being successful, the metrics will be synced automatically to W&B. Set to True by default. | | model_artifact_name | The name of the model artifact that is logged. Defaults to `"model-metadata"`. | | model_artifact_type | The type of the model artifact that is logged. Defaults to `"model"`. | -| \*\*kwargs_wandb_init | Aany additional argument passed directly to [`wandb.init()`](../../../ref/python/init.md) | +| \*\*kwargs_wandb_init | Aany additional argument passed directly to [`wandb.init()`]({{< relref "/ref/python/init.md" >}}) | ## Dataset Versioning and Visualization @@ -107,7 +107,7 @@ The datasets are visualized as W&B Tables, which allows you to explore, search, OpenAI gives you an id of the fine-tuned model. Since we don't have access to the model weights, the `WandbLogger` creates a `model_metadata.json` file with all the details (hyperparameters, data file ids, etc.) of the model along with the `fine_tuned_model`` id and is logged as a W&B Artifact. -This model (metadata) artifact can further be linked to a model in the [W&B Model Registry](../../model_registry/intro.md). +This model (metadata) artifact can further be linked to a model in the [W&B Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}). {{< img src="/images/integrations/openai_model_metadata.png" alt="" >}} diff --git a/content/guides/integrations/openai-gym.md b/content/guides/integrations/openai-gym.md index 16723d839..da79d80c7 100644 --- a/content/guides/integrations/openai-gym.md +++ b/content/guides/integrations/openai-gym.md @@ -14,9 +14,9 @@ weight: 260 Since Gym is no longer an actively maintained project, try out our integration with Gymnasium. {{% /alert %}} -If you're using [OpenAI Gym](https://github.com/openai/gym), Weights & Biases automatically logs videos of your environment generated by `gym.wrappers.Monitor`. Just set the `monitor_gym` keyword argument to [`wandb.init`](../../../ref/python/init.md) to `True` or call `wandb.gym.monitor()`. +If you're using [OpenAI Gym](https://github.com/openai/gym), Weights & Biases automatically logs videos of your environment generated by `gym.wrappers.Monitor`. Just set the `monitor_gym` keyword argument to [`wandb.init`]({{< relref "/ref/python/init.md" >}}) to `True` or call `wandb.gym.monitor()`. -Our gym integration is very light. We simply [look at the name of the video file](https://github.com/wandb/wandb/blob/master/wandb/integration/gym/__init__.py#L15) being logged from `gym` and name it after that or fall back to `"videos"` if we don't find a match. If you want more control, you can always just manually [log a video](../../track/log/media.md). +Our gym integration is very light. We simply [look at the name of the video file](https://github.com/wandb/wandb/blob/master/wandb/integration/gym/__init__.py#L15) being logged from `gym` and name it after that or fall back to `"videos"` if we don't find a match. If you want more control, you can always just manually [log a video]({{< relref "/guides/models/track/log/media.md" >}}). The [OpenRL Benchmark](http://wandb.me/openrl-benchmark-report) by[ CleanRL](https://github.com/vwxyzjn/cleanrl) uses this integration for its OpenAI Gym examples. You can find source code (including [the specific code used for specific runs](https://wandb.ai/cleanrl/cleanrl.benchmark/runs/2jrqfugg/code?workspace=user-costa-huang)) that demonstrates how to use gym with diff --git a/content/guides/integrations/paddledetection.md b/content/guides/integrations/paddledetection.md index c9af5df43..5bb54a3de 100644 --- a/content/guides/integrations/paddledetection.md +++ b/content/guides/integrations/paddledetection.md @@ -56,7 +56,7 @@ To use wandb via arguments to `train.py` in [PaddleDetection](https://github.com * Add the `--use_wandb` flag * The first wandb arguments must be preceded by `-o` (you only need to pass this once) -* Each individual wandb argument must contain the prefix `wandb-` . For example any argument to be passed to [`wandb.init`](/ref/python/init) would get the `wandb-` prefix +* Each individual wandb argument must contain the prefix `wandb-` . For example any argument to be passed to [`wandb.init`]({{< relref "/ref/python/init" >}}) would get the `wandb-` prefix ```shell python tools/train.py diff --git a/content/guides/integrations/paddleocr.md b/content/guides/integrations/paddleocr.md index 636b708b5..30ab46945 100644 --- a/content/guides/integrations/paddleocr.md +++ b/content/guides/integrations/paddleocr.md @@ -46,7 +46,7 @@ Global: use_wandb: True ``` -Any additional, optional arguments that you might like to pass to [`wandb.init`](/ref/python/init) can also be added under the `wandb` header in the yaml file: +Any additional, optional arguments that you might like to pass to [`wandb.init`]({{< relref "/ref/python/init" >}}) can also be added under the `wandb` header in the yaml file: ``` wandb: diff --git a/content/guides/integrations/prodigy.md b/content/guides/integrations/prodigy.md index e3bb6f654..32550f57d 100644 --- a/content/guides/integrations/prodigy.md +++ b/content/guides/integrations/prodigy.md @@ -8,7 +8,7 @@ title: Prodigy weight: 290 --- -[Prodigy](https://prodi.gy/) is an annotation tool for creating training and evaluation data for machine learning models, error analysis, data inspection & cleaning. [W&B Tables](../../tables/tables-walkthrough.md) allow you to log, visualize, analyze, and share datasets (and more!) inside W&B. +[Prodigy](https://prodi.gy/) is an annotation tool for creating training and evaluation data for machine learning models, error analysis, data inspection & cleaning. [W&B Tables]({{< relref "/guides/core/tables/tables-walkthrough.md" >}}) allow you to log, visualize, analyze, and share datasets (and more!) inside W&B. The [W&B integration with Prodigy](https://github.com/wandb/wandb/blob/master/wandb/integration/prodigy/prodigy.py) adds simple and easy-to-use functionality to upload your Prodigy-annotated dataset directly to W&B for use with Tables. @@ -28,9 +28,9 @@ and get visual, interactive, shareable tables like this one: ## Quickstart -Use `wandb.integration.prodigy.upload_dataset` to upload your annotated prodigy dataset directly from the local Prodigy database to W&B in our [Table](/ref/python/data-types/table) format. For more information on Prodigy, including installation & setup, please refer to the [Prodigy documentation](https://prodi.gy/docs/). +Use `wandb.integration.prodigy.upload_dataset` to upload your annotated prodigy dataset directly from the local Prodigy database to W&B in our [Table]({{< relref "/ref/python/data-types/table" >}}) format. For more information on Prodigy, including installation & setup, please refer to the [Prodigy documentation](https://prodi.gy/docs/). -W&B will automatically try to convert images and named entity fields to [`wandb.Image`](/ref/python/data-types/image) and [`wandb.Html`](/ref/python/data-types/html)respectively. Extra columns may be added to the resulting table to include these visualizations. +W&B will automatically try to convert images and named entity fields to [`wandb.Image`]({{< relref "/ref/python/data-types/image" >}}) and [`wandb.Html`]({{< relref "/ref/python/data-types/html" >}})respectively. Extra columns may be added to the resulting table to include these visualizations. ## Read through a detailed example @@ -38,4 +38,4 @@ Explore the [Visualizing Prodigy Datasets Using W&B Tables](https://wandb.ai/ksh ## Also using spaCy? -W&B also has an integration with spaCy, see the [docs here](/guides/integrations/spacy). \ No newline at end of file +W&B also has an integration with spaCy, see the [docs here]({{< relref "/guides/integrations/spacy" >}}). \ No newline at end of file diff --git a/content/guides/integrations/pytorch.md b/content/guides/integrations/pytorch.md index 5dcc51fb0..28746ce1f 100644 --- a/content/guides/integrations/pytorch.md +++ b/content/guides/integrations/pytorch.md @@ -18,7 +18,7 @@ You can also see our [example repo](https://github.com/wandb/examples) for scrip ## Log gradients with `wandb.watch` -To automatically log gradients, you can call [`wandb.watch`](../../ref/python/watch.md) and pass in your PyTorch model. +To automatically log gradients, you can call [`wandb.watch`]({{< relref "/ref/python/watch.md" >}}) and pass in your PyTorch model. ```python import wandb @@ -40,7 +40,7 @@ for batch_idx, (data, target) in enumerate(train_loader): wandb.log({"loss": loss}) ``` -If you need to track multiple models in the same script, you can call `wandb.watch` on each model separately. Reference documentation for this function is [here](../../ref/python/watch.md). +If you need to track multiple models in the same script, you can call `wandb.watch` on each model separately. Reference documentation for this function is [here]({{< relref "/ref/python/watch.md" >}}). {{% alert color="secondary" %}} Gradients, metrics, and the graph won't be logged until `wandb.log` is called after a forward _and_ backward pass. @@ -48,14 +48,14 @@ Gradients, metrics, and the graph won't be logged until `wandb.log` is called af ## Log images and media -You can pass PyTorch `Tensors` with image data into [`wandb.Image`](../../ref/python/data-types/image.md) and utilities from [`torchvision`](https://pytorch.org/vision/stable/index.html) will be used to convert them to images automatically: +You can pass PyTorch `Tensors` with image data into [`wandb.Image`]({{< relref "/ref/python/data-types/image.md" >}}) and utilities from [`torchvision`](https://pytorch.org/vision/stable/index.html) will be used to convert them to images automatically: ```python images_t = ... # generate or load images as PyTorch Tensors wandb.log({"examples": [wandb.Image(im) for im in images_t]}) ``` -For more on logging rich media to W&B in PyTorch and other frameworks, check out our [media logging guide](../track/log/media.md). +For more on logging rich media to W&B in PyTorch and other frameworks, check out our [media logging guide]({{< relref "/guides/models/track/log/media.md" >}}). If you also want to include information alongside media, like your model's predictions or derived metrics, use a `wandb.Table`. @@ -72,7 +72,7 @@ wandb.log({"mnist_predictions": my_table}) {{< img src="/images/integrations/pytorch_example_table.png" alt="The code above generates a table like this one. This model's looking good!" >}} -For more on logging and visualizing datasets and models, check out our [guide to W&B Tables](../tables/intro.md). +For more on logging and visualizing datasets and models, check out our [guide to W&B Tables]({{< relref "/guides/core/tables/" >}}). ## Profile PyTorch code diff --git a/content/guides/integrations/scikit.md b/content/guides/integrations/scikit.md index 2818551a7..df7e760b8 100644 --- a/content/guides/integrations/scikit.md +++ b/content/guides/integrations/scikit.md @@ -20,7 +20,7 @@ To get started: 3. Find your API key on the [Authorize page](https://wandb.ai/authorize). -4. If you are using Weights and Biases for the first time,check out a [quickstart](../../quickstart.md) +4. If you are using Weights and Biases for the first time,check out a [quickstart]({{< relref "/guides/quickstart.md" >}}) {{< tabpane text=true >}} {{% tab header="Command Line" value="cli" %}} diff --git a/content/guides/integrations/spacy.md b/content/guides/integrations/spacy.md index 901e447b6..9924014a1 100644 --- a/content/guides/integrations/spacy.md +++ b/content/guides/integrations/spacy.md @@ -52,7 +52,7 @@ model_log_interval = 1000 | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `project_name` | `str`. The name of the W&B Project. The project will be created automatically if it doesn’t exist yet. | | `remove_config_values` | `List[str]` . A list of values to exclude from the config before it is uploaded to W&B. `[]` by default. | -| `model_log_interval` | `Optional int`. `None` by default. If set, [model versioning](../model_registry/intro.md) with [Artifacts](../artifacts/intro.md)will be enabled. Pass in the number of steps to wait between logging model checkpoints. `None` by default. | +| `model_log_interval` | `Optional int`. `None` by default. If set, [model versioning]({{< relref "/guides/models/registry/model_registry/" >}}) with [Artifacts]({{< relref "/guides/core/artifacts/" >}})will be enabled. Pass in the number of steps to wait between logging model checkpoints. `None` by default. | | `log_dataset_dir` | `Optional str`. If passed a path, the dataset will be uploaded as an Artifact at the beginning of training. `None` by default. | | `entity` | `Optional str` . If passed, the run will be created in the specified entity | | `run_name` | `Optional str` . If specified, the run will be created with the specified name. | @@ -89,4 +89,4 @@ python -m spacy train \ {{< /tabpane >}} -When training begins, a link to your training run's [W&B page](../runs/intro.md) will be output which will take you to this run's experiment tracking [dashboard](../track/workspaces.md) in the Weights & Biases web UI. \ No newline at end of file +When training begins, a link to your training run's [W&B page]({{< relref "/guides/models/track/runs/" >}}) will be output which will take you to this run's experiment tracking [dashboard]({{< relref "/guides/models/track/workspaces.md" >}}) in the Weights & Biases web UI. \ No newline at end of file diff --git a/content/guides/integrations/torchtune.md b/content/guides/integrations/torchtune.md index e9f065d41..d69baf897 100644 --- a/content/guides/integrations/torchtune.md +++ b/content/guides/integrations/torchtune.md @@ -48,7 +48,7 @@ log_every_n_steps: 5 Enable W&B logging on the recipe's config file by modifying the `metric_logger` section. Change the `_component_` to `torchtune.utils.metric_logging.WandBLogger` class. You can also pass a `project` name and `log_every_n_steps` to customize the logging behavior. -You can also pass any other `kwargs` as you would to the [wandb.init](../../ref/python/init.md) method. For example, if you are working on a team, you can pass the `entity` argument to the `WandBLogger` class to specify the team name. +You can also pass any other `kwargs` as you would to the [wandb.init]({{< relref "/ref/python/init.md" >}}) method. For example, if you are working on a team, you can pass the `entity` argument to the `WandBLogger` class to specify the team name. {{< tabpane text=true >}} {{% tab header="Recipe's Config" value="config" %}} @@ -123,7 +123,7 @@ This is a fast evolving library, the current metrics are subject to change. If y The torchtune library supports various [checkpoint formats](https://pytorch.org/torchtune/stable/deep_dives/checkpointer.html). Depending on the origin of the model you are using, you should switch to the appropriate [checkpointer class](https://pytorch.org/torchtune/stable/deep_dives/checkpointer.html). -If you want to save the model checkpoints to [W&B Artifacts](../artifacts/intro.md), the simplest solution is to override the `save_checkpoint` functions inside the corresponding recipe. +If you want to save the model checkpoints to [W&B Artifacts]({{< relref "/guides/core/artifacts/" >}}), the simplest solution is to override the `save_checkpoint` functions inside the corresponding recipe. Here is an example of how you can override the `save_checkpoint` function to save the model checkpoints to W&B Artifacts. diff --git a/content/guides/integrations/ultralytics.md b/content/guides/integrations/ultralytics.md index 0c6dbabde..0bba81671 100644 --- a/content/guides/integrations/ultralytics.md +++ b/content/guides/integrations/ultralytics.md @@ -53,7 +53,7 @@ from wandb.integration.ultralytics import add_wandb_callback from ultralytics import YOLO ``` -Initialize the `YOLO` model of your choice, and invoke the `add_wandb_callback` function on it before performing inference with the model. This ensures that when you perform training, fine-tuning, validation, or inference, it automatically saves the experiment logs and the images, overlaid with both ground-truth and the respective prediction results using the [interactive overlays for computer vision tasks](../track/log/media#image-overlays-in-tables) on W&B along with additional insights in a [`wandb.Table`](../tables/intro.md). +Initialize the `YOLO` model of your choice, and invoke the `add_wandb_callback` function on it before performing inference with the model. This ensures that when you perform training, fine-tuning, validation, or inference, it automatically saves the experiment logs and the images, overlaid with both ground-truth and the respective prediction results using the [interactive overlays for computer vision tasks]({{< relref "/guides/models/track/log/media#image-overlays-in-tables" >}}) on W&B along with additional insights in a [`wandb.Table`]({{< relref "/guides/core/tables/" >}}). ```python # Initialize YOLO Model @@ -76,7 +76,7 @@ Here's how experiments tracked using W&B for an Ultralytics training or fine-tun
YOLO Fine-tuning Experiments
-Here's how epoch-wise validation results are visualized using a [W&B Table](../tables/intro.md): +Here's how epoch-wise validation results are visualized using a [W&B Table]({{< relref "/guides/core/tables/" >}}):
WandB Validation Visualization Table
@@ -108,14 +108,14 @@ Download a few images to test the integration on. You can use still images, vide !wget https://raw.githubusercontent.com/wandb/examples/ultralytics/colabs/ultralytics/assets/img5.png ``` -Next, initialize a W&B [run](../runs/intro.md) using `wandb.init`. +Next, initialize a W&B [run]({{< relref "/guides/models/track/runs/" >}}) using `wandb.init`. ```python # Initialize W&B run wandb.init(project="ultralytics", job_type="inference") ``` -Next, initialize your desired `YOLO` model and invoke the `add_wandb_callback` function on it before you perform inference with the model. This ensures that when you perform inference, it automatically logs the images overlaid with your [interactive overlays for computer vision tasks](../track/log/media#image-overlays-in-tables) along with additional insights in a [`wandb.Table`](../tables/intro.md). +Next, initialize your desired `YOLO` model and invoke the `add_wandb_callback` function on it before you perform inference with the model. This ensures that when you perform inference, it automatically logs the images overlaid with your [interactive overlays for computer vision tasks]({{< relref "/guides/models/track/log/media#image-overlays-in-tables" >}}) along with additional insights in a [`wandb.Table`]({{< relref "/guides/core/tables/" >}}). ```python # Initialize YOLO Model @@ -145,7 +145,7 @@ Here's how the interactive bbox overlay looks:
WandB Image Overlay
-You can fine more information on the W&B image overlays [here](../track/log/media.md#image-overlays). +You can fine more information on the W&B image overlays [here]({{< relref "/guides/models/track/log/media.md#image-overlays" >}}). ## More resources diff --git a/content/guides/integrations/xgboost.md b/content/guides/integrations/xgboost.md index cca58b168..6886bd84a 100644 --- a/content/guides/integrations/xgboost.md +++ b/content/guides/integrations/xgboost.md @@ -63,7 +63,7 @@ For additional examples, check out the [repository of examples on GitHub](https: ## Tune your hyperparameters with Sweeps -Attaining the maximum performance out of models requires tuning hyperparameters, like tree depth and learning rate. Weights & Biases includes [Sweeps](../sweeps/intro.md), a powerful toolkit for configuring, orchestrating, and analyzing large hyperparameter testing experiments. +Attaining the maximum performance out of models requires tuning hyperparameters, like tree depth and learning rate. Weights & Biases includes [Sweeps]({{< relref "/guides/models/sweeps/" >}}), a powerful toolkit for configuring, orchestrating, and analyzing large hyperparameter testing experiments. {{< cta-button colabLink="https://colab.research.google.com/github/wandb/examples/blob/master/colabs/boosting/Using_W%26B_Sweeps_with_XGBoost.ipynb" >}} diff --git a/content/guides/integrations/yolov5.md b/content/guides/integrations/yolov5.md index 200c4bd97..e2a44c705 100644 --- a/content/guides/integrations/yolov5.md +++ b/content/guides/integrations/yolov5.md @@ -18,7 +18,7 @@ All W&B logging features are compatible with data-parallel multi-GPU training, s {{% /alert %}} ## Track core experiments -Simply by installing `wandb`, you'll activate the built-in W&B [logging features](../track/log/intro.md): system metrics, model metrics, and media logged to interactive [Dashboards](../track/workspaces.md). +Simply by installing `wandb`, you'll activate the built-in W&B [logging features]({{< relref "/guides/models/track/log/" >}}): system metrics, model metrics, and media logged to interactive [Dashboards]({{< relref "/guides/models/track/workspaces.md" >}}). ```python pip install wandb @@ -34,9 +34,9 @@ Just follow the links printed to the standard out by wandb. By passing a few simple command line arguments to YOLO, you can take advantage of even more W&B features. -* Passing a number to `--save_period` will turn on [model versioning](../model_registry/intro.md). At the end of every `save_period` epochs, the model weights will be saved to W&B. The best-performing model on the validation set will be tagged automatically. +* Passing a number to `--save_period` will turn on [model versioning]({{< relref "/guides/models/registry/model_registry/" >}}). At the end of every `save_period` epochs, the model weights will be saved to W&B. The best-performing model on the validation set will be tagged automatically. * Turning on the `--upload_dataset` flag will also upload the dataset for data versioning. -* Passing a number to `--bbox_interval` will turn on [data visualization](../intro.md). At the end of every `bbox_interval` epochs, the outputs of the model on the validation set will be uploaded to W&B. +* Passing a number to `--bbox_interval` will turn on [data visualization]({{< relref "../" >}}). At the end of every `bbox_interval` epochs, the outputs of the model on the validation set will be uploaded to W&B. {{< tabpane text=true >}} {{% tab header="Model Versioning Only" value="modelversioning" %}} diff --git a/content/guides/integrations/yolox.md b/content/guides/integrations/yolox.md index 24a9e9394..9e8347a92 100644 --- a/content/guides/integrations/yolox.md +++ b/content/guides/integrations/yolox.md @@ -14,7 +14,7 @@ weight: 490 To use YOLOX with Weights & Biases you will first need to sign up for a Weights & Biases account [here](https://wandb.ai/site). -Then just use the `--logger wandb` command line argument to turn on logging with wandb. Optionally you can also pass all of the arguments that [wandb.init](/ref/python/init) would expect, just prepend `wandb-` to the start of each argument +Then just use the `--logger wandb` command line argument to turn on logging with wandb. Optionally you can also pass all of the arguments that [wandb.init]({{< relref "/ref/python/init" >}}) would expect, just prepend `wandb-` to the start of each argument `num_eval_imges` controls the number of validation set images and predictions that are logged to Weights & Biases tables for model evaluation. diff --git a/content/guides/models/_index.md b/content/guides/models/_index.md index 52075a467..b03bd5a2d 100644 --- a/content/guides/models/_index.md +++ b/content/guides/models/_index.md @@ -13,11 +13,9 @@ W&B Models is the system of record for ML Practitioners who want to organize the With W&B Models, you can: -- Track and visualize all [ML experiments](./track/intro.md). -- Optimize and fine-tune models at scale with [hyperparameter sweeps](./sweeps/intro.md). -- [Maintain a centralized hub of all models](./model_registry/intro.md), with a seamless handoff point to devops and deployment -- Configure custom automations that trigger key workflows for [model CI/CD](./model_registry/model-registry-automations.md). - - +- Track and visualize all [ML experiments]({{< relref "./track/" >}}). +- Optimize and fine-tune models at scale with [hyperparameter sweeps]({{< relref "./sweeps/" >}}). +- [Maintain a centralized hub of all models]({{< relref "./registry/" >}}), with a seamless handoff point to devops and deployment +- Configure custom automations that trigger key workflows for [model CI/CD]({{< relref "./automations/model-registry-automations.md" >}}). Machine learning practitioners rely on W&B Models as their ML system of record to track and visualize experiments, manage model versions and lineage, and optimize hyperparameters. \ No newline at end of file diff --git a/content/guides/models/app/_index.md b/content/guides/models/app/_index.md index 60dbf30db..5a297feb5 100644 --- a/content/guides/models/app/_index.md +++ b/content/guides/models/app/_index.md @@ -6,6 +6,6 @@ menu: title: W&B App UI Reference url: guides/app aliases: -- /guides/app/features +- /guides/models/app/features --- diff --git a/content/guides/models/app/features/cascade-settings.md b/content/guides/models/app/features/cascade-settings.md index e56654f04..93ae271e3 100644 --- a/content/guides/models/app/features/cascade-settings.md +++ b/content/guides/models/app/features/cascade-settings.md @@ -7,14 +7,14 @@ title: Manage workspace, section, and panel settings url: guides/app/features/cascade-settings --- -Within a given workspace page there are three different setting levels: workspaces, sections, and panels. [Workspace settings](#workspace-settings) apply to the entire workspace. [Section settings](#section-settings) apply to all panels within a section. [Panel settings](#panel-settings) apply to individual panels. +Within a given workspace page there are three different setting levels: workspaces, sections, and panels. [Workspace settings]({{< relref "#workspace-settings" >}}) apply to the entire workspace. [Section settings]({{< relref "#section-settings" >}}) apply to all panels within a section. [Panel settings]({{< relref "#panel-settings" >}}) apply to individual panels. ## Workspace settings -Workspace settings apply to all sections and all panels within those sections. You can edit two types of workspace settings: [**Workspace layout**](#workspace-layout-options) and [**Line plots**](#line-plots-options). **Workspace layouts** determine the structure of the workspace, while **Line plots** settings control the default settings for line plots in the workspace. +Workspace settings apply to all sections and all panels within those sections. You can edit two types of workspace settings: [**Workspace layout**]({{< relref "#workspace-layout-options" >}}) and [**Line plots**]({{< relref "#line-plots-options" >}}). **Workspace layouts** determine the structure of the workspace, while **Line plots** settings control the default settings for line plots in the workspace. To edit settings that apply to the overall structure of this workspace: @@ -29,7 +29,7 @@ Configure a workspaces layout to define the overall structure of the workspace. {{< img src="/images/app_ui/workspace_layout_settings.png" alt="" >}} -The workspace layout options page shows whether the workspace generates panels automatically or manually. To adjust a workspace's panel generation mode, refer to [Panels](panels/intro.md). +The workspace layout options page shows whether the workspace generates panels automatically or manually. To adjust a workspace's panel generation mode, refer to [Panels]({{< relref "panels/" >}}). This table describes each workspace layout option. @@ -55,9 +55,9 @@ You can edit two main settings within **Line plots** settings: **Data** and **Di | ----- | ----- | | **X axis** | The scale of the x-axis in line plots. The x-axis is set to **Step** by default. See the proceeding table for the list of x-axis options. | | **Range** | Minimum and maximum settings to display for x axis. | -| **Smoothing** | Change the smoothing on the line plot. For more information about smoothing, see [Smooth line plots](./panels/line-plot/smoothing.md). | +| **Smoothing** | Change the smoothing on the line plot. For more information about smoothing, see [Smooth line plots]({{< relref "./panels/line-plot/smoothing.md" >}}). | | **Outliers** | Rescale to exclude outliers from the default plot min and max scale. | -| **Point aggregation method** | Improve data visualization accuracy and performance. See [Point aggregation](./panels/line-plot/sampling.md) for more information. | +| **Point aggregation method** | Improve data visualization accuracy and performance. See [Point aggregation]({{< relref "./panels/line-plot/sampling.md" >}}) for more information. | | **Max number of runs or groups** | Limit the number of runs or groups displayed on the line plot. | In addition to **Step**, there are other options for the x-axis: @@ -72,7 +72,7 @@ In addition to **Step**, there are other options for the x-axis: {{% alert %}} -For information on how to edit an individual line plot, see [Edit line panel settings](./panels/line-plot/intro.md#edit-line-panel-settings) in Line plots. +For information on how to edit an individual line plot, see [Edit line panel settings]({{< relref "./panels/line-plot/#edit-line-panel-settings" >}}) in Line plots. {{% /alert %}} @@ -123,4 +123,4 @@ Customize an individual panel's settings to compare multiple lines on the same p 3. Within the modal that appears, you can edit settings related to the panel's data, display preferences, and more. {{< img src="/images/app_ui/panel_settings_modal.png" alt="" >}} -For a complete list of settings you can apply to a panel, see [Edit line panel settings](./panels/line-plot/intro.md#edit-line-panel-settings). \ No newline at end of file +For a complete list of settings you can apply to a panel, see [Edit line panel settings]({{< relref "./panels/line-plot/#edit-line-panel-settings" >}}). \ No newline at end of file diff --git a/content/guides/models/app/features/custom-charts/_index.md b/content/guides/models/app/features/custom-charts/_index.md index 787b5de5b..f8105f1cd 100644 --- a/content/guides/models/app/features/custom-charts/_index.md +++ b/content/guides/models/app/features/custom-charts/_index.md @@ -21,7 +21,7 @@ Use **Custom Charts** to create charts that aren't possible right now in the def ### How it works -1. **Log data**: From your script, log [config](../../../../guides/track/config.md) and summary data as you normally would when running with W&B. To visualize a list of multiple values logged at one specific time, use a custom`wandb.Table` +1. **Log data**: From your script, log [config]({{< relref "/guides/models/track/config.md" >}}) and summary data as you normally would when running with W&B. To visualize a list of multiple values logged at one specific time, use a custom`wandb.Table` 2. **Customize the chart**: Pull in any of this logged data with a [GraphQL](https://graphql.org) query. Visualize the results of your query with [Vega](https://vega.github.io/vega/), a powerful visualization grammar. 3. **Log the chart**: Call your own preset from your script with `wandb.plot_table()`. diff --git a/content/guides/models/app/features/custom-charts/walkthrough.md b/content/guides/models/app/features/custom-charts/walkthrough.md index b3586363b..df305173d 100644 --- a/content/guides/models/app/features/custom-charts/walkthrough.md +++ b/content/guides/models/app/features/custom-charts/walkthrough.md @@ -12,7 +12,7 @@ Use custom charts to control the data you're loading in to a panel and its visua ## 1. Log data to W&B -First, log data in your script. Use [wandb.config](../../../../guides/track/config.md) for single points set at the beginning of training, like hyperparameters. Use [wandb.log()](../../../../guides/track/log/intro.md) for multiple points over time, and log custom 2D arrays with `wandb.Table()`. We recommend logging up to 10,000 data points per logged key. +First, log data in your script. Use [wandb.config]({{< relref "/guides/models/track/config.md" >}}) for single points set at the beginning of training, like hyperparameters. Use [wandb.log()]({{< relref "/guides/models/track/log/" >}}) for multiple points over time, and log custom 2D arrays with `wandb.Table()`. We recommend logging up to 10,000 data points per logged key. ```python # Logging a custom table of data diff --git a/content/guides/models/app/features/panels/_index.md b/content/guides/models/app/features/panels/_index.md index 3605dba25..3be2f11ab 100644 --- a/content/guides/models/app/features/panels/_index.md +++ b/content/guides/models/app/features/panels/_index.md @@ -10,7 +10,7 @@ cascade: - url: guides/app/features/panels/:filename --- -Use workspace panel visualizations to explore your [logged data](/ref/python/log.md) by key, visualize the relationships between hyperparameters and output metrics, and more. +Use workspace panel visualizations to explore your [logged data]({{< relref "/ref/python/log.md" >}}) by key, visualize the relationships between hyperparameters and output metrics, and more. ## Workspace modes @@ -21,7 +21,7 @@ W&B projects support two different workspace modes. The icon next to the workspa | {{< img src="/images/app_ui/automated_workspace.svg" alt="automated workspace icon" width="32px" >}} | **Automated workspaces** automatically generate panels for all keys logged in the project. This can help you get started by visualizing all available data for the project. | | {{manual workspace icon}} | **Manual workspaces** start as blank slates and display only those panels intentionally added by users. Choose a manual workspace when you care mainly about a fraction of the keys logged in the project, or for a more focused analysis. | -To change how a workspace generates panels, [reset the workspace](#reset-a-workspace). +To change how a workspace generates panels, [reset the workspace]({{< relref "#reset-a-workspace" >}}). {{% alert title="Undo changes to your workspace" %}} To undo changes to your workspace, click the Undo button (arrow that points left) or type **CMD + Z** (macOS) or **CTRL + Z** (Windows / Linux). @@ -43,7 +43,8 @@ To add a panel: 1. To add a panel globally, click **Add panels** in the control bar near the panel search field. 1. To add a panel directly to a section instead, click the section's action `...` menu, then click **+ Add panels**. 1. Select the type of panel to add. -![](/images/app_ui/add_single_panel.gif) + + {{< img src="/images/app_ui/add_single_panel.gif" >}} ### Quick add @@ -58,7 +59,7 @@ To add a custom panel to your workspace: 1. Select the type of panel you’d like to create. 1. Follow the prompts to configure the panel. -To learn more about the options for each type of panel, refer to the relevant section below, such as [Line plots](line-plot/intro.md) or [Bar plots](bar-plot.md). +To learn more about the options for each type of panel, refer to the relevant section below, such as [Line plots]({{< relref "line-plot/" >}}) or [Bar plots]({{< relref "bar-plot.md" >}}). ## Manage panels @@ -88,7 +89,7 @@ To duplicate a panel: 1. At the top of the panel, click the action `...` menu. 1. Click **Duplicate**. -If desired, you can [customize](#edit-a-panel) or [move](#move-a-panel) the duplicated panel. +If desired, you can [customize]({{< relref "#edit-a-panel" >}}) or [move]({{< relref "#move-a-panel" >}}) the duplicated panel. ### Remove panels @@ -100,7 +101,7 @@ To remove a panel: To remove all panels from a manual workspace, click its action `...` menu, then click **Clear all panels**. -To remove all panels from an automatic or manual workspace, you can [reset the workspace](#reset-a-workspace). Select **Automatic** to start with the default set of panels, or select **Manual** to start with an empty workspace with no panels. +To remove all panels from an automatic or manual workspace, you can [reset the workspace]({{< relref "#reset-a-workspace" >}}). Select **Automatic** to start with the default set of panels, or select **Manual** to start with an empty workspace with no panels. ## Manage sections diff --git a/content/guides/models/app/features/panels/code.md b/content/guides/models/app/features/panels/code.md index 983caf1d5..2433b8df0 100644 --- a/content/guides/models/app/features/panels/code.md +++ b/content/guides/models/app/features/panels/code.md @@ -32,7 +32,7 @@ import wandb wandb.init(settings=wandb.Settings(code_dir=".")) ``` -This captures all python source code files in the current directory and all subdirectories as an [artifact](../../../../ref/python/artifact.md). For more control over the types and locations of source code files that are saved, see the [reference docs](../../../../ref/python/run.md#log_code). +This captures all python source code files in the current directory and all subdirectories as an [artifact]({{< relref "/ref/python/artifact.md" >}}). For more control over the types and locations of source code files that are saved, see the [reference docs]({{< relref "/ref/python/run.md#log_code" >}}). ### Set code saving in the UI diff --git a/content/guides/models/app/features/panels/line-plot/reference.md b/content/guides/models/app/features/panels/line-plot/reference.md index 2392f0bc4..44a6363c5 100644 --- a/content/guides/models/app/features/panels/line-plot/reference.md +++ b/content/guides/models/app/features/panels/line-plot/reference.md @@ -62,7 +62,7 @@ You can aggregate all of the runs by turning on grouping, or group over an indiv ## Smoothing -You can set the [smoothing coefficient](../../../../../support/formula_smoothing_algorithm.md) to be between 0 and 1 where 0 is no smoothing and 1 is maximum smoothing. +You can set the [smoothing coefficient]({{< relref "/support/formula_smoothing_algorithm.md" >}}) to be between 0 and 1 where 0 is no smoothing and 1 is maximum smoothing. ## Ignore outliers diff --git a/content/guides/models/app/features/panels/line-plot/sampling.md b/content/guides/models/app/features/panels/line-plot/sampling.md index b5124e5e3..1e9e1037c 100644 --- a/content/guides/models/app/features/panels/line-plot/sampling.md +++ b/content/guides/models/app/features/panels/line-plot/sampling.md @@ -8,7 +8,7 @@ weight: 20 --- -Use point aggregation methods within your line plots for improved data visualization accuracy and performance. There are two types of point aggregation modes: [full fidelity](#full-fidelity) and [random sampling](#random-sampling). W&B uses full fidelity mode by default. +Use point aggregation methods within your line plots for improved data visualization accuracy and performance. There are two types of point aggregation modes: [full fidelity]({{< relref "#full-fidelity" >}}) and [random sampling]({{< relref "#random-sampling" >}}). W&B uses full fidelity mode by default. ## Full fidelity @@ -120,7 +120,7 @@ By default, W&B uses full fidelity mode. To enable random sampling, follow these ### Access non sampled data -You can access the complete history of metrics logged during a run using the [W&B Run API](../../../../../ref/python/public-api/run.md). The following example demonstrates how to retrieve and process the loss values from a specific run: +You can access the complete history of metrics logged during a run using the [W&B Run API]({{< relref "/ref/python/public-api/run.md" >}}). The following example demonstrates how to retrieve and process the loss values from a specific run: ```python diff --git a/content/guides/models/app/features/panels/line-plot/smoothing.md b/content/guides/models/app/features/panels/line-plot/smoothing.md index 738688542..ce80cbaf5 100644 --- a/content/guides/models/app/features/panels/line-plot/smoothing.md +++ b/content/guides/models/app/features/panels/line-plot/smoothing.md @@ -10,10 +10,10 @@ weight: 30 W&B supports three types of smoothing: -- [exponential moving average](smoothing.md#exponential-moving-average-default) (default) -- [gaussian smoothing](smoothing.md#gaussian-smoothing) -- [running average](smoothing.md#running-average) -- [exponential moving average - Tensorboard](smoothing.md#exponential-moving-average-deprecated) (deprecated) +- [exponential moving average]({{< relref "smoothing.md#exponential-moving-average-default" >}}) (default) +- [gaussian smoothing]({{< relref "smoothing.md#gaussian-smoothing" >}}) +- [running average]({{< relref "smoothing.md#running-average" >}}) +- [exponential moving average - Tensorboard]({{< relref "smoothing.md#exponential-moving-average-deprecated" >}}) (deprecated) See these live in an [interactive W&B report](https://wandb.ai/carey/smoothing-example/reports/W-B-Smoothing-Features--Vmlldzo1MzY3OTc). diff --git a/content/guides/models/app/features/panels/parallel-coordinates.md b/content/guides/models/app/features/panels/parallel-coordinates.md index 71b485999..e2380bf31 100644 --- a/content/guides/models/app/features/panels/parallel-coordinates.md +++ b/content/guides/models/app/features/panels/parallel-coordinates.md @@ -12,7 +12,7 @@ Parallel coordinates charts summarize the relationship between large numbers of {{< img src="/images/app_ui/parallel_coordinates.gif" alt="" >}} -* **Axes**: Different hyperparameters from [`wandb.config`](../../../../guides/track/config.md) and metrics from [`wandb.log`](../../../../guides/track/log/intro.md). +* **Axes**: Different hyperparameters from [`wandb.config`]({{< relref "/guides/models/track/config.md" >}}) and metrics from [`wandb.log`]({{< relref "/guides/models/track/log/" >}}). * **Lines**: Each line represents a single run. Mouse over a line to see a tooltip with details about the run. All lines that match the current filters will be shown, but if you turn off the eye, lines will be grayed out. ## Panel Settings diff --git a/content/guides/models/app/features/panels/parameter-importance.md b/content/guides/models/app/features/panels/parameter-importance.md index 18b2c2e99..922d99a57 100644 --- a/content/guides/models/app/features/panels/parameter-importance.md +++ b/content/guides/models/app/features/panels/parameter-importance.md @@ -44,7 +44,7 @@ With the parameter manager, we can manually set the visible and hidden parameter {{< img src="/images/general/parameter-importance-4.png" alt="" >}} -This panel shows you all the parameters passed to the [wandb.config](/guides/track/config/) object in your training script. Next, it shows the feature importances and correlations of these config parameters with respect to the model metric you select (`val_loss` in this case). +This panel shows you all the parameters passed to the [wandb.config]({{< relref "/guides/models/track/config/" >}}) object in your training script. Next, it shows the feature importances and correlations of these config parameters with respect to the model metric you select (`val_loss` in this case). ### Importance diff --git a/content/guides/models/app/features/panels/query-panels/_index.md b/content/guides/models/app/features/panels/query-panels/_index.md index fd77e490d..8ccdf9773 100644 --- a/content/guides/models/app/features/panels/query-panels/_index.md +++ b/content/guides/models/app/features/panels/query-panels/_index.md @@ -74,7 +74,7 @@ runs.summary["cifar10_sample_table"] Breaking this down: -* `runs` is a variable automatically injected in Query Panel Expressions when the Query Panel is in a Workspace. Its "value" is the list of runs which are visible for that particular Workspace. [Read about the different attributes available within a run here](../../../../track/public-api-guide.md#understanding-the-different-attributes). +* `runs` is a variable automatically injected in Query Panel Expressions when the Query Panel is in a Workspace. Its "value" is the list of runs which are visible for that particular Workspace. [Read about the different attributes available within a run here]({{< relref "../../../../track/public-api-guide.md#understanding-the-different-attributes" >}}). * `summary` is an op which returns the Summary object for a Run. Ops are _mapped_, meaning this op is applied to each Run in the list, resulting in a list of Summary objects. * `["cifar10_sample_table"]` is a Pick op (denoted with brackets), with a parameter of `predictions`. Since Summary objects act like dictionaries or maps, this operation picks the `predictions` field off of each Summary object. diff --git a/content/guides/models/app/settings-page/_index.md b/content/guides/models/app/settings-page/_index.md index dfa0d9efa..df4868c61 100644 --- a/content/guides/models/app/settings-page/_index.md +++ b/content/guides/models/app/settings-page/_index.md @@ -8,6 +8,6 @@ menu: title: Settings --- -Within your individual user account you can edit: your profile picture, display name, geography location, biography information, emails associated to your account, and manage alerts for runs. You can also use the settings page to link your GitHub repository and delete your account. For more information, see [User settings](./user-settings.md). +Within your individual user account you can edit: your profile picture, display name, geography location, biography information, emails associated to your account, and manage alerts for runs. You can also use the settings page to link your GitHub repository and delete your account. For more information, see [User settings]({{< relref "./user-settings.md" >}}). -Use the team settings page to invite or remove new members to a team, manage alerts for team runs, change privacy settings, and view and manage storage usage. For more information about team settings, see [Team settings](./team-settings.md). \ No newline at end of file +Use the team settings page to invite or remove new members to a team, manage alerts for team runs, change privacy settings, and view and manage storage usage. For more information about team settings, see [Team settings]({{< relref "./team-settings.md" >}}). \ No newline at end of file diff --git a/content/guides/models/app/settings-page/storage.md b/content/guides/models/app/settings-page/storage.md index c0e42fcb3..4e3762f00 100644 --- a/content/guides/models/app/settings-page/storage.md +++ b/content/guides/models/app/settings-page/storage.md @@ -13,11 +13,11 @@ If you are approaching or exceeding your storage limit, there are multiple paths ## Manage storage consumption W&B offers different methods of optimizing your storage consumption: -- Use [reference artifacts](../../artifacts/track-external-files.md) to track files saved outside the W&B system, instead of uploading them to W&B storage. -- Use an [external cloud storage bucket](../features/teams.md) for storage. *(Enterprise only)* +- Use [reference artifacts]({{< relref "/guides/core/artifacts/track-external-files.md" >}}) to track files saved outside the W&B system, instead of uploading them to W&B storage. +- Use an [external cloud storage bucket]({{< relref "teams.md" >}}) for storage. *(Enterprise only)* ## Delete data You can also choose to delete data to remain under your storage limit. There are several ways to do this: - Delete data interactively with the app UI. -- [Set a TTL policy](../../artifacts/ttl.md) on Artifacts so they are automatically deleted. \ No newline at end of file +- [Set a TTL policy]({{< relref "/guides/core/artifacts/manage-data/ttl.md" >}}) on Artifacts so they are automatically deleted. \ No newline at end of file diff --git a/content/guides/models/app/settings-page/system-metrics.md b/content/guides/models/app/settings-page/system-metrics.md index c513ea32e..48a0a8f09 100644 --- a/content/guides/models/app/settings-page/system-metrics.md +++ b/content/guides/models/app/settings-page/system-metrics.md @@ -64,7 +64,7 @@ W&B assigns a `disk.in` tag to this metric. ### Disk Out Represents the total system disk write in megabytes (MB). -Similar to [Disk In](#disk-in), the initial disk write bytes are recorded when the first sample is taken. Subsequent samples calculate the difference between the current write bytes and the initial value. +Similar to [Disk In]({{< relref "#disk-in" >}}), the initial disk write bytes are recorded when the first sample is taken. Subsequent samples calculate the difference between the current write bytes and the initial value. W&B assigns a `disk.out` tag to this metric. @@ -104,7 +104,7 @@ W&B assigns a `network.sent` tag to this metric. ### Network Received Indicates the total bytes received over the network. -Similar to [Network Sent](#network-sent), the initial bytes received are recorded when the metric is first initialized. Subsequent samples calculate the difference between the current bytes received and the initial value. +Similar to [Network Sent]({{< relref "#network-sent" >}}), the initial bytes received are recorded when the metric is first initialized. Subsequent samples calculate the difference between the current bytes received and the initial value. W&B assigns a `network.recv` tag to this metric. diff --git a/content/guides/models/app/settings-page/team-settings.md b/content/guides/models/app/settings-page/team-settings.md index 285f2b3db..6e955996d 100644 --- a/content/guides/models/app/settings-page/team-settings.md +++ b/content/guides/models/app/settings-page/team-settings.md @@ -21,7 +21,7 @@ Only Administration account types can change team settings or remove a member fr ## Members The Members section shows a list of all pending invitations and the members that have either accepted the invitation to join the team. Each member listed displays a member’s name, username, email, team role, as well as their access privileges to Models and Weave, which is inherited by from the Organization. There are three standard team roles: Administrator (Admin), Member, and View-only. -See [Add and Manage teams](../../hosting/iam/manage-organization.md#add-and-manage-teams) for information on how to create a tea, invite users to a team, remove users from a team, and change a user's role. +See [Add and Manage teams]({{< relref "/guides/hosting/iam/access-management/manage-organization.md#add-and-manage-teams" >}}) for information on how to create a tea, invite users to a team, remove users from a team, and change a user's role. ## Avatar @@ -39,7 +39,7 @@ Toggle the switch next to the event type you want to receive alerts from. Weight * **Runs finished**: whether a Weights and Biases run successfully finished. * **Run crashed**: if a run has failed to finish. -For more information about how to set up and manage alerts, see [Send alerts with wandb.alert](../../runs/alert.md). +For more information about how to set up and manage alerts, see [Send alerts with wandb.alert]({{< relref "/guides/models/track/runs/alert.md" >}}). ## Privacy @@ -54,4 +54,4 @@ The **Usage** section describes the total memory usage the team has consumed on ## Storage -The **Storage** section describes the cloud storage bucket configuration that is being used for the team's data. For more information, see [Secure Storage Connector](../features/teams.md#secure-storage-connector) or check out our [W&B Server](../../hosting/data-security/secure-storage-connector.md) docs if you are self-hosting. \ No newline at end of file +The **Storage** section describes the cloud storage bucket configuration that is being used for the team's data. For more information, see [Secure Storage Connector]({{< relref "teams.md#secure-storage-connector" >}}) or check out our [W&B Server]({{< relref "/guides/hosting/data-security/secure-storage-connector.md" >}}) docs if you are self-hosting. \ No newline at end of file diff --git a/content/guides/models/app/settings-page/teams.md b/content/guides/models/app/settings-page/teams.md index c51c30b9c..d308876fa 100644 --- a/content/guides/models/app/settings-page/teams.md +++ b/content/guides/models/app/settings-page/teams.md @@ -53,7 +53,7 @@ Select a team role when you invite colleagues to join a team. There are followin - **Member**: A regular member of the team. An admin invites a team member by email. A team member cannot invite other members. Team members can only delete runs and sweep runs created by that member. Suppose you have two members A and B. Member B moves a Run from team B's project to a different project owned by Member A. Member A can not delete the Run Member B moved to Member A's project. Only the member that creates the Run, or the team admin, can delete the run. - **View-Only (Enterprise-only feature)**: View-Only members can view assets within the team such as runs, reports, and workspaces. They can follow and comment on reports, but they can not create, edit, or delete project overview, reports, or runs. View-Only members do not have an API key. - **Custom roles (Enterprise-only feature)**: Custom roles allow organization admins to compose new roles based on either of the **View-Only** or **Member** roles, together with additional permissions to achieve fine-grained access control. Team admins can then assign any of those custom roles to users in their respective teams. Refer to [Introducing Custom Roles for W&B Teams](https://wandb.ai/wandb_fc/announcements/reports/Introducing-Custom-Roles-for-W-B-Teams--Vmlldzo2MTMxMjQ3) for details. -- **Service accounts (Enterprise-only feature)**: Refer to [Use service accounts to automate workflows](../../hosting/iam/service-accounts.md). +- **Service accounts (Enterprise-only feature)**: Refer to [Use service accounts to automate workflows]({{< relref "/guides/hosting/iam/authentication/service-accounts.md" >}}). {{% alert %}} W&B recommends to have more than one admin in a team. It is a best practice to ensure that admin operations can continue when the primary admin is not available. @@ -80,7 +80,7 @@ The proceeding table lists permissions that apply to all projects across a given |Add/Remove Registry Admins | | | X | X | |Add/Remove Protected Aliases| | | X | | -See the [Model Registry](../../model_registry/access_controls.md) chapter for more information about protected aliases. +See the [Model Registry]({{< relref "/guides/models/registry/model_registry/access_controls.md" >}}) chapter for more information about protected aliases. ### Reports Report permissions grant access to create, view, and edit reports. The proceeding table lists permissions that apply to all reports across a given team. @@ -140,7 +140,7 @@ For example, to add a Twitter follow badge, add `[{{< img src="https://img.shiel ## Team trials -See the [pricing page](https://wandb.ai/site/pricing) for more information on W&B plans. You can download all your data at any time, either using the dashboard UI or the [Export API](../../../ref/python/public-api/README.md). +See the [pricing page](https://wandb.ai/site/pricing) for more information on W&B plans. You can download all your data at any time, either using the dashboard UI or the [Export API]({{< relref "/ref/python/public-api/" >}}). ## Privacy settings @@ -151,4 +151,4 @@ You can see the privacy settings of all team projects on the team settings page: ### Secure storage connector -The team-level secure storage connector allows teams to use their own cloud storage bucket with W&B. This provides greater data access control and data isolation for teams with highly sensitive data or strict compliance requirements. Refer to [Secure Storage Connector](../../hosting/data-security/secure-storage-connector.md) for more information. \ No newline at end of file +The team-level secure storage connector allows teams to use their own cloud storage bucket with W&B. This provides greater data access control and data isolation for teams with highly sensitive data or strict compliance requirements. Refer to [Secure Storage Connector]({{< relref "/guides/hosting/data-security/secure-storage-connector.md" >}}) for more information. \ No newline at end of file diff --git a/content/guides/models/app/settings-page/user-settings.md b/content/guides/models/app/settings-page/user-settings.md index 8219efc8b..9f46f9386 100644 --- a/content/guides/models/app/settings-page/user-settings.md +++ b/content/guides/models/app/settings-page/user-settings.md @@ -34,12 +34,12 @@ Within the **Beta Features** section you can optionally enable fun add-ons and s ## Alerts -Get notified when your runs crash, finish, or set custom alerts with [wandb.alert()](../../runs/alert.md). Receive notifications either through Email or Slack. Toggle the switch next to the event type you want to receive alerts from. +Get notified when your runs crash, finish, or set custom alerts with [wandb.alert()]({{< relref "/guides/models/track/runs/alert.md" >}}). Receive notifications either through Email or Slack. Toggle the switch next to the event type you want to receive alerts from. * **Runs finished**: whether a Weights and Biases run successfully finished. * **Run crashed**: notification if a run has failed to finish. -For more information about how to set up and manage alerts, see [Send alerts with wandb.alert](../../runs/alert.md). +For more information about how to set up and manage alerts, see [Send alerts with wandb.alert]({{< relref "/guides/models/track/runs/alert.md" >}}). ## Personal GitHub integration diff --git a/content/guides/models/automations/model-registry-automations.md b/content/guides/models/automations/model-registry-automations.md index cd97f1326..9ca508fd4 100644 --- a/content/guides/models/automations/model-registry-automations.md +++ b/content/guides/models/automations/model-registry-automations.md @@ -9,7 +9,7 @@ title: Model registry automations url: guides/model_registry/model-registry-automations --- -Create an automation to trigger workflow steps, such as automated model testing and deployment. To create an automation, define the action you want to occur based on an [event type](#event-types). +Create an automation to trigger workflow steps, such as automated model testing and deployment. To create an automation, define the action you want to occur based on an [event type]({{< relref "#event-types" >}}). For example, you can create a trigger that automatically deploys a model to GitHub when you add a new version of a registered model. @@ -26,7 +26,7 @@ An *event* is a change that takes place in the W&B ecosystem. The Model Registry - Use **Linking a new artifact to a registered model** to test new model candidates. - Use **Adding a new alias to a version of the registered model** to specify an alias that represents a special step of your workflow, like `deploy`, and any time a new model version has that alias applied. -See [Link a model version](./link-model-version.md) and [Create a custom alias](../artifacts/create-a-custom-alias.md). +See [Link a model version]({{< relref "/guides/models/registry/link_version.md" >}}) and [Create a custom alias]({{< relref "/guides/core/artifacts/create-a-custom-alias.md" >}}). ## Create a webhook automation @@ -44,7 +44,7 @@ To use a secret in your webhook, you must first add that secret to your team's s {{% alert %}} * Only W&B Admins can create, edit, or delete a secret. * Skip this section if the external server you send HTTP POST requests to does not use secrets. -* Secrets are also available if you use [W&B Server](../hosting/intro.md) in an Azure, GCP, or AWS deployment. Connect with your W&B account team to discuss how you can use secrets in W&B if you use a different deployment type. +* Secrets are also available if you use [W&B Server]({{< relref "/guides/hosting/" >}}) in an Azure, GCP, or AWS deployment. Connect with your W&B account team to discuss how you can use secrets in W&B if you use a different deployment type. {{% /alert %}} There are two types of secrets W&B suggests that you create when you use a webhook automation: @@ -62,7 +62,7 @@ Follow the instructions below to create a webhook: 6. Add your secret into the **Secret** field. 7. (Optional) Repeat steps 5 and 6 to create another secret (such as an access token) if your webhook requires additional secret keys or tokens to authenticate your webhook. -Specify the secrets you want to use for your webhook automation when you configure the webhook. See the [Configure a webhook](#configure-a-webhook) section for more information. +Specify the secrets you want to use for your webhook automation when you configure the webhook. See the [Configure a webhook]({{< relref "#configure-a-webhook" >}}) section for more information. {{% alert %}} Once you create a secret, you can access that secret in your W&B workflows with `$`. @@ -81,7 +81,7 @@ Before you can use a webhook, first configure that webhook in the W&B App UI. {{% alert %}} * Only W&B Admins can configure a webhook for a W&B Team. -* Ensure you already [created one or more secrets](#add-a-secret-for-authentication-or-authorization) if your webhook requires additional secret keys or tokens to authenticate your webhook. +* Ensure you already [created one or more secrets]({{< relref "#add-a-secret-for-authentication-or-authorization" >}}) if your webhook requires additional secret keys or tokens to authenticate your webhook. {{% /alert %}} 1. Navigate to the W&B App UI. @@ -95,7 +95,7 @@ Before you can use a webhook, first configure that webhook in the W&B App UI. 9. (Optional) From the **Access token** dropdown menu select additional secret keys or tokens required to authenticate a webhook (such as an access token). {{% alert %}} -See the [Troubleshoot your webhook](#troubleshoot-your-webhook) section to view where the secret and access token are specified in +See the [Troubleshoot your webhook]({{< relref "#troubleshoot-your-webhook" >}}) section to view where the secret and access token are specified in the POST request. {{% /alert %}} @@ -103,7 +103,7 @@ the POST request. ### Add a webhook Once you have a webhook configured and (optionally) a secret, navigate to the Model Registry App at [https://wandb.ai/registry/model](https://wandb.ai/registry/model). -1. From the **Event type** dropdown, select an [event type](#event-types). +1. From the **Event type** dropdown, select an [event type]({{< relref "#event-types" >}}). {{< img src="/images/models/webhook_select_event.png" alt="" >}} 2. (Optional) If you selected **A new version is added to a registered model** event, provide the name of a registered model from the **Registered model** dropdown. {{< img src="/images/models/webhook_new_version_reg_model.png" alt="" >}} @@ -111,7 +111,7 @@ Once you have a webhook configured and (optionally) a secret, navigate to the Mo 4. Click on the **Next step** button. 5. Select a webhook from the **Webhook** dropdown. {{< img src="/images/models/webhooks_select_from_dropdown.png" alt="" >}} -6. (Optional) Provide a payload in the JSON expression editor. See the [Example payload](#example-payloads) section for common use case examples. +6. (Optional) Provide a payload in the JSON expression editor. See the [Example payload]({{< relref "#example-payloads" >}}) section for common use case examples. 7. Click on **Next step**. 8. Provide a name for your webhook automation in the **Automation name** field. {{< img src="/images/models/webhook_name_automation.png" alt="" >}} @@ -179,7 +179,7 @@ Verify that your access tokens have required set of permissions to trigger your ${entity_name} --> "" ``` - Use template strings to dynamically pass context from W&B to GitHub Actions and other tools. If those tools can call Python scripts, they can consume the registered model artifacts through the [W&B API](../artifacts/download-and-use-an-artifact.md). + Use template strings to dynamically pass context from W&B to GitHub Actions and other tools. If those tools can call Python scripts, they can consume the registered model artifacts through the [W&B API]({{< relref "/guides/core/artifacts/download-and-use-an-artifact.md" >}}). For more information about repository dispatch, see the [official documentation on the GitHub Marketplace](https://github.com/marketplace/actions/repository-dispatch). @@ -278,7 +278,7 @@ Interactively troubleshoot your webhook with the W&B App UI or programmatically 3. Click on the horizontal three docs (meatball icon) next to the name of your webhook. 4. Select **Test**. 5. From the UI panel that appears, paste your POST request to the field that appears. - ![](/images/models/webhook_ui.png) + {{< img src="/images/models/webhook_ui.png" >}} 6. Click on **Test webhook**. Within the W&B App UI, W&B posts the response made by your endpoint. diff --git a/content/guides/models/automations/project-scoped-automations.md b/content/guides/models/automations/project-scoped-automations.md index bee07a588..4a5125650 100644 --- a/content/guides/models/automations/project-scoped-automations.md +++ b/content/guides/models/automations/project-scoped-automations.md @@ -9,12 +9,12 @@ title: Trigger CI/CD events when artifact changes url: guides/artifacts/project-scoped-automations --- -Create an automation that triggers when an artifact is changed. Use artifact automations when you want to automate downstream actions for versioning artifacts. To create an automation, define the action you want to occur based on an [event type](#event-types). +Create an automation that triggers when an artifact is changed. Use artifact automations when you want to automate downstream actions for versioning artifacts. To create an automation, define the action you want to occur based on an [event type]({{< relref "#event-types" >}}). {{% alert %}} Artifact automations are scoped to a project. This means that only events within a project will trigger an artifact automation. -This is in contrast to automations created in the W&B Model Registry. Automations created in the model registry are in scope of the Model Registry. They are triggered when events are performed on model versions linked to the [Model Registry](../model_registry/intro.md). For information on how to create an automations for model versions, see the [Automations for Model CI/CD](../model_registry/model-registry-automations.md) page in the [Model Registry chapter](../model_registry/intro.md). +This is in contrast to automations created in the W&B Model Registry. Automations created in the model registry are in scope of the Model Registry. They are triggered when events are performed on model versions linked to the [Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}). For information on how to create an automations for model versions, see the [Automations for Model CI/CD]({{< relref "/guides/models/automations/model-registry-automations.md" >}}) page in the [Model Registry chapter]({{< relref "/guides/models/registry/model_registry/" >}}). {{% /alert %}} @@ -42,7 +42,7 @@ To use a secret in your webhook, you must first add that secret to your team's s {{% alert %}} * Only W&B Admins can create, edit, or delete a secret. * Skip this section if the external server you send HTTP POST requests to does not use secrets. -* Secrets are also available if you use [W&B Server](../hosting/intro.md) in an Azure, GCP, or AWS deployment. Connect with your W&B account team to discuss how you can use secrets in W&B if you use a different deployment type. +* Secrets are also available if you use [W&B Server]({{< relref "/guides/hosting/" >}}) in an Azure, GCP, or AWS deployment. Connect with your W&B account team to discuss how you can use secrets in W&B if you use a different deployment type. {{% /alert %}} @@ -62,7 +62,7 @@ Follow the instructions below to create a webhook: 6. Add your secret into the **Secret** field. 7. (Optional) Repeat steps 5 and 6 to create another secret (such as an access token) if your webhook requires additional secret keys or tokens to authenticate your webhook. -Specify the secrets you want to use for your webhook automation when you configure the webhook. See the [Configure a webhook](#configure-a-webhook) section for more information. +Specify the secrets you want to use for your webhook automation when you configure the webhook. See the [Configure a webhook]({{< relref "#configure-a-webhook" >}}) section for more information. {{% alert %}} Once you create a secret, you can access that secret in your W&B workflows with `$`. @@ -73,7 +73,7 @@ Before you can use a webhook, you will first need to configure that webhook in t {{% alert %}} * Only W&B Admins can configure a webhook for a W&B Team. -* Ensure you already [created one or more secrets](#add-a-secret-for-authentication-or-authorization) if your webhook requires additional secret keys or tokens to authenticate your webhook. +* Ensure you already [created one or more secrets]({{< relref "#add-a-secret-for-authentication-or-authorization" >}}) if your webhook requires additional secret keys or tokens to authenticate your webhook. {{% /alert %}} 1. Navigate to the W&B App UI. @@ -87,14 +87,14 @@ Before you can use a webhook, you will first need to configure that webhook in t 9. (Optional) From the **Access token** dropdown menu select additional secret keys or tokens required to authenticate a webhook (such as an access token). {{% alert %}} -See the [Troubleshoot your webhook](#troubleshoot-your-webhook) section to view where the secret and access token are specified in +See the [Troubleshoot your webhook]({{< relref "#troubleshoot-your-webhook" >}}) section to view where the secret and access token are specified in the POST request. {{% /alert %}} ### Add a webhook Once you have a webhook configured and (optionally) a secret, navigate to your project workspace. Click on the **Automations** tab on the left sidebar. -1. From the **Event type** dropdown, select an [event type](#event-types). +1. From the **Event type** dropdown, select an [event type]({{< relref "#event-types" >}}). {{< img src="/images/artifacts/artifact_webhook_select_event.png" alt="" >}} 2. If you selected **A new version of an artifact is created in a collection** event, provide the name of the artifact collection that the automation should respond to from the **Artifact collection** dropdown. {{< img src="/images/artifacts/webhook_new_version_artifact.png" alt="" >}} @@ -102,7 +102,7 @@ Once you have a webhook configured and (optionally) a secret, navigate to your p 4. Click on the **Next step** button. 5. Select a webhook from the **Webhook** dropdown. {{< img src="/images/artifacts/artifacts_webhooks_select_from_dropdown.png" alt="" >}} -6. (Optional) Provide a payload in the JSON expression editor. See the [Example payload](#example-payloads) section for common use case examples. +6. (Optional) Provide a payload in the JSON expression editor. See the [Example payload]({{< relref "#example-payloads" >}}) section for common use case examples. 7. Click on **Next step**. 8. Provide a name for your webhook automation in the **Automation name** field. {{< img src="/images/artifacts/artifacts_webhook_name_automation.png" alt="" >}} @@ -170,7 +170,7 @@ ${project_name} --> "" ${entity_name} --> "" ``` -Use template strings to dynamically pass context from W&B to GitHub Actions and other tools. If those tools can call Python scripts, they can consume W&B artifacts through the [W&B API](../artifacts/download-and-use-an-artifact.md). +Use template strings to dynamically pass context from W&B to GitHub Actions and other tools. If those tools can call Python scripts, they can consume W&B artifacts through the [W&B API]({{< relref "/guides/core/artifacts/download-and-use-an-artifact.md" >}}). For more information about repository dispatch, see the [official documentation on the GitHub Marketplace](https://github.com/marketplace/actions/repository-dispatch). {{% /tab %}} diff --git a/content/guides/models/registry/_index.md b/content/guides/models/registry/_index.md index c719f1bf5..146cd9201 100644 --- a/content/guides/models/registry/_index.md +++ b/content/guides/models/registry/_index.md @@ -13,21 +13,21 @@ cascade: {{% alert %}} -W&B Registry is now in public preview. Visit [this](#enable-wb-registry) section to learn how to enable it for your deployment type. +W&B Registry is now in public preview. Visit [this]({{< relref "#enable-wb-registry" >}}) section to learn how to enable it for your deployment type. {{% /alert %}} -W&B Registry is a curated central repository of [artifact](../artifacts/intro.md) versions within your organization. Users who [have permission](./configure_registry.md) within your organization can [download](./download_use_artifact.md), share, and collaboratively manage the lifecycle of all artifacts, regardless of the team that user belongs to. +W&B Registry is a curated central repository of [artifact]({{< relref "/guides/core/artifacts/" >}}) versions within your organization. Users who [have permission]({{< relref "./configure_registry.md" >}}) within your organization can [download]({{< relref "./download_use_artifact.md" >}}), share, and collaboratively manage the lifecycle of all artifacts, regardless of the team that user belongs to. -You can use the Registry to [track artifact versions](./link_version.md), audit the history of an artifact's usage and changes, ensure governance and compliance of your artifacts, and [automate downstream processes such as model CI/CD](../automations/intro.md). +You can use the Registry to [track artifact versions]({{< relref "./link_version.md" >}}), audit the history of an artifact's usage and changes, ensure governance and compliance of your artifacts, and [automate downstream processes such as model CI/CD]({{< relref "../automations/" >}}). In summary, use W&B Registry to: -- [Promote](./link_version.md) artifact versions that satisfy a machine learning task to other users in your organization. -- Organize [artifacts with tags](./organize-with-tags.md) so that you can find or reference specific artifacts. -- Track an [artifact’s lineage](../model_registry/model-lineage.md) and audit the history of changes. -- [Automate](../model_registry/model-registry-automations.md) downstream processes such as model CI/CD. -- [Limit who in your organization](./configure_registry.md) can access artifacts in each registry. +- [Promote]({{< relref "./link_version.md" >}}) artifact versions that satisfy a machine learning task to other users in your organization. +- Organize [artifacts with tags]({{< relref "./organize-with-tags.md" >}}) so that you can find or reference specific artifacts. +- Track an [artifact’s lineage]({{< relref "/guides/models/registry/model_registry/model-lineage.md" >}}) and audit the history of changes. +- [Automate]({{< relref "/guides/models/automations/model-registry-automations.md" >}}) downstream processes such as model CI/CD. +- [Limit who in your organization]({{< relref "./configure_registry.md" >}}) can access artifacts in each registry. @@ -37,13 +37,13 @@ The preceding image shows the Registry App with "Model" and "Dataset" core regis ## Learn the basics -Each organization initially contains two registries that you can use to organize your model and dataset artifacts called **Models** and **Datasets**, respectively. You can create [additional registries to organize other artifact types based on your organization's needs](./registry_types.md). +Each organization initially contains two registries that you can use to organize your model and dataset artifacts called **Models** and **Datasets**, respectively. You can create [additional registries to organize other artifact types based on your organization's needs]({{< relref "./registry_types.md" >}}). -Each [registry](./configure_registry.md) consists of one or more [collections](./create_collection.md). Each collection represents a distinct task or use case. +Each [registry]({{< relref "./configure_registry.md" >}}) consists of one or more [collections]({{< relref "./create_collection.md" >}}). Each collection represents a distinct task or use case. {{< img src="/images/registry/homepage_registry.png" >}} -To add an artifact to a registry, you first log a [specific artifact version to W&B](../artifacts/create-a-new-artifact-version.md). Each time you log an artifact, W&B automatically assigns a version to that artifact. Artifact versions use 0 indexing, so the first version is `v0`, the second version is `v1`, and so on. +To add an artifact to a registry, you first log a [specific artifact version to W&B]({{< relref "/guides/core/artifacts/create-a-new-artifact-version.md" >}}). Each time you log an artifact, W&B automatically assigns a version to that artifact. Artifact versions use 0 indexing, so the first version is `v0`, the second version is `v1`, and so on. Once you log an artifact to W&B, you can then link that specific artifact version to a collection in the registry. @@ -51,7 +51,7 @@ Once you log an artifact to W&B, you can then link that specific artifact versio The term "link" refers to pointers that connect where W&B stores the artifact and where the artifact is accessible in the registry. W&B does not duplicate artifacts when you link an artifact to a collection. {{% /alert %}} -As an example, the proceeding code example shows how to log and link a fake model artifact called "my_model.txt" to a collection named "first-collection" in the [core Model registry](./registry_types.md). More specifically, the code accomplishes the following: +As an example, the proceeding code example shows how to log and link a fake model artifact called "my_model.txt" to a collection named "first-collection" in the [core Model registry]({{< relref "./registry_types.md" >}}). More specifically, the code accomplishes the following: 1. Initialize a W&B run. 2. Log the artifact to W&B. @@ -108,7 +108,7 @@ Based on your deployment type, satisfy the following conditions to enable W&B Re | ----- | ----- | | Multi-tenant Cloud | No action required. W&B Registry is available on the W&B App. | | Dedicated Cloud | Contact your account team. The Solutions Architect (SA) Team enables W&B Registry within your instance's operator console. Ensure your instance is on server release version 0.59.2 or newer.| -| Self-Managed | Enable the environment variable called `ENABLE_REGISTRY_UI`. To learn more about enabling environment variables in server, visit [these docs](/guides/hosting/env-vars). In self-managed instances, your infrastructure administrator should enable this environment variable and set it to `true`. Ensure your instance is on server release version 0.59.2 or newer.| +| Self-Managed | Enable the environment variable called `ENABLE_REGISTRY_UI`. To learn more about enabling environment variables in server, visit [these docs]({{< relref "/guides/hosting/env-vars/" >}}). In self-managed instances, your infrastructure administrator should enable this environment variable and set it to `true`. Ensure your instance is on server release version 0.59.2 or newer.| ## Resources to get started @@ -129,7 +129,7 @@ Depending on your use case, explore the following resources to get started with The legacy Model Registry is scheduled for deprecation with the exact date not yet decided. Before deprecating the legacy Model Registry, W&B will migrate the contents of the legacy Model Registry to the W&B Registry. -See [Migrating from legacy Model Registry](./model_registry_eol.md) for more information about the migration process from the legacy Model Registry to W&B Registry. +See [Migrating from legacy Model Registry]({{< relref "./model_registry_eol.md" >}}) for more information about the migration process from the legacy Model Registry to W&B Registry. Until the migration occurs, W&B supports both the legacy Model Registry and the new Registry. diff --git a/content/guides/models/registry/configure_registry.md b/content/guides/models/registry/configure_registry.md index 4bcbe74b2..c6585c491 100644 --- a/content/guides/models/registry/configure_registry.md +++ b/content/guides/models/registry/configure_registry.md @@ -9,10 +9,10 @@ weight: 3 -Registry admins can limit who can access a registry by navigating to a registry's settings and assigning a user's role to [Admin, Member, or Viewer](#registry-roles-permissions). Users can have different roles in different registries. For example, a user can have a view role in "Registry A" and a member role in the "Registry B". +Registry admins can limit who can access a registry by navigating to a registry's settings and assigning a user's role to [Admin, Member, or Viewer]({{< relref "#registry-roles-permissions" >}}). Users can have different roles in different registries. For example, a user can have a view role in "Registry A" and a member role in the "Registry B". {{% alert %}} -Only registry admins can [restrict visibility](#restrict-visibility-to-a-registry), [configure user roles](#configure-user-roles-in-a-registry), or [remove users](#remove-a-user-from-a-registry) from registries in an organization. +Only registry admins can [restrict visibility]({{< relref "#restrict-visibility-to-a-registry" >}}), [configure user roles]({{< relref "#configure-user-roles-in-a-registry" >}}), or [remove users]({{< relref "#remove-a-user-from-a-registry" >}}) from registries in an organization. {{% /alert %}} ## Registry roles permissions @@ -20,7 +20,7 @@ Only registry admins can [restrict visibility](#restrict-visibility-to-a-registr A user within an organization can have different roles, and therefore permissions, for each registry in their organization. {{% alert title="W&B role types" %}} -W&B has three different types of roles: Organization roles, [team roles](../app/features/teams.md#team-roles-and-permissions), and [registry roles](#registry-roles-permissions). +W&B has three different types of roles: Organization roles, [team roles]({{< relref "/guides/models/app/settings-page/teams.md#team-roles-and-permissions" >}}), and [registry roles]({{< relref "#registry-roles-permissions" >}}). Your role in a team has no impact or relationship on your role in any registry. {{% /alert %}} @@ -90,7 +90,7 @@ There are two registry visibility types: restricted or organization visibility. ## Restrict visibility to a registry -Restrict who can view and access a custom registry. You can restrict visibility to a registry when you create a custom registry or after you create a custom registry. A custom registry can have either restricted or organization visibility. For more information on registry visibilities, see [Registry visibility types](./configure_registry.md#registry-visibility-types). +Restrict who can view and access a custom registry. You can restrict visibility to a registry when you create a custom registry or after you create a custom registry. A custom registry can have either restricted or organization visibility. For more information on registry visibilities, see [Registry visibility types]({{< relref "./configure_registry.md#registry-visibility-types" >}}). {{% /alert %}} @@ -23,14 +23,14 @@ The term "type" refers to the artifact object's type. When you create an artifac Link an artifact version to a collection interactively or programmatically. {{% alert %}} -Before you link an artifact to a registry, check the types of artifacts that collection permits. For more information about collection types, see "Collection types" within [Create a collection](./create_collection.md). +Before you link an artifact to a registry, check the types of artifacts that collection permits. For more information about collection types, see "Collection types" within [Create a collection]({{< relref "./create_collection.md" >}}). {{% /alert %}} Based on your use case, follow the instructions described in the tabs below to link an artifact version. {{< tabpane text=true >}} {{% tab header="Python SDK" %}} -Programmatically link an artifact version to a collection with [`wandb.init.Run.link_artifact()`](../../ref/python/run.md#link_artifact). +Programmatically link an artifact version to a collection with [`wandb.init.Run.link_artifact()`]({{< relref "/ref/python/run.md#link_artifact" >}}). {{% alert %}} Before you link an artifact to a collection, ensure that the registry that the collection belongs to already exists. To check that the registry exists, navigate to the Registry app on the W&B App UI and search for the name of the registry. @@ -106,7 +106,7 @@ If you want to link an artifact version to the Model registry or the Dataset reg @@ -156,7 +156,7 @@ You can confirm the name of your team by: 2. Copy the site's URL. It has the form of `https://wandb.ai/`. Where `` is the both the name of your team and the team's entity. #### Log from a team entity -1. Specify the team as the entity when you initialize a run with [`wandb.init()`](/ref/python/init). If you do not specify the `entity` when you initialize a run, the run uses your default entity which may or may not be your team entity. +1. Specify the team as the entity when you initialize a run with [`wandb.init()`]({{< relref "/ref/python/init" >}}). If you do not specify the `entity` when you initialize a run, the run uses your default entity which may or may not be your team entity. ```python import wandb @@ -170,7 +170,7 @@ You can confirm the name of your team by: ```python artifact = wandb.Artifact(name="", type="") ``` - For more information on how to log artifacts, see [Construct artifacts](../artifacts/construct-an-artifact.md). + For more information on how to log artifacts, see [Construct artifacts]({{< relref "/guides/core/artifacts/construct-an-artifact.md" >}}). 3. If an artifact is logged to your personal entity, you will need to re-log it to an entity within your organization. ### Confirm the path of a registry in the W&B App UI diff --git a/content/guides/models/registry/model_registry/_index.md b/content/guides/models/registry/model_registry/_index.md index 495d6598c..d75d40187 100644 --- a/content/guides/models/registry/model_registry/_index.md +++ b/content/guides/models/registry/model_registry/_index.md @@ -12,10 +12,10 @@ cascade: --- {{% alert %}} -W&B will no longer support W&B Model Registry after 2024. Users are encouraged to instead use [W&B Registry](../registry/intro.md) for linking and sharing their model artifacts versions. W&B Registry broadens the capabilities of the legacy W&B Model Registry. For more information about W&B Registry, see the [Registry docs](../registry/intro.md). +W&B will no longer support W&B Model Registry after 2024. Users are encouraged to instead use [W&B Registry]({{< relref "/guides/models/registry/" >}}) for linking and sharing their model artifacts versions. W&B Registry broadens the capabilities of the legacy W&B Model Registry. For more information about W&B Registry, see the [Registry docs]({{< relref "/guides/models/registry/" >}}). -W&B will migrate existing model artifacts linked to the legacy Model Registry to the new W&B Registry in the Fall or early Winter of 2024. See [Migrating from legacy Model Registry](../registry/model_registry_eol.md) for information about the migration process. +W&B will migrate existing model artifacts linked to the legacy Model Registry to the new W&B Registry in the Fall or early Winter of 2024. See [Migrating from legacy Model Registry]({{< relref "/guides/models/registry/model_registry_eol.md" >}}) for information about the migration process. {{% /alert %}} The W&B Model Registry houses a team's trained models where ML Practitioners can publish candidates for production to be consumed by downstream teams and stakeholders. It is used to house staged/candidate models and manage workflows associated with staging. @@ -24,8 +24,8 @@ The W&B Model Registry houses a team's trained models where ML Practitioners can With W&B Model Registry, you can: -* [Bookmark your best model versions for each machine learning task.](./link-model-version.md) -* [Automate](./model-registry-automations.md) downstream processes and model CI/CD. +* [Bookmark your best model versions for each machine learning task.]({{< relref "./link-model-version.md" >}}) +* [Automate]({{< relref "/guides/models/automations/model-registry-automations.md" >}}) downstream processes and model CI/CD. * Move model versions through its ML lifecycle; from staging to production. * Track a model's lineage and audit the history of changes to production models. @@ -60,7 +60,7 @@ run.link_model(path="./my_model.h5", registered_model_name="MNIST") run.finish() ``` -4. **Connect model transitions to CI/DC workflows**: transition candidate models through workflow stages and [automate downstream actions](./model-registry-automations.md) with webhooks or jobs. +4. **Connect model transitions to CI/DC workflows**: transition candidate models through workflow stages and [automate downstream actions]({{< relref "/guides/models/automations/model-registry-automations.md" >}}) with webhooks or jobs. ## How to get started @@ -69,11 +69,11 @@ Depending on your use case, explore the following resources to get started with * Check out the two-part video series: 1. [Logging and registering models](https://www.youtube.com/watch?si=MV7nc6v-pYwDyS-3&v=ZYipBwBeSKE&feature=youtu.be) 2. [Consuming models and automating downstream processes](https://www.youtube.com/watch?v=8PFCrDSeHzw) in the Model Registry. -* Read the [models walkthrough](./walkthrough.md) for a step-by-step outline of the W&B Python SDK commands you could use to create, track, and use a dataset artifact. +* Read the [models walkthrough]({{< relref "./walkthrough.md" >}}) for a step-by-step outline of the W&B Python SDK commands you could use to create, track, and use a dataset artifact. * Learn about: - * [Protected models and access control](./access_controls.md). - * [How to connect the Model Registry to CI/CD processes](./model-registry-automations.md). - * Set up [Slack notifications](./notifications.md) when a new model version is linked to a registered model. + * [Protected models and access control]({{< relref "./access_controls.md" >}}). + * [How to connect the Model Registry to CI/CD processes]({{< relref "/guides/models/automations/model-registry-automations.md" >}}). + * Set up [Slack notifications]({{< relref "./notifications.md" >}}) when a new model version is linked to a registered model. * Review [this](https://wandb.ai/wandb_fc/model-registry-reports/reports/What-is-an-ML-Model-Registry---Vmlldzo1MTE5MjYx) report on how the Model Registry fits into your ML workflow and the benefits of using one for model management. * Take the W&B [Enterprise Model Management](https://www.wandb.courses/courses/enterprise-model-management) course and learn how to: * Use the W&B Model Registry to manage and version your models, track lineage, and promote models through different lifecycle stages diff --git a/content/guides/models/registry/model_registry/consume-models.md b/content/guides/models/registry/model_registry/consume-models.md index aaf72a6f7..a8a7901cd 100644 --- a/content/guides/models/registry/model_registry/consume-models.md +++ b/content/guides/models/registry/model_registry/consume-models.md @@ -13,7 +13,7 @@ Use the W&B Python SDK to download a model artifact that you linked to the Model {{% alert %}} You are responsible for providing additional Python functions, API calls to reconstruct, deserialize your model into a form that you can work with. -W&B suggests that you document information on how to load models into memory with model cards. For more information, see the [Document machine learning models](./create-model-cards.md) page. +W&B suggests that you document information on how to load models into memory with model cards. For more information, see the [Document machine learning models]({{< relref "./create-model-cards.md" >}}) page. {{% /alert %}} @@ -35,7 +35,7 @@ Reference a model version with one of following formats listed: * `v#` - Use `v0`, `v1`, `v2`, and so on to fetch a specific version in the Registered Model * `alias` - Specify the custom alias that you and your team assigned to your model version -See [`use_model`](../../ref/python/run.md#use_model) in the API Reference guide for more information on possible parameters and return type. +See [`use_model`]({{< relref "/ref/python/run.md#use_model" >}}) in the API Reference guide for more information on possible parameters and return type.
Example: Download and use a logged model @@ -62,7 +62,7 @@ downloaded_model_path = run.use_model(name=f"{entity/project/model_artifact_name {{% alert title="Planned deprecation for W&B Model Registry in 2024" %}} The proceeding tabs demonstrate how to consume model artifacts using the soon to be deprecated Model Registry. -Use the W&B Registry to track, organize and consume model artifacts. For more information see the [Registry docs](../registry/intro.md). +Use the W&B Registry to track, organize and consume model artifacts. For more information see the [Registry docs]({{< relref "/guides/models/registry/" >}}). {{% /alert %}} {{< tabpane text=true >}} @@ -81,7 +81,7 @@ Reference a model version with one of following formats listed: * `v#` - Use `v0`, `v1`, `v2`, and so on to fetch a specific version in the Registered Model * `alias` - Specify the custom alias that you and your team assigned to your model version -See [`use_model`](../../ref/python/run.md#use_model) in the API Reference guide for more information on possible parameters and return type. +See [`use_model`]({{< relref "/ref/python/run.md#use_model" >}}) in the API Reference guide for more information on possible parameters and return type. {{% /tab %}} {{% tab header="W&B App" %}} 1. Navigate to the Model Registry App at [https://wandb.ai/registry/model](https://wandb.ai/registry/model). diff --git a/content/guides/models/registry/model_registry/create-registered-model.md b/content/guides/models/registry/model_registry/create-registered-model.md index 45f6aee86..65b0dd3d5 100644 --- a/content/guides/models/registry/model_registry/create-registered-model.md +++ b/content/guides/models/registry/model_registry/create-registered-model.md @@ -10,7 +10,7 @@ weight: 4 --- -Create a [registered model](./model-management-concepts.md#registered-model) to hold all the candidate models for your modeling tasks. You can create a registered model interactively within the Model Registry or programmatically with the Python SDK. +Create a [registered model]({{< relref "./model-management-concepts.md#registered-model" >}}) to hold all the candidate models for your modeling tasks. You can create a registered model interactively within the Model Registry or programmatically with the Python SDK. ## Programmatically create registered a model Programmatically register a model with the W&B Python SDK. W&B automatically creates a registered model for you if the registered model doesn't exist. @@ -44,7 +44,7 @@ Interactively create a registered model within the [Model Registry App](https:// {{% alert %}} -Manual linking a model to the model registry is useful for one-off models. However, it is often useful to [programmatically link model versions to the model registry](link-model-version#programmatically-link-a-model). +Manual linking a model to the model registry is useful for one-off models. However, it is often useful to [programmatically link model versions to the model registry]({{< relref "link-model-version#programmatically-link-a-model" >}}). For example, suppose you have a nightly job. It is tedious to manually link a model created each night. Instead, you could create a script that evaluates the model, and if the model improves in performance, link that model to the model registry with the W&B Python SDK. {{% /alert %}} \ No newline at end of file diff --git a/content/guides/models/registry/model_registry/link-model-version.md b/content/guides/models/registry/model_registry/link-model-version.md index d09973257..bcc1efa25 100644 --- a/content/guides/models/registry/model_registry/link-model-version.md +++ b/content/guides/models/registry/model_registry/link-model-version.md @@ -14,7 +14,7 @@ Link a model version to a registered model with the W&B App or programmatically ## Programmatically link a model -Use the [`link_model`](../../ref/python/run.md#link_model) method to programmatically log model files to a W&B run and link it to the [W&B Model Registry](./intro.md). +Use the [`link_model`]({{< relref "/ref/python/run.md#link_model" >}}) method to programmatically log model files to a W&B run and link it to the [W&B Model Registry]({{< relref "./" >}}). Ensure to replace other the values enclosed in `<>` with your own: diff --git a/content/guides/models/registry/model_registry/log-model-to-experiment.md b/content/guides/models/registry/model_registry/log-model-to-experiment.md index a730cc956..39bc31c03 100644 --- a/content/guides/models/registry/model_registry/log-model-to-experiment.md +++ b/content/guides/models/registry/model_registry/log-model-to-experiment.md @@ -12,11 +12,11 @@ weight: 3 Track a model, the model's dependencies, and other information relevant to that model with the W&B Python SDK. -Under the hood, W&B creates a lineage of [model artifact](./model-management-concepts.md#model-artifact) that you can view with the W&B App UI or programmatically with the W&B Python SDK. See the [Create model lineage map](./model-lineage.md) for more information. +Under the hood, W&B creates a lineage of [model artifact]({{< relref "./model-management-concepts.md#model-artifact" >}}) that you can view with the W&B App UI or programmatically with the W&B Python SDK. See the [Create model lineage map]({{< relref "./model-lineage.md" >}}) for more information. ## How to log a model -Use the `run.log_model` API to log a model. Provide the path where your model files are saved to the `path` parameter. The path can be a local file, directory, or [reference URI](../artifacts/track-external-files.md#amazon-s3--gcs--azure-blob-storage-references) to an external bucket such as `s3://bucket/path`. +Use the `run.log_model` API to log a model. Provide the path where your model files are saved to the `path` parameter. The path can be a local file, directory, or [reference URI]({{< relref "/guides/core/artifacts/track-external-files.md#amazon-s3--gcs--azure-blob-storage-references" >}}) to an external bucket such as `s3://bucket/path`. Optionally provide a name for the model artifact for the `name` parameter. If `name` is not specified, W&B uses the basename of the input path prepended with the run ID. diff --git a/content/guides/models/registry/model_registry/model-lineage.md b/content/guides/models/registry/model_registry/model-lineage.md index e316d5f9b..2083339d3 100644 --- a/content/guides/models/registry/model_registry/model-lineage.md +++ b/content/guides/models/registry/model_registry/model-lineage.md @@ -10,7 +10,7 @@ weight: 7 A useful feature of logging model artifacts to W&B are lineage graphs. Lineage graphs show artifacts logged by a run as well as artifacts used by specific run. -This means that, when you log a model artifact, you at a minimum have access to view the W&B run that used or produced the model artifact. If you [track a dependency](#track-an-artifact-dependency), you also see the inputs used by the model artifact. +This means that, when you log a model artifact, you at a minimum have access to view the W&B run that used or produced the model artifact. If you [track a dependency]({{< relref "#track-an-artifact-dependency" >}}), you also see the inputs used by the model artifact. For example, the proceeding image shows artifacts created and used throughout an ML experiment: diff --git a/content/guides/models/registry/model_registry/model-management-concepts.md b/content/guides/models/registry/model_registry/model-management-concepts.md index 33714d60b..e9d75385c 100644 --- a/content/guides/models/registry/model_registry/model-management-concepts.md +++ b/content/guides/models/registry/model_registry/model-management-concepts.md @@ -8,14 +8,14 @@ title: Model Registry Terms and Concepts weight: 2 --- -The following terms describe key components of the W&B Model Registry: [*model version*](#model-version), [*model artifact*](#model-artifact), and [*registered model*](#registered-model). +The following terms describe key components of the W&B Model Registry: [*model version*]({{< relref "#model-version" >}}), [*model artifact*]({{< relref "#model-artifact" >}}), and [*registered model*]({{< relref "#registered-model" >}}). ## Model version A model version represents a single model checkpoint. Model versions are a snapshot at a point in time of a model and its files within an experiment. A model version is an immutable directory of data and metadata that describes a trained model. W&B suggests that you add files to your model version that let you store (and restore) your model architecture and learned parameters at a later date. -A model version belongs to one, and only one, [model artifact](#model-artifact). A model version can belong to zero or more, [registered models](#registered-model). Model versions are stored in a model artifact in the order they are logged to the model artifact. W&B automatically creates a new model version if it detects that a model you log (to the same model artifact) has different contents than a previous model version. +A model version belongs to one, and only one, [model artifact]({{< relref "#model-artifact" >}}). A model version can belong to zero or more, [registered models]({{< relref "#registered-model" >}}). Model versions are stored in a model artifact in the order they are logged to the model artifact. W&B automatically creates a new model version if it detects that a model you log (to the same model artifact) has different contents than a previous model version. Store files within model versions that are produced from the serialization process provided by your modeling library (for example, [PyTorch](https://pytorch.org/tutorials/beginner/saving_loading_models.html) and [Keras](https://www.tensorflow.org/guide/keras/save_and_serialize)). @@ -42,16 +42,16 @@ Model tags are keywords or labels that belong to one or more registered models. Use model tags to organize registered models into categories and to search over those categories in the Model Registry's search bar. Model tags appear at the top of the Registered Model Card. You might choose to use them to group your registered models by ML task, owning team, or priority. The same model tag can be added to multiple registered models to allow for grouping. {{% alert %}} -Model tags, which are labels applied to registered models for grouping and discoverability, are different from [model aliases](#model-alias). Model aliases are unique identifiers or nicknames that you use to fetch a model version programatically. To learn more about using tags to organize the tasks in your Model Registry, see [Organize models](./organize-models.md). +Model tags, which are labels applied to registered models for grouping and discoverability, are different from [model aliases]({{< relref "#model-alias" >}}). Model aliases are unique identifiers or nicknames that you use to fetch a model version programatically. To learn more about using tags to organize the tasks in your Model Registry, see [Organize models]({{< relref "./organize-models.md" >}}). {{% /alert %}} ## Model artifact -A model artifact is a collection of logged [model versions](#model-version). Model versions are stored in a model artifact in the order they are logged to the model artifact. +A model artifact is a collection of logged [model versions]({{< relref "#model-version" >}}). Model versions are stored in a model artifact in the order they are logged to the model artifact. A model artifact can contain one or more model versions. A model artifact can be empty if no model versions are logged to it. -For example, suppose you create a model artifact. During model training, you periodically save your model during checkpoints. Each checkpoint corresponds to its own [model version](#model-version). All of the model versions created during your model training and checkpoint saving are stored in the same model artifact you created at the beginning of your training script. +For example, suppose you create a model artifact. During model training, you periodically save your model during checkpoints. Each checkpoint corresponds to its own [model version]({{< relref "#model-version" >}}). All of the model versions created during your model training and checkpoint saving are stored in the same model artifact you created at the beginning of your training script. @@ -63,7 +63,7 @@ The proceeding image shows a model artifact that contains three model versions: View an [example model artifact here](https://wandb.ai/timssweeney/model_management_docs_official_v0/artifacts/model/mnist-zws7gt0n). ## Registered model -A registered model is a collection of pointers (links) to model versions. You can think of a registered model as a folder of "bookmarks" of candidate models for the same ML task. Each "bookmark" of a registered model is a pointer to a [model version](#model-version) that belongs to a [model artifact](#model-artifact). You can use [model tags](#model-tags) to group your registered models. +A registered model is a collection of pointers (links) to model versions. You can think of a registered model as a folder of "bookmarks" of candidate models for the same ML task. Each "bookmark" of a registered model is a pointer to a [model version]({{< relref "#model-version" >}}) that belongs to a [model artifact]({{< relref "#model-artifact" >}}). You can use [model tags]({{< relref "#model-tags" >}}) to group your registered models. Registered models often represent candidate models for a single modeling use case or task. For example, you might create registered model for different image classification task based on the model you use: `ImageClassifier-ResNet50`, `ImageClassifier-VGG16`, `DogBreedClassifier-MobileNetV2` and so on. Model versions are assigned version numbers in the order in which they were linked to the registered model. diff --git a/content/guides/models/registry/model_registry/walkthrough.md b/content/guides/models/registry/model_registry/walkthrough.md index fd54ea2e9..64ff926e6 100644 --- a/content/guides/models/registry/model_registry/walkthrough.md +++ b/content/guides/models/registry/model_registry/walkthrough.md @@ -68,7 +68,7 @@ def generate_raw_data(train_size=6000): (x_train, y_train), (x_eval, y_eval) = generate_raw_data() ``` -Next, upload the dataset to W&B. To do this, create an [artifact](../artifacts/intro.md) object and add the dataset to that artifact. +Next, upload the dataset to W&B. To do this, create an [artifact]({{< relref "/guides/core/artifacts/" >}}) object and add the dataset to that artifact. ```python project = "model-registry-dev" @@ -116,7 +116,7 @@ Train a model with the artifact dataset you created in the previous step. ### Declare dataset artifact as an input to the run -Declare the dataset artifact you created in a previous step as the input to the W&B run. This is particularly useful in the context of logging models because declaring an artifact as an input to a run lets you track the dataset (and the version of the dataset) used to train a specific model. W&B uses the information collected to create a [lineage map](./model-lineage.md). +Declare the dataset artifact you created in a previous step as the input to the W&B run. This is particularly useful in the context of logging models because declaring an artifact as an input to a run lets you track the dataset (and the version of the dataset) used to train a specific model. W&B uses the information collected to create a [lineage map]({{< relref "./model-lineage.md" >}}). Use the `use_artifact` API to both declare the dataset artifact as the input of the run and to retrieve the artifact itself. @@ -143,7 +143,7 @@ x_train = train_table.get_column("x_train", convert_to="numpy") y_train = train_table.get_column("y_train", convert_to="numpy") ``` -For more information about tracking the inputs and output of a model, see [Create model lineage](./model-lineage.md) map. +For more information about tracking the inputs and output of a model, see [Create model lineage]({{< relref "./model-lineage.md" >}}) map. ### Define and train model @@ -210,7 +210,7 @@ model.save(path) ## Log and link a model to the Model Registry -Use the [`link_model`](../../ref/python/run.md#link_model) API to log model one ore more files to a W&B run and link it to the [W&B Model Registry](./intro.md). +Use the [`link_model`]({{< relref "/ref/python/run.md#link_model" >}}) API to log model one ore more files to a W&B run and link it to the [W&B Model Registry]({{< relref "./" >}}). ```python path = "./model.h5" @@ -222,7 +222,7 @@ run.finish() W&B creates a registered model for you if the name you specify for `registered-model-name` does not already exist. -See [`link_model`](../../ref/python/run.md#link_model) in the API Reference guide for more information on optional parameters. +See [`link_model`]({{< relref "/ref/python/run.md#link_model" >}}) in the API Reference guide for more information on optional parameters. ## Evaluate the performance of a model It is common practice to evaluate the performance of a one or more models. @@ -248,7 +248,7 @@ x_eval = eval_table.get_column("x_eval", convert_to="numpy") y_eval = eval_table.get_column("y_eval", convert_to="numpy") ``` -Download the [model version](./model-management-concepts.md#model-version) from W&B that you want to evaluate. Use the `use_model` API to access and download your model. +Download the [model version]({{< relref "./model-management-concepts.md#model-version" >}}) from W&B that you want to evaluate. Use the `use_model` API to access and download your model. ```python alias = "latest" # alias @@ -277,7 +277,7 @@ run.log(data={"loss": (loss, _)}) ## Promote a model version -Mark a model version ready for the next stage of your machine learning workflow with a [*model alias*](./model-management-concepts.md#model-alias). Each registered model can have one or more model aliases. A model alias can only belong to a single model version at a time. +Mark a model version ready for the next stage of your machine learning workflow with a [*model alias*]({{< relref "./model-management-concepts.md#model-alias" >}}). Each registered model can have one or more model aliases. A model alias can only belong to a single model version at a time. For example, suppose that after evaluating a model's performance, you are confident that the model is ready for production. To promote that model version, add the `production` alias to that specific model version. diff --git a/content/guides/models/registry/model_registry_eol.md b/content/guides/models/registry/model_registry_eol.md index 612bf0871..7c5b24c2c 100644 --- a/content/guides/models/registry/model_registry_eol.md +++ b/content/guides/models/registry/model_registry_eol.md @@ -7,7 +7,7 @@ title: Migrate from legacy Model Registry weight: 8 --- -W&B will transition assets from the legacy [W&B Model Registry](../model_registry/intro.md) to the new [W&B Registry](./intro.md). This migration will be fully managed and triggered by W&B, requiring no intervention from users. The process is designed to be as seamless as possible, with minimal disruption to existing workflows. +W&B will transition assets from the legacy [W&B Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}) to the new [W&B Registry]({{< relref "./" >}}). This migration will be fully managed and triggered by W&B, requiring no intervention from users. The process is designed to be as seamless as possible, with minimal disruption to existing workflows. The transition will take place once the new W&B Registry includes all the functionalities currently available in the Model Registry. W&B will attempt to preserve current workflows, codebases, and references. @@ -28,10 +28,10 @@ Artifacts linked to the legacy Model Registry have team level visibility. This m ### Restrict visibility to a registry -Restrict who can view and access a custom registry. You can restrict visibility to a registry when you create a custom registry or after you create a custom registry. In a Restricted registry, only selected members can access the content, maintaining privacy and control. For more information about registry visibility, see [Registry visibility types](./configure_registry.md#registry-visibility-types). +Restrict who can view and access a custom registry. You can restrict visibility to a registry when you create a custom registry or after you create a custom registry. In a Restricted registry, only selected members can access the content, maintaining privacy and control. For more information about registry visibility, see [Registry visibility types]({{< relref "./configure_registry.md#registry-visibility-types" >}}). ### Create custom registries -Unlike the legacy Model Registry, W&B Registry is not limited to models or dataset registries. You can create custom registries tailored to specific workflows or project needs, capable of holding any arbitrary object type. This flexibility allows teams to organize and manage artifacts according to their unique requirements. For more information on how to create a custom registry, see [Create a custom registry](./create_registry.md). +Unlike the legacy Model Registry, W&B Registry is not limited to models or dataset registries. You can create custom registries tailored to specific workflows or project needs, capable of holding any arbitrary object type. This flexibility allows teams to organize and manage artifacts according to their unique requirements. For more information on how to create a custom registry, see [Create a custom registry]({{< relref "./create_registry.md" >}}). {{< img src="/images/registry/mode_reg_eol.png" alt="" >}} @@ -61,7 +61,7 @@ W&B will migrate registered models (now called collections) and associated artif ### Team visibility to organization visibility -After the migration, your model registry will have organization level visibility. You can restrict who has access to a registry by [assigning roles](./configure_registry.md). This helps ensure that only specific members have access to specific registries. +After the migration, your model registry will have organization level visibility. You can restrict who has access to a registry by [assigning roles]({{< relref "./configure_registry.md" >}}). This helps ensure that only specific members have access to specific registries. The migration will preserve existing permission boundaries of your current team-level registered models (soon to be called collections) in the legacy W&B Model Registry. Permissions currently defined in the legacy Model Registry will be preserved in the new Registry. This means that collections currently restricted to specific team members will remain protected during and after the migration. @@ -83,7 +83,7 @@ Users are encouraged to explore the new features and capabilities available in t Support is available if you are interested in trying the W&B Registry early, or for new users that prefer to start with Registry and not the legacy W&B Model Registry. Contact support@wandb.com or your Sales MLE to enable this functionality. Note that any early migration will be into a BETA version. The BETA version of W&B Registry might not have all the functionality or features of the legacy Model Registry. -For more details and to learn about the full range of features in the W&B Registry, visit the [W&B Registry Guide](./intro.md). +For more details and to learn about the full range of features in the W&B Registry, visit the [W&B Registry Guide]({{< relref "./" >}}). ## FAQs diff --git a/content/guides/models/registry/organize-with-tags.md b/content/guides/models/registry/organize-with-tags.md index 8b1de0b04..5546b7165 100644 --- a/content/guides/models/registry/organize-with-tags.md +++ b/content/guides/models/registry/organize-with-tags.md @@ -66,7 +66,7 @@ collection.save() Update a tag programmatically by reassigning or by mutating the `tags` attribute. W&B recommends, and it is good Python practice, that you reassign the `tags` attribute instead of in-place mutation. -For example, the proceeding code snippet shows common ways to update a list with reassignment. For brevity, we continue the code example from the [Add a tag to a collection section](#add-a-tag-to-a-collection): +For example, the proceeding code snippet shows common ways to update a list with reassignment. For brevity, we continue the code example from the [Add a tag to a collection section]({{< relref "#add-a-tag-to-a-collection" >}}): ```python collection.tags = [*collection.tags, "new-tag", "other-tag"] @@ -190,7 +190,7 @@ artifact.save() Update a tag programmatically by reassigning or by mutating the `tags` attribute. W&B recommends, and it is good Python practice, that you reassign the `tags` attribute instead of in-place mutation. -For example, the proceeding code snippet shows common ways to update a list with reassignment. For brevity, we continue the code example from the [Add a tag to an artifact version section](#add-a-tag-to-an-artifact-version): +For example, the proceeding code snippet shows common ways to update a list with reassignment. For brevity, we continue the code example from the [Add a tag to an artifact version section]({{< relref "#add-a-tag-to-an-artifact-version" >}}): ```python artifact.tags = [*artifact.tags, "new-tag", "other-tag"] diff --git a/content/guides/models/registry/registry_types.md b/content/guides/models/registry/registry_types.md index 785b11ab2..e6e376ec5 100644 --- a/content/guides/models/registry/registry_types.md +++ b/content/guides/models/registry/registry_types.md @@ -7,7 +7,7 @@ title: Registry types weight: 1 --- -W&B supports two types of registries: [Core registries](#core-registry) and [Custom registries](#custom-registry). +W&B supports two types of registries: [Core registries]({{< relref "#core-registry" >}}) and [Custom registries]({{< relref "#custom-registry" >}}). ## Core registry A core registry is a template for specific use cases: **Models** and **Datasets**. @@ -20,7 +20,7 @@ By default, the **Models** registry is configured to accept `"model"` artifact t The preceding image shows the **Models** and the **Dataset** core registry along with a custom registry called **Fine_Tuned_Models** in the W&B Registry App UI. -A core registry has [organization visibility](./configure_registry.md#registry-visibility-types). A registry admin can not change the visibility of a core registry. +A core registry has [organization visibility]({{< relref "./configure_registry.md#registry-visibility-types" >}}). A registry admin can not change the visibility of a core registry. ## Custom registry Custom registries are not restricted to `"model"` artifact types or `"dataset"` artifact types. @@ -31,9 +31,9 @@ For example, you might create a registry called "Benchmark_Datasets" for organiz {{< img src="/images/registry/custom_registry_example.png" alt="" >}} -A custom registry can have either [organization or restricted visibility](./configure_registry.md#registry-visibility-types). A registry admin can change the visibility of a custom registry from organization to restricted. However, the registry admin can not change a custom registry's visibility from restricted to organizational visibility. +A custom registry can have either [organization or restricted visibility]({{< relref "./configure_registry.md#registry-visibility-types" >}}). A registry admin can change the visibility of a custom registry from organization to restricted. However, the registry admin can not change a custom registry's visibility from restricted to organizational visibility. -For information on how to create a custom registry, see [Create a custom registry](./create_collection.md). +For information on how to create a custom registry, see [Create a custom registry]({{< relref "./create_collection.md" >}}). ## Summary diff --git a/content/guides/models/sweeps/_index.md b/content/guides/models/sweeps/_index.md index 043f1b5fa..00a586697 100644 --- a/content/guides/models/sweeps/_index.md +++ b/content/guides/models/sweeps/_index.md @@ -18,7 +18,7 @@ Use W&B Sweeps to automate hyperparameter search and visualize rich, interactive {{< img src="/images/sweeps/intro_what_it_is.png" alt="Draw insights from large hyperparameter tuning experiments with interactive dashboards." >}} ### How it works -Create a sweep with two [W&B CLI](../../ref/cli/README.md) commands: +Create a sweep with two [W&B CLI]({{< relref "/ref/cli/" >}}) commands: 1. Initialize a sweep @@ -34,7 +34,7 @@ wandb agent ``` {{% alert %}} -The preceding code snippet, and the colab linked on this page, show how to initialize and create a sweep with wht W&B CLI. See the Sweeps [Walkthrough](./walkthrough.md) for a step-by-step outline of the W&B Python SDK commands to use to define a sweep configuration, initialize a sweep, and start a sweep. +The preceding code snippet, and the colab linked on this page, show how to initialize and create a sweep with wht W&B CLI. See the Sweeps [Walkthrough]({{< relref "./walkthrough.md" >}}) for a step-by-step outline of the W&B Python SDK commands to use to define a sweep configuration, initialize a sweep, and start a sweep. {{% /alert %}} @@ -43,14 +43,14 @@ The preceding code snippet, and the colab linked on this page, show how to initi Depending on your use case, explore the following resources to get started with W&B Sweeps: -* Read through the [sweeps walkthrough](./walkthrough.md) for a step-by-step outline of the W&B Python SDK commands to use to define a sweep configuration, initialize a sweep, and start a sweep. +* Read through the [sweeps walkthrough]({{< relref "./walkthrough.md" >}}) for a step-by-step outline of the W&B Python SDK commands to use to define a sweep configuration, initialize a sweep, and start a sweep. * Explore this chapter to learn how to: - * [Add W&B to your code](./add-w-and-b-to-your-code.md) - * [Define sweep configuration](./define-sweep-configuration.md) - * [Initialize sweeps](./initialize-sweeps.md) - * [Start sweep agents](./start-sweep-agents.md) - * [Visualize sweep results](./visualize-sweep-results.md) -* Explore a [curated list of Sweep experiments](./useful-resources.md) that explore hyperparameter optimization with W&B Sweeps. Results are stored in W&B Reports. + * [Add W&B to your code]({{< relref "./add-w-and-b-to-your-code.md" >}}) + * [Define sweep configuration]({{< relref "./define-sweep-configuration.md" >}}) + * [Initialize sweeps]({{< relref "./initialize-sweeps.md" >}}) + * [Start sweep agents]({{< relref "./start-sweep-agents.md" >}}) + * [Visualize sweep results]({{< relref "./visualize-sweep-results.md" >}}) +* Explore a [curated list of Sweep experiments]({{< relref "./useful-resources.md" >}}) that explore hyperparameter optimization with W&B Sweeps. Results are stored in W&B Reports. For a step-by-step video, see: [Tune Hyperparameters Easily with W&B Sweeps](https://www.youtube.com/watch?v=9zrmUIlScdY\&ab_channel=Weights%26Biases). diff --git a/content/guides/models/sweeps/add-w-and-b-to-your-code.md b/content/guides/models/sweeps/add-w-and-b-to-your-code.md index aca36518f..6abc96287 100644 --- a/content/guides/models/sweeps/add-w-and-b-to-your-code.md +++ b/content/guides/models/sweeps/add-w-and-b-to-your-code.md @@ -67,12 +67,12 @@ The following code examples demonstrate how to add the W&B Python SDK into your To create a W&B Sweep, we added the following to the code example: 1. Line 1: Import the Weights & Biases Python SDK. -2. Line 6: Create a dictionary object where the key-value pairs define the sweep configuration. In the proceeding example, the batch size (`batch_size`), epochs (`epochs`), and the learning rate (`lr`) hyperparameters are varied during each sweep. For more information on how to create a sweep configuration, see [Define sweep configuration](./define-sweep-configuration.md). -3. Line 19: Pass the sweep configuration dictionary to [`wandb.sweep`](../../ref/python/sweep.md). This initializes the sweep. This returns a sweep ID (`sweep_id`). For more information on how to initialize sweeps, see [Initialize sweeps](./initialize-sweeps.md). -4. Line 33: Use the [`wandb.init()`](../../ref/python/init.md) API to generate a background process to sync and log data as a [W&B Run](../../ref/python/run.md). +2. Line 6: Create a dictionary object where the key-value pairs define the sweep configuration. In the proceeding example, the batch size (`batch_size`), epochs (`epochs`), and the learning rate (`lr`) hyperparameters are varied during each sweep. For more information on how to create a sweep configuration, see [Define sweep configuration]({{< relref "./define-sweep-configuration.md" >}}). +3. Line 19: Pass the sweep configuration dictionary to [`wandb.sweep`]({{< relref "/ref/python/sweep.md" >}}). This initializes the sweep. This returns a sweep ID (`sweep_id`). For more information on how to initialize sweeps, see [Initialize sweeps]({{< relref "./initialize-sweeps.md" >}}). +4. Line 33: Use the [`wandb.init()`]({{< relref "/ref/python/init.md" >}}) API to generate a background process to sync and log data as a [W&B Run]({{< relref "/ref/python/run.md" >}}). 5. Line 37-39: (Optional) define values from `wandb.config` instead of defining hard coded values. -6. Line 45: Log the metric we want to optimize with [`wandb.log`](../../ref/python/log.md). You must log the metric defined in your configuration. Within the configuration dictionary (`sweep_configuration` in this example) we defined the sweep to maximize the `val_acc` value). -7. Line 54: Start the sweep with the [`wandb.agent`](../../ref/python/agent.md) API call. Provide the sweep ID (line 19), the name of the function the sweep will execute (`function=main`), and set the maximum number of runs to try to four (`count=4`). For more information on how to start W&B Sweep, see [Start sweep agents](./start-sweep-agents.md). +6. Line 45: Log the metric we want to optimize with [`wandb.log`]({{< relref "/ref/python/log.md" >}}). You must log the metric defined in your configuration. Within the configuration dictionary (`sweep_configuration` in this example) we defined the sweep to maximize the `val_acc` value). +7. Line 54: Start the sweep with the [`wandb.agent`]({{< relref "/ref/python/agent.md" >}}) API call. Provide the sweep ID (line 19), the name of the function the sweep will execute (`function=main`), and set the maximum number of runs to try to four (`count=4`). For more information on how to start W&B Sweep, see [Start sweep agents]({{< relref "./start-sweep-agents.md" >}}). ```python showLineNumbers @@ -163,7 +163,7 @@ parameters: values: [5, 10, 15] ``` -For more information on how to create a W&B Sweep configuration, see [Define sweep configuration](./define-sweep-configuration.md). +For more information on how to create a W&B Sweep configuration, see [Define sweep configuration]({{< relref "./define-sweep-configuration.md" >}}). Note that you must provide the name of your Python script for the `program` key in your YAML file. @@ -171,9 +171,9 @@ Next, we add the following to the code example: 1. Line 1-2: Import the Wieghts & Biases Python SDK (`wandb`) and PyYAML (`yaml`). PyYAML is used to read in our YAML configuration file. 2. Line 18: Read in the configuration file. -3. Line 21: Use the [`wandb.init()`](../../ref/python/init.md) API to generate a background process to sync and log data as a [W&B Run](../../ref/python/run.md). We pass the config object to the config parameter. +3. Line 21: Use the [`wandb.init()`]({{< relref "/ref/python/init.md" >}}) API to generate a background process to sync and log data as a [W&B Run]({{< relref "/ref/python/run.md" >}}). We pass the config object to the config parameter. 4. Line 25 - 27: Define hyperparameter values from `wandb.config` instead of using hard coded values. -5. Line 33-39: Log the metric we want to optimize with [`wandb.log`](../../ref/python/log.md). You must log the metric defined in your configuration. Within the configuration dictionary (`sweep_configuration` in this example) we defined the sweep to maximize the `val_acc` value. +5. Line 33-39: Log the metric we want to optimize with [`wandb.log`]({{< relref "/ref/python/log.md" >}}). You must log the metric defined in your configuration. Within the configuration dictionary (`sweep_configuration` in this example) we defined the sweep to maximize the `val_acc` value. ```python showLineNumbers @@ -233,21 +233,21 @@ Navigate to your CLI. Within your CLI, set a maximum number of runs the sweep ag NUM=5 ``` -Next, initialize the sweep with the [`wandb sweep`](../../ref/cli/wandb-sweep.md) command. Provide the name of the YAML file. Optionally provide the name of the project for the project flag (`--project`): +Next, initialize the sweep with the [`wandb sweep`]({{< relref "/ref/cli/wandb-sweep.md" >}}) command. Provide the name of the YAML file. Optionally provide the name of the project for the project flag (`--project`): ```bash wandb sweep --project sweep-demo-cli config.yaml ``` -This returns a sweep ID. For more information on how to initialize sweeps, see [Initialize sweeps](./initialize-sweeps.md). +This returns a sweep ID. For more information on how to initialize sweeps, see [Initialize sweeps]({{< relref "./initialize-sweeps.md" >}}). -Copy the sweep ID and replace `sweepID` in the proceeding code snippet to start the sweep job with the [`wandb agent`](../../ref/cli/wandb-agent.md) command: +Copy the sweep ID and replace `sweepID` in the proceeding code snippet to start the sweep job with the [`wandb agent`]({{< relref "/ref/cli/wandb-agent.md" >}}) command: ```bash wandb agent --count $NUM your-entity/sweep-demo-cli/sweepID ``` -For more information on how to start sweep jobs, see [Start sweep jobs](./start-sweep-agents.md). +For more information on how to start sweep jobs, see [Start sweep jobs]({{< relref "./start-sweep-agents.md" >}}). {{% /tab %}} {{< /tabpane >}} diff --git a/content/guides/models/sweeps/define-sweep-configuration/_index.md b/content/guides/models/sweeps/define-sweep-configuration/_index.md index 6b59e69cd..520b6fd28 100644 --- a/content/guides/models/sweeps/define-sweep-configuration/_index.md +++ b/content/guides/models/sweeps/define-sweep-configuration/_index.md @@ -18,13 +18,13 @@ Define a sweep configuration either in a [Python dictionary](https://docs.python Define your sweep configuration in a YAML file if you want to initialize a sweep and start a sweep agent from the command line. Define your sweep in a Python dictionary if you initialize a sweep and start a sweep entirely within a Python script or Jupyter notebook. {{% /alert %}} -The following guide describes how to format your sweep configuration. See [Sweep configuration options](./sweep-config-keys.md) for a comprehensive list of top-level sweep configuration keys. +The following guide describes how to format your sweep configuration. See [Sweep configuration options]({{< relref "./sweep-config-keys.md" >}}) for a comprehensive list of top-level sweep configuration keys. ## Basic structure Both sweep configuration format options (YAML and Python dictionary) utilize key-value pairs and nested structures. -Use top-level keys within your sweep configuration to define qualities of your sweep search such as the name of the sweep ([`name`](./sweep-config-keys.md) key), the parameters to search through ([`parameters`](./sweep-config-keys.md#parameters) key), the methodology to search the parameter space ([`method`](./sweep-config-keys.md#method) key), and more. +Use top-level keys within your sweep configuration to define qualities of your sweep search such as the name of the sweep ([`name`]({{< relref "./sweep-config-keys.md" >}}) key), the parameters to search through ([`parameters`]({{< relref "./sweep-config-keys.md#parameters" >}}) key), the methodology to search the parameter space ([`method`]({{< relref "./sweep-config-keys.md#method" >}}) key), and more. For example, the proceeding code snippets show the same sweep configuration defined within a YAML file and within a Python dictionary. Within the sweep configuration there are five top level keys specified: `program`, `name`, `method`, `metric` and `parameters`. @@ -75,7 +75,7 @@ parameters: {{< /tabpane >}} -Within the top level `parameters` key, the following keys are nested: `learning_rate`, `batch_size`, `epoch`, and `optimizer`. For each of the nested keys you specify, you can provide one or more values, a distribution, a probability, and more. For more information, see the [parameters](./sweep-config-keys.md#parameters) section in [Sweep configuration options](./sweep-config-keys.md). +Within the top level `parameters` key, the following keys are nested: `learning_rate`, `batch_size`, `epoch`, and `optimizer`. For each of the nested keys you specify, you can provide one or more values, a distribution, a probability, and more. For more information, see the [parameters]({{< relref "./sweep-config-keys.md#parameters" >}}) section in [Sweep configuration options]({{< relref "./sweep-config-keys.md" >}}). ## Double nested parameters @@ -375,7 +375,7 @@ command: {{% /tab %}} {{% tab header="Hydra" %}} -You can change the command to pass arguments the way tools like [Hydra](https://hydra.cc) expect. See [Hydra with W&B](../integrations/other/hydra.md) for more information. +You can change the command to pass arguments the way tools like [Hydra](https://hydra.cc) expect. See [Hydra with W&B]({{< relref "/guides/integrations/hydra.md" >}}) for more information. ```yaml command: diff --git a/content/guides/models/sweeps/define-sweep-configuration/sweep-config-keys.md b/content/guides/models/sweeps/define-sweep-configuration/sweep-config-keys.md index b79a6029e..b590628d2 100644 --- a/content/guides/models/sweeps/define-sweep-configuration/sweep-config-keys.md +++ b/content/guides/models/sweeps/define-sweep-configuration/sweep-config-keys.md @@ -7,7 +7,7 @@ title: Sweep configuration options --- -A sweep configuration consists of nested key-value pairs. Use top-level keys within your sweep configuration to define qualities of your sweep search such as the parameters to search through ([`parameter`](./sweep-config-keys.md#parameters) key), the methodology to search the parameter space ([`method`](./sweep-config-keys.md#method) key), and more. +A sweep configuration consists of nested key-value pairs. Use top-level keys within your sweep configuration to define qualities of your sweep search such as the parameters to search through ([`parameter`]({{< relref "./sweep-config-keys.md#parameters" >}}) key), the methodology to search the parameter space ([`method`]({{< relref "./sweep-config-keys.md#method" >}}) key), and more. The proceeding table lists top-level sweep configuration keys and a brief description. See the respective sections for more information about each key. @@ -19,14 +19,14 @@ The proceeding table lists top-level sweep configuration keys and a brief descri | `project` | The project for this sweep | | `description` | Text description of the sweep | | `name` | The name of the sweep, displayed in the W&B UI. | -| [`method`](#method) | (required) The search strategy | -| [`metric`](#metric) | The metric to optimize (only used by certain search strategies and stopping criteria) | -| [`parameters`](#parameters) | (required) Parameter bounds to search | -| [`early_terminate`](#early_terminate) | Any early stopping criteria | -| [`command`](#command) | Command structure for invoking and passing arguments to the training script | +| [`method`]({{< relref "#method" >}}) | (required) The search strategy | +| [`metric`]({{< relref "#metric" >}}) | The metric to optimize (only used by certain search strategies and stopping criteria) | +| [`parameters`]({{< relref "#parameters" >}}) | (required) Parameter bounds to search | +| [`early_terminate`]({{< relref "#early_terminate" >}}) | Any early stopping criteria | +| [`command`]({{< relref "#command" >}}) | Command structure for invoking and passing arguments to the training script | | `run_cap` | Maximum number of runs for this sweep | -See the [Sweep configuration](./sweep-config-keys.md) structure for more information on how to structure your sweep configuration. +See the [Sweep configuration]({{< relref "./sweep-config-keys.md" >}}) structure for more information on how to structure your sweep configuration. -Bayesian search runs forever unless you stop the process from the command line, within your python script, or [the W&B App UI](./sweeps-ui.md). +Bayesian search runs forever unless you stop the process from the command line, within your python script, or [the W&B App UI]({{< relref "../sweeps-ui.md" >}}). ### Distribution options for random and Bayesian search Within the `parameter` key, nest the name of the hyperparameter. Next, specify the `distribution` key and specify a distribution for the value. @@ -159,12 +159,12 @@ Specify either `min_iter` or `max_iter` to create a bracket schedule. {{% alert %}} -Hyperband checks which [W&B runs](../../ref/python/run.md) to end once every few minutes. The end run timestamp might differ from the specified brackets if your run or iteration are short. +Hyperband checks which [W&B runs]({{< relref "/ref/python/run.md" >}}) to end once every few minutes. The end run timestamp might differ from the specified brackets if your run or iteration are short. {{% /alert %}} ## `command` - + Modify the format and contents with nested values within the `command` key. You can directly include fixed components such as filenames. diff --git a/content/guides/models/sweeps/existing-project.md b/content/guides/models/sweeps/existing-project.md index 7d70f6aa1..439212f5f 100644 --- a/content/guides/models/sweeps/existing-project.md +++ b/content/guides/models/sweeps/existing-project.md @@ -25,7 +25,7 @@ Optionally explore the example appear in the W&B App UI dashboard. ## 2. Create a sweep -From your project page, open the [Sweep tab](./sweeps-ui.md) in the sidebar and select **Create Sweep**. +From your project page, open the [Sweep tab]({{< relref "./sweeps-ui.md" >}}) in the sidebar and select **Create Sweep**. {{< img src="/images/sweeps/sweep1.png" alt="" >}} diff --git a/content/guides/models/sweeps/initialize-sweeps.md b/content/guides/models/sweeps/initialize-sweeps.md index 97c9dcd35..fe599e13a 100644 --- a/content/guides/models/sweeps/initialize-sweeps.md +++ b/content/guides/models/sweeps/initialize-sweeps.md @@ -13,8 +13,8 @@ W&B uses a _Sweep Controller_ to manage sweeps on the cloud (standard), locally The following code snippets demonstrate how to initialize sweeps with the CLI and within a Jupyter Notebook or Python script. {{% alert color="secondary" %}} -1. Before you initialize a sweep, make sure you have a sweep configuration defined either in a YAML file or a nested Python dictionary object in your script. For more information see, [Define sweep configuration](../../guides/sweeps/define-sweep-configuration.md). -2. Both the W&B Sweep and the W&B Run must be in the same project. Therefore, the name you provide when you initialize W&B ([`wandb.init`](../../ref/python/init.md)) must match the name of the project you provide when you initialize a W&B Sweep ([`wandb.sweep`](../../ref/python/sweep.md)). +1. Before you initialize a sweep, make sure you have a sweep configuration defined either in a YAML file or a nested Python dictionary object in your script. For more information see, [Define sweep configuration]({{< relref "/guides/models/sweeps/define-sweep-configuration.md" >}}). +2. Both the W&B Sweep and the W&B Run must be in the same project. Therefore, the name you provide when you initialize W&B ([`wandb.init`]({{< relref "/ref/python/init.md" >}})) must match the name of the project you provide when you initialize a W&B Sweep ([`wandb.sweep`]({{< relref "/ref/python/sweep.md" >}})). {{% /alert %}} @@ -41,14 +41,14 @@ sweep_configuration = { sweep_id = wandb.sweep(sweep=sweep_configuration, project="project-name") ``` -The [`wandb.sweep`](../../ref/python/sweep) function returns the sweep ID. The sweep ID includes the entity name and the project name. Make a note of the sweep ID. +The [`wandb.sweep`]({{< relref "/ref/python/sweep" >}}) function returns the sweep ID. The sweep ID includes the entity name and the project name. Make a note of the sweep ID. {{% /tab %}} {{% tab header="CLI" %}} Use the W&B CLI to initialize a sweep. Provide the name of your configuration file. Optionally provide the name of the project for the `project` flag. If the project is not specified, the W&B Run is put in an "Uncategorized" project. -Use the [`wandb sweep`](../../ref/cli/wandb-sweep.md) command to initialize a sweep. The proceeding code example initializes a sweep for a `sweeps_demo` project and uses a `config.yaml` file for the configuration. +Use the [`wandb sweep`]({{< relref "/ref/cli/wandb-sweep.md" >}}) command to initialize a sweep. The proceeding code example initializes a sweep for a `sweeps_demo` project and uses a `config.yaml` file for the configuration. ```bash wandb sweep --project sweeps_demo config.yaml diff --git a/content/guides/models/sweeps/local-controller.md b/content/guides/models/sweeps/local-controller.md index bc7641fc1..2f673ddcf 100644 --- a/content/guides/models/sweeps/local-controller.md +++ b/content/guides/models/sweeps/local-controller.md @@ -22,7 +22,7 @@ Before you get start, you must install the W&B SDK(`wandb`). Type the following pip install wandb sweeps ``` -The following examples assume you already have a configuration file and a training loop defined in a python script or Jupyter Notebook. For more information about how to define a configuration file, see [Define sweep configuration](./define-sweep-configuration.md). +The following examples assume you already have a configuration file and a training loop defined in a python script or Jupyter Notebook. For more information about how to define a configuration file, see [Define sweep configuration]({{< relref "./define-sweep-configuration.md" >}}). ### Run the local controller from the command line @@ -47,14 +47,14 @@ Next, initialize the sweep: wandb sweep config.yaml ``` -After you initialized the sweep, start a controller with [`wandb controller`](../../ref/python/controller.md): +After you initialized the sweep, start a controller with [`wandb controller`]({{< relref "/ref/python/controller.md" >}}): ```bash # wandb sweep command will print a sweep_id wandb controller {user}/{entity}/{sweep_id} ``` -Once you have specified you want to use a local controller, start one or more Sweep agents to execute the sweep. Start a W&B Sweep similar to how you normally would. See [Start sweep agents](../../guides/sweeps/start-sweep-agents.md), for more information. +Once you have specified you want to use a local controller, start one or more Sweep agents to execute the sweep. Start a W&B Sweep similar to how you normally would. See [Start sweep agents]({{< relref "/guides/models/sweeps/start-sweep-agents.md" >}}), for more information. ```bash wandb sweep sweep_ID @@ -64,7 +64,7 @@ wandb sweep sweep_ID The following code snippets demonstrate how to specify and use a local controller with the W&B Python SDK. -The simplest way to use a controller with the Python SDK is to pass the sweep ID to the [`wandb.controller`](../../ref/python/controller.md) method. Next, use the return objects `run` method to start the sweep job: +The simplest way to use a controller with the Python SDK is to pass the sweep ID to the [`wandb.controller`]({{< relref "/ref/python/controller.md" >}}) method. Next, use the return objects `run` method to start the sweep job: ```python sweep = wandb.controller(sweep_id) diff --git a/content/guides/models/sweeps/parallelize-agents.md b/content/guides/models/sweeps/parallelize-agents.md index 48c516eb9..4e10f2282 100644 --- a/content/guides/models/sweeps/parallelize-agents.md +++ b/content/guides/models/sweeps/parallelize-agents.md @@ -9,7 +9,7 @@ weight: 6 --- -Parallelize your W&B Sweep agents on a multi-core or multi-GPU machine. Before you get started, ensure you have initialized your W&B Sweep. For more information on how to initialize a W&B Sweep, see [Initialize sweeps](./initialize-sweeps.md). +Parallelize your W&B Sweep agents on a multi-core or multi-GPU machine. Before you get started, ensure you have initialized your W&B Sweep. For more information on how to initialize a W&B Sweep, see [Initialize sweeps]({{< relref "./initialize-sweeps.md" >}}). ### Parallelize on a multi-CPU machine @@ -18,7 +18,7 @@ Depending on your use case, explore the proceeding tabs to learn how to parallel {{< tabpane text=true >}} {{% tab header="CLI" %}} -Use the [`wandb agent`](../../ref/cli/wandb-agent.md) command to parallelize your W&B Sweep agent across multiple CPUs with the terminal. Provide the sweep ID that was returned when you [initialized the sweep](./initialize-sweeps.md). +Use the [`wandb agent`]({{< relref "/ref/cli/wandb-agent.md" >}}) command to parallelize your W&B Sweep agent across multiple CPUs with the terminal. Provide the sweep ID that was returned when you [initialized the sweep]({{< relref "./initialize-sweeps.md" >}}). 1. Open more than one terminal window on your local machine. 2. Copy and paste the code snippet below and replace `sweep_id` with your sweep ID: @@ -28,7 +28,7 @@ wandb agent sweep_id ``` {{% /tab %}} {{% tab header="Jupyter Notebook" %}} -Use the W&B Python SDK library to parallelize your W&B Sweep agent across multiple CPUs within Jupyter Notebooks. Ensure you have the sweep ID that was returned when you [initialized the sweep](./initialize-sweeps.md). In addition, provide the name of the function the sweep will execute for the `function` parameter: +Use the W&B Python SDK library to parallelize your W&B Sweep agent across multiple CPUs within Jupyter Notebooks. Ensure you have the sweep ID that was returned when you [initialized the sweep]({{< relref "./initialize-sweeps.md" >}}). In addition, provide the name of the function the sweep will execute for the `function` parameter: 1. Open more than one Jupyter Notebook. 2. Copy and past the W&B Sweep ID on multiple Jupyter Notebooks to parallelize a W&B Sweep. For example, you can paste the following code snippet on multiple jupyter notebooks to paralleliz your sweep if you have the sweep ID stored in a variable called `sweep_id` and the name of the function is `function_name`: @@ -46,7 +46,7 @@ wandb.agent(sweep_id=sweep_id, function=function_name) Follow the procedure outlined to parallelize your W&B Sweep agent across multiple GPUs with a terminal using CUDA Toolkit: 1. Open more than one terminal window on your local machine. -2. Specify the GPU instance to use with `CUDA_VISIBLE_DEVICES` when you start a W&B Sweep job ([`wandb agent`](../../ref/cli/wandb-agent.md)). Assign `CUDA_VISIBLE_DEVICES` an integer value corresponding to the GPU instance to use. +2. Specify the GPU instance to use with `CUDA_VISIBLE_DEVICES` when you start a W&B Sweep job ([`wandb agent`]({{< relref "/ref/cli/wandb-agent.md" >}})). Assign `CUDA_VISIBLE_DEVICES` an integer value corresponding to the GPU instance to use. For example, suppose you have two NVIDIA GPUs on your local machine. Open a terminal window and set `CUDA_VISIBLE_DEVICES` to `0` (`CUDA_VISIBLE_DEVICES=0`). Replace `sweep_ID` in the proceeding example with the W&B Sweep ID that is returned when you initialized a W&B Sweep: diff --git a/content/guides/models/sweeps/pause-resume-and-cancel-sweeps.md b/content/guides/models/sweeps/pause-resume-and-cancel-sweeps.md index a66ecaa82..9df617916 100644 --- a/content/guides/models/sweeps/pause-resume-and-cancel-sweeps.md +++ b/content/guides/models/sweeps/pause-resume-and-cancel-sweeps.md @@ -46,7 +46,7 @@ Cancel a sweep to kill all running runs and stop running new runs. Use the `wand wandb sweep --cancel entity/project/sweep_ID ``` -For a full list of CLI command options, see the [wandb sweep](../../ref/cli/wandb-sweep.md) CLI Reference Guide. +For a full list of CLI command options, see the [wandb sweep]({{< relref "/ref/cli/wandb-sweep.md" >}}) CLI Reference Guide. ### Pause, resume, stop, and cancel a sweep across multiple agents @@ -64,4 +64,4 @@ Specify the `--resume` flag along with the Sweep ID to resume the Sweep across y wandb sweep --resume entity/project/sweep_ID ``` -For more information on how to parallelize W&B agents, see [Parallelize agents](./parallelize-agents.md). \ No newline at end of file +For more information on how to parallelize W&B agents, see [Parallelize agents]({{< relref "./parallelize-agents.md" >}}). \ No newline at end of file diff --git a/content/guides/models/sweeps/start-sweep-agents.md b/content/guides/models/sweeps/start-sweep-agents.md index ac80446c7..131a9c923 100644 --- a/content/guides/models/sweeps/start-sweep-agents.md +++ b/content/guides/models/sweeps/start-sweep-agents.md @@ -25,7 +25,7 @@ Where: Provide the name of the function the W&B Sweep will execute if you start a W&B Sweep agent within a Jupyter Notebook or Python script. -The proceeding code snippets demonstrate how to start an agent with W&B. We assume you already have a configuration file and you have already initialized a W&B Sweep. For more information about how to define a configuration file, see [Define sweep configuration](./define-sweep-configuration.md). +The proceeding code snippets demonstrate how to start an agent with W&B. We assume you already have a configuration file and you have already initialized a W&B Sweep. For more information about how to define a configuration file, see [Define sweep configuration]({{< relref "./define-sweep-configuration.md" >}}). {{< tabpane text=true >}} {{% tab header="CLI" %}} @@ -49,14 +49,14 @@ wandb.agent(sweep_id=sweep_id, function=function_name) ### Stop W&B agent {{% alert color="secondary" %}} -Random and Bayesian searches will run forever. You must stop the process from the command line, within your python script, or the [Sweeps UI](./visualize-sweep-results.md). +Random and Bayesian searches will run forever. You must stop the process from the command line, within your python script, or the [Sweeps UI]({{< relref "./visualize-sweep-results.md" >}}). {{% /alert %}} -Optionally specify the number of W&B Runs a Sweep agent should try. The following code snippets demonstrate how to set a maximum number of [W&B Runs](../../ref/python/run.md) with the CLI and within a Jupyter Notebook, Python script. +Optionally specify the number of W&B Runs a Sweep agent should try. The following code snippets demonstrate how to set a maximum number of [W&B Runs]({{< relref "/ref/python/run.md" >}}) with the CLI and within a Jupyter Notebook, Python script. {{< tabpane text=true >}} {{% tab header="Python script or notebook" %}} -First, initialize your sweep. For more information, see [Initialize sweeps](./initialize-sweeps.md). +First, initialize your sweep. For more information, see [Initialize sweeps]({{< relref "./initialize-sweeps.md" >}}). ``` sweep_id = wandb.sweep(sweep_config) @@ -74,7 +74,7 @@ If you start a new run after the sweep agent has finished, within the same scrip {{% /alert %}} {{% /tab %}} {{% tab header="CLI" %}} -First, initialize your sweep with the [`wandb sweep`](../../ref/cli/wandb-sweep.md) command. For more information, see [Initialize sweeps](./initialize-sweeps.md). +First, initialize your sweep with the [`wandb sweep`]({{< relref "/ref/cli/wandb-sweep.md" >}}) command. For more information, see [Initialize sweeps]({{< relref "./initialize-sweeps.md" >}}). ``` wandb sweep config.yaml diff --git a/content/guides/models/sweeps/troubleshoot-sweeps.md b/content/guides/models/sweeps/troubleshoot-sweeps.md index 765a700e9..900adb254 100644 --- a/content/guides/models/sweeps/troubleshoot-sweeps.md +++ b/content/guides/models/sweeps/troubleshoot-sweeps.md @@ -60,7 +60,7 @@ Navigate to your CLI and initialize a W&B Sweep with wandb sweep: wandb sweep config.yaml ``` -Make a note of the W&B Sweep ID that is returned. Next, start the Sweep job with [`wandb agent`](../../ref/cli/wandb-agent.md) with the CLI instead of the Python SDK ([`wandb.agent`](../../ref/python/agent.md)). Replace `sweep_ID` in the code snippet below with the Sweep ID that was returned in the previous step: +Make a note of the W&B Sweep ID that is returned. Next, start the Sweep job with [`wandb agent`]({{< relref "/ref/cli/wandb-agent.md" >}}) with the CLI instead of the Python SDK ([`wandb.agent`]({{< relref "/ref/python/agent.md" >}})). Replace `sweep_ID` in the code snippet below with the Sweep ID that was returned in the previous step: ```shell wandb agent sweep_ID @@ -75,4 +75,4 @@ wandb: ERROR Error while calling W&B API: anaconda 400 error: {"code": 400, "message": "TypeError: bad operand type for unary -: 'NoneType'"} ``` -Within your YAML file or nested dictionary you specify a key named "metric" to optimize. Ensure that you log (`wandb.log`) this metric. In addition, ensure you use the _exact_ metric name that you defined the sweep to optimize within your Python script or Jupyter Notebook. For more information about configuration files, see [Define sweep configuration](./define-sweep-configuration.md). \ No newline at end of file +Within your YAML file or nested dictionary you specify a key named "metric" to optimize. Ensure that you log (`wandb.log`) this metric. In addition, ensure you use the _exact_ metric name that you defined the sweep to optimize within your Python script or Jupyter Notebook. For more information about configuration files, see [Define sweep configuration]({{< relref "./define-sweep-configuration.md" >}}). \ No newline at end of file diff --git a/content/guides/models/sweeps/visualize-sweep-results.md b/content/guides/models/sweeps/visualize-sweep-results.md index 92764a284..dbf94b3de 100644 --- a/content/guides/models/sweeps/visualize-sweep-results.md +++ b/content/guides/models/sweeps/visualize-sweep-results.md @@ -8,23 +8,23 @@ title: Visualize sweep results weight: 7 --- -Visualize the results of your W&B Sweeps with the W&B App UI. Navigate to the W&B App UI at [https://wandb.ai/home](https://wandb.ai/home). Choose the project that you specified when you initialized a W&B Sweep. You will be redirected to your project [workspace](../track/workspaces.md). Select the **Sweep icon** on the left panel (broom icon). From the [Sweep UI](./visualize-sweep-results.md), select the name of your Sweep from the list. +Visualize the results of your W&B Sweeps with the W&B App UI. Navigate to the W&B App UI at [https://wandb.ai/home](https://wandb.ai/home). Choose the project that you specified when you initialized a W&B Sweep. You will be redirected to your project [workspace]({{< relref "/guides/models/track/workspaces.md" >}}). Select the **Sweep icon** on the left panel (broom icon). From the [Sweep UI]({{< relref "./visualize-sweep-results.md" >}}), select the name of your Sweep from the list. By default, W&B will automatically create a parallel coordinates plot, a parameter importance plot, and a scatter plot when you start a W&B Sweep job. {{< img src="/images/sweeps/navigation_sweeps_ui.gif" alt="Animation that shows how to navigate to the Sweep UI interface and view autogenerated plots." >}} -Parallel coordinates charts summarize the relationship between large numbers of hyperparameters and model metrics at a glance. For more information on parallel coordinates plots, see [Parallel coordinates](../app/features/panels/parallel-coordinates.md). +Parallel coordinates charts summarize the relationship between large numbers of hyperparameters and model metrics at a glance. For more information on parallel coordinates plots, see [Parallel coordinates]({{< relref "/guides/models/app/features/panels/parallel-coordinates.md" >}}). {{< img src="/images/sweeps/example_parallel_coordiantes_plot.png" alt="Example parallel coordinates plot." >}} -The scatter plot(left) compares the W&B Runs that were generated during the Sweep. For more information about scatter plots, see [Scatter Plots](../app/features/panels/scatter-plot.md). +The scatter plot(left) compares the W&B Runs that were generated during the Sweep. For more information about scatter plots, see [Scatter Plots]({{< relref "/guides/models/app/features/panels/scatter-plot.md" >}}). -The parameter importance plot(right) lists the hyperparameters that were the best predictors of, and highly correlated to desirable values of your metrics. For more information parameter importance plots, see [Parameter Importance](../app/features/panels/parameter-importance.md). +The parameter importance plot(right) lists the hyperparameters that were the best predictors of, and highly correlated to desirable values of your metrics. For more information parameter importance plots, see [Parameter Importance]({{< relref "/guides/models/app/features/panels/parameter-importance.md" >}}). {{< img src="/images/sweeps/scatter_and_parameter_importance.png" alt="Example scatter plot (left) and parameter importance plot (right)." >}} You can alter the dependent and independent values (x and y axis) that are automatically used. Within each panel there is a pencil icon called **Edit panel**. Choose **Edit panel**. A model will appear. Within the modal, you can alter the behavior of the graph. -For more information on all default W&B visualization options, see [Panels](../app/features/panels/intro.md). See the [Data Visualization docs](../tables/intro.md) for information on how to create plots from W&B Runs that are not part of a W&B Sweep. \ No newline at end of file +For more information on all default W&B visualization options, see [Panels]({{< relref "/guides/models/app/features/panels/" >}}). See the [Data Visualization docs]({{< relref "/guides/core/tables/" >}}) for information on how to create plots from W&B Runs that are not part of a W&B Sweep. \ No newline at end of file diff --git a/content/guides/models/sweeps/walkthrough.md b/content/guides/models/sweeps/walkthrough.md index de2f6eaa1..4176d9eec 100644 --- a/content/guides/models/sweeps/walkthrough.md +++ b/content/guides/models/sweeps/walkthrough.md @@ -11,10 +11,10 @@ weight: 1 This page shows how to define, initialize, and run a sweep. There are four main steps: -1. [Set up your training code](#set-up-your-training-code) -2. [Define the search space with a sweep configuration](#define-the-search-space-with-a-sweep-configuration) -3. [Initialize the sweep](#initialize-the-sweep) -4. [Start the sweep agent](#start-the-sweep) +1. [Set up your training code]({{< relref "#set-up-your-training-code" >}}) +2. [Define the search space with a sweep configuration]({{< relref "#define-the-search-space-with-a-sweep-configuration" >}}) +3. [Initialize the sweep]({{< relref "#initialize-the-sweep" >}}) +4. [Start the sweep agent]({{< relref "#start-the-sweep" >}}) Copy and paste the following code into a Jupyter Notebook or Python script: @@ -57,7 +57,7 @@ The following sections break down and explains each step in the code sample. ## Set up your training code Define a training function that takes in hyperparameter values from `wandb.config` and uses them to train a model and return metrics. -Optionally provide the name of the project where you want the output of the W&B Run to be stored (project parameter in [`wandb.init`](../../ref/python/init.md)). If the project is not specified, the run is put in an "Uncategorized" project. +Optionally provide the name of the project where you want the output of the W&B Run to be stored (project parameter in [`wandb.init`]({{< relref "/ref/python/init.md" >}})). If the project is not specified, the run is put in an "Uncategorized" project. {{% alert %}} Both the sweep and the run must be in the same project. Therefore, the name you provide when you initialize W&B must match the name of the project you provide when you initialize a sweep. @@ -77,7 +77,7 @@ def main(): ``` ## Define the search space with a sweep configuration -Within a dictionary, specify what hyperparameters you want to sweep over and. For more information about configuration options, see [Define sweep configuration](./define-sweep-configuration.md). +Within a dictionary, specify what hyperparameters you want to sweep over and. For more information about configuration options, see [Define sweep configuration]({{< relref "./define-sweep-configuration.md" >}}). The proceeding example demonstrates a sweep configuration that uses a random search (`'method':'random'`). The sweep will randomly select a random set of values listed in the configuration for the batch size, epoch, and the learning rate. @@ -98,7 +98,7 @@ sweep_configuration = { ## Initialize the Sweep -W&B uses a _Sweep Controller_ to manage sweeps on the cloud (standard), locally (local) across one or more machines. For more information about Sweep Controllers, see [Search and stop algorithms locally](./local-controller.md). +W&B uses a _Sweep Controller_ to manage sweeps on the cloud (standard), locally (local) across one or more machines. For more information about Sweep Controllers, see [Search and stop algorithms locally]({{< relref "./local-controller.md" >}}). A sweep identification number is returned when you initialize a sweep: @@ -106,11 +106,11 @@ A sweep identification number is returned when you initialize a sweep: sweep_id = wandb.sweep(sweep=sweep_configuration, project="my-first-sweep") ``` -For more information about initializing sweeps, see [Initialize sweeps](./initialize-sweeps.md). +For more information about initializing sweeps, see [Initialize sweeps]({{< relref "./initialize-sweeps.md" >}}). ## Start the Sweep -Use the [`wandb.agent`](../../ref/python/agent.md) API call to start a sweep. +Use the [`wandb.agent`]({{< relref "/ref/python/agent.md" >}}) API call to start a sweep. ```python wandb.agent(sweep_id, function=main, count=10) @@ -118,11 +118,11 @@ wandb.agent(sweep_id, function=main, count=10) ## Visualize results (optional) -Open your project to see your live results in the W&B App dashboard. With just a few clicks, construct rich, interactive charts like [parallel coordinates plots](../app/features/panels/parallel-coordinates.md),[ parameter importance analyzes](../app/features/panels/parameter-importance.md), and [more](../app/features/panels/intro.md). +Open your project to see your live results in the W&B App dashboard. With just a few clicks, construct rich, interactive charts like [parallel coordinates plots]({{< relref "/guides/models/app/features/panels/parallel-coordinates.md" >}}),[ parameter importance analyzes]({{< relref "/guides/models/app/features/panels/parameter-importance.md" >}}), and [more]({{< relref "/guides/models/app/features/panels/" >}}). {{< img src="/images/sweeps/quickstart_dashboard_example.png" alt="Sweeps Dashboard example" >}} -For more information about how to visualize results, see [Visualize sweep results](./visualize-sweep-results.md). For an example dashboard, see this sample [Sweeps Project](https://wandb.ai/anmolmann/pytorch-cnn-fashion/sweeps/pmqye6u3). +For more information about how to visualize results, see [Visualize sweep results]({{< relref "./visualize-sweep-results.md" >}}). For an example dashboard, see this sample [Sweeps Project](https://wandb.ai/anmolmann/pytorch-cnn-fashion/sweeps/pmqye6u3). ## Stop the agent (optional) diff --git a/content/guides/models/track/_index.md b/content/guides/models/track/_index.md index 71f623f1e..55fbb4294 100644 --- a/content/guides/models/track/_index.md +++ b/content/guides/models/track/_index.md @@ -12,20 +12,20 @@ cascade: --- {{< cta-button productLink="https://wandb.ai/stacey/deep-drive/workspace?workspace=user-lavanyashukla" colabLink="https://colab.research.google.com/github/wandb/examples/blob/master/colabs/intro/Intro_to_Weights_%26_Biases.ipynb" >}} -Track machine learning experiments with a few lines of code. You can then review the results in an [interactive dashboard](../track/workspaces.md) or export your data to Python for programmatic access using our [Public API](../../ref/python/public-api/README.md). +Track machine learning experiments with a few lines of code. You can then review the results in an [interactive dashboard]({{< relref "/guides/models/track/workspaces.md" >}}) or export your data to Python for programmatic access using our [Public API]({{< relref "/ref/python/public-api/" >}}). -Utilize W&B Integrations if you use popular frameworks such as [PyTorch](../integrations/pytorch.md), [Keras](../integrations/keras.md), or [Scikit](../integrations/scikit.md). See our [Integration guides](../integrations/intro.md) for a for a full list of integrations and information on how to add W&B to your code. +Utilize W&B Integrations if you use popular frameworks such as [PyTorch]({{< relref "/guides/integrations/pytorch.md" >}}), [Keras]({{< relref "/guides/integrations/keras.md" >}}), or [Scikit]({{< relref "/guides/integrations/scikit.md" >}}). See our [Integration guides]({{< relref "/guides/integrations/" >}}) for a for a full list of integrations and information on how to add W&B to your code. {{< img src="/images/experiments/experiments_landing_page.png" alt="" >}} -The image above shows an example dashboard where you can view and compare metrics across multiple [runs](../runs/intro.md). +The image above shows an example dashboard where you can view and compare metrics across multiple [runs]({{< relref "/guides/models/track/runs/" >}}). ## How it works Track a machine learning experiment with a few lines of code: -1. Create a [W&B run](../runs/intro.md). -2. Store a dictionary of hyperparameters, such as learning rate or model type, into your configuration ([`wandb.config`](./config.md)). -3. Log metrics ([`wandb.log()`](./log/intro.md)) over time in a training loop, such as accuracy and loss. +1. Create a [W&B run]({{< relref "/guides/models/track/runs/" >}}). +2. Store a dictionary of hyperparameters, such as learning rate or model type, into your configuration ([`wandb.config`]({{< relref "./config.md" >}})). +3. Log metrics ([`wandb.log()`]({{< relref "./log/" >}})) over time in a training loop, such as accuracy and loss. 4. Save outputs of a run, like the model weights or a table of predictions. The proceeding pseudocode demonstrates a common W&B Experiment tracking workflow: @@ -53,10 +53,10 @@ wandb.log_artifact(model) Depending on your use case, explore the following resources to get started with W&B Experiments: -* Read the [W&B Quickstart](../../quickstart.md) for a step-by-step outline of the W&B Python SDK commands you could use to create, track, and use a dataset artifact. +* Read the [W&B Quickstart]({{< relref "/guides/quickstart.md" >}}) for a step-by-step outline of the W&B Python SDK commands you could use to create, track, and use a dataset artifact. * Explore this chapter to learn how to: * Create an experiment * Configure experiments * Log data from experiments * View results from experiments -* Explore the [W&B Python Library](../../ref/python/README.md) within the [W&B API Reference Guide](../../ref/README.md). \ No newline at end of file +* Explore the [W&B Python Library]({{< relref "/ref/python/" >}}) within the [W&B API Reference Guide]({{< relref "/ref/" >}}). \ No newline at end of file diff --git a/content/guides/models/track/config.md b/content/guides/models/track/config.md index 1ccdf0014..f1430fcb4 100644 --- a/content/guides/models/track/config.md +++ b/content/guides/models/track/config.md @@ -179,9 +179,9 @@ wandb.config.update({"lr": 0.1, "channels": 16}) ``` ### Set the configuration after your Run has finished -Use the [W&B Public API](../../ref/python/public-api/README.md) to update your config (or anything else about from a complete Run) after your Run. This is particularly useful if you forgot to log a value during a Run. +Use the [W&B Public API]({{< relref "/ref/python/public-api/" >}}) to update your config (or anything else about from a complete Run) after your Run. This is particularly useful if you forgot to log a value during a Run. -Provide your `entity`, `project name`, and the `Run ID` to update your configuration after a Run has finished. Find these values directly from the Run object itself `wandb.run` or from the [W&B App UI](../track/workspaces.md): +Provide your `entity`, `project name`, and the `Run ID` to update your configuration after a Run has finished. Find these values directly from the Run object itself `wandb.run` or from the [W&B App UI]({{< relref "/guides/models/track/workspaces.md" >}}): ```python api = wandb.Api() diff --git a/content/guides/models/track/environment-variables.md b/content/guides/models/track/environment-variables.md index 09d8a9fcc..329fe12c2 100644 --- a/content/guides/models/track/environment-variables.md +++ b/content/guides/models/track/environment-variables.md @@ -37,10 +37,10 @@ Use these optional environment variables to do things like set up authentication | --------------------------- | ---------- | | **WANDB_ANONYMOUS** | Set this to `allow`, `never`, or `must` to let users create anonymous runs with secret urls. | | **WANDB_API_KEY** | Sets the authentication key associated with your account. You can find your key on [your settings page](https://app.wandb.ai/settings). This must be set if `wandb login` hasn't been run on the remote machine. | -| **WANDB_BASE_URL** | If you're using [wandb/local](../hosting/intro.md) you should set this environment variable to `http://YOUR_IP:YOUR_PORT` | +| **WANDB_BASE_URL** | If you're using [wandb/local]({{< relref "/guides/hosting/" >}}) you should set this environment variable to `http://YOUR_IP:YOUR_PORT` | | **WANDB_CACHE_DIR** | This defaults to \~/.cache/wandb, you can override this location with this environment variable | | **WANDB_CONFIG_DIR** | This defaults to \~/.config/wandb, you can override this location with this environment variable | -| **WANDB_CONFIG_PATHS** | Comma separated list of yaml files to load into wandb.config. See [config](./config.md#file-based-configs). | +| **WANDB_CONFIG_PATHS** | Comma separated list of yaml files to load into wandb.config. See [config]({{< relref "./config.md#file-based-configs" >}}). | | **WANDB_CONSOLE** | Set this to "off" to disable stdout / stderr logging. This defaults to "on" in environments that support it. | | **WANDB_DIR** | Set this to an absolute path to store all generated files here instead of the _wandb_ directory relative to your training script. _be sure this directory exists and the user your process runs as can write to it_ | | **WANDB_DISABLE_GIT** | Prevent wandb from probing for a git repository and capturing the latest commit / diff. | @@ -51,14 +51,14 @@ Use these optional environment variables to do things like set up authentication | **WANDB_HOST** | Set this to the hostname you want to see in the wandb interface if you don't want to use the system provided hostname | | **WANDB_IGNORE_GLOBS** | Set this to a comma separated list of file globs to ignore. These files will not be synced to the cloud. | | **WANDB_JOB_NAME** | Specify a name for any jobs created by `wandb`. | -| **WANDB_JOB_TYPE** | Specify the job type, like "training" or "evaluation" to indicate different types of runs. See [grouping](../runs/grouping.md) for more info. | +| **WANDB_JOB_TYPE** | Specify the job type, like "training" or "evaluation" to indicate different types of runs. See [grouping]({{< relref "/guides/models/track/runs/grouping.md" >}}) for more info. | | **WANDB_MODE** | If you set this to "offline" wandb will save your run metadata locally and not sync to the server. If you set this to `disabled` wandb will turn off completely. | | **WANDB_NAME** | The human-readable name of your run. If not set it will be randomly generated for you | | **WANDB_NOTEBOOK_NAME** | If you're running in jupyter you can set the name of the notebook with this variable. We attempt to auto detect this. | | **WANDB_NOTES** | Longer notes about your run. Markdown is allowed and you can edit this later in the UI. | | **WANDB_PROJECT** | The project associated with your run. This can also be set with `wandb init`, but the environmental variable will override the value. | | **WANDB_RESUME** | By default this is set to _never_. If set to _auto_ wandb will automatically resume failed runs. If set to _must_ forces the run to exist on startup. If you want to always generate your own unique ids, set this to _allow_ and always set **WANDB_RUN_ID**. | -| **WANDB_RUN_GROUP** | Specify the experiment name to automatically group runs together. See [grouping](../runs/grouping.md) for more info. | +| **WANDB_RUN_GROUP** | Specify the experiment name to automatically group runs together. See [grouping]({{< relref "/guides/models/track/runs/grouping.md" >}}) for more info. | | **WANDB_RUN_ID** | Set this to a globally unique string (per project) corresponding to a single run of your script. It must be no longer than 64 characters. All non-word characters will be converted to dashes. This can be used to resume an existing run in cases of failure. | | **WANDB_SILENT** | Set this to **true** to silence wandb log statements. If this is set all logs will be written to **WANDB_DIR**/debug.log | | **WANDB_SHOW_RUN** | Set this to **true** to automatically open a browser with the run url if your operating system supports it. | diff --git a/content/guides/models/track/jupyter.md b/content/guides/models/track/jupyter.md index 6b90df9df..2cfa3ba1d 100644 --- a/content/guides/models/track/jupyter.md +++ b/content/guides/models/track/jupyter.md @@ -14,7 +14,7 @@ Use W&B with Jupyter to get interactive visualizations without leaving your note ## Use cases for W&B with Jupyter notebooks 1. **Iterative experimentation**: Run and re-run experiments, tweaking parameters, and have all the runs you do saved automatically to W&B without having to take manual notes along the way. -2. **Code saving**: When reproducing a model, it's hard to know which cells in a notebook ran, and in which order. Turn on code saving on your [settings page](../app/settings-page/intro.md) to save a record of cell execution for each experiment. +2. **Code saving**: When reproducing a model, it's hard to know which cells in a notebook ran, and in which order. Turn on code saving on your [settings page]({{< relref "/guides/models/app/settings-page/" >}}) to save a record of cell execution for each experiment. 3. **Custom analysis**: Once runs are logged to W&B, it's easy to get a dataframe from the API and do custom analysis, then log those results to W&B to save and share in reports. ## Getting started in a notebook @@ -80,7 +80,7 @@ wandb.run ``` {{% alert %}} -Want to know more about what you can do with W&B? Check out our [guide to logging data and media](log/intro.md), learn [how to integrate us with your favorite ML toolkits](../integrations/intro.md), or just dive straight into the [reference docs](../../ref/python/README.md) or our [repo of examples](https://github.com/wandb/examples). +Want to know more about what you can do with W&B? Check out our [guide to logging data and media]({{< relref "log/" >}}), learn [how to integrate us with your favorite ML toolkits]({{< relref "/guides/integrations/" >}}), or just dive straight into the [reference docs]({{< relref "/ref/python/" >}}) or our [repo of examples](https://github.com/wandb/examples). {{% /alert %}} ## Additional Jupyter features in W&B diff --git a/content/guides/models/track/launch.md b/content/guides/models/track/launch.md index 03c49c847..d168d86bc 100644 --- a/content/guides/models/track/launch.md +++ b/content/guides/models/track/launch.md @@ -8,7 +8,7 @@ weight: 1 title: Create an experiment --- -Use the W&B Python SDK to track machine learning experiments. You can then review the results in an interactive dashboard or export your data to Python for programmatic access with the [W&B Public API](../../ref/python/public-api/README.md). +Use the W&B Python SDK to track machine learning experiments. You can then review the results in an interactive dashboard or export your data to Python for programmatic access with the [W&B Public API]({{< relref "/ref/python/public-api/" >}}). This guide describes how to use W&B building blocks to create a W&B Experiment. @@ -16,13 +16,13 @@ This guide describes how to use W&B building blocks to create a W&B Experiment. Create a W&B Experiment in four steps: -1. [Initialize a W&B Run](#initialize-a-wb-run) -2. [Capture a dictionary of hyperparameters](#capture-a-dictionary-of-hyperparameters) -3. [Log metrics inside your training loop](#log-metrics-inside-your-training-loop) -4. [Log an artifact to W&B](#log-an-artifact-to-wb) +1. [Initialize a W&B Run]({{< relref "#initialize-a-wb-run" >}}) +2. [Capture a dictionary of hyperparameters]({{< relref "#capture-a-dictionary-of-hyperparameters" >}}) +3. [Log metrics inside your training loop]({{< relref "#log-metrics-inside-your-training-loop" >}}) +4. [Log an artifact to W&B]({{< relref "#log-an-artifact-to-wb" >}}) ### Initialize a W&B run -At the beginning of your script call, the [`wandb.init()`](../../ref/python/init.md) API to generate a background process to sync and log data as a W&B Run. +At the beginning of your script call, the [`wandb.init()`]({{< relref "/ref/python/init.md" >}}) API to generate a background process to sync and log data as a W&B Run. The proceeding code snippet demonstrates how to create a new W&B project named `“cat-classification”`. A note `“My first experiment”` was added to help identify this run. Tags `“baseline”` and `“paper1”` are included to remind us that this run is a baseline experiment intended for a future paper publication. @@ -37,7 +37,7 @@ run = wandb.init( tags=["baseline", "paper1"], ) ``` -A [Run](../../ref/python/run.md) object is returned when you initialize W&B with `wandb.init()`. Additionally, W&B creates a local directory where all logs and files are saved and streamed asynchronously to a W&B server. +A [Run]({{< relref "/ref/python/run.md" >}}) object is returned when you initialize W&B with `wandb.init()`. Additionally, W&B creates a local directory where all logs and files are saved and streamed asynchronously to a W&B server. {{% alert %}} Note: Runs are added to pre-existing projects if that project already exists when you call wandb.init(). For example, if you already have a project called `“cat-classification”`, that project will continue to exist and not be deleted. Instead, a new run is added to that project. @@ -50,10 +50,10 @@ Save a dictionary of hyperparameters such as learning rate or model type. The mo #  2. Capture a dictionary of hyperparameters wandb.config = {"epochs": 100, "learning_rate": 0.001, "batch_size": 128} ``` -For more information on how to configure an experiment, see [Configure Experiments](./config.md). +For more information on how to configure an experiment, see [Configure Experiments]({{< relref "./config.md" >}}). ### Log metrics inside your training loop -Log metrics during each `for` loop (epoch), the accuracy and loss values are computed and logged to W&B with [`wandb.log()`](../../ref/python/log.md). By default, when you call wandb.log it appends a new step to the history object and updates the summary object. +Log metrics during each `for` loop (epoch), the accuracy and loss values are computed and logged to W&B with [`wandb.log()`]({{< relref "/ref/python/log.md" >}}). By default, when you call wandb.log it appends a new step to the history object and updates the summary object. The following code example shows how to log metrics with `wandb.log`. @@ -72,14 +72,14 @@ for epoch in range(wandb.config.epochs): # model performance wandb.log({"accuracy": accuracy, "loss": loss}) ``` -For more information on different data types you can log with W&B, see [Log Data During Experiments](./log/intro.md). +For more information on different data types you can log with W&B, see [Log Data During Experiments]({{< relref "./log/" >}}). ### Log an artifact to W&B Optionally log a W&B Artifact. Artifacts make it easy to version datasets and models. ```python wandb.log_artifact(model) ``` -For more information about Artifacts, see the [Artifacts Chapter](../artifacts/intro.md). For more information about versioning models, see [Model Management](../model_registry/intro.md). +For more information about Artifacts, see the [Artifacts Chapter]({{< relref "/guides/core/artifacts/" >}}). For more information about versioning models, see [Model Management]({{< relref "/guides/models/registry/model_registry/" >}}). ### Putting it all together @@ -113,11 +113,11 @@ wandb.save("model.onnx") ``` ## Next steps: Visualize your experiment -Use the W&B Dashboard as a central place to organize and visualize results from your machine learning models. With just a few clicks, construct rich, interactive charts like [parallel coordinates plots](../app/features/panels/parallel-coordinates.md),[ parameter importance analyzes](../app/features/panels/parameter-importance.md), and [more](../app/features/panels/intro.md). +Use the W&B Dashboard as a central place to organize and visualize results from your machine learning models. With just a few clicks, construct rich, interactive charts like [parallel coordinates plots]({{< relref "/guides/models/app/features/panels/parallel-coordinates.md" >}}),[ parameter importance analyzes]({{< relref "/guides/models/app/features/panels/parameter-importance.md" >}}), and [more]({{< relref "/guides/models/app/features/panels/" >}}). {{< img src="/images/sweeps/quickstart_dashboard_example.png" alt="Quickstart Sweeps Dashboard example" >}} -For more information on how to view experiments and specific runs, see [Visualize results from experiments](../track/workspaces.md). +For more information on how to view experiments and specific runs, see [Visualize results from experiments]({{< relref "/guides/models/track/workspaces.md" >}}). ## Best practices @@ -145,4 +145,4 @@ wandb.init( ) ``` -For more information about available parameters when defining a W&B Experiment, see the [`wandb.init`](../../ref/python/init.md) API docs in the [API Reference Guide](../../ref/python/README.md). \ No newline at end of file +For more information about available parameters when defining a W&B Experiment, see the [`wandb.init`]({{< relref "/ref/python/init.md" >}}) API docs in the [API Reference Guide]({{< relref "/ref/python/" >}}). \ No newline at end of file diff --git a/content/guides/models/track/limits.md b/content/guides/models/track/limits.md index 00da7c5e4..e6d0a3180 100644 --- a/content/guides/models/track/limits.md +++ b/content/guides/models/track/limits.md @@ -117,7 +117,7 @@ for step in range(1000000): ) # Commit batched, per-step metrics together ``` - + {{% alert %}} W&B continues to accept your logged data but pages may load more slowly if you exceed guidelines. @@ -154,7 +154,7 @@ with f as open("large_config.json", "r"): For faster loading times, keep the total number of runs in a single project under 10,000. Large run counts can slow down project workspaces and runs table operations, especially when grouping is enabled or runs have a large count of distinct metrics. -If you find that you or your team are frequently accessing the same set of runs (for example, recent runs), consider [bulk moving _other_ runs](../runs/manage-runs.md) to a new project used as an archive, leaving a smaller set of runs in your working project. +If you find that you or your team are frequently accessing the same set of runs (for example, recent runs), consider [bulk moving _other_ runs]({{< relref "/guides/models/track/runs/manage-runs.md" >}}) to a new project used as an archive, leaving a smaller set of runs in your working project. ### Section count @@ -202,7 +202,7 @@ The preceding table describes rate limit HTTP headers: ### Rate limits on metric logging API -The `wandb.log` calls in your script utilize a metrics logging API to log your training data to W&B. This API is engaged through either online or [offline syncing](../../ref/cli/wandb-sync.md). In either case, it imposes a rate limit quota limit in a rolling time window. This includes limits on total request size and request rate, where latter refers to the number of requests in a time duration. +The `wandb.log` calls in your script utilize a metrics logging API to log your training data to W&B. This API is engaged through either online or [offline syncing]({{< relref "/ref/cli/wandb-sync.md" >}}). In either case, it imposes a rate limit quota limit in a rolling time window. This includes limits on total request size and request rate, where latter refers to the number of requests in a time duration. W&B applies rate limits per W&B project. So if you have 3 projects in a team, each project has its own rate limit quota. Users on [Teams and Enterprise plans](https://wandb.ai/site/pricing) have higher rate limits than those on the Free plan. @@ -221,7 +221,7 @@ if epoch % 5 == 0: # Log metrics every 5 epochs wandb.log({"acc": accuracy, "loss": loss}) ``` -- Manual data syncing: W&B store your run data locally if you are rate limited. You can manually sync your data with the command `wandb sync `. For more details, see the [`wandb sync`](../../ref/cli/wandb-sync.md) reference. +- Manual data syncing: W&B store your run data locally if you are rate limited. You can manually sync your data with the command `wandb sync `. For more details, see the [`wandb sync`]({{< relref "/ref/cli/wandb-sync.md" >}}) reference. ### Rate limits on GraphQL API diff --git a/content/guides/models/track/log/_index.md b/content/guides/models/track/log/_index.md index 6c171ac60..8f75e6f2e 100644 --- a/content/guides/models/track/log/_index.md +++ b/content/guides/models/track/log/_index.md @@ -11,13 +11,13 @@ cascade: - url: guides/track/log/:filename --- -Log a dictionary of metrics, media, or custom objects to a step with the W&B Python SDK. W&B collects the key-value pairs during each step and stores them in one unified dictionary each time you log data with `wandb.log()`. Data logged from your script is saved locally to your machine in a directory called `wandb`, then synced to the W&B cloud or your [private server](../../hosting/intro.md). +Log a dictionary of metrics, media, or custom objects to a step with the W&B Python SDK. W&B collects the key-value pairs during each step and stores them in one unified dictionary each time you log data with `wandb.log()`. Data logged from your script is saved locally to your machine in a directory called `wandb`, then synced to the W&B cloud or your [private server]({{< relref "/guides/hosting/" >}}). {{% alert %}} Key-value pairs are stored in one unified dictionary only if you pass the same value for each step. W&B writes all of the collected keys and values to memory if you log a different value for `step`. {{% /alert %}} -Each call to `wandb.log` is a new `step` by default. W&B uses steps as the default x-axis when it creates charts and panels. You can optionally create and use a custom x-axis or capture a custom summary metric. For more information, see [Customize log axes](./customize-logging-axes.md). +Each call to `wandb.log` is a new `step` by default. W&B uses steps as the default x-axis when it creates charts and panels. You can optionally create and use a custom x-axis or capture a custom summary metric. For more information, see [Customize log axes]({{< relref "./customize-logging-axes.md" >}}). -You can also specify `resume="auto"` to let W&B to automatically try to restart the run on your behalf. However, you will need to ensure that you restart your run from the same directory. See the [Enable runs to automatically resume](#enable-runs-to-automatically-resume) section for more information. +You can also specify `resume="auto"` to let W&B to automatically try to restart the run on your behalf. However, you will need to ensure that you restart your run from the same directory. See the [Enable runs to automatically resume]({{< relref "#enable-runs-to-automatically-resume" >}}) section for more information. For all the examples below, replace values enclosed within `<>` with your own. @@ -53,7 +53,7 @@ run = wandb.init(entity="", \ Unexpected results will occur if multiple processes use the same `id` concurrently. -For more information on how to manage multiple processes, see the [Log distributed training experiments](../track/log/distributed-training.md) +For more information on how to manage multiple processes, see the [Log distributed training experiments]({{< relref "/guides/models/track/log/distributed-training.md" >}}) {{% /alert %}} ## Resume a run without overriding the existing run @@ -111,7 +111,7 @@ For example, suppose you execute a python script called `train.py` in a director {{% alert %}} -If you can not share a filesystem, specify the `WANDB_RUN_ID` environment variable or pass the run ID with the W&B Python SDK. See the [Custom run IDs](./intro.md#custom-run-ids) section in the "What are runs?" page for more information on run IDs. +If you can not share a filesystem, specify the `WANDB_RUN_ID` environment variable or pass the run ID with the W&B Python SDK. See the [Custom run IDs]({{< relref "./#custom-run-ids" >}}) section in the "What are runs?" page for more information on run IDs. {{% /alert %}} @@ -119,9 +119,9 @@ If you can not share a filesystem, specify the `WANDB_RUN_ID` environment variab ## Resume preemptible Sweeps runs -Automatically requeue interrupted [sweep](../sweeps/intro.md) runs. This is particularly useful if you run a sweep agent in a compute environment that is subject to preemption such as a SLURM job in a preemptible queue, an EC2 spot instance, or a Google Cloud preemptible VM. +Automatically requeue interrupted [sweep]({{< relref "/guides/models/sweeps/" >}}) runs. This is particularly useful if you run a sweep agent in a compute environment that is subject to preemption such as a SLURM job in a preemptible queue, an EC2 spot instance, or a Google Cloud preemptible VM. -Use the [`mark_preempting`](../../ref/python/run.md#mark_preempting) function to enable W&B to automatically requeue interrupted sweep runs. For example, the following code snippet +Use the [`mark_preempting`]({{< relref "/ref/python/run.md#mark_preempting" >}}) function to enable W&B to automatically requeue interrupted sweep runs. For example, the following code snippet ```python run = wandb.init() # Initialize a run diff --git a/content/guides/models/track/runs/rewind.md b/content/guides/models/track/runs/rewind.md index ea90e5409..e3540d401 100644 --- a/content/guides/models/track/runs/rewind.md +++ b/content/guides/models/track/runs/rewind.md @@ -19,7 +19,7 @@ W&B currently does not support: {{% alert %}} * To rewind a run, you must have [W&B Python SDK](https://pypi.org/project/wandb/) version >= `0.17.1`. -* You must use monotonically increasing steps. You can not use non-monotonic steps defined with [`define_metric()`](/ref/python/run#define_metric) because it disrupts the required chronological order of run history and system metrics. +* You must use monotonically increasing steps. You can not use non-monotonic steps defined with [`define_metric()`]({{< relref "/ref/python/run#define_metric" >}}) because it disrupts the required chronological order of run history and system metrics. {{% /alert %}} Rewind a run to correct or modify the history of a run without losing the original data. In addition, when you @@ -31,7 +31,7 @@ rewind a run, you can log new data from that point in time. W&B recomputes the s When you rewind a run, W&B resets the state of the run to the specified step, preserving the original data and maintaining a consistent run ID. This means that: -- **Run archiving**: W&B archives the original runs. Runs are accessible from the [**Run Overview**](./intro.md#overview-tab) tab. +- **Run archiving**: W&B archives the original runs. Runs are accessible from the [**Run Overview**]({{< relref "./#overview-tab" >}}) tab. - **Artifact association**: Associates artifacts with the run that produce them. - **Immutable run IDs**: Introduced for consistent forking from a precise state. - **Copy immutable run ID**: A button to copy the immutable run ID for improved run management. @@ -48,7 +48,7 @@ When you rewind a run, W&B lets you correct or modify the run history itself. ## Rewind a run -Use `resume_from` with [`wandb.init()`](/ref/python/init) to "rewind" a run’s history to a specific step. Specify the name of the run and the step you want to rewind from: +Use `resume_from` with [`wandb.init()`]({{< relref "/ref/python/init" >}}) to "rewind" a run’s history to a specific step. Specify the name of the run and the step you want to rewind from: ```python import wandb @@ -84,14 +84,14 @@ run2.finish() After you rewind a run, you can explore archived run with the W&B App UI. Follow these steps to view archived runs: -1. **Access the Overview Tab:** Navigate to the [**Overview tab**](./intro.md#overview-tab) on the run's page. This tab provides a comprehensive view of the run's details and history. +1. **Access the Overview Tab:** Navigate to the [**Overview tab**]({{< relref "./#overview-tab" >}}) on the run's page. This tab provides a comprehensive view of the run's details and history. 2. **Locate the Forked From field:** Within the **Overview** tab, find the `Forked From` field. This field captures the history of the resumptions. The **Forked From** field includes a link to the source run, allowing you to trace back to the original run and understand the entire rewind history. By using the `Forked From` field, you can effortlessly navigate the tree of archived resumptions and gain insights into the sequence and origin of each rewind. ## Fork from a run that you rewind -To fork from a rewound run, use the [**`fork_from`**](/guides/runs/forking) argument in `wandb.init()` and specify the source run ID and the step from the source run to fork from: +To fork from a rewound run, use the [**`fork_from`**]({{< relref "/guides/models/track/runs/forking" >}}) argument in `wandb.init()` and specify the source run ID and the step from the source run to fork from: ```python import wandb diff --git a/content/guides/models/track/runs/tags.md b/content/guides/models/track/runs/tags.md index 219caf57f..45dca60a0 100644 --- a/content/guides/models/track/runs/tags.md +++ b/content/guides/models/track/runs/tags.md @@ -48,7 +48,7 @@ if current_loss < threshold: ``` {{% /tab %}} {{% tab header="Public API" %}} -After you create a run, you can update tags using [the Public API](../track/public-api-guide.md). For example: +After you create a run, you can update tags using [the Public API]({{< relref "/guides/models/track/public-api-guide.md" >}}). For example: ```python run = wandb.Api().run("{entity}/{project}/{run-id}") diff --git a/content/guides/models/track/workspaces.md b/content/guides/models/track/workspaces.md index 39e651273..5b0bc5f47 100644 --- a/content/guides/models/track/workspaces.md +++ b/content/guides/models/track/workspaces.md @@ -11,7 +11,7 @@ weight: 4 W&B workspace is your personal sandbox to customize charts and explore model results. A W&B workspace consists of *Tables* and *Panel sections*: * **Tables**: All runs logged to your project are listed in the project's table. Turn on and off runs, change colors, and expand the table to see notes, config, and summary metrics for each run. -* **Panel sections**: A section that contains one or more [panels](../app/features/panels/intro.md). Create new panels, organize them, and export to reports to save snapshots of your workspace. +* **Panel sections**: A section that contains one or more [panels]({{< relref "/guides/models/app/features/panels/" >}}). Create new panels, organize them, and export to reports to save snapshots of your workspace. {{< img src="/images/app_ui/workspace_table_and_panels.png" alt="" >}} @@ -130,4 +130,4 @@ new_workspace = ws.Workspace(entity="new-entity", project="new-project", views=[ new_workspace.save() ``` -See [`wandb-workspace examples`](https://github.com/wandb/wandb-workspaces/tree/main/examples/workspaces) for comprehensive workspace API examples. For an end to end tutorial, see [Programmatic Workspaces](../../tutorials/workspaces.md) tutorial. \ No newline at end of file +See [`wandb-workspace examples`](https://github.com/wandb/wandb-workspaces/tree/main/examples/workspaces) for comprehensive workspace API examples. For an end to end tutorial, see [Programmatic Workspaces]({{< relref "/tutorials/workspaces.md" >}}) tutorial. \ No newline at end of file diff --git a/content/guides/quickstart.md b/content/guides/quickstart.md index eaebdcc72..10a0f2264 100644 --- a/content/guides/quickstart.md +++ b/content/guides/quickstart.md @@ -48,7 +48,7 @@ Next, log in to W&B: wandb login ``` -Or if you are using [W&B Server](./guides/hosting/intro.md) (including **Dedicated Cloud** or **Self-managed**): +Or if you are using [W&B Server]({{< relref "/guides/hosting/" >}}) (including **Dedicated Cloud** or **Self-managed**): ```bash wandb login --relogin --host=http://your-shared-local-host.com @@ -72,7 +72,7 @@ Provide [your API key](https://wandb.ai/authorize) when prompted. ## 3. Start a run and track hyperparameters -Initialize a W&B Run object in your Python script or notebook with [`wandb.init()`](./ref/python/run.md) and pass a dictionary to the `config` parameter with key-value pairs of hyperparameter names and values: +Initialize a W&B Run object in your Python script or notebook with [`wandb.init()`]({{< relref "/ref/python/run.md" >}}) and pass a dictionary to the `config` parameter with key-value pairs of hyperparameter names and values: ```python run = wandb.init( @@ -87,7 +87,7 @@ run = wandb.init( ``` -A [run](./guides/runs/intro.md) is the basic building block of W&B. You will use them often to [track metrics](./guides/track/intro.md), [create logs](./guides/artifacts/intro.md), and more. +A [run]({{< relref "/guides/models/track/runs/" >}}) is the basic building block of W&B. You will use them often to [track metrics]({{< relref "/guides/models/track/" >}}), [create logs]({{< relref "/guides/core/artifacts/" >}}), and more. @@ -146,11 +146,11 @@ The image above (click to expand) shows the loss and accuracy that was tracked f Explore the rest of the W&B ecosystem. -1. Check out [W&B Integrations](guides/integrations/intro.md) to learn how to integrate W&B with your ML framework such as PyTorch, ML library such as Hugging Face, or ML service such as SageMaker. -2. Organize runs, embed and automate visualizations, describe your findings, and share updates with collaborators with [W&B Reports](./guides/reports/intro.md). -2. Create [W&B Artifacts](./guides/artifacts/intro.md) to track datasets, models, dependencies, and results through each step of your machine learning pipeline. -3. Automate hyperparameter search and explore the space of possible models with [W&B Sweeps](./guides/sweeps/intro.md). -4. Understand your datasets, visualize model predictions, and share insights in a [central dashboard](./guides/tables/intro.md). +1. Check out [W&B Integrations]({{< relref "guides/integrations/" >}}) to learn how to integrate W&B with your ML framework such as PyTorch, ML library such as Hugging Face, or ML service such as SageMaker. +2. Organize runs, embed and automate visualizations, describe your findings, and share updates with collaborators with [W&B Reports]({{< relref "/guides/core/reports/" >}}). +2. Create [W&B Artifacts]({{< relref "/guides/core/artifacts/" >}}) to track datasets, models, dependencies, and results through each step of your machine learning pipeline. +3. Automate hyperparameter search and explore the space of possible models with [W&B Sweeps]({{< relref "/guides/models/sweeps/" >}}). +4. Understand your datasets, visualize model predictions, and share insights in a [central dashboard]({{< relref "/guides/core/tables/" >}}). 5. Navigate to W&B AI Academy and learn about LLMs, MLOps and W&B Models from hands-on [courses](https://wandb.me/courses). {{< img src="/images/quickstart/wandb_demo_experiments.gif" alt="" >}} \ No newline at end of file diff --git a/content/launch/create-and-deploy-jobs/add-job-to-queue.md b/content/launch/create-and-deploy-jobs/add-job-to-queue.md index 78ef30048..f3400d5d1 100644 --- a/content/launch/create-and-deploy-jobs/add-job-to-queue.md +++ b/content/launch/create-and-deploy-jobs/add-job-to-queue.md @@ -9,7 +9,7 @@ url: guides/launch/add-job-to-queue The following page describes how to add launch jobs to a launch queue. {{% alert %}} -Ensure that you, or someone on your team, has already configured a launch queue. For more information, see the [Set up Launch](./setup-launch.md) page. +Ensure that you, or someone on your team, has already configured a launch queue. For more information, see the [Set up Launch]({{< relref "../set-up-launch/" >}}) page. {{% /alert %}} ## Add jobs to your queue @@ -26,7 +26,7 @@ Add a job to your queue programmatically with the W&B App. 3. The **Jobs** page displays a list of W&B launch jobs that were created from previously executed W&B runs. {{< img src="/images/launch/view_jobs.png" alt="" >}} 4. Select the **Launch** button next to the name of the Job name. A modal will appear on the right side of the page. -5. From the **Job version** dropdown, select the version of hte launch job you want to use. Launch jobs are versioned like any other [W&B Artifact](../artifacts/create-a-new-artifact-version.md). Different versions of the same launch job will be created if you make modifications to the software dependencies or source code used to run the job. +5. From the **Job version** dropdown, select the version of hte launch job you want to use. Launch jobs are versioned like any other [W&B Artifact]({{< relref "/guides/core/artifacts/create-a-new-artifact-version.md" >}}). Different versions of the same launch job will be created if you make modifications to the software dependencies or source code used to run the job. 6. Within the **Overrides** section, provide new values for any inputs that are configured for your launch job. Common overrides include a new entrypoint command, arguments, or values in the `wandb.config` of your new W&B run. {{< img src="/images/launch/create_starter_queue_gs.png" alt="" >}} You can copy and paste values from other W&B runs that used your launch job by clicking on the **Paste from...** button. @@ -42,7 +42,7 @@ For example, in the following example, the team admin configured AWS instance ty {{% /tab %}} {{% tab "W&B CLI" %}} -Use the `wandb launch` command to add jobs to a queue. Create a JSON configuration with hyperparameter overrides. For example, using the script from the [Quickstart](./walkthrough.md) guide, we create a JSON file with the following overrides: +Use the `wandb launch` command to add jobs to a queue. Create a JSON configuration with hyperparameter overrides. For example, using the script from the [Quickstart]({{< relref "../walkthrough.md" >}}) guide, we create a JSON file with the following overrides: ```json title="config.json" { diff --git a/content/launch/create-and-deploy-jobs/create-launch-job.md b/content/launch/create-and-deploy-jobs/create-launch-job.md index fb300b713..bc082fe89 100644 --- a/content/launch/create-and-deploy-jobs/create-launch-job.md +++ b/content/launch/create-and-deploy-jobs/create-launch-job.md @@ -13,7 +13,7 @@ Launch jobs are blueprints for reproducing W&B runs. Jobs are W&B Artifacts that Create and run jobs with the `wandb launch` command. {{% alert %}} -To create a job without submitting it for execution, use the `wandb job create` command. See the [command reference docs](../../ref/cli/wandb-job/wandb-job-create.md) for more information. +To create a job without submitting it for execution, use the `wandb job create` command. See the [command reference docs]({{< relref "/ref/cli/wandb-job/wandb-job-create.md" >}}) for more information. {{% /alert %}} @@ -94,7 +94,7 @@ wandb launch --docker-image "wandb/job_hello_world:main" --project "hello-world" W&B will automatically create and track a job for any run with tracked source code, even if that run was not created with Launch. Runs are considered to have tracked source code if any of the three following conditions are met: - The run has an associated git remote and commit hash -- The run logged a code artifact (see [`Run.log_code`](../../ref/python/run.md#log_code) for more information) +- The run logged a code artifact (see [`Run.log_code`]({{< relref "/ref/python/run.md#log_code" >}}) for more information) - The run was executed in a Docker container with the `WANDB_DOCKER` environment variable set to an image tag The Git remote URL is inferred from the local git repository if your Launch job is created automatically by a W&B run. @@ -148,14 +148,14 @@ The term build context refers to the tree of files and directories that are sent {{% alert %}} The `--build-context` argument is particularly useful for working with Git jobs that refer to a monorepo with multiple projects. By specifying a subdirectory as the build context, you can build a container image for a specific project within the monorepo. -See the [example above](#git-jobs) for a demonstration of how to use the `--build-context` argument with the official W&B Launch jobs repository. +See the [example above]({{< relref "#git-jobs" >}}) for a demonstration of how to use the `--build-context` argument with the official W&B Launch jobs repository. {{% /alert %}} ### Dockerfile The Dockerfile is a text file that contains instructions for building a Docker image. By default, Launch uses a default Dockerfile that installs the `requirements.txt` file. To use a custom Dockerfile, specify the path to the file with the `--dockerfile` argument of `wandb launch`. -The Dockerfile path is specified relative to the build context. For example, if the build context is `jobs/hello_world`, and the Dockerfile is located in the `jobs/hello_world` directory, the `--dockerfile` argument should be set to `Dockerfile.wandb`. See the [example above](#git-jobs) for a demonstration of how to use the `--dockerfile` argument with the official W&B Launch jobs repository. +The Dockerfile path is specified relative to the build context. For example, if the build context is `jobs/hello_world`, and the Dockerfile is located in the `jobs/hello_world` directory, the `--dockerfile` argument should be set to `Dockerfile.wandb`. See the [example above]({{< relref "#git-jobs" >}}) for a demonstration of how to use the `--dockerfile` argument with the official W&B Launch jobs repository. ### Requirements file diff --git a/content/launch/create-and-deploy-jobs/job-inputs.md b/content/launch/create-and-deploy-jobs/job-inputs.md index 2871670cd..30dd2c954 100644 --- a/content/launch/create-and-deploy-jobs/job-inputs.md +++ b/content/launch/create-and-deploy-jobs/job-inputs.md @@ -6,7 +6,7 @@ menu: title: Manage job inputs url: guides/launch/job-inputs --- -The core experience of Launch is easily experimenting with different job inputs like hyperparameters and datasets, and routing these jobs to appropriate hardware. Once a job is created, users beyond the original author can adjust these inputs via the W&B GUI or CLI. For information on how job inputs can be set when launching from the CLI or UI, see the [Enqueue jobs](./add-job-to-queue.md) guide. +The core experience of Launch is easily experimenting with different job inputs like hyperparameters and datasets, and routing these jobs to appropriate hardware. Once a job is created, users beyond the original author can adjust these inputs via the W&B GUI or CLI. For information on how job inputs can be set when launching from the CLI or UI, see the [Enqueue jobs]({{< relref "./add-job-to-queue.md" >}}) guide. This section describes how to programmatically control the inputs that can be tweaked for a job. @@ -95,7 +95,7 @@ to load the run config input values anywhere in the job code. The Launch SDK also provides a way to manage input values stored in config files in the job code. This is a common pattern in many deep learning and large language model use cases, like this [torchtune](https://github.com/pytorch/torchtune/blob/main/recipes/configs/llama3/8B_lora.yaml) example or this [Axolotl config](https://github.com/OpenAccess-AI-Collective/axolotl/blob/main/examples/llama-3/qlora-fsdp-70b.yaml)). {{% alert %}} -[Sweeps on Launch](./sweeps-on-launch.md) does not support the use of config file inputs as sweep parameters. Sweep parameters must be controlled through the `Run.config` object. +[Sweeps on Launch]({{< relref "../sweeps-on-launch.md" >}}) does not support the use of config file inputs as sweep parameters. Sweep parameters must be controlled through the `Run.config` object. {{% /alert %}} The `launch.manage_config_file` function can be used to add a config file as an input to the Launch job, giving you access to edit values within the config file when launching the job. diff --git a/content/launch/create-and-deploy-jobs/launch-queue-observability.md b/content/launch/create-and-deploy-jobs/launch-queue-observability.md index bc68dede3..da705a310 100644 --- a/content/launch/create-and-deploy-jobs/launch-queue-observability.md +++ b/content/launch/create-and-deploy-jobs/launch-queue-observability.md @@ -29,7 +29,7 @@ For example, suppose you have a fixed resource (such as DGX BasePod). If you obs On the other hand, suppose you use a cloud resource and you see periodic bursts of activity. Periodic bursts of activity might suggest an opportunity to save money by reserving resources for particular times. -To the right of the plot is a key that shows which colors represent the [status of a launch job](./launch-view-jobs.md#check-the-status-of-a-job). +To the right of the plot is a key that shows which colors represent the [status of a launch job]({{< relref "./launch-view-jobs.md#check-the-status-of-a-job" >}}). {{% alert %}} `Queued` items might indicate opportunities to shift workloads to other queues. A spike in failures can identify users who might need help with their launch job setup. diff --git a/content/launch/integration-guides/dagster.md b/content/launch/integration-guides/dagster.md index 03d60c0ba..59180731b 100644 --- a/content/launch/integration-guides/dagster.md +++ b/content/launch/integration-guides/dagster.md @@ -9,10 +9,10 @@ url: guides/integrations/dagster --- Use Dagster and W&B (W&B) to orchestrate your MLOps pipelines and maintain ML assets. The integration with W&B makes it easy within Dagster to: -* Use and create [W&B Artifacts](../artifacts/intro.md). -* Use and create Registered Models in [W&B Model Registry](../model_registry/intro.md). -* Run training jobs on dedicated compute using [W&B Launch](../launch/intro.md). -* Use the [wandb](../../ref/python/README.md) client in ops and assets. +* Use and create [W&B Artifacts]({{< relref "/guides/core/artifacts/" >}}). +* Use and create Registered Models in [W&B Model Registry]({{< relref "/guides/models/registry/model_registry/" >}}). +* Run training jobs on dedicated compute using [W&B Launch]({{< relref "/launch/" >}}). +* Use the [wandb]({{< relref "/ref/python/" >}}) client in ops and assets. The W&B Dagster integration provides a W&B-specific Dagster resource and IO Manager: @@ -25,13 +25,13 @@ The following guide demonstrates how to satisfy prerequisites to use W&B in Dags You will need the following resources to use Dagster within Weights and Biases: 1. **W&B API Key**. 2. **W&B entity (user or team)**: An entity is a username or team name where you send W&B Runs and Artifacts. Make sure to create your account or team entity in the W&B App UI before you log runs. If you do not specify ain entity, the run will be sent to your default entity, which is usually your username. Change your default entity in your settings under **Project Defaults**. -3. **W&B project**: The name of the project where [W&B Runs](../runs/intro.md) are stored. +3. **W&B project**: The name of the project where [W&B Runs]({{< relref "/guides/models/track/runs/" >}}) are stored. Find your W&B entity by checking the profile page for that user or team in the W&B App. You can use a pre-existing W&B project or create a new one. New projects can be created on the W&B App homepage or on user/team profile page. If a project does not exist it will be automatically created when you first use it. The proceeding instructions demonstrate how to get an API key: ### How to get an API key 1. [Log in to W&B](https://wandb.ai/login). Note: if you are using W&B Server ask your admin for the instance host name. -2. Collect your API key by navigating to the [authorize page](https://wandb.ai/authorize) or in your user/team settings. For a production environment we recommend using a [service account](../../support/service_account_useful.md) to own that key. +2. Collect your API key by navigating to the [authorize page](https://wandb.ai/authorize) or in your user/team settings. For a production environment we recommend using a [service account]({{< relref "../../support/service_account_useful.md" >}}) to own that key. 3. Set an environment variable for that API key export `WANDB_API_KEY=YOUR_KEY`. @@ -154,7 +154,7 @@ def create_dataset(): You can annotate your `@op`, `@asset` and `@multi_asset` with a metadata configuration in order to write Artifacts. Similarly you can also consume W&B Artifacts even if they were created outside Dagster. ## Write W&B Artifacts -Before continuing, we recommend you to have a good understanding of how to use W&B Artifacts. Consider reading the [Guide on Artifacts](../artifacts/intro.md). +Before continuing, we recommend you to have a good understanding of how to use W&B Artifacts. Consider reading the [Guide on Artifacts]({{< relref "/guides/core/artifacts/" >}}). Return an object from a Python function to write a W&B Artifact. The following objects are supported by W&B: * Python objects (int, dict, list…) @@ -166,7 +166,7 @@ The proceeding examples demonstrate how to write W&B Artifacts with Dagster asse {{< tabpane text=true >}} {{% tab "Python objects" %}} -Anything that can be serialized with the [pickle](https://docs.python.org/3/library/pickle.html) module is pickled and added to an Artifact created by the integration. The content is unpickled when you read that Artifact inside Dagster (see [Read artifacts](#read-wb-artifacts) for more details). +Anything that can be serialized with the [pickle](https://docs.python.org/3/library/pickle.html) module is pickled and added to an Artifact created by the integration. The content is unpickled when you read that Artifact inside Dagster (see [Read artifacts]({{< relref "#read-wb-artifacts" >}}) for more details). ```python @asset( @@ -183,10 +183,10 @@ def create_dataset(): ``` -W&B supports multiple Pickle-based serialization modules ([pickle](https://docs.python.org/3/library/pickle.html), [dill](https://github.com/uqfoundation/dill), [cloudpickle](https://github.com/cloudpipe/cloudpickle), [joblib](https://github.com/joblib/joblib)). You can also use more advanced serialization like [ONNX](https://onnx.ai/) or [PMML](https://en.wikipedia.org/wiki/Predictive_Model_Markup_Language). Please refer to the [Serialization](#serialization-configuration) section for more information. +W&B supports multiple Pickle-based serialization modules ([pickle](https://docs.python.org/3/library/pickle.html), [dill](https://github.com/uqfoundation/dill), [cloudpickle](https://github.com/cloudpipe/cloudpickle), [joblib](https://github.com/joblib/joblib)). You can also use more advanced serialization like [ONNX](https://onnx.ai/) or [PMML](https://en.wikipedia.org/wiki/Predictive_Model_Markup_Language). Please refer to the [Serialization]({{< relref "#serialization-configuration" >}}) section for more information. {{% /tab %}} {{% tab "W&B Object" %}} -Any native W&B object (e.g [Table](../../ref/python/data-types/table.md), [Image](../../ref/python/data-types/image.md), or [Graph](../../ref/python/data-types/graph.md)) is added to an Artifact created by the integration. Here’s an example using a Table. +Any native W&B object (e.g [Table]({{< relref "/ref/python/data-types/table.md" >}}), [Image]({{< relref "/ref/python/data-types/image.md" >}}), or [Graph]({{< relref "/ref/python/data-types/graph.md" >}})) is added to an Artifact created by the integration. Here’s an example using a Table. ```python import wandb @@ -317,9 +317,9 @@ Supported properties: * `type`: (str) The type of the artifact, which is used to organize and differentiate artifacts. Common types include dataset or model, but you can use any string containing letters, numbers, underscores, hyphens, and dots. Required when the output is not already an Artifact. * `description`: (str) Free text that offers a description of the artifact. The description is markdown rendered in the UI, so this is a good place to place tables, links, etc. * `aliases`: (list[str]) An array containing one or more aliases you want to apply on the Artifact. The integration will also add the “latest” tag to that list whether it’s set or not. This is an effective way for you to manage versioning of models and datasets. -* [`add_dirs`](../../ref/python/artifact.md#add_dir): (list[dict[str, Any]]): An array containing configuration for each local directory to include in the Artifact. It supports the same arguments as the homonymous method in the SDK. -* [`add_files`](../../ref/python/artifact.md#add_file): (list[dict[str, Any]]): An array containing configuration for each local file to include in the Artifact. It supports the same arguments as the homonymous method in the SDK. -* [`add_references`](../../ref/python/artifact.md#add_reference): (list[dict[str, Any]]): An array containing configuration for each external reference to include in the Artifact. It supports the same arguments as the homonymous method in the SDK. +* [`add_dirs`]({{< relref "/ref/python/artifact.md#add_dir" >}}): (list[dict[str, Any]]): An array containing configuration for each local directory to include in the Artifact. It supports the same arguments as the homonymous method in the SDK. +* [`add_files`]({{< relref "/ref/python/artifact.md#add_file" >}}): (list[dict[str, Any]]): An array containing configuration for each local file to include in the Artifact. It supports the same arguments as the homonymous method in the SDK. +* [`add_references`]({{< relref "/ref/python/artifact.md#add_reference" >}}): (list[dict[str, Any]]): An array containing configuration for each external reference to include in the Artifact. It supports the same arguments as the homonymous method in the SDK. * `serialization_module`: (dict) Configuration of the serialization module to be used. Refer to the Serialization section for more information. * `name`: (str) Name of the serialization module. Accepted values: `pickle`, `dill`, `cloudpickle`, `joblib`. The module needs to be available locally. * `parameters`: (dict[str, Any]) Optional arguments passed to the serialization function. It accepts the same parameters as the dump method for that module. For example, `{"compress": 3, "protocol": 4}`. @@ -849,7 +849,7 @@ The integration provides an importable `@op` called `run_launch_agent`. It start Agents are processes that poll launch queues and execute the jobs (or dispatch them to external services to be executed) in order. -Refer to the [reference documentation](../launch/intro.md) for configuration +Refer to the [reference documentation]({{< relref "/launch/" >}}) for configuration You can also view useful descriptions for all properties in Launchpad. @@ -897,7 +897,7 @@ The integration provides an importable `@op` called `run_launch_job`. It execute A Launch job is assigned to a queue in order to be executed. You can create a queue or use the default one. Make sure you have an active agent listening to that queue. You can run an agent inside your Dagster instance but can also consider using a deployable agent in Kubernetes. -Refer to the [reference documentation](../launch/intro.md) for configuration. +Refer to the [reference documentation]({{< relref "/launch/" >}}) for configuration. You can also view useful descriptions for all properties in Launchpad. @@ -947,14 +947,14 @@ def run_launch_job_example(): ## Best practices 1. Use the IO Manager to read and write Artifacts. -You should never need to use [`Artifact.download()`](../../ref/python/artifact.md#download) or [`Run.log_artifact()`](../../ref/python/run.md#log_artifact) directly. Those methods are handled by integration. Simply return the data you wish to store in Artifact and let the integration do the rest. This will provide better lineage for the Artifact in W&B. +You should never need to use [`Artifact.download()`]({{< relref "/ref/python/artifact.md#download" >}}) or [`Run.log_artifact()`]({{< relref "/ref/python/run.md#log_artifact" >}}) directly. Those methods are handled by integration. Simply return the data you wish to store in Artifact and let the integration do the rest. This will provide better lineage for the Artifact in W&B. 2. Only build an Artifact object yourself for complex use cases. Python objects and W&B objects should be returned from your ops/assets. The integration handles bundling the Artifact. For complex use cases, you can build an Artifact directly in a Dagster job. We recommend you pass an Artifact object to the integration for metadata enrichment such as the source integration name and version, the python version used, the pickle protocol version and more. 3. Add files, directories and external references to your Artifacts through the metadata. -Use the integration `wandb_artifact_configuration` object to add any file, directory or external references (Amazon S3, GCS, HTTP…). See the advanced example in the [Artifact configuration section](#configuration-1) for more information. +Use the integration `wandb_artifact_configuration` object to add any file, directory or external references (Amazon S3, GCS, HTTP…). See the advanced example in the [Artifact configuration section]({{< relref "#configuration-1" >}}) for more information. 4. Use an @asset instead of an @op when an Artifact is produced. Artifacts are assets. It is recommended to use an asset when Dagster maintains that asset. This will provide better observability in the Dagit Asset Catalog. @@ -966,7 +966,7 @@ This allows you to take advantage of the integration to read externally created You can train small models inside your Dagster cluster and you can run Dagster in a Kubernetes cluster with GPU nodes. We recommend using W&B Launch for large model training. This will prevent overloading your instance and provide access to more adequate compute. 7. When experiment tracking within Dagster, set your W&B Run ID to the value of your Dagster Run ID. -We recommend that you both: make the [Run resumable](../runs/resuming.md) and set the W&B Run ID to the Dagster Run ID or to a string of your choice. Following this recommendation ensures your W&B metrics and W&B Artifacts are stored in the same W&B Run when you train models inside of Dagster. +We recommend that you both: make the [Run resumable]({{< relref "/guides/models/track/runs/resuming.md" >}}) and set the W&B Run ID to the Dagster Run ID or to a string of your choice. Following this recommendation ensures your W&B metrics and W&B Artifacts are stored in the same W&B Run when you train models inside of Dagster. Either set the W&B Run ID to the Dagster Run ID. diff --git a/content/launch/integration-guides/minikube_gpu.md b/content/launch/integration-guides/minikube_gpu.md index 95deba1d5..f5d658e51 100644 --- a/content/launch/integration-guides/minikube_gpu.md +++ b/content/launch/integration-guides/minikube_gpu.md @@ -13,7 +13,7 @@ This tutorial is intended to guide users with direct access to a machine that ha W&B recommends you create a Kubernetes cluster with GPU support that uses your cloud provider, if you want to set up a minikube cluster on a cloud machine. For example, AWS, GCP, Azure, Coreweave, and other cloud providers have tools to create Kubernetes clusters with GPU support. -W&B recommends you use a [Launch Docker queue](/guides/launch/setup-launch-docker) if you want to set up a minikube cluster for scheduling GPUs on a machine that has a single GPU. You can still follow the tutorial for fun, but the GPU scheduling will not be very useful. +W&B recommends you use a [Launch Docker queue]({{< relref "../set-up-launch/setup-launch-docker" >}}) if you want to set up a minikube cluster for scheduling GPUs on a machine that has a single GPU. You can still follow the tutorial for fun, but the GPU scheduling will not be very useful. {{% /alert %}} ## Background @@ -90,7 +90,7 @@ spec: {{% /tab %}} {{< /tabpane >}} -For more information about queue configurations, see the [Set up Launch on Kubernetes](/guides/launch/setup-launch-kubernetes.md) and the [Advanced queue setup guide](/guides/launch/setup-queue-advanced.md). +For more information about queue configurations, see the [Set up Launch on Kubernetes]({{< relref "../set-up-launch/setup-launch-kubernetes.md" >}}) and the [Advanced queue setup guide]({{< relref "../set-up-launch/setup-queue-advanced.md" >}}). The `${image_uri}` and `{{gpus}}` strings are examples of the two kinds of diff --git a/content/launch/integration-guides/nim.md b/content/launch/integration-guides/nim.md index 36832a0c1..b8a92d5d6 100644 --- a/content/launch/integration-guides/nim.md +++ b/content/launch/integration-guides/nim.md @@ -20,7 +20,7 @@ Deployment time varies by model and machine type. The base Llama2-7b config take ## Quickstart -1. [Create a launch queue](../launch/add-job-to-queue.md) if you don't have one already. See an example queue config below. +1. [Create a launch queue]({{< relref "../create-and-deploy-jobs/add-job-to-queue.md" >}}) if you don't have one already. See an example queue config below. ```yaml net: host diff --git a/content/launch/integration-guides/volcano.md b/content/launch/integration-guides/volcano.md index 6801323ce..14a99fb81 100644 --- a/content/launch/integration-guides/volcano.md +++ b/content/launch/integration-guides/volcano.md @@ -31,7 +31,7 @@ The first step is to create a launch queue. Head to [wandb.ai/launch](https://wa In the configuration section, we will enter a [volcano job](https://volcano.sh/en/docs/vcjob/) template. Any runs launched from this queue will be created using this job specification, so you can modify this configuration as needed to customize your jobs. -This configuration block can accept a Kubernetes job specification, volcano job specification, or any other custom resource definition (CRD) that you are interested in launching. You can make use of [macros in the configuration block](../guides/launch/setup-launch.md) to dynamically set the contents of this spec. +This configuration block can accept a Kubernetes job specification, volcano job specification, or any other custom resource definition (CRD) that you are interested in launching. You can make use of [macros in the configuration block]({{< relref "../set-up-launch/" >}}) to dynamically set the contents of this spec. In this tutorial, we will use a configuration for multinode pytorch training that makes use of [volcano's pytorch plugin](https://github.com/volcano-sh/volcano/blob/master/docs/user-guide/how_to_use_pytorch_plugin.md). You can copy and paste the following config as YAML or JSON: diff --git a/content/launch/launch-faq/best_practices_launch_effectively.md b/content/launch/launch-faq/best_practices_launch_effectively.md index 6ef25ecf0..c895f7206 100644 --- a/content/launch/launch-faq/best_practices_launch_effectively.md +++ b/content/launch/launch-faq/best_practices_launch_effectively.md @@ -10,4 +10,4 @@ title: Are there best practices for using Launch effectively? 2. Create a W&B service account to initiate the agent, ensuring it is not linked to an individual user account. -3. Use `wandb.config` to manage hyperparameters, allowing for overwriting during job re-runs. Refer to [this guide](/guides/track/config/#set-the-configuration-with-argparse) for details on using argparse. \ No newline at end of file +3. Use `wandb.config` to manage hyperparameters, allowing for overwriting during job re-runs. Refer to [this guide]({{< relref "/guides/models/track/config/#set-the-configuration-with-argparse" >}}) for details on using argparse. \ No newline at end of file diff --git a/content/launch/launch-faq/launch_d_wandb_job_create_image_uploading_whole_docker.md b/content/launch/launch-faq/launch_d_wandb_job_create_image_uploading_whole_docker.md index c4042bbac..425d77806 100644 --- a/content/launch/launch-faq/launch_d_wandb_job_create_image_uploading_whole_docker.md +++ b/content/launch/launch-faq/launch_d_wandb_job_create_image_uploading_whole_docker.md @@ -20,6 +20,6 @@ docker push : wandb launch -d : ``` -The launch agent then spins up a job pointing to the specified container. See [Advanced agent setup](../guides/launch/setup-agent-advanced.md#agent-configuration) for examples on configuring agent access to pull images from a container registry. +The launch agent then spins up a job pointing to the specified container. See [Advanced agent setup]({{< relref "/launch/set-up-launch/setup-agent-advanced.md#agent-configuration" >}}) for examples on configuring agent access to pull images from a container registry. For Kubernetes, ensure that the Kubernetes cluster pods have access to the registry where the image is pushed. \ No newline at end of file diff --git a/content/launch/launch-faq/launch_support_parallelization_limit_resources_consumed_job.md b/content/launch/launch-faq/launch_support_parallelization_limit_resources_consumed_job.md index 671f2fad1..6d4e2e7fd 100644 --- a/content/launch/launch-faq/launch_support_parallelization_limit_resources_consumed_job.md +++ b/content/launch/launch-faq/launch_support_parallelization_limit_resources_consumed_job.md @@ -7,11 +7,11 @@ title: Does Launch support parallelization? How can I limit the resources consu by a job? --- -Launch supports scaling jobs across multiple GPUs and nodes. Refer to [this guide](/tutorials/volcano) for details. +Launch supports scaling jobs across multiple GPUs and nodes. Refer to [this guide]({{< relref "/launch/integration-guides/volcano.md" >}}) for details. Each launch agent is configured with a `max_jobs` parameter, which determines the maximum number of simultaneous jobs it can run. Multiple agents can point to a single queue as long as they connect to an appropriate launching infrastructure. -You can set limits on CPU, GPU, memory, and other resources at the queue or job run level in the resource configuration. For information on setting up queues with resource limits on Kubernetes, refer to [this guide](../guides/launch/setup-launch-kubernetes.md). +You can set limits on CPU, GPU, memory, and other resources at the queue or job run level in the resource configuration. For information on setting up queues with resource limits on Kubernetes, refer to [this guide]({{< relref "/launch/set-up-launch/setup-launch-kubernetes.md" >}}). For sweeps, include the following block in the queue configuration to limit the number of concurrent runs: diff --git a/content/launch/launch-faq/launcherror_permission_denied.md b/content/launch/launch-faq/launcherror_permission_denied.md index f313615a8..bdd41ce21 100644 --- a/content/launch/launch-faq/launcherror_permission_denied.md +++ b/content/launch/launch-faq/launcherror_permission_denied.md @@ -8,6 +8,6 @@ title: How do I fix a "permission denied" error in Launch? If you encounter the error message `Launch Error: Permission denied`, it indicates insufficient permissions to log to the desired project. Possible causes include: -1. You are not logged in on this machine. Run [`wandb login`](../ref/cli/wandb-login.md) in the command line. +1. You are not logged in on this machine. Run [`wandb login`]({{< relref "/ref/cli/wandb-login.md" >}}) in the command line. 2. The specified entity does not exist. The entity must be your username or an existing team's name. Create a team if necessary with the [Subscriptions page](https://app.wandb.ai/billing). 3. You lack project permissions. Request the project creator to change the privacy setting to **Open** to allow logging runs to the project. \ No newline at end of file diff --git a/content/launch/launch-faq/restrict_access_modify_example.md b/content/launch/launch-faq/restrict_access_modify_example.md index 5ff700fc6..b0f093e56 100644 --- a/content/launch/launch-faq/restrict_access_modify_example.md +++ b/content/launch/launch-faq/restrict_access_modify_example.md @@ -6,4 +6,4 @@ menu: title: How can admins restrict which users have modify access? --- -Control access to certain queue fields for users who are not team administrators through [queue config templates](../guides/launch/setup-queue-advanced.md). Team administrators define which fields non-admin users can view, and set the editing limits. Only team administrators have the ability to create or edit queues. \ No newline at end of file +Control access to certain queue fields for users who are not team administrators through [queue config templates]({{< relref "/launch/set-up-launch/setup-queue-advanced.md" >}}). Team administrators define which fields non-admin users can view, and set the editing limits. Only team administrators have the ability to create or edit queues. \ No newline at end of file diff --git a/content/launch/launch-terminology.md b/content/launch/launch-terminology.md index 1f886c087..6fd27565c 100644 --- a/content/launch/launch-terminology.md +++ b/content/launch/launch-terminology.md @@ -8,10 +8,10 @@ url: guides/launch/launch-terminology weight: 2 --- -With W&B Launch, you enqueue [jobs](#launch-job) onto [queues](#launch-queue) to create runs. Jobs are python scripts instrumented with W&B. Queues hold a list of jobs to execute on a [target resource](#target-resources). [Agents](#launch-agent) pull jobs from queues and execute the jobs on target resources. W&B tracks launch jobs similarly to how W&B tracks [runs](../runs/intro.md). +With W&B Launch, you enqueue [jobs]({{< relref "#launch-job" >}}) onto [queues]({{< relref "#launch-queue" >}}) to create runs. Jobs are python scripts instrumented with W&B. Queues hold a list of jobs to execute on a [target resource]({{< relref "#target-resources" >}}). [Agents]({{< relref "#launch-agent" >}}) pull jobs from queues and execute the jobs on target resources. W&B tracks launch jobs similarly to how W&B tracks [runs]({{< relref "/guides/models/track/runs/" >}}). ### Launch job -A launch job is a specific type of [W&B Artifact](../artifacts/intro.md) that represents a task to complete. For example, common launch jobs include training a model or triggering a model evaluation. Job definitions include: +A launch job is a specific type of [W&B Artifact]({{< relref "/guides/core/artifacts/" >}}) that represents a task to complete. For example, common launch jobs include training a model or triggering a model evaluation. Job definitions include: - Python code and other file assets, including at least one runnable entrypoint. - Information about the input (config parameter) and output (metrics logged). @@ -29,7 +29,7 @@ There are three main kinds of job definitions: While Launch jobs can perform activities not related to model training--for example, deploy a model to a Triton inference server--all jobs must call `wandb.init` to complete successfully. This creates a run for tracking purposes in a W&B workspace. {{% /alert %}} -Find jobs you created in the W&B App under the `Jobs` tab of your project workspace. From there, jobs can be configured and sent to a [launch queue](#launch-queue) to be executed on a variety of [target resources](#target-resources). +Find jobs you created in the W&B App under the `Jobs` tab of your project workspace. From there, jobs can be configured and sent to a [launch queue]({{< relref "#launch-queue" >}}) to be executed on a variety of [target resources]({{< relref "#target-resources" >}}). ### Launch queue Launch *queues* are ordered lists of jobs to execute on a specific target resource. Launch queues are first-in, first-out. (FIFO). There is no practical limit to the number of queues you can have, but a good guideline is one queue per target resource. Jobs can be enqueued with the W&B App UI, W&B CLI or Python SDK. You can then configure one or more Launch agents to pull items from the queue and execute them on the queue's target resource. @@ -39,10 +39,10 @@ The compute environment that a Launch queue is configured to execute jobs on is W&B Launch supports the following target resources: -- [Docker](./setup-launch-docker.md) -- [Kubernetes](./setup-launch-kubernetes.md) -- [AWS SageMaker](./setup-launch-sagemaker.md) -- [GCP Vertex](./setup-vertex.md) +- [Docker]({{< relref "./set-up-launch/setup-launch-docker.md" >}}) +- [Kubernetes]({{< relref "./set-up-launch/setup-launch-kubernetes.md" >}}) +- [AWS SageMaker]({{< relref "./set-up-launch/setup-launch-sagemaker.md" >}}) +- [GCP Vertex]({{< relref "./set-up-launch/setup-vertex.md" >}}) Each target resource accepts a different set of configuration parameters called *resource configurations*. Resource configurations take on default values defined by each Launch queue, but can be overridden independently by each job. See the documentation for each target resource for more details. diff --git a/content/launch/set-up-launch/_index.md b/content/launch/set-up-launch/_index.md index ad1dc31ba..a51c2dfe1 100644 --- a/content/launch/set-up-launch/_index.md +++ b/content/launch/set-up-launch/_index.md @@ -15,7 +15,7 @@ This page describes the high-level steps required to set up W&B Launch: ## Set up a queue Launch queues must be configured to point to a specific target resource along with any additional configuration specific to that resource. For example, a launch queue that points to a Kubernetes cluster might include environment variables or set a custom namespace its launch queue configuration. When you create a queue, you will specify both the target resource you want to use and the configuration for that resource to use. -When an agent receives a job from a queue, it also receives the queue configuration. When the agent submits the job to the target resource, it includes the queue configuration along with any overrides from the job itself. For example, you can use a job configuration to specify the Amazon SageMaker instance type for that job instance only. In this case, it is common to use [queue config templates](./setup-queue-advanced.md#configure-queue-template) as the end user interface. +When an agent receives a job from a queue, it also receives the queue configuration. When the agent submits the job to the target resource, it includes the queue configuration along with any overrides from the job itself. For example, you can use a job configuration to specify the Amazon SageMaker instance type for that job instance only. In this case, it is common to use [queue config templates]({{< relref "./setup-queue-advanced.md#configure-queue-template" >}}) as the end user interface. ### Create a queue 1. Navigate to Launch App at [wandb.ai/launch](https://wandb.ai/launch). @@ -33,7 +33,7 @@ When an agent receives a job from a queue, it also receives the queue configurat Launch agents are long running processes that poll one or more launch queues for jobs. Launch agents dequeue jobs in first in, first out (FIFO) order or in priority order depending on the queues they pull from. When an agent dequeues a job from a queue, it optionally builds an image for that job. The agent then submits the job to the target resource along with configuration options specified in the queue configuration. {{% alert %}} -Agents are highly flexible and can be configured to support a wide variety of use cases. The required configuration for your agent will depend on your specific use case. See the dedicated page for [Docker](./setup-launch-docker.md), [Amazon SageMaker](./setup-launch-sagemaker.md), [Kubernetes](./setup-launch-kubernetes.md), or [Vertex AI](./setup-vertex.md). +Agents are highly flexible and can be configured to support a wide variety of use cases. The required configuration for your agent will depend on your specific use case. See the dedicated page for [Docker]({{< relref "./setup-launch-docker.md" >}}), [Amazon SageMaker]({{< relref "./setup-launch-sagemaker.md" >}}), [Kubernetes]({{< relref "./setup-launch-kubernetes.md" >}}), or [Vertex AI]({{< relref "./setup-vertex.md" >}}). {{% /alert %}} {{% alert %}} @@ -53,7 +53,7 @@ Independent of your use case, there are core configurable options for the launch * `queues`: the name of one or more queues for the agent to watch {{% alert %}} -You can use the W&B CLI to specify universal configurable options for the launch agent (instead of the config YAML file): maximum number of jobs, W&B entity, and launch queues. See the [`wandb launch-agent`](../../ref/cli/wandb-launch-agent.md) command for more information. +You can use the W&B CLI to specify universal configurable options for the launch agent (instead of the config YAML file): maximum number of jobs, W&B entity, and launch queues. See the [`wandb launch-agent`]({{< relref "/ref/cli/wandb-launch-agent.md" >}}) command for more information. {{% /alert %}} @@ -71,7 +71,7 @@ queues: ``` ### Configure a container builder -The launch agent can be configured to build images. You must configure the agent to use a container builder if you intend to use launch jobs created from git repositories or code artifacts. See the [Create a launch job](./create-launch-job.md) for more information on how to create a launch job. +The launch agent can be configured to build images. You must configure the agent to use a container builder if you intend to use launch jobs created from git repositories or code artifacts. See the [Create a launch job]({{< relref "../create-and-deploy-jobs/create-launch-job.md" >}}) for more information on how to create a launch job. W&B Launch supports three builder options: @@ -82,7 +82,7 @@ W&B Launch supports three builder options: {{% alert %}} Use the Kaniko builder if your agent is polling in an environment where a Docker daemon is unavailable (for example, a Kubernetes cluster). -See the [Set up Kubernetes](./setup-launch-kubernetes.md) for details about the Kaniko builder. +See the [Set up Kubernetes]({{< relref "./setup-launch-kubernetes.md" >}}) for details about the Kaniko builder. {{% /alert %}} To specify an image builder, include the builder key in your agent configuration. For example, the following code snippet shows a portion of the launch config (`launch-config.yaml`) that specifies to use Docker or Kaniko: @@ -99,7 +99,7 @@ In some cases, you might want to connect a launch agent to a cloud registry. Com * You want to use the agent to build images and run these images on Amazon SageMaker or VertexAI. * You want the launch agent to provide credentials to pull from an image repository. -To learn more about how to configure the agent to interact with a container registry, see the [Advanced agent set](./setup-agent-advanced.md) up page. +To learn more about how to configure the agent to interact with a container registry, see the [Advanced agent set]({{< relref "./setup-agent-advanced.md" >}}) up page. ## Activate the launch agent Activate the launch agent with the `launch-agent` W&B CLI command: @@ -108,4 +108,4 @@ Activate the launch agent with the `launch-agent` W&B CLI command: wandb launch-agent -q -q --max-jobs 5 ``` -In some use cases, you might want to have a launch agent polling queues from within a Kubernetes cluster. See the [Advanced queue set up page](./setup-queue-advanced.md) for more information. \ No newline at end of file +In some use cases, you might want to have a launch agent polling queues from within a Kubernetes cluster. See the [Advanced queue set up page]({{< relref "./setup-queue-advanced.md" >}}) for more information. \ No newline at end of file diff --git a/content/launch/set-up-launch/setup-agent-advanced.md b/content/launch/set-up-launch/setup-agent-advanced.md index 45aad3a62..2567b8034 100644 --- a/content/launch/set-up-launch/setup-agent-advanced.md +++ b/content/launch/set-up-launch/setup-agent-advanced.md @@ -13,7 +13,7 @@ This guide provides information on how to set up the W&B Launch agent to build c {{% alert %}} Build is only required for git and code artifact jobs. Image jobs do not require build. -See [Create a launch job](./create-launch-job.md) for more information on job types. +See [Create a launch job]({{< relref "../create-and-deploy-jobs/create-launch-job.md" >}}) for more information on job types. {{% /alert %}} ## Builders diff --git a/content/launch/set-up-launch/setup-launch-docker.md b/content/launch/set-up-launch/setup-launch-docker.md index 06539797b..e1300e764 100644 --- a/content/launch/set-up-launch/setup-launch-docker.md +++ b/content/launch/set-up-launch/setup-launch-docker.md @@ -23,7 +23,7 @@ When you use Docker with W&B Launch, W&B will first build an image, and then bui ## Configure a Docker queue -The launch queue configuration (for a Docker target resource) accepts the same options defined in the [`docker run`](../../ref/cli/wandb-docker-run.md) CLI command. +The launch queue configuration (for a Docker target resource) accepts the same options defined in the [`docker run`]({{< relref "/ref/cli/wandb-docker-run.md" >}}) CLI command. The agent receives options defined in the queue configuration. The agent then merges the received options with any overrides from the launch job’s configuration to produce a final `docker run` command that is executed on the target resource (in this case, a local machine). @@ -63,7 +63,7 @@ The `--gpus` flag of the `docker run` command allows you to specify GPUs that ar {{% alert %}} * Install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker) to use GPUs within a Docker container. -* If you build images from a code or artifact-sourced job, you can override the base image used by the [agent](#configure-a-launch-agent-on-a-local-machine) to include the NVIDIA Container Toolkit. +* If you build images from a code or artifact-sourced job, you can override the base image used by the [agent]({{< relref "#configure-a-launch-agent-on-a-local-machine" >}}) to include the NVIDIA Container Toolkit. For example, within your launch queue, you can override the base image to `tensorflow/tensorflow:latest-gpu`: ```json @@ -97,7 +97,7 @@ Create a queue that uses Docker as compute resource with the W&B CLI: Configure the launch agent with a YAML config file named `launch-config.yaml`. By default, W&B will check for the config file in `~/.config/wandb/launch-config.yaml`. You can optionally specify a different directory when you activate the launch agent. {{% alert %}} -You can use the W&B CLI to specify core configurable options for the launch agent (instead of the config YAML file): maximum number of jobs, W&B entity, and launch queues. See the [`wandb launch-agent`](../../ref/cli/wandb-launch-agent.md) command for more information. +You can use the W&B CLI to specify core configurable options for the launch agent (instead of the config YAML file): maximum number of jobs, W&B entity, and launch queues. See the [`wandb launch-agent`]({{< relref "/ref/cli/wandb-launch-agent.md" >}}) command for more information. {{% /alert %}} @@ -142,4 +142,4 @@ Launch uses external container registeries such as Dockerhub, Google Container R If you want to run a job on a different environment from where you built it, configure your agent to be able to pull from a container registry. -To learn more about how connect the launch agent with a cloud registry, see the [Advanced agent setup](./setup-agent-advanced.md#agent-configuration) page. \ No newline at end of file +To learn more about how connect the launch agent with a cloud registry, see the [Advanced agent setup]({{< relref "./setup-agent-advanced.md#agent-configuration" >}}) page. \ No newline at end of file diff --git a/content/launch/set-up-launch/setup-launch-kubernetes.md b/content/launch/set-up-launch/setup-launch-kubernetes.md index b9d67248f..7a96f1ce3 100644 --- a/content/launch/set-up-launch/setup-launch-kubernetes.md +++ b/content/launch/set-up-launch/setup-launch-kubernetes.md @@ -10,7 +10,7 @@ You can use W&B Launch to push ML workloads to a Kubernetes cluster, giving ML e W&B maintains an [official Launch agent image](https://hub.docker.com/r/wandb/launch-agent) that can be deployed to your cluster with a [Helm chart](https://github.com/wandb/helm-charts/tree/main/charts/launch-agent) that W&B maintains. -W&B uses the [Kaniko](https://github.com/GoogleContainerTools/kaniko) builder to enable the Launch agent to build Docker images in a Kubernetes cluster. To learn more on how to set up Kaniko for the Launch agent, or how to turn off job building and only use prebuilt Docker images, see [Advanced agent set up](./setup-agent-advanced.md). +W&B uses the [Kaniko](https://github.com/GoogleContainerTools/kaniko) builder to enable the Launch agent to build Docker images in a Kubernetes cluster. To learn more on how to set up Kaniko for the Launch agent, or how to turn off job building and only use prebuilt Docker images, see [Advanced agent set up]({{< relref "./setup-agent-advanced.md" >}}). {{% alert %}} To install Helm and apply or upgrade W&B's Launch agent Helm chart, you need `kubectl` access to the cluster with sufficient permissions to create, update, and delete Kubernetes resources. Typically, a user with cluster-admin or a custom role with equivalent permissions is required. @@ -111,7 +111,7 @@ Create a queue in the W&B App that uses Kubernetes as its compute resource: 3. Select the **Entity** you would like to create the queue in. 4. Provide a name for your queue in the **Name** field. 5. Select **Kubernetes** as the **Resource**. -6. Within the **Configuration** field, provide the Kubernetes Job workflow spec or Custom Resource spec you [configured in the previous section](#configure-a-queue-for-kubernetes). +6. Within the **Configuration** field, provide the Kubernetes Job workflow spec or Custom Resource spec you [configured in the previous section]({{< relref "#configure-a-queue-for-kubernetes" >}}). ## Configure a Launch agent with Helm @@ -194,4 +194,4 @@ serviceAccount: azureStorageAccessKey: '' ``` -For more information on registries, environments, and required agent permissions see [Advanced agent set up](./setup-agent-advanced.md). \ No newline at end of file +For more information on registries, environments, and required agent permissions see [Advanced agent set up]({{< relref "./setup-agent-advanced.md" >}}). \ No newline at end of file diff --git a/content/launch/set-up-launch/setup-launch-sagemaker.md b/content/launch/set-up-launch/setup-launch-sagemaker.md index 56cafa3fb..3d99adf39 100644 --- a/content/launch/set-up-launch/setup-launch-sagemaker.md +++ b/content/launch/set-up-launch/setup-launch-sagemaker.md @@ -21,9 +21,9 @@ This guide shows how to execute SageMaker Training Jobs. For information on how Before you get started, ensure you satisfy the following prerequisites: -* [Decide if you want the Launch agent to build a Docker image for you.](#decide-if-you-want-the-launch-agent-to-build-a-docker-images) -* [Set up AWS resources and gather information about S3, ECR, and Sagemaker IAM roles.](#set-up-aws-resources) -* [Create an IAM role for the Launch agent](#create-an-iam-role-for-launch-agent). +* [Decide if you want the Launch agent to build a Docker image for you.]({{< relref "#decide-if-you-want-the-launch-agent-to-build-a-docker-images" >}}) +* [Set up AWS resources and gather information about S3, ECR, and Sagemaker IAM roles.]({{< relref "#set-up-aws-resources" >}}) +* [Create an IAM role for the Launch agent]({{< relref "#create-an-iam-role-for-launch-agent" >}}). ### Decide if you want the Launch agent to build a Docker images @@ -41,7 +41,7 @@ Ensure you have the following AWS resources configured in your preferred AWS reg 2. One or more [S3 buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) to store inputs and outputs for your SageMaker Training jobs. 3. An IAM role for Amazon SageMaker that permits SageMaker to run training jobs and interact with Amazon ECR and Amazon S3. -Make a note of the ARNs for these resources. You will need the ARNs when you define the [Launch queue configuration](#configure-launch-queue-for-sagemaker). +Make a note of the ARNs for these resources. You will need the ARNs when you define the [Launch queue configuration]({{< relref "#configure-launch-queue-for-sagemaker" >}}). @@ -162,7 +162,7 @@ The Launch agent needs permission to create Amazon SageMaker training jobs. Foll For more information on how to create IAM role, see the [AWS Identity and Access Management Documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html). {{% alert %}} -* If you want the launch agent to build images, see the [Advanced agent set up](./setup-agent-advanced.md) for additional permissions required. +* If you want the launch agent to build images, see the [Advanced agent set up]({{< relref "./setup-agent-advanced.md" >}}) for additional permissions required. * The `kms:CreateGrant` permission for SageMaker queues is required only if the associated ResourceConfig has a specified VolumeKmsKeyId and the associated role does not have a policy that permits this action. {{% /alert %}} @@ -196,7 +196,7 @@ Next, create a queue in the W&B App that uses SageMaker as its compute resource: ``` You must at minimum specify: -- `RoleArn` : ARN of the SageMaker execution IAM role (see [prerequisites](#prerequisites)). Not to be confused with the launch **agent** IAM role. +- `RoleArn` : ARN of the SageMaker execution IAM role (see [prerequisites]({{< relref "#prerequisites" >}})). Not to be confused with the launch **agent** IAM role. - `OutputDataConfig.S3OutputPath` : An Amazon S3 URI specifying where SageMaker outputs will be stored. - `ResourceConfig`: Required specification of a resource config. Options for resource config are outlined [here](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ResourceConfig.html). - `StoppingCondition`: Required specification of the stopping conditions for the training job. Options outlined [here](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_StoppingCondition.html). @@ -207,7 +207,7 @@ You must at minimum specify: The following section describes where you can deploy your agent and how to configure your agent based on where it is deployed. -There are [several options for how the Launch agent is deployed for a Amazon SageMaker](#decide-where-to-run-the-launch-agent) queue: on a local machine, on an EC2 instance, or in an EKS cluster. [Configure your launch agent appropriately](#configure-a-launch-agent) based on the where you deploy your agent. +There are [several options for how the Launch agent is deployed for a Amazon SageMaker]({{< relref "#decide-where-to-run-the-launch-agent" >}}) queue: on a local machine, on an EC2 instance, or in an EKS cluster. [Configure your launch agent appropriately]({{< relref "#configure-a-launch-agent" >}}) based on the where you deploy your agent. ### Decide where to run the Launch agent @@ -302,7 +302,7 @@ Now start the agent with `wandb launch-agent` ## (Optional) Push your launch job Docker image to Amazon ECR {{% alert %}} -This section applies only if your launch agent uses existing Docker images that contain your training or inference logic. [There are two options on how your launch agent behaves.](#decide-if-you-want-the-launch-agent-to-build-a-docker-images) +This section applies only if your launch agent uses existing Docker images that contain your training or inference logic. [There are two options on how your launch agent behaves.]({{< relref "#decide-if-you-want-the-launch-agent-to-build-a-docker-images" >}}) {{% /alert %}} Upload your Docker image that contains your launch job to your Amazon ECR repo. Your Docker image needs to be in your ECR registry before you submit new launch jobs if you are using image-based jobs. diff --git a/content/launch/set-up-launch/setup-vertex.md b/content/launch/set-up-launch/setup-vertex.md index 4494891eb..8160f156c 100644 --- a/content/launch/set-up-launch/setup-vertex.md +++ b/content/launch/set-up-launch/setup-vertex.md @@ -91,7 +91,7 @@ queues: - ``` -If you want the launch agent to build images for you that are executed in Vertex AI, see [Advanced agent set up](./setup-agent-advanced.md). +If you want the launch agent to build images for you that are executed in Vertex AI, see [Advanced agent set up]({{< relref "./setup-agent-advanced.md" >}}). ## Set up agent permissions diff --git a/content/launch/sweeps-on-launch.md b/content/launch/sweeps-on-launch.md index 07835e199..50384831c 100644 --- a/content/launch/sweeps-on-launch.md +++ b/content/launch/sweeps-on-launch.md @@ -9,15 +9,15 @@ url: guides/launch/sweeps-on-launch --- {{< cta-button colabLink="https://colab.research.google.com/drive/1WxLKaJlltThgZyhc7dcZhDQ6cjVQDfil#scrollTo=AFEzIxA6foC7" >}} -Create a hyperparameter tuning job ([sweeps](../sweeps/intro.md)) with W&B Launch. With sweeps on launch, a sweep scheduler is pushed to a Launch Queue with the specified hyperparameters to sweep over. The sweep scheduler starts as it is picked up by the agent, launching sweep runs onto the same queue with chosen hyperparameters. This continues until the sweep finishes or is stopped. +Create a hyperparameter tuning job ([sweeps]({{< relref "/guides/models/sweeps/" >}})) with W&B Launch. With sweeps on launch, a sweep scheduler is pushed to a Launch Queue with the specified hyperparameters to sweep over. The sweep scheduler starts as it is picked up by the agent, launching sweep runs onto the same queue with chosen hyperparameters. This continues until the sweep finishes or is stopped. You can use the default W&B Sweep scheduling engine or implement your own custom scheduler: -1. Standard sweep scheduler: Use the default W&B Sweep scheduling engine that controls [W&B Sweeps](../sweeps/intro.md). The familiar `bayes`, `grid`, and `random` methods are available. +1. Standard sweep scheduler: Use the default W&B Sweep scheduling engine that controls [W&B Sweeps]({{< relref "/guides/models/sweeps/" >}}). The familiar `bayes`, `grid`, and `random` methods are available. 2. Custom sweep scheduler: Configure the sweep scheduler to run as a job. This option enables full customization. An example of how to extend the standard sweep scheduler to include more logging can be found in the section below. {{% alert %}} -This guide assumes that W&B Launch has been previously configured. If W&B Launch has is not configured, see the [how to get started](./intro.md#how-to-get-started) section of the launch documentation. +This guide assumes that W&B Launch has been previously configured. If W&B Launch has is not configured, see the [how to get started]({{< relref "./#how-to-get-started" >}}) section of the launch documentation. {{% /alert %}} {{% alert %}} @@ -28,7 +28,7 @@ We recommend you create a sweep on launch using the 'basic' method if you are a Create W&B Sweeps with Launch. You can create a sweep interactively with the W&B App or programmatically with the W&B CLI. For advanced configurations of Launch sweeps, including the ability to customize the scheduler, use the CLI. {{% alert %}} -Before you create a sweep with W&B Launch, ensure that you first create a job to sweep over. See the [Create a Job](./create-launch-job.md) page for more information. +Before you create a sweep with W&B Launch, ensure that you first create a job to sweep over. See the [Create a Job]({{< relref "./create-and-deploy-jobs/create-launch-job.md" >}}) page for more information. {{% /alert %}} {{< tabpane text=true >}} @@ -100,7 +100,7 @@ parameters: # registry: ``` -For information on how to create a sweep configuration, see the [Define sweep configuration](../sweeps/define-sweep-configuration.md) page. +For information on how to create a sweep configuration, see the [Define sweep configuration]({{< relref "/guides/models/sweeps/define-sweep-configuration.md" >}}) page. 4. Next, initialize a sweep. Provide the path to your config file, the name of your job queue, your W&B entity, and the name of the project. @@ -108,7 +108,7 @@ For information on how to create a sweep configuration, see the [Define sweep co wandb launch-sweep --queue --entity --project ``` -For more information on W&B Sweeps, see the [Tune Hyperparameters](../sweeps/intro.md) chapter. +For more information on W&B Sweeps, see the [Tune Hyperparameters]({{< relref "/guides/models/sweeps/" >}}) chapter. {{% /tab %}} {{< /tabpane >}} diff --git a/content/launch/walkthrough.md b/content/launch/walkthrough.md index 34f493934..c139df3f6 100644 --- a/content/launch/walkthrough.md +++ b/content/launch/walkthrough.md @@ -12,29 +12,29 @@ weight: 1 {{< cta-button colabLink="https://colab.research.google.com/drive/1wX0OSVxZJDHRsZaOaOEDx-lLUrO1hHgP" >}} -Easily scale training [runs](../runs/intro.md) from your desktop to a compute resource like Amazon SageMaker, Kubernetes and more with W&B Launch. Once W&B Launch is configured, you can quickly run training scripts, model evaluation suites, prepare models for production inference, and more with a few clicks and commands. +Easily scale training [runs]({{< relref "/guides/models/track/runs/" >}}) from your desktop to a compute resource like Amazon SageMaker, Kubernetes and more with W&B Launch. Once W&B Launch is configured, you can quickly run training scripts, model evaluation suites, prepare models for production inference, and more with a few clicks and commands. ## How it works Launch is composed of three fundamental components: **launch jobs**, **queues**, and **agents**. -A [*launch job*](./launch-terminology.md#launch-job) is a blueprint for configuring and running tasks in your ML workflow. Once you have a launch job, you can add it to a [*launch queue*](./launch-terminology.md#launch-queue). A launch queue is a first-in, first-out (FIFO) queue where you can configure and submit your jobs to a particular compute target resource, such as Amazon SageMaker or a Kubernetes cluster. +A [*launch job*]({{< relref "./launch-terminology.md#launch-job" >}}) is a blueprint for configuring and running tasks in your ML workflow. Once you have a launch job, you can add it to a [*launch queue*]({{< relref "./launch-terminology.md#launch-queue" >}}). A launch queue is a first-in, first-out (FIFO) queue where you can configure and submit your jobs to a particular compute target resource, such as Amazon SageMaker or a Kubernetes cluster. -As jobs are added to the queue, [*launch agents*](./launch-terminology.md#launch-agent) poll that queue and execute the job on the system targeted by the queue. +As jobs are added to the queue, [*launch agents*]({{< relref "./launch-terminology.md#launch-agent" >}}) poll that queue and execute the job on the system targeted by the queue. {{< img src="/images/launch/launch_overview.png" alt="" >}} -Based on your use case, you (or someone on your team) will configure the launch queue according to your chosen [compute resource target](./launch-terminology.md#target-resources) (for example Amazon SageMaker) and deploy a launch agent on your own infrastructure. +Based on your use case, you (or someone on your team) will configure the launch queue according to your chosen [compute resource target]({{< relref "./launch-terminology.md#target-resources" >}}) (for example Amazon SageMaker) and deploy a launch agent on your own infrastructure. -See the [Terms and concepts](./launch-terminology.md) page for more information on launch jobs, how queues work, launch agents, and additional information on how W&B Launch works. +See the [Terms and concepts]({{< relref "./launch-terminology.md" >}}) page for more information on launch jobs, how queues work, launch agents, and additional information on how W&B Launch works. ## How to get started Depending on your use case, explore the following resources to get started with W&B Launch: -* If this is your first time using W&B Launch, we recommend you go through the [Walkthrough](#walkthrough) guide. -* Learn how to set up [W&B Launch](./setup-launch.md). -* Create a [launch job](./create-launch-job.md). +* If this is your first time using W&B Launch, we recommend you go through the [Walkthrough]({{< relref "#walkthrough" >}}) guide. +* Learn how to set up [W&B Launch]({{< relref "./set-up-launch/" >}}). +* Create a [launch job]({{< relref "./create-and-deploy-jobs/create-launch-job.md" >}}). * Check out the W&B Launch [public jobs GitHub repository](https://github.com/wandb/launch-jobs) for templates of common tasks like [deploying to Triton](https://github.com/wandb/launch-jobs/tree/main/jobs/deploy_to_nvidia_triton), [evaluating an LLM](https://github.com/wandb/launch-jobs/tree/main/jobs/openai_evals), or more. * View launch jobs created from this repository in this public [`wandb/jobs` project](https://wandb.ai/wandb/jobs/jobs) W&B project. @@ -77,7 +77,7 @@ pip install wandb>=0.17.1 {{% /tabpane %}} ## Create a launch job -Create a [launch job](./launch-terminology.md#launch-job) in one of three ways: with a Docker image, from a git repository or from local source code: +Create a [launch job]({{< relref "./launch-terminology.md#launch-job" >}}) in one of three ways: with a Docker image, from a git repository or from local source code: {{< tabpane text=true >}} {{% tab "With a Docker image" %}} @@ -157,7 +157,7 @@ After clicking the button, the browser will redirect to the **Agents** tab of th {{< img src="/images/launch/create_docker_queue.gif" alt="" >}} -For advanced queue configuration options, see the [advanced queue setup page](./setup-queue-advanced.md). +For advanced queue configuration options, see the [advanced queue setup page]({{< relref "./set-up-launch/setup-queue-advanced.md" >}}). ## Connect an agent to the queue diff --git a/content/ref/_index.md b/content/ref/_index.md index 6b6a500da..06f6757d9 100644 --- a/content/ref/_index.md +++ b/content/ref/_index.md @@ -18,13 +18,13 @@ These docs are automatically generated from the [`wandb` library](https://github ### Reference sections -1. [Python Library](./python/README.md): Add `wandb` to your script to capture metrics and save artifacts -2. [Command Line Interface](./cli/README.md): Log in, run jobs, execute sweeps, and more using shell commands -3. [JavaScript Library](./js/README.md): A beta JavaScript/TypeScript client to track metrics from your Node server -4. [Query panels](./query-panel/README.md): A beta query language to select and aggregate data +1. [Python Library]({{< relref "./python/" >}}): Add `wandb` to your script to capture metrics and save artifacts +2. [Command Line Interface]({{< relref "./cli/" >}}): Log in, run jobs, execute sweeps, and more using shell commands +3. [JavaScript Library]({{< relref "./js/" >}}): A beta JavaScript/TypeScript client to track metrics from your Node server +4. [Query panels]({{< relref "./query-panel/" >}}): A beta query language to select and aggregate data ### Examples and guides [Our examples repo](https://github.com/wandb/examples) has scripts and colabs to try W&B features, and see integrations with various libraries. -[Our developer guide](../guides/intro.md) has guides, tutorials, and FAQs for the various W&B products. +[Our developer guide]({{< relref "/guides/" >}}) has guides, tutorials, and FAQs for the various W&B products. diff --git a/content/ref/js/_index.md b/content/ref/js/_index.md index b1b687d8a..8bcc64067 100644 --- a/content/ref/js/_index.md +++ b/content/ref/js/_index.md @@ -52,7 +52,7 @@ Node/CommonJS: const wandb = require('@wandb/sdk').default; ``` -We're currently missing a lot of the functionality found in our Python SDK, but basic logging functionality is available. We'll be adding additional features like [Tables](/guides/tables?utm_source=github&utm_medium=code&utm_campaign=wandb&utm_content=readme) soon. +We're currently missing a lot of the functionality found in our Python SDK, but basic logging functionality is available. We'll be adding additional features like [Tables]({{< relref "/guides/core/tables?utm_source=github&utm_medium=code&utm_campaign=wandb&utm_content=readme" >}}) soon. ### Authentication and Settings @@ -60,7 +60,7 @@ In node environments we look for `process.env.WANDB_API_KEY` and prompt for it's # Integrations -Our [Python integrations](/guides/integrations) are widely used by our community, and we hope to build out more JavaScript integrations to help LLM app builders leverage whatever tool they want. +Our [Python integrations]({{< relref "/guides/integrations/" >}}) are widely used by our community, and we hope to build out more JavaScript integrations to help LLM app builders leverage whatever tool they want. If you have any requests for additional integrations, we'd love you to open an issue with details about the request. diff --git a/content/ref/python/_index.md b/content/ref/python/_index.md index 8bc62f587..63b74841d 100644 --- a/content/ref/python/_index.md +++ b/content/ref/python/_index.md @@ -18,29 +18,29 @@ For reference documentation, see https://docs.wandb.com/ref/python. ## Classes -[`class Artifact`](./artifact.md): Flexible and lightweight building block for dataset and model versioning. +[`class Artifact`]({{< relref "./artifact.md" >}}): Flexible and lightweight building block for dataset and model versioning. -[`class Run`](./run.md): A unit of computation logged by wandb. Typically, this is an ML experiment. +[`class Run`]({{< relref "./run.md" >}}): A unit of computation logged by wandb. Typically, this is an ML experiment. ## Functions -[`agent(...)`](./agent.md): Start one or more sweep agents. +[`agent(...)`]({{< relref "./agent.md" >}}): Start one or more sweep agents. -[`controller(...)`](./controller.md): Public sweep controller constructor. +[`controller(...)`]({{< relref "./controller.md" >}}): Public sweep controller constructor. -[`finish(...)`](./finish.md): Finish a run and upload any remaining data. +[`finish(...)`]({{< relref "./finish.md" >}}): Finish a run and upload any remaining data. -[`init(...)`](./init.md): Start a new run to track and log to W&B. +[`init(...)`]({{< relref "./init.md" >}}): Start a new run to track and log to W&B. -[`log(...)`](./log.md): Upload run data. +[`log(...)`]({{< relref "./log.md" >}}): Upload run data. -[`login(...)`](./login.md): Set up W&B login credentials. +[`login(...)`]({{< relref "./login.md" >}}): Set up W&B login credentials. -[`save(...)`](./save.md): Sync one or more files to W&B. +[`save(...)`]({{< relref "./save.md" >}}): Sync one or more files to W&B. -[`sweep(...)`](./sweep.md): Initialize a hyperparameter sweep. +[`sweep(...)`]({{< relref "./sweep.md" >}}): Initialize a hyperparameter sweep. -[`watch(...)`](./watch.md): Hooks into the given PyTorch models to monitor gradients and the model's computational graph. +[`watch(...)`]({{< relref "./watch.md" >}}): Hooks into the given PyTorch models to monitor gradients and the model's computational graph. | Other Members | | | :--- | :--- | diff --git a/content/ref/python/artifact.md b/content/ref/python/artifact.md index 88f93c560..870427cef 100644 --- a/content/ref/python/artifact.md +++ b/content/ref/python/artifact.md @@ -35,7 +35,7 @@ begin with `add`. Once the artifact has all the desired files, you can call | Attributes | | | :--- | :--- | -| `aliases` | List of one or more semantically friendly references or identifying "nicknames" assigned to an artifact version. Aliases are mutable references that you can programmatically reference. Change an artifact's alias with the W&B App UI or programmatically. See [Create new artifact versions](https://docs.wandb.ai/guides/artifacts/create-a-new-artifact-version) for more information. | +| `aliases` | List of one or more semantically friendly references or identifying "nicknames" assigned to an artifact version. Aliases are mutable references that you can programmatically reference. Change an artifact's alias with the W&B App UI or programmatically. See [Create new artifact versions](https://docs.wandb.ai/guides/core/artifacts/create-a-new-artifact-version) for more information. | | `collection` | The collection this artifact was retrieved from. A collection is an ordered group of artifact versions. If this artifact was retrieved from a portfolio / linked collection, that collection will be returned rather than the collection that an artifact version originated from. The collection that an artifact originates from is known as the source sequence. | | `commit_hash` | The hash returned when this artifact was committed. | | `created_at` | Timestamp when the artifact was created. | @@ -172,7 +172,7 @@ Add a reference denoted by a URI to the artifact. Unlike files or directories that you add to an artifact, references are not uploaded to W&B. For more information, -see [Track external files](https://docs.wandb.ai/guides/artifacts/track-external-files). +see [Track external files](https://docs.wandb.ai/guides/core/artifacts/track-external-files). By default, the following schemes are supported: diff --git a/content/ref/python/data-types/_index.md b/content/ref/python/data-types/_index.md index d682efbca..a8e5c349a 100644 --- a/content/ref/python/data-types/_index.md +++ b/content/ref/python/data-types/_index.md @@ -10,10 +10,10 @@ This module defines data types for logging rich, interactive visualizations to W Data types include common media types, like images, audio, and videos, flexible containers for information, like tables and HTML, and more. -For more on logging media, see [our guide](https://docs.wandb.com/guides/track/log/media) +For more on logging media, see [our guide](https://docs.wandb.com/guides/models/track/log/media) For more on logging structured data for interactive dataset and model analysis, -see [our guide to W&B Tables](https://docs.wandb.com/guides/tables/). +see [our guide to W&B Tables](https://docs.wandb.com/guides/core/tables/). All of these special data types are subclasses of WBValue. All the data types serialize to JSON, since that is what wandb uses to save the objects locally diff --git a/content/ref/python/init.md b/content/ref/python/init.md index 5ddfa4668..e43dc3386 100644 --- a/content/ref/python/init.md +++ b/content/ref/python/init.md @@ -51,7 +51,7 @@ When you're done logging data, call `wandb.finish()` to end the run. If you don't call `wandb.finish()`, the run ends when your script exits. For more on using `wandb.init()`, including detailed examples, check out our -[guide and FAQs](https://docs.wandb.ai/guides/track/launch). +[guide and FAQs](https://docs.wandb.ai/guides/models/track/launch). #### Examples: @@ -99,13 +99,13 @@ call `wandb.finish()` at the end of the block. | `config_exclude_keys` | A list of specific keys to exclude from `wandb.config`. | | `config_include_keys` | A list of specific keys to include in `wandb.config`. | | `allow_val_change` | Controls whether config values can be modified after their initial set. By default, an exception is raised if a config value is overwritten. For tracking variables that change during training, such as a learning rate, consider using `wandb.log()` instead. By default, this is `False` in scripts and `True` in Notebook environments. | -| `group` | Specify a group name to organize individual runs as part of a larger experiment. This is useful for cases like cross-validation or running multiple jobs that train and evaluate a model on different test sets. Grouping allows you to manage related runs collectively in the UI, making it easy to toggle and review results as a unified experiment. For more information, refer to the [guide to grouping runs](https://docs.wandb.com/guides/runs/grouping). | +| `group` | Specify a group name to organize individual runs as part of a larger experiment. This is useful for cases like cross-validation or running multiple jobs that train and evaluate a model on different test sets. Grouping allows you to manage related runs collectively in the UI, making it easy to toggle and review results as a unified experiment. For more information, refer to the [guide to grouping runs](https://docs.wandb.com/guides/models/track/runs/grouping). | | `job_type` | Specify the type of run, especially helpful when organizing runs within a group as part of a larger experiment. For example, in a group, you might label runs with job types such as "train" and "eval". Defining job types enables you to easily filter and group similar runs in the UI, facilitating direct comparisons. | | `mode` | Specifies how run data is managed, with the following options: - `"online"` (default): Enables live syncing with W&B when a network connection is available, with real-time updates to visualizations. - `"offline"`: Suitable for air-gapped or offline environments; data is saved locally and can be synced later. Ensure the run folder is preserved to enable future syncing. - `"disabled"`: Disables all W&B functionality, making the run’s methods no-ops. Typically used in testing to bypass W&B operations. | | `force` | Determines if a W&B login is required to run the script. If `True`, the user must be log in to W&B; otherwise, the script will not proceed. If `False` (default), the script can proceed without a login, switching to offline mode if the user is not logged in. | | `anonymous` | Specifies the level of control over anonymous data logging. Available options are: - `"never"` (default): Requires you to link your W&B account before tracking the run. This prevents unintentional creation of anonymous runs by ensuring each run is associated with an account. - `"allow"`: Enables a logged-in user to track runs with their account, but also allows someone running the script without a W&B account to view the charts and data in the UI. - `"must"`: Forces the run to be logged to an anonymous account, even if the user is logged in. | | `reinit` | Determines if multiple `wandb.init()` calls can start new runs within the same process. By default (`False`), if an active run exists, calling `wandb.init()` returns the existing run instead of creating a new one. When `reinit=True`, the active run is finished before a new run is initialized. In notebook environments, runs are reinitialized by default unless `reinit` is explicitly set to `False`. | -| `resume` | Controls the behavior when resuming a run with the specified `id`. Available options are: - `"allow"`: If a run with the specified `id` exists, it will resume from the last step; otherwise, a new run will be created. - `"never"`: If a run with the specified `id` exists, an error will be raised. If no such run is found, a new run will be created. - `"must"`: If a run with the specified `id` exists, it will resume from the last step. If no run is found, an error will be raised. - `"auto"`: Automatically resumes the previous run if it crashed on this machine; otherwise, starts a new run. - `True`: Deprecated. Use `"auto"` instead. - `False`: Deprecated. Use the default behavior (leaving `resume` unset) to always start a new run. Note: If `resume` is set, `fork_from` and `resume_from` cannot be used. When `resume` is unset, the system will always start a new run. For more details, see the [guide to resuming runs](https://docs.wandb.com/guides/runs/resuming). | +| `resume` | Controls the behavior when resuming a run with the specified `id`. Available options are: - `"allow"`: If a run with the specified `id` exists, it will resume from the last step; otherwise, a new run will be created. - `"never"`: If a run with the specified `id` exists, an error will be raised. If no such run is found, a new run will be created. - `"must"`: If a run with the specified `id` exists, it will resume from the last step. If no run is found, an error will be raised. - `"auto"`: Automatically resumes the previous run if it crashed on this machine; otherwise, starts a new run. - `True`: Deprecated. Use `"auto"` instead. - `False`: Deprecated. Use the default behavior (leaving `resume` unset) to always start a new run. Note: If `resume` is set, `fork_from` and `resume_from` cannot be used. When `resume` is unset, the system will always start a new run. For more details, see the [guide to resuming runs](https://docs.wandb.com/guides/models/track/runs/resuming). | | `resume_from` | Specifies a moment in a previous run to resume a run from, using the format `{run_id}?_step={step}`. This allows users to truncate the history logged to a run at an intermediate step and resume logging from that step. The target run must be in the same project. If an `id` argument is also provided, the `resume_from` argument will take precedence. `resume`, `resume_from` and `fork_from` cannot be used together, only one of them can be used at a time. Note: This feature is in beta and may change in the future. | | `fork_from` | Specifies a point in a previous run from which to fork a new run, using the format `{id}?_step={step}`. This creates a new run that resumes logging from the specified step in the target run’s history. The target run must be part of the current project. If an `id` argument is also provided, it must be different from the `fork_from` argument, an error will be raised if they are the same. `resume`, `resume_from` and `fork_from` cannot be used together, only one of them can be used at a time. Note: This feature is in beta and may change in the future. | | `save_code` | Enables saving the main script or notebook to W&B, aiding in experiment reproducibility and allowing code comparisons across runs in the UI. By default, this is disabled, but you can change the default to enable on your [settings page](https://wandb.ai/settings). | diff --git a/content/ref/python/integrations/README.md b/content/ref/python/integrations/README.md index 1cf8c2a12..5edb70d1d 100644 --- a/content/ref/python/integrations/README.md +++ b/content/ref/python/integrations/README.md @@ -8,4 +8,4 @@ title: Integrations ## Modules -[`keras`](./keras) module: Tools for integrating `wandb` with [`Keras`](https://keras.io/). +[`keras`]({{< relref "./keras/" >}}) module: Tools for integrating `wandb` with [`Keras`](https://keras.io/). diff --git a/content/ref/python/integrations/_index.md b/content/ref/python/integrations/_index.md index c7c821865..941a69456 100644 --- a/content/ref/python/integrations/_index.md +++ b/content/ref/python/integrations/_index.md @@ -7,4 +7,4 @@ title: Integrations ## Modules -[`keras`](./keras) module: Tools for integrating `wandb` with [`Keras`](https://keras.io/). +[`keras`]({{< relref "./keras/" >}}) module: Tools for integrating `wandb` with [`Keras`](https://keras.io/). diff --git a/content/ref/python/integrations/keras/wandbmodelcheckpoint.md b/content/ref/python/integrations/keras/wandbmodelcheckpoint.md index 4bc229183..775415e19 100644 --- a/content/ref/python/integrations/keras/wandbmodelcheckpoint.md +++ b/content/ref/python/integrations/keras/wandbmodelcheckpoint.md @@ -30,7 +30,7 @@ here: https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/ModelCheckpo This callback is to be used in conjunction with training using `model.fit()` to save a model or weights (in a checkpoint file) at some interval. The model checkpoints will be logged as W&B Artifacts. You can learn more here: -https://docs.wandb.ai/guides/artifacts +https://docs.wandb.ai/guides/core/artifacts This callback provides the following features: - Save the model that has achieved `best performance` based on `monitor`. diff --git a/content/ref/python/log.md b/content/ref/python/log.md index d214cde6a..ad32398ec 100644 --- a/content/ref/python/log.md +++ b/content/ref/python/log.md @@ -18,7 +18,7 @@ log( Use `log` to log data from runs, such as scalars, images, video, histograms, plots, and tables. -See our [guides to logging](https://docs.wandb.ai/guides/track/log) for +See our [guides to logging](https://docs.wandb.ai/guides/models/track/log) for live examples, code snippets, best practices, and more. The most basic usage is `run.log({"train-loss": 0.5, "accuracy": 0.9})`. @@ -27,17 +27,17 @@ the summary values for these metrics. Visualize logged data in the workspace at [wandb.ai](https://wandb.ai), or locally on a [self-hosted instance](https://docs.wandb.ai/guides/hosting) -of the W&B app, or export data to visualize and explore locally, such as in a Jupyter notebook, with [our API](https://docs.wandb.ai/guides/track/public-api-guide). +of the W&B app, or export data to visualize and explore locally, such as in a Jupyter notebook, with [our API](https://docs.wandb.ai/guides/models/track/public-api-guide). Logged values don't have to be scalars. Logging any wandb object is supported. For example `run.log({"example": wandb.Image("myimage.jpg")})` will log an example image which will be displayed nicely in the W&B UI. See the [reference documentation](https://docs.wandb.com/ref/python/data-types) for all of the different supported types or check out our -[guides to logging](https://docs.wandb.ai/guides/track/log) for examples, +[guides to logging](https://docs.wandb.ai/guides/models/track/log) for examples, from 3D molecular structures and segmentation masks to PR curves and histograms. You can use `wandb.Table` to log structured data. See our -[guide to logging tables](https://docs.wandb.ai/guides/tables/tables-walkthrough) +[guide to logging tables](https://docs.wandb.ai/guides/core/tables/tables-walkthrough) for details. The W&B UI organizes metrics with a forward slash (`/`) in their name @@ -112,7 +112,7 @@ run.log({"accuracy": 0.9}, step=current_step) #### Examples: For more and more detailed examples, see -[our guides to logging](https://docs.wandb.com/guides/track/log). +[our guides to logging](https://docs.wandb.com/guides/models/track/log). ### Basic usage diff --git a/content/ref/python/run.md b/content/ref/python/run.md index 37942d707..34bdba49a 100644 --- a/content/ref/python/run.md +++ b/content/ref/python/run.md @@ -66,13 +66,13 @@ assert wandb.run is None ``` See the documentation for `wandb.init` for more on creating runs, or check out -[our guide to `wandb.init`](https://docs.wandb.ai/guides/track/launch). +[our guide to `wandb.init`](https://docs.wandb.ai/guides/models/track/launch). In distributed training, you can either create a single run in the rank 0 process and then log information only from that process, or you can create a run in each process, logging from each separately, and group the results together with the `group` argument to `wandb.init`. For more details on distributed training with W&B, check out -[our guide](https://docs.wandb.ai/guides/track/log/distributed-training). +[our guide](https://docs.wandb.ai/guides/models/track/log/distributed-training). Currently, there is a parallel `Run` object in the `wandb.Api`. Eventually these two objects will be merged. @@ -396,7 +396,7 @@ Upload run data. Use `log` to log data from runs, such as scalars, images, video, histograms, plots, and tables. -See our [guides to logging](https://docs.wandb.ai/guides/track/log) for +See our [guides to logging](https://docs.wandb.ai/guides/models/track/log) for live examples, code snippets, best practices, and more. The most basic usage is `run.log({"train-loss": 0.5, "accuracy": 0.9})`. @@ -405,17 +405,17 @@ the summary values for these metrics. Visualize logged data in the workspace at [wandb.ai](https://wandb.ai) or locally on a [self-hosted instance](https://docs.wandb.ai/guides/hosting) -of the W&B app. Use our [API](https://docs.wandb.ai/guides/track/public-api-guide) to export data to visualize and explore locally +of the W&B app. Use our [API](https://docs.wandb.ai/guides/models/track/public-api-guide) to export data to visualize and explore locally Logged values don't have to be scalars. Logging any wandb object is supported. For example `run.log({"example": wandb.Image("myimage.jpg")})` will log an example image which will be displayed nicely in the W&B UI. See the [reference documentation](https://docs.wandb.com/ref/python/data-types) for all of the different supported types or check out our -[guides to logging](https://docs.wandb.ai/guides/track/log) for examples, +[guides to logging](https://docs.wandb.ai/guides/models/track/log) for examples, from 3D molecular structures and segmentation masks to PR curves and histograms. You can use `wandb.Table` to log structured data. See our -[guide to logging tables](https://docs.wandb.ai/guides/tables/tables-walkthrough) +[guide to logging tables](https://docs.wandb.ai/guides/core/tables/tables-walkthrough) for details. The W&B UI organizes metrics with a forward slash (`/`) in their name @@ -490,7 +490,7 @@ run.log({"accuracy": 0.9}, step=current_step) #### Examples: For more and more detailed examples, see -[our guides to logging](https://docs.wandb.com/guides/track/log). +[our guides to logging](https://docs.wandb.com/guides/models/track/log). ### Basic usage diff --git a/content/ref/python/sweep.md b/content/ref/python/sweep.md index 952b65210..c9a91d880 100644 --- a/content/ref/python/sweep.md +++ b/content/ref/python/sweep.md @@ -23,7 +23,7 @@ At a later step provide the `sweep_id` to a sweep agent. | Args | | | :--- | :--- | -| `sweep` | The configuration of a hyperparameter search. (or configuration generator). See [Sweep configuration structure](https://docs.wandb.ai/guides/sweeps/define-sweep-configuration) for information on how to define your sweep. If you provide a callable, ensure that the callable does not take arguments and that it returns a dictionary that conforms to the W&B sweep config spec. | +| `sweep` | The configuration of a hyperparameter search. (or configuration generator). See [Sweep configuration structure](https://docs.wandb.ai/guides/models/sweeps/define-sweep-configuration) for information on how to define your sweep. If you provide a callable, ensure that the callable does not take arguments and that it returns a dictionary that conforms to the W&B sweep config spec. | | `entity` | The username or team name where you want to send W&B runs created by the sweep to. Ensure that the entity you specify already exists. If you don't specify an entity, the run will be sent to your default entity, which is usually your username. | | `project` | The name of the project where W&B runs created from the sweep are sent to. If the project is not specified, the run is sent to a project labeled 'Uncategorized'. | | `prior_runs` | The run IDs of existing runs to add to this sweep. | diff --git a/content/ref/query-panel/_index.md b/content/ref/query-panel/_index.md index e3ecb8373..7f4994fc2 100644 --- a/content/ref/query-panel/_index.md +++ b/content/ref/query-panel/_index.md @@ -10,7 +10,7 @@ cascade: parent: qel --- Use the query expressions to select and aggregate data across runs and projects. -Learn more about Query Panels here: https://docs.wandb.ai/guides/app/features/panels/query-panel +Learn more about Query Panels here: https://docs.wandb.ai/guides/models/app/features/panels/query-panel ## Data Types diff --git a/content/support/_index.md b/content/support/_index.md index 2be706d57..b89b719b7 100644 --- a/content/support/_index.md +++ b/content/support/_index.md @@ -23,13 +23,13 @@ and the W&B community. Here are the most commonly asked questions across all categories. -* [What does `wandb.init` do to my training process?](./wandbinit_training_process.md) -* [How do I use custom CLI commands with sweeps?](./custom_cli_commands_sweeps.md) -* [Is it possible to save metrics offline and sync them to W&B later?](./same_metric_appearing_more.md) -* [How can I configure the name of the run in my training code?](./configure_name_run_training_code.md) +* [What does `wandb.init` do to my training process?]({{< relref "./wandbinit_training_process.md" >}}) +* [How do I use custom CLI commands with sweeps?]({{< relref "./custom_cli_commands_sweeps.md" >}}) +* [Is it possible to save metrics offline and sync them to W&B later?]({{< relref "./same_metric_appearing_more.md" >}}) +* [How can I configure the name of the run in my training code?]({{< relref "./configure_name_run_training_code.md" >}}) -If you can't find what you are looking for, browse through the [popular categories](#popular-categories) below or search through articles based on categories. +If you can't find what you are looking for, browse through the [popular categories]({{< relref "#popular-categories" >}}) below or search through articles based on categories. ## Popular categories diff --git a/content/support/access_artifacts.md b/content/support/access_artifacts.md index 2b582ebf8..b38385083 100644 --- a/content/support/access_artifacts.md +++ b/content/support/access_artifacts.md @@ -13,4 +13,4 @@ Artifacts inherit access permissions from their parent project: ## Artifacts Workflows -This section outlines workflows for managing and editing Artifacts. Many workflows utilize [the W&B API](../guides/track/public-api-guide.md), a component of [the client library](../ref/python/README.md) that provides access to W&B-stored data. \ No newline at end of file +This section outlines workflows for managing and editing Artifacts. Many workflows utilize [the W&B API]({{< relref "/guides/models/track/public-api-guide.md" >}}), a component of [the client library]({{< relref "/ref/python/" >}}) that provides access to W&B-stored data. \ No newline at end of file diff --git a/content/support/admin_local_instance_manage.md b/content/support/admin_local_instance_manage.md index bd3ccc339..8a07231b7 100644 --- a/content/support/admin_local_instance_manage.md +++ b/content/support/admin_local_instance_manage.md @@ -5,4 +5,4 @@ type: docs tags: - administrator --- -If you are the admin for your instance, review the [User Management](../guides/hosting/iam/manage-organization.md) section for instructions on adding users and creating teams. \ No newline at end of file +If you are the admin for your instance, review the [User Management]({{< relref "/guides/hosting/iam/access-management/manage-organization.md" >}}) section for instructions on adding users and creating teams. \ No newline at end of file diff --git a/content/support/best_log_models_runs_sweep.md b/content/support/best_log_models_runs_sweep.md index 651c40427..10479b1a6 100644 --- a/content/support/best_log_models_runs_sweep.md +++ b/content/support/best_log_models_runs_sweep.md @@ -6,7 +6,7 @@ tags: - artifacts - sweeps --- -One effective approach for logging models in a [sweep](../guides/sweeps/intro.md) involves creating a model artifact for the sweep. Each version represents a different run from the sweep. Implement it as follows: +One effective approach for logging models in a [sweep]({{< relref "/guides/models/sweeps/" >}}) involves creating a model artifact for the sweep. Each version represents a different run from the sweep. Implement it as follows: ```python wandb.Artifact(name="sweep_name", type="model") diff --git a/content/support/best_practices_organize_hyperparameter_searches.md b/content/support/best_practices_organize_hyperparameter_searches.md index 44216899b..f97ed738e 100644 --- a/content/support/best_practices_organize_hyperparameter_searches.md +++ b/content/support/best_practices_organize_hyperparameter_searches.md @@ -10,4 +10,4 @@ tags: Set unique tags with `wandb.init(tags='your_tag')`. This allows efficient filtering of project runs by selecting the corresponding tag in a Project Page's Runs Table. -For more information on wandb.int, see the [documentation](../ref/python/init.md). \ No newline at end of file +For more information on wandb.int, see the [documentation]({{< relref "/ref/python/init.md" >}}). \ No newline at end of file diff --git a/content/support/create_team_add_delete_people_team.md b/content/support/create_team_add_delete_people_team.md index d5297982d..21578bfeb 100644 --- a/content/support/create_team_add_delete_people_team.md +++ b/content/support/create_team_add_delete_people_team.md @@ -5,4 +5,4 @@ type: docs tags: - team management --- -Refer to the link for details on roles and permissions: [Team Roles and Permissions](../guides/app/features/teams.md#team-roles-and-permissions). \ No newline at end of file +Refer to the link for details on roles and permissions: [Team Roles and Permissions]({{< relref "/guides/models/app/settings-page/teams.md#team-roles-and-permissions" >}}). \ No newline at end of file diff --git a/content/support/deal_network_issues.md b/content/support/deal_network_issues.md index aa70223b9..3b89ea1ac 100644 --- a/content/support/deal_network_issues.md +++ b/content/support/deal_network_issues.md @@ -8,8 +8,8 @@ tags: If you encounter SSL or network errors, such as `wandb: Network error (ConnectionError), entering retry loop`, use the following solutions: 1. Upgrade the SSL certificate. On an Ubuntu server, run `update-ca-certificates`. A valid SSL certificate is essential for syncing training logs to mitigate security risks. -2. If the network connection is unstable, operate in offline mode by setting the [optional environment variable](../guides/track/environment-variables.md#optional-environment-variables) `WANDB_MODE` to `offline`, and sync files later from a device with Internet access. -3. Consider using [W&B Private Hosting](../guides/hosting/intro.md), which runs locally and avoids syncing to cloud servers. +2. If the network connection is unstable, operate in offline mode by setting the [optional environment variable]({{< relref "/guides/models/track/environment-variables.md#optional-environment-variables" >}}) `WANDB_MODE` to `offline`, and sync files later from a device with Internet access. +3. Consider using [W&B Private Hosting]({{< relref "/guides/hosting/" >}}), which runs locally and avoids syncing to cloud servers. For the `SSL CERTIFICATE_VERIFY_FAILED` error, this issue might stem from a company firewall. Configure local CAs and execute: diff --git a/content/support/difference_wandbinit_modes.md b/content/support/difference_wandbinit_modes.md index ec8947b9d..7b06e3a35 100644 --- a/content/support/difference_wandbinit_modes.md +++ b/content/support/difference_wandbinit_modes.md @@ -8,5 +8,5 @@ tags: These modes are available: * `online` (default): The client sends data to the wandb server. -* `offline`: The client stores data locally on the machine instead of sending it to the wandb server. Use the [`wandb sync`](../ref/cli/wandb-sync.md) command to synchronize the data later. +* `offline`: The client stores data locally on the machine instead of sending it to the wandb server. Use the [`wandb sync`]({{< relref "/ref/cli/wandb-sync.md" >}}) command to synchronize the data later. * `disabled`: The client simulates operation by returning mocked objects and prevents any network communication. All logging is turned off, but all API method stubs remain callable. This mode is typically used for testing. \ No newline at end of file diff --git a/content/support/enable_code_logging_sweeps.md b/content/support/enable_code_logging_sweeps.md index 8ee655b92..94d39cbc5 100644 --- a/content/support/enable_code_logging_sweeps.md +++ b/content/support/enable_code_logging_sweeps.md @@ -5,4 +5,4 @@ type: docs tags: - sweeps --- -To enable code logging for sweeps, add `wandb.log_code()` after initializing the W&B Run. This action is necessary even when code logging is enabled in the W&B profile settings. For advanced code logging, refer to the [docs for `wandb.log_code()` here](../ref/python/run.md#log_code). \ No newline at end of file +To enable code logging for sweeps, add `wandb.log_code()` after initializing the W&B Run. This action is necessary even when code logging is enabled in the W&B profile settings. For advanced code logging, refer to the [docs for `wandb.log_code()` here]({{< relref "/ref/python/run.md#log_code" >}}). \ No newline at end of file diff --git a/content/support/group_runs_tags.md b/content/support/group_runs_tags.md index 171f5c134..9621f0d8c 100644 --- a/content/support/group_runs_tags.md +++ b/content/support/group_runs_tags.md @@ -5,4 +5,4 @@ type: docs tags: - runs --- -A run can have multiple tags, so grouping by tags is not supported. Add a value to the [`config`](../guides/track/config.md) object for these runs and group by this config value instead. This can be accomplished using [the API](../guides/track/config.md#set-the-configuration-after-your-run-has-finished). \ No newline at end of file +A run can have multiple tags, so grouping by tags is not supported. Add a value to the [`config`]({{< relref "/guides/models/track/config.md" >}}) object for these runs and group by this config value instead. This can be accomplished using [the API]({{< relref "/guides/models/track/config.md#set-the-configuration-after-your-run-has-finished" >}}). \ No newline at end of file diff --git a/content/support/index_academic.md b/content/support/index_academic.md index 9d37f4967..95afe41e9 100644 --- a/content/support/index_academic.md +++ b/content/support/index_academic.md @@ -13,5 +13,5 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can I get an academic plan as a student?](academic_plan_student.md) -- [How can I change my account from corporate to academic?](change_account_from_corporate_academic.md) \ No newline at end of file +- [Can I get an academic plan as a student?]({{< relref "academic_plan_student.md" >}}) +- [How can I change my account from corporate to academic?]({{< relref "change_account_from_corporate_academic.md" >}}) \ No newline at end of file diff --git a/content/support/index_administrator.md b/content/support/index_administrator.md index 77a6cca98..4940c3970 100644 --- a/content/support/index_administrator.md +++ b/content/support/index_administrator.md @@ -13,25 +13,25 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can I get an academic plan as a student?](academic_plan_student.md) -- [Can I just log metrics, no code or dataset examples?](just_log_metrics_no_code_dataset_examples.md) -- [How can I change my account from corporate to academic?](change_account_from_corporate_academic.md) -- [How can I remove projects from a team space without admin privileges?](remove_projects_from_team_space_without_admin_privileges.md) -- [How can I rotate or revoke access?](rotate_revoke_access.md) -- [How can I see the bytes stored, bytes tracked and tracked hours of my organization?](see_bytes_stored_bytes_tracked_tracked_hours_organization.md) -- [How do I cancel my subscription?](cancel_subscription.md) -- [How do I change my billing address?](change_billing_address.md) -- [How do I delete a team from my account?](delete_team_from_account.md) -- [How do I delete my organization account?](delete_organization_account.md) -- [How do I downgrade my subscription plan?](downgrade_subscription_plan.md) -- [How do I export a list of users from my W&B Organisation?](export_list_users_account.md) -- [How do I get added to a team on W&B?](join_team.md) -- [How do I renew my expired license?](renew_expired_license.md) -- [If I am the admin of my local instance, how should I manage it?](admin_local_instance_manage.md) -- [Is it possible to add the same service account to multiple teams?](add_same_service_account_multiple_teams.md) -- [Is it possible to change the username?](change_username.md) -- [Is there a monthly subscription option for the teams plan?](monthly_subscription_option_teams_plan.md) -- [Is there a way to add more seats?](add_more_seats.md) -- [On a local instance, which files should I check when I have issues?](local_instance_files_check_issues.md) -- [What is a service account, and why is it useful?](service_account_useful.md) -- [What is the difference between team and organization?](difference_team_organization.md) \ No newline at end of file +- [Can I get an academic plan as a student?]({{< relref "academic_plan_student.md" >}}) +- [Can I just log metrics, no code or dataset examples?]({{< relref "just_log_metrics_no_code_dataset_examples.md" >}}) +- [How can I change my account from corporate to academic?]({{< relref "change_account_from_corporate_academic.md" >}}) +- [How can I remove projects from a team space without admin privileges?]({{< relref "remove_projects_from_team_space_without_admin_privileges.md" >}}) +- [How can I rotate or revoke access?]({{< relref "rotate_revoke_access.md" >}}) +- [How can I see the bytes stored, bytes tracked and tracked hours of my organization?]({{< relref "see_bytes_stored_bytes_tracked_tracked_hours_organization.md" >}}) +- [How do I cancel my subscription?]({{< relref "cancel_subscription.md" >}}) +- [How do I change my billing address?]({{< relref "change_billing_address.md" >}}) +- [How do I delete a team from my account?]({{< relref "delete_team_from_account.md" >}}) +- [How do I delete my organization account?]({{< relref "delete_organization_account.md" >}}) +- [How do I downgrade my subscription plan?]({{< relref "downgrade_subscription_plan.md" >}}) +- [How do I export a list of users from my W&B Organisation?]({{< relref "export_list_users_account.md" >}}) +- [How do I get added to a team on W&B?]({{< relref "join_team.md" >}}) +- [How do I renew my expired license?]({{< relref "renew_expired_license.md" >}}) +- [If I am the admin of my local instance, how should I manage it?]({{< relref "admin_local_instance_manage.md" >}}) +- [Is it possible to add the same service account to multiple teams?]({{< relref "add_same_service_account_multiple_teams.md" >}}) +- [Is it possible to change the username?]({{< relref "change_username.md" >}}) +- [Is there a monthly subscription option for the teams plan?]({{< relref "monthly_subscription_option_teams_plan.md" >}}) +- [Is there a way to add more seats?]({{< relref "add_more_seats.md" >}}) +- [On a local instance, which files should I check when I have issues?]({{< relref "local_instance_files_check_issues.md" >}}) +- [What is a service account, and why is it useful?]({{< relref "service_account_useful.md" >}}) +- [What is the difference between team and organization?]({{< relref "difference_team_organization.md" >}}) \ No newline at end of file diff --git a/content/support/index_alerts.md b/content/support/index_alerts.md index 05ffc6129..640042223 100644 --- a/content/support/index_alerts.md +++ b/content/support/index_alerts.md @@ -13,5 +13,5 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Do "Run Finished" alerts work in notebooks?](run_finished_alerts.md) -- [How can I send run alerts to Microsoft Teams?](send_alerts_microsoft_teams_using_wb.md) \ No newline at end of file +- [Do "Run Finished" alerts work in notebooks?]({{< relref "run_finished_alerts.md" >}}) +- [How can I send run alerts to Microsoft Teams?]({{< relref "send_alerts_microsoft_teams_using_wb.md" >}}) \ No newline at end of file diff --git a/content/support/index_anonymous.md b/content/support/index_anonymous.md index ef799f703..2467734a7 100644 --- a/content/support/index_anonymous.md +++ b/content/support/index_anonymous.md @@ -13,5 +13,5 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How does someone without an account see run results?](run_results_anonymous_mode.md) -- [What are features that are not available to anonymous users?](anon_users_unavailable_features.md) \ No newline at end of file +- [How does someone without an account see run results?]({{< relref "run_results_anonymous_mode.md" >}}) +- [What are features that are not available to anonymous users?]({{< relref "anon_users_unavailable_features.md" >}}) \ No newline at end of file diff --git a/content/support/index_artifacts.md b/content/support/index_artifacts.md index 352802817..e3b32071d 100644 --- a/content/support/index_artifacts.md +++ b/content/support/index_artifacts.md @@ -13,15 +13,15 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can I turn off wandb when testing my code?](testing_code_turn_off.md) -- [How can I fetch these Version IDs and ETags in W&B?](fetch_version_ids_etags_wb.md) -- [How can I find the artifacts logged or consumed by a run? How can I find the runs that produced or consumed an artifact?](find_artifacts_logged_consumed_run_find.md) -- [How do I best log models from runs in a sweep?](best_log_models_runs_sweep.md) -- [How do I find an artifact from the best run in a sweep?](find_artifact_best_run_sweep.md) -- [How do I log an artifact to an existing run?](log_artifact_existing_run.md) -- [How do I save code?‌](save_code‌.md) -- [How do I set a retention or expiration policy on my artifact?](retention_expiration_policy_artifact.md) -- [How much storage does each artifact version use?](artifact_storage_version.md) -- [Is it possible to recover an artifact after it has been deleted with a run?](recover_artifact_after_deleted.md) -- [Using artifacts with multiple architectures and runs?](artifacts_multiple_architectures_runs.md) -- [Who has access to my artifacts?](access_artifacts.md) \ No newline at end of file +- [Can I turn off wandb when testing my code?]({{< relref "testing_code_turn_off.md" >}}) +- [How can I fetch these Version IDs and ETags in W&B?]({{< relref "fetch_version_ids_etags_wb.md" >}}) +- [How can I find the artifacts logged or consumed by a run? How can I find the runs that produced or consumed an artifact?]({{< relref "find_artifacts_logged_consumed_run_find.md" >}}) +- [How do I best log models from runs in a sweep?]({{< relref "best_log_models_runs_sweep.md" >}}) +- [How do I find an artifact from the best run in a sweep?]({{< relref "find_artifact_best_run_sweep.md" >}}) +- [How do I log an artifact to an existing run?]({{< relref "log_artifact_existing_run.md" >}}) +- [How do I save code?‌]({{< relref "save_code‌.md" >}}) +- [How do I set a retention or expiration policy on my artifact?]({{< relref "retention_expiration_policy_artifact.md" >}}) +- [How much storage does each artifact version use?]({{< relref "artifact_storage_version.md" >}}) +- [Is it possible to recover an artifact after it has been deleted with a run?]({{< relref "recover_artifact_after_deleted.md" >}}) +- [Using artifacts with multiple architectures and runs?]({{< relref "artifacts_multiple_architectures_runs.md" >}}) +- [Who has access to my artifacts?]({{< relref "access_artifacts.md" >}}) \ No newline at end of file diff --git a/content/support/index_aws.md b/content/support/index_aws.md index da97709d1..e2f8760f7 100644 --- a/content/support/index_aws.md +++ b/content/support/index_aws.md @@ -13,5 +13,5 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can I use Sweeps and SageMaker?](sweeps_sagemaker.md) -- [Can you use W&B Sweeps with cloud infrastructures such as AWS Batch, ECS, etc.?](sweeps_cloud_infrastructures_such_aws_batch_ecs.md) \ No newline at end of file +- [Can I use Sweeps and SageMaker?]({{< relref "sweeps_sagemaker.md" >}}) +- [Can you use W&B Sweeps with cloud infrastructures such as AWS Batch, ECS, etc.?]({{< relref "sweeps_cloud_infrastructures_such_aws_batch_ecs.md" >}}) \ No newline at end of file diff --git a/content/support/index_billing.md b/content/support/index_billing.md index 82e14612a..22ea60671 100644 --- a/content/support/index_billing.md +++ b/content/support/index_billing.md @@ -13,7 +13,7 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How do I change my billing address?](change_billing_address.md) -- [How do I downgrade my subscription plan?](downgrade_subscription_plan.md) -- [How do we update our payment method?](update_payment_method.md) -- [Is there a monthly subscription option for the teams plan?](monthly_subscription_option_teams_plan.md) \ No newline at end of file +- [How do I change my billing address?]({{< relref "change_billing_address.md" >}}) +- [How do I downgrade my subscription plan?]({{< relref "downgrade_subscription_plan.md" >}}) +- [How do we update our payment method?]({{< relref "update_payment_method.md" >}}) +- [Is there a monthly subscription option for the teams plan?]({{< relref "monthly_subscription_option_teams_plan.md" >}}) \ No newline at end of file diff --git a/content/support/index_charts.md b/content/support/index_charts.md index 99c29b44e..2e736790f 100644 --- a/content/support/index_charts.md +++ b/content/support/index_charts.md @@ -13,7 +13,7 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How do I add Plotly or Bokeh Charts into Tables?](add_plotlybokeh_charts_tables.md) -- [How do you delete a custom chart preset?](delete_custom_chart_preset.md) -- [How do you show a "step slider" in a custom chart?](show_step_slider_custom_chart.md) -- [What is the difference between `.log()` and `.summary`?](difference_log_summary.md) \ No newline at end of file +- [How do I add Plotly or Bokeh Charts into Tables?]({{< relref "add_plotlybokeh_charts_tables.md" >}}) +- [How do you delete a custom chart preset?]({{< relref "delete_custom_chart_preset.md" >}}) +- [How do you show a "step slider" in a custom chart?]({{< relref "show_step_slider_custom_chart.md" >}}) +- [What is the difference between `.log()` and `.summary`?]({{< relref "difference_log_summary.md" >}}) \ No newline at end of file diff --git a/content/support/index_connectivity.md b/content/support/index_connectivity.md index 62073ab3b..7cfd39da4 100644 --- a/content/support/index_connectivity.md +++ b/content/support/index_connectivity.md @@ -13,7 +13,7 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How can I resolve the Filestream rate limit exceeded error?](filestream_rate_limit_exceeded_error.md) -- [How do I deal with network issues?](deal_network_issues.md) -- [How do I resolve a run initialization timeout error in wandb?](initialization_timeout_error.md) -- [Is there a W&B outage?](wandb_status.md) \ No newline at end of file +- [How can I resolve the Filestream rate limit exceeded error?]({{< relref "filestream_rate_limit_exceeded_error.md" >}}) +- [How do I deal with network issues?]({{< relref "deal_network_issues.md" >}}) +- [How do I resolve a run initialization timeout error in wandb?]({{< relref "initialization_timeout_error.md" >}}) +- [Is there a W&B outage?]({{< relref "wandb_status.md" >}}) \ No newline at end of file diff --git a/content/support/index_crashing and hanging runs.md b/content/support/index_crashing and hanging runs.md index d922be78c..48cae684e 100644 --- a/content/support/index_crashing and hanging runs.md +++ b/content/support/index_crashing and hanging runs.md @@ -13,7 +13,7 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How do I kill a job with wandb?](kill_job_wandb.md) -- [How do I resolve a run initialization timeout error in wandb?](initialization_timeout_error.md) -- [If wandb crashes, will it possibly crash my training run?](crashes_crash_training_run.md) -- [Why is a run marked crashed in W&B when it’s training fine locally?](run_marked_crashed_wb’s_training_fine_locally.md) \ No newline at end of file +- [How do I kill a job with wandb?]({{< relref "kill_job_wandb.md" >}}) +- [How do I resolve a run initialization timeout error in wandb?]({{< relref "initialization_timeout_error.md" >}}) +- [If wandb crashes, will it possibly crash my training run?]({{< relref "crashes_crash_training_run.md" >}}) +- [Why is a run marked crashed in W&B when it’s training fine locally?]({{< relref "run_marked_crashed_wb’s_training_fine_locally.md" >}}) \ No newline at end of file diff --git a/content/support/index_environment variables.md b/content/support/index_environment variables.md index 1f26cc874..81bb7cefa 100644 --- a/content/support/index_environment variables.md +++ b/content/support/index_environment variables.md @@ -13,12 +13,12 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Do environment variables overwrite the parameters passed to wandb.init()?](environment_variables_overwrite_parameters.md) -- [How do I handle the 'Failed to query for notebook' error?](query_notebook_failed.md) -- [How do I silence W&B info messages?](silence_info_messages.md) -- [How do I stop wandb from writing to my terminal or my jupyter notebook output?](stop_wandb_writing_terminal_jupyter_notebook_output.md) -- [How do I switch between accounts on the same machine?](switch_accounts_same_machine.md) -- [How does wandb stream logs and writes to disk?](stream_logs_writes_disk.md) -- [Is it possible to save metrics offline and sync them to W&B later?](save_metrics_offline_sync_them_wb_later.md) -- [What does wandb.init do to my training process?](wandbinit_training_process.md) -- [What happens if internet connection is lost while I'm training a model?](internet_connection_lost_while_im_training_model.md) \ No newline at end of file +- [Do environment variables overwrite the parameters passed to wandb.init()?]({{< relref "environment_variables_overwrite_parameters.md" >}}) +- [How do I handle the 'Failed to query for notebook' error?]({{< relref "query_notebook_failed.md" >}}) +- [How do I silence W&B info messages?]({{< relref "silence_info_messages.md" >}}) +- [How do I stop wandb from writing to my terminal or my jupyter notebook output?]({{< relref "stop_wandb_writing_terminal_jupyter_notebook_output.md" >}}) +- [How do I switch between accounts on the same machine?]({{< relref "switch_accounts_same_machine.md" >}}) +- [How does wandb stream logs and writes to disk?]({{< relref "stream_logs_writes_disk.md" >}}) +- [Is it possible to save metrics offline and sync them to W&B later?]({{< relref "save_metrics_offline_sync_them_wb_later.md" >}}) +- [What does wandb.init do to my training process?]({{< relref "wandbinit_training_process.md" >}}) +- [What happens if internet connection is lost while I'm training a model?]({{< relref "internet_connection_lost_while_im_training_model.md" >}}) \ No newline at end of file diff --git a/content/support/index_experiments.md b/content/support/index_experiments.md index efb0a0b31..63875d573 100644 --- a/content/support/index_experiments.md +++ b/content/support/index_experiments.md @@ -13,35 +13,35 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can I just set the run name to the run ID?](just_set_run_name_run_id.md) -- [Can I log metrics on two different time scales?](log_metrics_two_different_time_scales_example_log_training.md) -- [Can I run wandb offline?](run_wandb_offline.md) -- [Does W&B use the `multiprocessing` library?](multiprocessing_library.md) -- [Does logging block my training?](logging_block_training.md) -- [Does your tool track or store training data?](tool_track_store_training_data.md) -- [How can I access the data logged to my runs directly and programmatically?](access_data_logged_runs_directly_programmatically.md) -- [How can I compare images or media across epochs or steps?](compare_images_media_across_epochs_steps.md) -- [How can I configure the name of the run in my training code?](configure_name_run_training_code.md) -- [How can I organize my logged charts and media in the W&B UI?](organize_logged_charts_media_wb_ui.md) -- [How can I save the git commit associated with my run?](save_git_commit_associated_run.md) -- [How can I use wandb with multiprocessing, such as in distributed training?](multiprocessing_eg_distributed_training.md) -- [How do I add Plotly or Bokeh Charts into Tables?](add_plotlybokeh_charts_tables.md) -- [How do I get the random run name in my script?](random_run_name_script.md) -- [How do I launch multiple runs from one script?](launch_multiple_runs_one_script.md) -- [How do I log a list of values?](log_list_values.md) -- [How do I plot multiple lines on a plot with a legend?](plot_multiple_lines_plot_legend.md) -- [How do I programmatically access the human-readable run name?](programmatically_access_humanreadable_run_name.md) -- [I didn't name my run. Where is the run name coming from?](didnt_name_run_run_name_coming.md) -- [InitStartError: Error communicating with wandb process](initstarterror_error_communicating_wandb_process.md) -- [Is it possible to save metrics offline and sync them to W&B later?](save_metrics_offline_sync_them_wb_later.md) -- [My run's state is `crashed` on the UI but is still running on my machine. What do I do to get my data back?](runs_state_crashed_ui_running_machine_get_data.md) -- [What does wandb.init do to my training process?](wandbinit_training_process.md) -- [What happens if I pass a class attribute into wandb.log()?](pass_class_attribute_wandblog.md) -- [What happens when I log millions of steps to W&B? How is that rendered in the browser?](log_millions_steps_wb_rendered_browser.md) -- [What if I want to integrate W&B into my project, but I don't want to upload any images or media?](integrate_wb_project_dont_upload_any_images_media.md) -- [What if I want to log some metrics on batches and some metrics only on epochs?](log_metrics_batches_some_metrics_epochs.md) -- [What is the difference between wandb.init modes?](difference_wandbinit_modes.md) -- [Why am I seeing fewer data points than I logged?](seeing_fewer_data_points_logged.md) -- [Why is nothing showing up in my graphs?](graphs_nothing_showing.md) -- [Why is the same metric appearing more than once?](same_metric_appearing_more.md) -- [Will wandb slow down my training?](slow_training.md) \ No newline at end of file +- [Can I just set the run name to the run ID?]({{< relref "just_set_run_name_run_id.md" >}}) +- [Can I log metrics on two different time scales?]({{< relref "log_metrics_two_different_time_scales_example_log_training.md" >}}) +- [Can I run wandb offline?]({{< relref "run_wandb_offline.md" >}}) +- [Does W&B use the `multiprocessing` library?]({{< relref "multiprocessing_library.md" >}}) +- [Does logging block my training?]({{< relref "logging_block_training.md" >}}) +- [Does your tool track or store training data?]({{< relref "tool_track_store_training_data.md" >}}) +- [How can I access the data logged to my runs directly and programmatically?]({{< relref "access_data_logged_runs_directly_programmatically.md" >}}) +- [How can I compare images or media across epochs or steps?]({{< relref "compare_images_media_across_epochs_steps.md" >}}) +- [How can I configure the name of the run in my training code?]({{< relref "configure_name_run_training_code.md" >}}) +- [How can I organize my logged charts and media in the W&B UI?]({{< relref "organize_logged_charts_media_wb_ui.md" >}}) +- [How can I save the git commit associated with my run?]({{< relref "save_git_commit_associated_run.md" >}}) +- [How can I use wandb with multiprocessing, such as in distributed training?]({{< relref "multiprocessing_eg_distributed_training.md" >}}) +- [How do I add Plotly or Bokeh Charts into Tables?]({{< relref "add_plotlybokeh_charts_tables.md" >}}) +- [How do I get the random run name in my script?]({{< relref "random_run_name_script.md" >}}) +- [How do I launch multiple runs from one script?]({{< relref "launch_multiple_runs_one_script.md" >}}) +- [How do I log a list of values?]({{< relref "log_list_values.md" >}}) +- [How do I plot multiple lines on a plot with a legend?]({{< relref "plot_multiple_lines_plot_legend.md" >}}) +- [How do I programmatically access the human-readable run name?]({{< relref "programmatically_access_humanreadable_run_name.md" >}}) +- [I didn't name my run. Where is the run name coming from?]({{< relref "didnt_name_run_run_name_coming.md" >}}) +- [InitStartError: Error communicating with wandb process]({{< relref "initstarterror_error_communicating_wandb_process.md" >}}) +- [Is it possible to save metrics offline and sync them to W&B later?]({{< relref "save_metrics_offline_sync_them_wb_later.md" >}}) +- [My run's state is `crashed` on the UI but is still running on my machine. What do I do to get my data back?]({{< relref "runs_state_crashed_ui_running_machine_get_data.md" >}}) +- [What does wandb.init do to my training process?]({{< relref "wandbinit_training_process.md" >}}) +- [What happens if I pass a class attribute into wandb.log()?]({{< relref "pass_class_attribute_wandblog.md" >}}) +- [What happens when I log millions of steps to W&B? How is that rendered in the browser?]({{< relref "log_millions_steps_wb_rendered_browser.md" >}}) +- [What if I want to integrate W&B into my project, but I don't want to upload any images or media?]({{< relref "integrate_wb_project_dont_upload_any_images_media.md" >}}) +- [What if I want to log some metrics on batches and some metrics only on epochs?]({{< relref "log_metrics_batches_some_metrics_epochs.md" >}}) +- [What is the difference between wandb.init modes?]({{< relref "difference_wandbinit_modes.md" >}}) +- [Why am I seeing fewer data points than I logged?]({{< relref "seeing_fewer_data_points_logged.md" >}}) +- [Why is nothing showing up in my graphs?]({{< relref "graphs_nothing_showing.md" >}}) +- [Why is the same metric appearing more than once?]({{< relref "same_metric_appearing_more.md" >}}) +- [Will wandb slow down my training?]({{< relref "slow_training.md" >}}) \ No newline at end of file diff --git a/content/support/index_hyperparameter.md b/content/support/index_hyperparameter.md index 9fd1e142b..9ec0cef49 100644 --- a/content/support/index_hyperparameter.md +++ b/content/support/index_hyperparameter.md @@ -13,6 +13,6 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Best practices to organize hyperparameter searches](best_practices_organize_hyperparameter_searches.md) -- [Can I rerun a grid search?](rerun_grid_search.md) -- [What is the `Est. Runs` column?](est_runs_column.md) \ No newline at end of file +- [Best practices to organize hyperparameter searches]({{< relref "best_practices_organize_hyperparameter_searches.md" >}}) +- [Can I rerun a grid search?]({{< relref "rerun_grid_search.md" >}}) +- [What is the `Est. Runs` column?]({{< relref "est_runs_column.md" >}}) \ No newline at end of file diff --git a/content/support/index_logs.md b/content/support/index_logs.md index 13ef0b535..32d37f8d8 100644 --- a/content/support/index_logs.md +++ b/content/support/index_logs.md @@ -13,7 +13,7 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How do I log runs launched by continuous integration or internal tools?](log_automated_runs_service_account.md) -- [How do I log to the right wandb user on a shared machine?](log_shared_machine.md) -- [How do I turn off logging?](logging_turn_off.md) -- [Which files should I check when my code crashes?](files_check_code_crashes.md) \ No newline at end of file +- [How do I log runs launched by continuous integration or internal tools?]({{< relref "log_automated_runs_service_account.md" >}}) +- [How do I log to the right wandb user on a shared machine?]({{< relref "log_shared_machine.md" >}}) +- [How do I turn off logging?]({{< relref "logging_turn_off.md" >}}) +- [Which files should I check when my code crashes?]({{< relref "files_check_code_crashes.md" >}}) \ No newline at end of file diff --git a/content/support/index_metrics.md b/content/support/index_metrics.md index e2e50626b..99bf6b0f4 100644 --- a/content/support/index_metrics.md +++ b/content/support/index_metrics.md @@ -13,12 +13,12 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can I just log metrics, no code or dataset examples?](just_log_metrics_no_code_dataset_examples.md) -- [Can I log metrics on two different time scales?](log_metrics_two_different_time_scales_example_log_training.md) -- [How can I log additional metrics after a run completes?](log_additional_metrics_run_completes.md) -- [How often are system metrics collected?](system_metrics_collected.md) -- [Is it possible to plot the max of a metric rather than plot step by step?](plot_max_metric.md) -- [Is it possible to save metrics offline and sync them to W&B later?](save_metrics_offline_sync_them_wb_later.md) -- [Optimizing multiple metrics](optimizing_multiple_metrics.md) -- [What if I want to log some metrics on batches and some metrics only on epochs?](log_metrics_batches_some_metrics_epochs.md) -- [Why am I seeing fewer data points than I logged?](seeing_fewer_data_points_logged.md) \ No newline at end of file +- [Can I just log metrics, no code or dataset examples?]({{< relref "just_log_metrics_no_code_dataset_examples.md" >}}) +- [Can I log metrics on two different time scales?]({{< relref "log_metrics_two_different_time_scales_example_log_training.md" >}}) +- [How can I log additional metrics after a run completes?]({{< relref "log_additional_metrics_run_completes.md" >}}) +- [How often are system metrics collected?]({{< relref "system_metrics_collected.md" >}}) +- [Is it possible to plot the max of a metric rather than plot step by step?]({{< relref "plot_max_metric.md" >}}) +- [Is it possible to save metrics offline and sync them to W&B later?]({{< relref "save_metrics_offline_sync_them_wb_later.md" >}}) +- [Optimizing multiple metrics]({{< relref "optimizing_multiple_metrics.md" >}}) +- [What if I want to log some metrics on batches and some metrics only on epochs?]({{< relref "log_metrics_batches_some_metrics_epochs.md" >}}) +- [Why am I seeing fewer data points than I logged?]({{< relref "seeing_fewer_data_points_logged.md" >}}) \ No newline at end of file diff --git a/content/support/index_notebooks.md b/content/support/index_notebooks.md index cfea26bcd..c672c9161 100644 --- a/content/support/index_notebooks.md +++ b/content/support/index_notebooks.md @@ -13,6 +13,6 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Do "Run Finished" alerts work in notebooks?](run_finished_alerts.md) -- [How do I handle the 'Failed to query for notebook' error?](query_notebook_failed.md) -- [How do I silence W&B info messages?](silence_info_messages.md) \ No newline at end of file +- [Do "Run Finished" alerts work in notebooks?]({{< relref "run_finished_alerts.md" >}}) +- [How do I handle the 'Failed to query for notebook' error?]({{< relref "query_notebook_failed.md" >}}) +- [How do I silence W&B info messages?]({{< relref "silence_info_messages.md" >}}) \ No newline at end of file diff --git a/content/support/index_outage.md b/content/support/index_outage.md index 29dc2cd44..cc2fd1c1e 100644 --- a/content/support/index_outage.md +++ b/content/support/index_outage.md @@ -13,5 +13,5 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How can I resolve the Filestream rate limit exceeded error?](filestream_rate_limit_exceeded_error.md) -- [Is there a W&B outage?](wandb_status.md) \ No newline at end of file +- [How can I resolve the Filestream rate limit exceeded error?]({{< relref "filestream_rate_limit_exceeded_error.md" >}}) +- [Is there a W&B outage?]({{< relref "wandb_status.md" >}}) \ No newline at end of file diff --git a/content/support/index_privacy.md b/content/support/index_privacy.md index cc2ce3f1c..abfa9105a 100644 --- a/content/support/index_privacy.md +++ b/content/support/index_privacy.md @@ -13,5 +13,5 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can W&B team members see my data?](wandb_see_data.md) -- [How can I make my project public?](project_make_public.md) \ No newline at end of file +- [Can W&B team members see my data?]({{< relref "wandb_see_data.md" >}}) +- [How can I make my project public?]({{< relref "project_make_public.md" >}}) \ No newline at end of file diff --git a/content/support/index_projects.md b/content/support/index_projects.md index f2ae282c7..db5154f8a 100644 --- a/content/support/index_projects.md +++ b/content/support/index_projects.md @@ -13,6 +13,6 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How can I make my project public?](project_make_public.md) -- [How do I rename a project?](rename_project.md) -- [How many runs can I create per project?](runs_create_per_project.md) \ No newline at end of file +- [How can I make my project public?]({{< relref "project_make_public.md" >}}) +- [How do I rename a project?]({{< relref "rename_project.md" >}}) +- [How many runs can I create per project?]({{< relref "runs_create_per_project.md" >}}) \ No newline at end of file diff --git a/content/support/index_python.md b/content/support/index_python.md index 70ec4d467..162460373 100644 --- a/content/support/index_python.md +++ b/content/support/index_python.md @@ -13,9 +13,9 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Does the W&B client support Python 2?](client_support_python_2.md) -- [Does the W&B client support Python 3.5?](client_support_python_35.md) -- [Does this only work for Python?](work_python.md) -- [How can I resume a sweep using Python code?](resume_sweep_using_python_code.md) -- [How do I install the wandb Python library in environments without gcc?](install_wandb_python_library_environments_without_gcc.md) -- [Is there an anaconda package for Weights and Biases?](anaconda_package.md) \ No newline at end of file +- [Does the W&B client support Python 2?]({{< relref "client_support_python_2.md" >}}) +- [Does the W&B client support Python 3.5?]({{< relref "client_support_python_35.md" >}}) +- [Does this only work for Python?]({{< relref "work_python.md" >}}) +- [How can I resume a sweep using Python code?]({{< relref "resume_sweep_using_python_code.md" >}}) +- [How do I install the wandb Python library in environments without gcc?]({{< relref "install_wandb_python_library_environments_without_gcc.md" >}}) +- [Is there an anaconda package for Weights and Biases?]({{< relref "anaconda_package.md" >}}) \ No newline at end of file diff --git a/content/support/index_reports.md b/content/support/index_reports.md index 1052d527d..038eba8d6 100644 --- a/content/support/index_reports.md +++ b/content/support/index_reports.md @@ -13,18 +13,18 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Adding multiple authors to a report](adding_multiple_authors_report.md) -- [Can I use Markdown in my reports?](reports_markdown.md) -- [Embedding Reports](embedding_reports.md) -- [Filter and delete unwanted reports](filter_delete_unwanted_reports.md) -- [How do I delete a panel grid?](delete_panel_grid.md) -- [How do I insert a table?](insert_table.md) -- [How to get multiple charts with different selected runs?](multiple_charts_different_selected_runs.md) -- [I converted my report to WYSIWYG but want to revert back to Markdown](convert_from_wysiwyg_markdown.md) -- [Incorporating LaTeX](incorporating_latex.md) -- [My report is running slowly after the change to WYSIWYG](report_slow_after_wysiwyg.md) -- [My report looks different after converting from Markdown.](report_looks_different_converting_markdown.md) -- [Refreshing data](refreshing_data.md) -- [Upload a CSV to a report](upload_csv_report.md) -- [Upload an image to a report](upload_image_report.md) -- [Who can edit and share reports?](edit_share_reports.md) \ No newline at end of file +- [Adding multiple authors to a report]({{< relref "adding_multiple_authors_report.md" >}}) +- [Can I use Markdown in my reports?]({{< relref "reports_markdown.md" >}}) +- [Embedding Reports]({{< relref "embedding_reports.md" >}}) +- [Filter and delete unwanted reports]({{< relref "filter_delete_unwanted_reports.md" >}}) +- [How do I delete a panel grid?]({{< relref "delete_panel_grid.md" >}}) +- [How do I insert a table?]({{< relref "insert_table.md" >}}) +- [How to get multiple charts with different selected runs?]({{< relref "multiple_charts_different_selected_runs.md" >}}) +- [I converted my report to WYSIWYG but want to revert back to Markdown]({{< relref "convert_from_wysiwyg_markdown.md" >}}) +- [Incorporating LaTeX]({{< relref "incorporating_latex.md" >}}) +- [My report is running slowly after the change to WYSIWYG]({{< relref "report_slow_after_wysiwyg.md" >}}) +- [My report looks different after converting from Markdown.]({{< relref "report_looks_different_converting_markdown.md" >}}) +- [Refreshing data]({{< relref "refreshing_data.md" >}}) +- [Upload a CSV to a report]({{< relref "upload_csv_report.md" >}}) +- [Upload an image to a report]({{< relref "upload_image_report.md" >}}) +- [Who can edit and share reports?]({{< relref "edit_share_reports.md" >}}) \ No newline at end of file diff --git a/content/support/index_resuming.md b/content/support/index_resuming.md index 1de51777c..8ea3ce855 100644 --- a/content/support/index_resuming.md +++ b/content/support/index_resuming.md @@ -13,4 +13,4 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How do I use the resume parameter when resuming a run in W&B?](resume_parameter.md) \ No newline at end of file +- [How do I use the resume parameter when resuming a run in W&B?]({{< relref "resume_parameter.md" >}}) \ No newline at end of file diff --git a/content/support/index_runs.md b/content/support/index_runs.md index f66a7fe70..f6ddf28e3 100644 --- a/content/support/index_runs.md +++ b/content/support/index_runs.md @@ -13,16 +13,16 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Best practices to organize hyperparameter searches](best_practices_organize_hyperparameter_searches.md) -- [Can I group runs without using the "Group" feature?](group_runs_without_group_feature.md) -- [Can I rerun a grid search?](rerun_grid_search.md) -- [Can you group runs by tags?](group_runs_tags.md) -- [How can I change the colors of each run in the same group?](group_runs_custom_meter.md) -- [How can I log a metric that doesn't change over time such as a final evaluation accuracy?](log_metric_doesnt_change_time_such_final.md) -- [How can I log additional metrics after a run completes?](log_additional_metrics_run_completes.md) -- [How can I recover deleted runs?](recover_deleted_runs.md) -- [How do I log runs launched by continuous integration or internal tools?](log_automated_runs_service_account.md) -- [How do I resolve permission errors when logging a run?](resolve_permission_errors_when_logging_wandb_entity.md) -- [How many runs can I create per project?](runs_create_per_project.md) -- [Is it possible to change the group assigned to a run after completion?](change_group_after_completion.md) -- [Is it possible to move a run from one project to another?](move_from_project_another.md) \ No newline at end of file +- [Best practices to organize hyperparameter searches]({{< relref "best_practices_organize_hyperparameter_searches.md" >}}) +- [Can I group runs without using the "Group" feature?]({{< relref "group_runs_without_group_feature.md" >}}) +- [Can I rerun a grid search?]({{< relref "rerun_grid_search.md" >}}) +- [Can you group runs by tags?]({{< relref "group_runs_tags.md" >}}) +- [How can I change the colors of each run in the same group?]({{< relref "group_runs_custom_meter.md" >}}) +- [How can I log a metric that doesn't change over time such as a final evaluation accuracy?]({{< relref "log_metric_doesnt_change_time_such_final.md" >}}) +- [How can I log additional metrics after a run completes?]({{< relref "log_additional_metrics_run_completes.md" >}}) +- [How can I recover deleted runs?]({{< relref "recover_deleted_runs.md" >}}) +- [How do I log runs launched by continuous integration or internal tools?]({{< relref "log_automated_runs_service_account.md" >}}) +- [How do I resolve permission errors when logging a run?]({{< relref "resolve_permission_errors_when_logging_wandb_entity.md" >}}) +- [How many runs can I create per project?]({{< relref "runs_create_per_project.md" >}}) +- [Is it possible to change the group assigned to a run after completion?]({{< relref "change_group_after_completion.md" >}}) +- [Is it possible to move a run from one project to another?]({{< relref "move_from_project_another.md" >}}) \ No newline at end of file diff --git a/content/support/index_security.md b/content/support/index_security.md index 8cd69478b..27a88f58d 100644 --- a/content/support/index_security.md +++ b/content/support/index_security.md @@ -13,9 +13,9 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can W&B team members see my data?](wandb_see_data.md) -- [Do you have a bug bounty program?](bounty_program.md) -- [Does W&B support SSO for Multi-tenant?](sso_multitenant.md) -- [How can I rotate or revoke access?](rotate_revoke_access.md) -- [How do I find my API key?](find_api_key.md) -- [How do I resolve permission errors when logging a run?](resolve_permission_errors_when_logging_wandb_entity.md) \ No newline at end of file +- [Can W&B team members see my data?]({{< relref "wandb_see_data.md" >}}) +- [Do you have a bug bounty program?]({{< relref "bounty_program.md" >}}) +- [Does W&B support SSO for Multi-tenant?]({{< relref "sso_multitenant.md" >}}) +- [How can I rotate or revoke access?]({{< relref "rotate_revoke_access.md" >}}) +- [How do I find my API key?]({{< relref "find_api_key.md" >}}) +- [How do I resolve permission errors when logging a run?]({{< relref "resolve_permission_errors_when_logging_wandb_entity.md" >}}) \ No newline at end of file diff --git a/content/support/index_storage.md b/content/support/index_storage.md index 120c463ed..b6ff7f5f0 100644 --- a/content/support/index_storage.md +++ b/content/support/index_storage.md @@ -13,5 +13,5 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How much storage does each artifact version use?](artifact_storage_version.md) -- [Why does the storage meter not update after deleting runs?](storage_meter_update_after_deleting_runs.md) \ No newline at end of file +- [How much storage does each artifact version use?]({{< relref "artifact_storage_version.md" >}}) +- [Why does the storage meter not update after deleting runs?]({{< relref "storage_meter_update_after_deleting_runs.md" >}}) \ No newline at end of file diff --git a/content/support/index_sweeps.md b/content/support/index_sweeps.md index 7e80ae67b..4a04cf4d8 100644 --- a/content/support/index_sweeps.md +++ b/content/support/index_sweeps.md @@ -13,18 +13,18 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Best practices to organize hyperparameter searches](best_practices_organize_hyperparameter_searches.md) -- [Can I rerun a grid search?](rerun_grid_search.md) -- [Can I use Sweeps and SageMaker?](sweeps_sagemaker.md) -- [Can we flag boolean variables as hyperparameters?](flag_boolean_variables_hyperparameters.md) -- [Can you use W&B Sweeps with cloud infrastructures such as AWS Batch, ECS, etc.?](sweeps_cloud_infrastructures_such_aws_batch_ecs.md) -- [Do I need to provide values for all hyperparameters as part of the W&B Sweep. Can I set defaults?](need_provide_values_all_hyperparameters_part_wb_sweep_set.md) -- [How can I change the directory my sweep logs to locally?](change_directory_sweep_logs_locally.md) -- [How can I resume a sweep using Python code?](resume_sweep_using_python_code.md) -- [How do I best log models from runs in a sweep?](best_log_models_runs_sweep.md) -- [How do I enable code logging with Sweeps?](enable_code_logging_sweeps.md) -- [How do I use custom CLI commands with sweeps?](custom_cli_commands_sweeps.md) -- [How should I run sweeps on SLURM?](run_sweeps_slurm.md) -- [Is there a way to add extra values to a sweep, or do I need to start a new one?](add_extra_values_sweep_start_new_one.md) -- [Optimizing multiple metrics](optimizing_multiple_metrics.md) -- [What is the `Est. Runs` column?](est_runs_column.md) \ No newline at end of file +- [Best practices to organize hyperparameter searches]({{< relref "best_practices_organize_hyperparameter_searches.md" >}}) +- [Can I rerun a grid search?]({{< relref "rerun_grid_search.md" >}}) +- [Can I use Sweeps and SageMaker?]({{< relref "sweeps_sagemaker.md" >}}) +- [Can we flag boolean variables as hyperparameters?]({{< relref "flag_boolean_variables_hyperparameters.md" >}}) +- [Can you use W&B Sweeps with cloud infrastructures such as AWS Batch, ECS, etc.?]({{< relref "sweeps_cloud_infrastructures_such_aws_batch_ecs.md" >}}) +- [Do I need to provide values for all hyperparameters as part of the W&B Sweep. Can I set defaults?]({{< relref "need_provide_values_all_hyperparameters_part_wb_sweep_set.md" >}}) +- [How can I change the directory my sweep logs to locally?]({{< relref "change_directory_sweep_logs_locally.md" >}}) +- [How can I resume a sweep using Python code?]({{< relref "resume_sweep_using_python_code.md" >}}) +- [How do I best log models from runs in a sweep?]({{< relref "best_log_models_runs_sweep.md" >}}) +- [How do I enable code logging with Sweeps?]({{< relref "enable_code_logging_sweeps.md" >}}) +- [How do I use custom CLI commands with sweeps?]({{< relref "custom_cli_commands_sweeps.md" >}}) +- [How should I run sweeps on SLURM?]({{< relref "run_sweeps_slurm.md" >}}) +- [Is there a way to add extra values to a sweep, or do I need to start a new one?]({{< relref "add_extra_values_sweep_start_new_one.md" >}}) +- [Optimizing multiple metrics]({{< relref "optimizing_multiple_metrics.md" >}}) +- [What is the `Est. Runs` column?]({{< relref "est_runs_column.md" >}}) \ No newline at end of file diff --git a/content/support/index_tables.md b/content/support/index_tables.md index 7dac9cdc4..407a51cac 100644 --- a/content/support/index_tables.md +++ b/content/support/index_tables.md @@ -13,5 +13,5 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How do I add Plotly or Bokeh Charts into Tables?](add_plotlybokeh_charts_tables.md) -- [How do I insert a table?](insert_table.md) \ No newline at end of file +- [How do I add Plotly or Bokeh Charts into Tables?]({{< relref "add_plotlybokeh_charts_tables.md" >}}) +- [How do I insert a table?]({{< relref "insert_table.md" >}}) \ No newline at end of file diff --git a/content/support/index_team management.md b/content/support/index_team management.md index 5db4f7356..f7e0d98a2 100644 --- a/content/support/index_team management.md +++ b/content/support/index_team management.md @@ -13,14 +13,14 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can I just log metrics, no code or dataset examples?](just_log_metrics_no_code_dataset_examples.md) -- [How do I delete a team from my account?](delete_team_from_account.md) -- [How do I get added to a team on W&B?](join_team.md) -- [Is it possible to add the same service account to multiple teams?](add_same_service_account_multiple_teams.md) -- [Is there a monthly subscription option for the teams plan?](monthly_subscription_option_teams_plan.md) -- [What is a team and where can I find more information about it?](team_find_more_information.md) -- [What is the difference between team and entity? As a user - what does entity mean for me?](difference_team_entity_user_entity_mean_me.md) -- [What is the difference between team and organization?](difference_team_organization.md) -- [What type of roles are available and what are the differences between them?](type_roles_available_differences.md) -- [When should I log to my personal entity against my team entity?](log_personal_entity_team_entity.md) -- [Who can create a team? Who can add or delete people from a team? Who can delete projects?](create_team_add_delete_people_team.md) \ No newline at end of file +- [Can I just log metrics, no code or dataset examples?]({{< relref "just_log_metrics_no_code_dataset_examples.md" >}}) +- [How do I delete a team from my account?]({{< relref "delete_team_from_account.md" >}}) +- [How do I get added to a team on W&B?]({{< relref "join_team.md" >}}) +- [Is it possible to add the same service account to multiple teams?]({{< relref "add_same_service_account_multiple_teams.md" >}}) +- [Is there a monthly subscription option for the teams plan?]({{< relref "monthly_subscription_option_teams_plan.md" >}}) +- [What is a team and where can I find more information about it?]({{< relref "team_find_more_information.md" >}}) +- [What is the difference between team and entity? As a user - what does entity mean for me?]({{< relref "difference_team_entity_user_entity_mean_me.md" >}}) +- [What is the difference between team and organization?]({{< relref "difference_team_organization.md" >}}) +- [What type of roles are available and what are the differences between them?]({{< relref "type_roles_available_differences.md" >}}) +- [When should I log to my personal entity against my team entity?]({{< relref "log_personal_entity_team_entity.md" >}}) +- [Who can create a team? Who can add or delete people from a team? Who can delete projects?]({{< relref "create_team_add_delete_people_team.md" >}}) \ No newline at end of file diff --git a/content/support/index_tensorboard.md b/content/support/index_tensorboard.md index b9fc94fe7..6bf8e2685 100644 --- a/content/support/index_tensorboard.md +++ b/content/support/index_tensorboard.md @@ -13,5 +13,5 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How is W&B different from TensorBoard?](different_tensorboard.md) -- [What formula do you use for your smoothing algorithm?](formula_smoothing_algorithm.md) \ No newline at end of file +- [How is W&B different from TensorBoard?]({{< relref "different_tensorboard.md" >}}) +- [What formula do you use for your smoothing algorithm?]({{< relref "formula_smoothing_algorithm.md" >}}) \ No newline at end of file diff --git a/content/support/index_user management.md b/content/support/index_user management.md index 120f7440f..7e3ffd88f 100644 --- a/content/support/index_user management.md +++ b/content/support/index_user management.md @@ -13,14 +13,14 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can I get an academic plan as a student?](academic_plan_student.md) -- [How can I change my account from corporate to academic?](change_account_from_corporate_academic.md) -- [How can I regain access to my account if I cannot receive a password reset email?](regain_access_account_cannot_receive_password_reset_email.md) -- [How can I resolve login issues with my account?](resolve_login_issues_with_account.md) -- [How do I export a list of users from my W&B Organisation?](export_list_users_account.md) -- [How do I find my API key?](find_api_key.md) -- [How do I request the complete deletion of my W&B account?](request_complete_deletion_account.md) -- [Is it possible to change the username?](change_username.md) -- [Is it possible to create a new account with an email that was previously used for a deleted account?](create_account_email_that_previously_used_deleted_account.md) -- [Is there a way to add more seats?](add_more_seats.md) -- [What type of roles are available and what are the differences between them?](type_roles_available_differences.md) \ No newline at end of file +- [Can I get an academic plan as a student?]({{< relref "academic_plan_student.md" >}}) +- [How can I change my account from corporate to academic?]({{< relref "change_account_from_corporate_academic.md" >}}) +- [How can I regain access to my account if I cannot receive a password reset email?]({{< relref "regain_access_account_cannot_receive_password_reset_email.md" >}}) +- [How can I resolve login issues with my account?]({{< relref "resolve_login_issues_with_account.md" >}}) +- [How do I export a list of users from my W&B Organisation?]({{< relref "export_list_users_account.md" >}}) +- [How do I find my API key?]({{< relref "find_api_key.md" >}}) +- [How do I request the complete deletion of my W&B account?]({{< relref "request_complete_deletion_account.md" >}}) +- [Is it possible to change the username?]({{< relref "change_username.md" >}}) +- [Is it possible to create a new account with an email that was previously used for a deleted account?]({{< relref "create_account_email_that_previously_used_deleted_account.md" >}}) +- [Is there a way to add more seats?]({{< relref "add_more_seats.md" >}}) +- [What type of roles are available and what are the differences between them?]({{< relref "type_roles_available_differences.md" >}}) \ No newline at end of file diff --git a/content/support/index_workspaces.md b/content/support/index_workspaces.md index 90d0575b3..7efd0c53b 100644 --- a/content/support/index_workspaces.md +++ b/content/support/index_workspaces.md @@ -13,9 +13,9 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [Can I group runs without using the "Group" feature?](group_runs_without_group_feature.md) -- [How can I change the colors of each run in the same group?](group_runs_custom_meter.md) -- [How do I fix the "overflows maximum values of a signed 64 bits integer" error?](overflows_maximum_values_error.md) -- [Is there a dark mode?](dark_mode.md) -- [Refreshing data](refreshing_data.md) -- [What really good functionalities are hidden and where can I find those?](functionalities_hidden_find.md) \ No newline at end of file +- [Can I group runs without using the "Group" feature?]({{< relref "group_runs_without_group_feature.md" >}}) +- [How can I change the colors of each run in the same group?]({{< relref "group_runs_custom_meter.md" >}}) +- [How do I fix the "overflows maximum values of a signed 64 bits integer" error?]({{< relref "overflows_maximum_values_error.md" >}}) +- [Is there a dark mode?]({{< relref "dark_mode.md" >}}) +- [Refreshing data]({{< relref "refreshing_data.md" >}}) +- [What really good functionalities are hidden and where can I find those?]({{< relref "functionalities_hidden_find.md" >}}) \ No newline at end of file diff --git a/content/support/index_wysiwyg.md b/content/support/index_wysiwyg.md index 7aa22f4c3..828f74c8d 100644 --- a/content/support/index_wysiwyg.md +++ b/content/support/index_wysiwyg.md @@ -13,8 +13,8 @@ your question answered, try [asking the community](https://community.wandb.ai/), or email [support@wandb.com](mailto:support@wandb.com). {{% /card %}} -- [How do I delete a panel grid?](delete_panel_grid.md) -- [How do I insert a table?](insert_table.md) -- [I converted my report to WYSIWYG but want to revert back to Markdown](convert_from_wysiwyg_markdown.md) -- [My report is running slowly after the change to WYSIWYG](report_slow_after_wysiwyg.md) -- [My report looks different after converting from Markdown.](report_looks_different_converting_markdown.md) \ No newline at end of file +- [How do I delete a panel grid?]({{< relref "delete_panel_grid.md" >}}) +- [How do I insert a table?]({{< relref "insert_table.md" >}}) +- [I converted my report to WYSIWYG but want to revert back to Markdown]({{< relref "convert_from_wysiwyg_markdown.md" >}}) +- [My report is running slowly after the change to WYSIWYG]({{< relref "report_slow_after_wysiwyg.md" >}}) +- [My report looks different after converting from Markdown.]({{< relref "report_looks_different_converting_markdown.md" >}}) \ No newline at end of file diff --git a/content/support/log_additional_metrics_run_completes.md b/content/support/log_additional_metrics_run_completes.md index 9e83ebe01..c73f0073e 100644 --- a/content/support/log_additional_metrics_run_completes.md +++ b/content/support/log_additional_metrics_run_completes.md @@ -8,7 +8,7 @@ tags: --- There are several ways to manage experiments. -For complex workflows, use multiple runs and set the group parameters in [`wandb.init`](../guides/track/launch.md) to a unique value for all processes within a single experiment. The [**Runs** tab](../guides/track/project-page.md#runs-tab) will group the table by group ID, ensuring that visualizations function properly. This approach enables concurrent experiments and training runs while logging results in one location. +For complex workflows, use multiple runs and set the group parameters in [`wandb.init`]({{< relref "/guides/models/track/launch.md" >}}) to a unique value for all processes within a single experiment. The [**Runs** tab]({{< relref "/guides/models/track/project-page.md#runs-tab" >}}) will group the table by group ID, ensuring that visualizations function properly. This approach enables concurrent experiments and training runs while logging results in one location. -For simpler workflows, call `wandb.init` with `resume=True` and `id=UNIQUE_ID`, then call `wandb.init` again with the same `id=UNIQUE_ID`. Log normally with [`wandb.log`](../guides/track/log/intro.md) or `wandb.summary`, and the run values will update accordingly. +For simpler workflows, call `wandb.init` with `resume=True` and `id=UNIQUE_ID`, then call `wandb.init` again with the same `id=UNIQUE_ID`. Log normally with [`wandb.log`]({{< relref "/guides/models/track/log/" >}}) or `wandb.summary`, and the run values will update accordingly. diff --git a/content/support/log_list_values.md b/content/support/log_list_values.md index 53f433b9a..84b8c7ec3 100644 --- a/content/support/log_list_values.md +++ b/content/support/log_list_values.md @@ -6,7 +6,7 @@ tags: - logs - experiments --- -These examples show logging losses a couple of different ways using [`wandb.log()`](/ref/python/log/). +These examples show logging losses a couple of different ways using [`wandb.log()`]({{< relref "/ref/python/log/" >}}). {{< tabpane text=true >}} {{% tab "Using a dictionary" %}} @@ -23,4 +23,4 @@ wandb.log({"losses": wandb.Histogram(losses)}) {{% /tab %}} {{< /tabpane >}} -For more, see [the documentation on logging](/guides/track/log/). \ No newline at end of file +For more, see [the documentation on logging]({{< relref "/guides/models/track/log/" >}}). \ No newline at end of file diff --git a/content/support/log_metrics_two_different_time_scales_example_log_training.md b/content/support/log_metrics_two_different_time_scales_example_log_training.md index 8543d2f3a..6f23b8943 100644 --- a/content/support/log_metrics_two_different_time_scales_example_log_training.md +++ b/content/support/log_metrics_two_different_time_scales_example_log_training.md @@ -8,7 +8,7 @@ tags: --- For example, I want to log training accuracy per batch and validation accuracy per epoch. -Yes, log indices like `batch` and `epoch` alongside your metrics. Use `wandb.log({'train_accuracy': 0.9, 'batch': 200})` in one step and `wandb.log({'val_accuracy': 0.8, 'epoch': 4})` in another. In the UI, set the desired value as the x-axis for each chart. To set a default x-axis for a specific index, use [Run.define_metric()](../ref/python/run.md#define_metric). For the example provided, use the following code: +Yes, log indices like `batch` and `epoch` alongside your metrics. Use `wandb.log({'train_accuracy': 0.9, 'batch': 200})` in one step and `wandb.log({'val_accuracy': 0.8, 'epoch': 4})` in another. In the UI, set the desired value as the x-axis for each chart. To set a default x-axis for a specific index, use [Run.define_metric()]({{< relref "/ref/python/run.md#define_metric" >}}). For the example provided, use the following code: ```python wandb.init() diff --git a/content/support/multiprocessing_eg_distributed_training.md b/content/support/multiprocessing_eg_distributed_training.md index dffadfbfd..200328d97 100644 --- a/content/support/multiprocessing_eg_distributed_training.md +++ b/content/support/multiprocessing_eg_distributed_training.md @@ -9,9 +9,9 @@ If a training program uses multiple processes, structure the program to avoid ma Manage multiprocess training using these approaches: -1. Call `wandb.init` in all processes and use the [group](../guides/runs/grouping.md) keyword argument to create a shared group. Each process will have its own wandb run, and the UI will group the training processes together. +1. Call `wandb.init` in all processes and use the [group]({{< relref "/guides/models/track/runs/grouping.md" >}}) keyword argument to create a shared group. Each process will have its own wandb run, and the UI will group the training processes together. 2. Call `wandb.init` from only one process and pass data to log through [multiprocessing queues](https://docs.python.org/3/library/multiprocessing.html#exchanging-objects-between-processes). {{% alert %}} -Refer to the [Distributed Training Guide](../guides/track/log/distributed-training.md) for detailed explanations of these approaches, including code examples with Torch DDP. +Refer to the [Distributed Training Guide]({{< relref "/guides/models/track/log/distributed-training.md" >}}) for detailed explanations of these approaches, including code examples with Torch DDP. {{% /alert %}} \ No newline at end of file diff --git a/content/support/organize_logged_charts_media_wb_ui.md b/content/support/organize_logged_charts_media_wb_ui.md index b619d1f06..72a82bc8e 100644 --- a/content/support/organize_logged_charts_media_wb_ui.md +++ b/content/support/organize_logged_charts_media_wb_ui.md @@ -12,4 +12,4 @@ wandb.log({"val/loss": 1.1, "val/acc": 0.3}) wandb.log({"train/loss": 0.1, "train/acc": 0.94}) ``` -In the [Workspace](../guides/track/project-page.md#workspace-tab) settings, adjust the grouping of panels based on either the first segment or all segments separated by `/`. \ No newline at end of file +In the [Workspace]({{< relref "/guides/models/track/project-page.md#workspace-tab" >}}) settings, adjust the grouping of panels based on either the first segment or all segments separated by `/`. \ No newline at end of file diff --git a/content/support/plot_multiple_lines_plot_legend.md b/content/support/plot_multiple_lines_plot_legend.md index 41ff570e9..7f2d4fe18 100644 --- a/content/support/plot_multiple_lines_plot_legend.md +++ b/content/support/plot_multiple_lines_plot_legend.md @@ -6,7 +6,7 @@ tags: - experiments --- -Create a multi-line custom chart with `wandb.plot.line_series()`. Navigate to the [project page](../guides/track/project-page.md) to view the line chart. To add a legend, include the `keys` argument in `wandb.plot.line_series()`. For example: +Create a multi-line custom chart with `wandb.plot.line_series()`. Navigate to the [project page]({{< relref "/guides/models/track/project-page.md" >}}) to view the line chart. To add a legend, include the `keys` argument in `wandb.plot.line_series()`. For example: ```python wandb.log( @@ -18,4 +18,4 @@ wandb.log( ) ``` -Refer to additional details about multi-line plots [here](../guides/track/log/plots.md#basic-charts) under the **Multi-line** tab. \ No newline at end of file +Refer to additional details about multi-line plots [here]({{< relref "/guides/models/track/log/plots.md#basic-charts" >}}) under the **Multi-line** tab. \ No newline at end of file diff --git a/content/support/programmatically_access_humanreadable_run_name.md b/content/support/programmatically_access_humanreadable_run_name.md index 08ab1e14a..e390685a0 100644 --- a/content/support/programmatically_access_humanreadable_run_name.md +++ b/content/support/programmatically_access_humanreadable_run_name.md @@ -5,7 +5,7 @@ type: docs tags: - experiments --- -The `.name` attribute of a [`wandb.Run`](../ref/python/run.md) is accessible as follows: +The `.name` attribute of a [`wandb.Run`]({{< relref "/ref/python/run.md" >}}) is accessible as follows: ```python import wandb diff --git a/content/support/rerun_grid_search.md b/content/support/rerun_grid_search.md index 02061cd8f..a0ec02a6f 100644 --- a/content/support/rerun_grid_search.md +++ b/content/support/rerun_grid_search.md @@ -7,6 +7,6 @@ tags: - hyperparameter - runs --- -If a grid search completes but some W&B Runs need re-execution due to crashes, delete the specific W&B Runs to re-run. Then, select the **Resume** button on the [sweep control page](../guides/sweeps/sweeps-ui.md). Start new W&B Sweep agents using the new Sweep ID. +If a grid search completes but some W&B Runs need re-execution due to crashes, delete the specific W&B Runs to re-run. Then, select the **Resume** button on the [sweep control page]({{< relref "/guides/models/sweeps/sweeps-ui.md" >}}). Start new W&B Sweep agents using the new Sweep ID. W&B Run parameter combinations that completed are not re-executed. \ No newline at end of file diff --git a/content/support/retention_expiration_policy_artifact.md b/content/support/retention_expiration_policy_artifact.md index f61807b3d..e454676e1 100644 --- a/content/support/retention_expiration_policy_artifact.md +++ b/content/support/retention_expiration_policy_artifact.md @@ -5,4 +5,4 @@ type: docs tags: - artifacts --- -To manage artifacts that contain sensitive data or to schedule the deletion of artifact versions, set a TTL (time-to-live) policy. For detailed instructions, refer to the [TTL guide](../guides/artifacts/ttl.md). \ No newline at end of file +To manage artifacts that contain sensitive data or to schedule the deletion of artifact versions, set a TTL (time-to-live) policy. For detailed instructions, refer to the [TTL guide]({{< relref "/guides/core/artifacts/manage-data/ttl.md" >}}). \ No newline at end of file diff --git a/content/support/runs_state_crashed_ui_running_machine_get_data.md b/content/support/runs_state_crashed_ui_running_machine_get_data.md index b8cb96bef..dc8323f93 100644 --- a/content/support/runs_state_crashed_ui_running_machine_get_data.md +++ b/content/support/runs_state_crashed_ui_running_machine_get_data.md @@ -5,4 +5,4 @@ type: docs tags: - experiments --- -You likely lost connection to your machine during training. Recover data by running [`wandb sync [PATH_TO_RUN]`](../ref/cli/wandb-sync.md). The path to your run is a folder in your `wandb` directory that matches the Run ID of the ongoing run. \ No newline at end of file +You likely lost connection to your machine during training. Recover data by running [`wandb sync [PATH_TO_RUN]`]({{< relref "/ref/cli/wandb-sync.md" >}}). The path to your run is a folder in your `wandb` directory that matches the Run ID of the ongoing run. \ No newline at end of file diff --git a/content/support/save_git_commit_associated_run.md b/content/support/save_git_commit_associated_run.md index 267337d91..c2b4f6a2a 100644 --- a/content/support/save_git_commit_associated_run.md +++ b/content/support/save_git_commit_associated_run.md @@ -5,6 +5,6 @@ type: docs tags: - experiments --- -When `wandb.init` is invoked, the system automatically collects git information, including the remote repository link and the SHA of the latest commit. This information appears on the [run page](../guides/runs/intro.md#view-logged-runs). Ensure the current working directory when executing the script is within a git-managed folder to view this information. +When `wandb.init` is invoked, the system automatically collects git information, including the remote repository link and the SHA of the latest commit. This information appears on the [run page]({{< relref "/guides/models/track/runs/#view-logged-runs" >}}). Ensure the current working directory when executing the script is within a git-managed folder to view this information. The git commit and the command used to run the experiment remain visible to the user but are hidden from external users. In public projects, these details remain private. \ No newline at end of file diff --git a/content/support/service_account_useful.md b/content/support/service_account_useful.md index 6d980889e..48c290414 100644 --- a/content/support/service_account_useful.md +++ b/content/support/service_account_useful.md @@ -9,11 +9,11 @@ tags: A service account (Enterprise-only feature) represents a non-human or machine user, which can automate common tasks across teams and projects or ones that are not specific to a particular human user. You can create a service account within a team and use its API key to read from and write to projects within that team. -Among other things, service accounts are useful for tracking automated jobs logged to wandb, like periodic retraining, nightly builds, and so on. If you'd like, you can associate a username with one of these machine-launched runs with the [environment variables](../guides/track/environment-variables.md) `WANDB_USERNAME` or `WANDB_USER_EMAIL`. +Among other things, service accounts are useful for tracking automated jobs logged to wandb, like periodic retraining, nightly builds, and so on. If you'd like, you can associate a username with one of these machine-launched runs with the [environment variables]({{< relref "/guides/models/track/environment-variables.md" >}}) `WANDB_USERNAME` or `WANDB_USER_EMAIL`. -Refer to [Team Service Account Behavior](../guides/app/features/teams.md#team-service-account-behavior) for more information. +Refer to [Team Service Account Behavior]({{< relref "/guides/models/app/settings-page/teams.md#team-service-account-behavior" >}}) for more information. You can get the API key for a service account in your team at `//service-accounts`. Alternatively you can go to the **Team settings** for your team and then refer to the **Service Accounts** tab. @@ -25,5 +25,5 @@ To create a new service account for your team: * Click the **Copy API key** button for the newly created service account and store it in a secret manager or another safe but accessible location {{% alert %}} -Apart from the **Built-in** service accounts, W&B also supports **External service accounts** using [identity federation for SDK and CLI](../guides/hosting/iam/identity_federation.md#external-service-accounts). Use external service accounts if you are looking to automate W&B tasks using service identities managed in your identity provider that can issue JSON Web Tokens (JWT). +Apart from the **Built-in** service accounts, W&B also supports **External service accounts** using [identity federation for SDK and CLI]({{< relref "/guides/hosting/iam/authentication/identity_federation.md#external-service-accounts" >}}). Use external service accounts if you are looking to automate W&B tasks using service identities managed in your identity provider that can issue JSON Web Tokens (JWT). {{% /alert %}} diff --git a/content/support/stop_wandb_writing_terminal_jupyter_notebook_output.md b/content/support/stop_wandb_writing_terminal_jupyter_notebook_output.md index 6a2985e2c..744342c0a 100644 --- a/content/support/stop_wandb_writing_terminal_jupyter_notebook_output.md +++ b/content/support/stop_wandb_writing_terminal_jupyter_notebook_output.md @@ -5,7 +5,7 @@ type: docs tags: - environment variables --- -Set the environment variable [`WANDB_SILENT`](../guides/track/environment-variables.md) to `true`. +Set the environment variable [`WANDB_SILENT`]({{< relref "/guides/models/track/environment-variables.md" >}}) to `true`. {{< tabpane text=true langEqualsHeader=true >}} {{% tab header="Python" %}} diff --git a/content/support/sweeps_sagemaker.md b/content/support/sweeps_sagemaker.md index 4fd387fc2..cfa18328d 100644 --- a/content/support/sweeps_sagemaker.md +++ b/content/support/sweeps_sagemaker.md @@ -6,7 +6,7 @@ tags: - sweeps - aws --- -To authenticate W&B, complete the following steps: create a `requirements.txt` file if using a built-in Amazon SageMaker estimator. For details on authentication and setting up the `requirements.txt` file, refer to the [SageMaker integration](../guides/integrations/other/sagemaker.md) guide. +To authenticate W&B, complete the following steps: create a `requirements.txt` file if using a built-in Amazon SageMaker estimator. For details on authentication and setting up the `requirements.txt` file, refer to the [SageMaker integration]({{< relref "/guides/integrations/sagemaker.md" >}}) guide. {{% alert %}} Find a complete example on [GitHub](https://github.com/wandb/examples/tree/master/examples/pytorch/pytorch-cifar10-sagemaker) and additional insights on our [blog](https://wandb.ai/site/articles/running-sweeps-with-sagemaker).\ diff --git a/content/support/team_find_more_information.md b/content/support/team_find_more_information.md index 6f87790d0..06c1ab9e7 100644 --- a/content/support/team_find_more_information.md +++ b/content/support/team_find_more_information.md @@ -5,4 +5,4 @@ type: docs tags: - team management --- -For additional information about teams, visit the [teams section](../guides/app/features/teams.md). \ No newline at end of file +For additional information about teams, visit the [teams section]({{< relref "/guides/models/app/settings-page/teams.md" >}}). \ No newline at end of file diff --git a/content/support/type_roles_available_differences.md b/content/support/type_roles_available_differences.md index cf1911aa2..46f9905ac 100644 --- a/content/support/type_roles_available_differences.md +++ b/content/support/type_roles_available_differences.md @@ -6,4 +6,4 @@ tags: - user management - team management --- -Visit [this](../guides/app/features/teams.md#team-roles-and-permissions) page for an overview of the available roles and permissions. \ No newline at end of file +Visit [this]({{< relref "/guides/models/app/settings-page/teams.md#team-roles-and-permissions" >}}) page for an overview of the available roles and permissions. \ No newline at end of file diff --git a/content/tutorials/_index.md b/content/tutorials/_index.md index 66c409328..c6c7eacbd 100644 --- a/content/tutorials/_index.md +++ b/content/tutorials/_index.md @@ -17,27 +17,27 @@ no_list: true The following tutorials take you through the fundamentals of W&B for machine learning experiment tracking, model evaluation, hyperparameter tuning, model and dataset versioning, and more. -1. [Track experiments](/tutorials/experiments) -2. [Visualize predictions](/tutorials/tables) -3. [Tune hyperparameters](/tutorials/sweeps) -4. [Track models and datasets](/tutorials/artifacts) +1. [Track experiments]({{< relref "/tutorials/experiments/" >}}) +2. [Visualize predictions]({{< relref "/tutorials/tables/" >}}) +3. [Tune hyperparameters]({{< relref "/tutorials/sweeps/" >}}) +4. [Track models and datasets]({{< relref "/tutorials/artifacts/" >}}) ## Popular ML framework tutorials See the following tutorials for step by step information on how to use popular ML frameworks and libraries with W&B: -- [PyTorch](/tutorials/pytorch) -- [PyTorch Lightning](/tutorials/lightning) -- [HuggingFace Transformers](/tutorials/huggingface) +- [PyTorch]({{< relref "/tutorials/integration-tutorials/pytorch" >}}) +- [PyTorch Lightning]({{< relref "/tutorials/integration-tutorials/lightning" >}}) +- [HuggingFace Transformers]({{< relref "/tutorials/integration-tutorials/huggingface" >}}) - Tensorflow - - [Track experiments](/tutorials/tensorflow) - - [Tune hyperparameters](/tutorials/tensorflow_sweeps) + - [Track experiments]({{< relref "/tutorials/integration-tutorials/tensorflow" >}}) + - [Tune hyperparameters]({{< relref "/tutorials/integration-tutorials/tensorflow_sweeps/" >}}) - Keras - - [Track experiments](/tutorials/keras) - - [Visualize predictions](/tutorials/keras_tables) - - [Model Checkpointing](/tutorials/keras_models) + - [Track experiments]({{< relref "/tutorials/integration-tutorials/keras/" >}}) + - [Visualize predictions]({{< relref "/tutorials/integration-tutorials/keras_tables/" >}}) + - [Model Checkpointing]({{< relref "/tutorials/integration-tutorials/keras_models/" >}}) - XGBoost - - [Tune hyperparameters](/tutorials/xgboost_sweeps) + - [Tune hyperparameters]({{< relref "/tutorials/integration-tutorials/xgboost_sweeps/" >}}) diff --git a/content/tutorials/experiments.md b/content/tutorials/experiments.md index f21418b81..7bd546e07 100644 --- a/content/tutorials/experiments.md +++ b/content/tutorials/experiments.md @@ -10,7 +10,7 @@ weight: 1 colabLink="https://colab.research.google.com/github/wandb/examples/blob/master/colabs/intro/Intro_to_Weights_&_Biases.ipynb" >}} -Use [W&B](https://wandb.ai/site) for machine learning experiment tracking, model checkpointing, collaboration with your team and more. See the full W&B Documentation [here](/). +Use [W&B](https://wandb.ai/site) for machine learning experiment tracking, model checkpointing, collaboration with your team and more. In this notebook, you will create and track a machine learning experiment using a simple PyTorch model. By the end of the notebook, you will have an interactive project dashboard that you can share and customize with other members of your team. [View an example dashboard here](https://wandb.ai/wandb/wandb_example). @@ -43,7 +43,7 @@ wandb.login() Create, track, and visualize a machine learning experiment. To do this: -1. Initialize a [W&B run](/guides/runs) and pass in the hyperparameters you want to track. +1. Initialize a [W&B run]({{< relref "/guides/models/track/runs/" >}}) and pass in the hyperparameters you want to track. 2. Within your training loop, log metrics such as the accuruacy and loss. @@ -94,11 +94,11 @@ Now that we know how to integrate W&B into a psuedo machine learning training lo The following code cell defines and trains a simple MNIST classifier. During training, you will see W&B prints out URLs. Click on the project page link to see your results stream in live to a W&B project. -W&B runs automatically log [metrics](/guides/runs/intro.md#workspace-tab), -[system information](/guides/runs/intro.md#system-tab), -[hyperparameters](/guides/runs/intro.md#overview-tab), -[terminal output](/guides/runs/intro.md#logs-tab) and -you'll see an [interactive table](/guides/tables) +W&B runs automatically log [metrics]({{< relref "/guides/models/track/runs/#workspace-tab" >}}), +[system information]({{< relref "/guides/models/track/runs/#system-tab" >}}), +[hyperparameters]({{< relref "/guides/models/track/runs/#overview-tab" >}}), +[terminal output]({{< relref "/guides/models/track/runs/#logs-tab" >}}) and +you'll see an [interactive table]({{< relref "/guides/core/tables/" >}}) with model inputs and outputs. ### Set up PyTorch Dataloader @@ -200,7 +200,7 @@ def log_image_table(images, predicted, labels, probs): The following code trains and saves model checkpoints to your project. Use model checkpoints like you normally would to assess how the model performed during training. -W&B also makes it easy to share your saved models and model checkpoints with other members of your team or organization. To learn how to share your model and model checkpoints with members outside of your team, see [W&B Registry](/guides/registry). +W&B also makes it easy to share your saved models and model checkpoints with other members of your team or organization. To learn how to share your model and model checkpoints with members outside of your team, see [W&B Registry]({{< relref "/guides/models/registry/" >}}). ```python @@ -291,7 +291,7 @@ You have now trained your first model using W&B. Click on one of the links above ## (Optional) Set up a W&B Alert -Create a [W&B Alerts](/guides/runs/alert/) to send alerts to your Slack or email from your Python code. +Create a [W&B Alerts]({{< relref "/guides/models/track/runs/alert/" >}}) to send alerts to your Slack or email from your Python code. There are 2 steps to follow the first time you'd like to send a Slack or email alert, triggered from your code: @@ -334,7 +334,7 @@ for training_step in range(1000): wandb.finish() ``` -You can find the full docs for [W&B Alerts here](/guides/runs/alert). +You can find the full docs for [W&B Alerts here]({{< relref "/guides/models/track/runs/alert" >}}). ## Next steps The next tutorial you will learn how to do hyperparameter optimization using W&B Sweeps: diff --git a/content/tutorials/integration-tutorials/huggingface.md b/content/tutorials/integration-tutorials/huggingface.md index 60c6b92ef..3321f7702 100644 --- a/content/tutorials/integration-tutorials/huggingface.md +++ b/content/tutorials/integration-tutorials/huggingface.md @@ -38,7 +38,7 @@ Here's an in depth look at how the integration works: [Hugging Face + W&B Report Install the Hugging Face and Weights & Biases libraries, and the GLUE dataset and training script for this tutorial. - [Hugging Face Transformers](https://github.com/huggingface/transformers): Natural language models and datasets -- [Weights & Biases](/): Experiment tracking and visualization +- [Weights & Biases]({{< relref "/" >}}): Experiment tracking and visualization - [GLUE dataset](https://gluebenchmark.com/): A language understanding benchmark dataset - [GLUE script](https://github.com/huggingface/transformers/blob/master/examples/run_glue.py): Model training script for sequence classification @@ -66,7 +66,7 @@ import wandb wandb.login() ``` -Optionally, we can set environment variables to customize W&B logging. See [documentation](/guides/integrations/huggingface). +Optionally, we can set environment variables to customize W&B logging. See [documentation]({{< relref "/guides/integrations/huggingface/" >}}). ```python @@ -117,6 +117,6 @@ Weights & Biases saves a new run for each experiment. Here's the information tha - **System Metrics**: GPU and CPU utilization, memory, temperature etc. ## Learn more -- [Documentation](/guides/integrations/huggingface): docs on the Weights & Biases and Hugging Face integration +- [Documentation]({{< relref "/guides/integrations/huggingface" >}}): docs on the Weights & Biases and Hugging Face integration - [Videos](http://wandb.me/youtube): tutorials, interviews with practitioners, and more on our YouTube channel - Contact: Message us at contact@wandb.com with questions \ No newline at end of file diff --git a/content/tutorials/integration-tutorials/keras.md b/content/tutorials/integration-tutorials/keras.md index d1235f66f..c0ea2437c 100644 --- a/content/tutorials/integration-tutorials/keras.md +++ b/content/tutorials/integration-tutorials/keras.md @@ -11,7 +11,7 @@ Use Weights & Biases for machine learning experiment tracking, dataset versionin {{< img src="/images/tutorials/huggingface-why.png" alt="" >}} -This Colab notebook introduces the `WandbMetricsLogger` callback. Use this callback for [Experiment Tracking](/guides/track). It will log your training and validation metrics along with system metrics to Weights and Biases. +This Colab notebook introduces the `WandbMetricsLogger` callback. Use this callback for [Experiment Tracking]({{< relref "/guides/models/track" >}}). It will log your training and validation metrics along with system metrics to Weights and Biases. ## Setup and Installation diff --git a/content/tutorials/integration-tutorials/keras_models.md b/content/tutorials/integration-tutorials/keras_models.md index 19dde3ad1..96c3262e5 100644 --- a/content/tutorials/integration-tutorials/keras_models.md +++ b/content/tutorials/integration-tutorials/keras_models.md @@ -11,7 +11,7 @@ Use Weights & Biases for machine learning experiment tracking, dataset versionin {{< img src="/images/tutorials/huggingface-why.png" alt="" >}} -This Colab notebook introduces the `WandbModelCheckpoint` callback. Use this callback to log your model checkpoints to Weight and Biases [Artifacts](/guides/artifacts). +This Colab notebook introduces the `WandbModelCheckpoint` callback. Use this callback to log your model checkpoints to Weight and Biases [Artifacts]({{< relref "/guides/core/artifacts/" >}}). ## Setup and Installation diff --git a/content/tutorials/integration-tutorials/monai_3d_segmentation.md b/content/tutorials/integration-tutorials/monai_3d_segmentation.md index 772a78c89..e3714398f 100644 --- a/content/tutorials/integration-tutorials/monai_3d_segmentation.md +++ b/content/tutorials/integration-tutorials/monai_3d_segmentation.md @@ -231,7 +231,7 @@ val_dataset = DecathlonDataset( ### Visualizing the Dataset -Weights & Biases supports images, video, audio, and more. You can log rich media to explore your results and visually compare our runs, models, and datasets. Use the [segmentation mask overlay system](/guides/track/log/media#image-overlays-in-tables) to visualize our data volumes. To log segmentation masks in [tables](/guides/tables), you must provide a `wandb.Image` object for each row in the table. +Weights & Biases supports images, video, audio, and more. You can log rich media to explore your results and visually compare our runs, models, and datasets. Use the [segmentation mask overlay system]({{< relref "/guides/models/track/log/media/#image-overlays-in-tables" >}}) to visualize our data volumes. To log segmentation masks in [tables]({{< relref "/guides/core/tables/" >}}), you must provide a `wandb.Image` object for each row in the table. An example is provided in the pseudocode below: @@ -359,7 +359,7 @@ for data_idx, sample in progress_bar: wandb.log({"Tumor-Segmentation-Data": table}) ``` -The data appears on the W&B dashboard in an interactive tabular format. We can see each channel of a particular slice from a data volume overlaid with the respective segmentation mask in each row. You can write [Weave queries](/guides/weave) to filter the data on the table and focus on one particular row. +The data appears on the W&B dashboard in an interactive tabular format. We can see each channel of a particular slice from a data volume overlaid with the respective segmentation mask in each row. You can write [Weave queries]({{< relref "/guides/weave" >}}) to filter the data on the table and focus on one particular row. | {{< img src="/images/tutorials/monai/viz-1.gif" alt="An example of logged table data." >}} | |:--:| diff --git a/content/tutorials/integration-tutorials/pytorch.md b/content/tutorials/integration-tutorials/pytorch.md index e88e21f05..500764330 100644 --- a/content/tutorials/integration-tutorials/pytorch.md +++ b/content/tutorials/integration-tutorials/pytorch.md @@ -434,7 +434,7 @@ You can use Weights & Biases Sweeps to automate hyperparameter testing and explo Running a hyperparameter sweep with Weights & Biases is very easy. There are just 3 simple steps: -1. **Define the sweep:** We do this by creating a dictionary or a [YAML file](/guides/sweeps/define-sweep-configuration) that specifies the parameters to search through, the search strategy, the optimization metric et all. +1. **Define the sweep:** We do this by creating a dictionary or a [YAML file]({{< relref "/guides/models/sweeps/define-sweep-configuration" >}}) that specifies the parameters to search through, the search strategy, the optimization metric et all. 2. **Initialize the sweep:** `sweep_id = wandb.sweep(sweep_config)` @@ -452,7 +452,7 @@ That's all there is to running a hyperparameter sweep. See examples of projects tracked and visualized with W&B in our [Gallery →](https://app.wandb.ai/gallery) ## Advanced Setup -1. [Environment variables](/guides/hosting/env-vars): Set API keys in environment variables so you can run training on a managed cluster. -2. [Offline mode](../support/run_wandb_offline.md): Use `dryrun` mode to train offline and sync results later. -3. [On-prem](/guides/hosting/hosting-options/self-managed): Install W&B in a private cloud or air-gapped servers in your own infrastructure. We have local installations for everyone from academics to enterprise teams. -4. [Sweeps](/guides/sweeps): Set up hyperparameter search quickly with our lightweight tool for tuning. \ No newline at end of file +1. [Environment variables]({{< relref "/guides/hosting/env-vars/" >}}): Set API keys in environment variables so you can run training on a managed cluster. +2. [Offline mode]({{< relref "/support/run_wandb_offline.md" >}}): Use `dryrun` mode to train offline and sync results later. +3. [On-prem]({{< relref "/guides/hosting/hosting-options/self-managed" >}}): Install W&B in a private cloud or air-gapped servers in your own infrastructure. We have local installations for everyone from academics to enterprise teams. +4. [Sweeps]({{< relref "/guides/models/sweeps/" >}}): Set up hyperparameter search quickly with our lightweight tool for tuning. \ No newline at end of file diff --git a/content/tutorials/integration-tutorials/tensorflow.md b/content/tutorials/integration-tutorials/tensorflow.md index 171c08c46..6179e28e9 100644 --- a/content/tutorials/integration-tutorials/tensorflow.md +++ b/content/tutorials/integration-tutorials/tensorflow.md @@ -166,7 +166,7 @@ def train(train_dataset, val_dataset, model, optimizer, This lets us know you're launching an experiment, so we can give it a unique ID and a dashboard. -[Check out the official documentation](/ref/python/init) +[Check out the official documentation]({{< relref "/ref/python/init" >}}) ```python # initialize wandb with your project name and optionally with configutations. @@ -211,7 +211,7 @@ run.finish() # In Jupyter/Colab, let us know you're finished! ### Visualize Results -Click on the [**run page**](/guides/runs/intro.md#view-logged-runs) link above to see your live results. +Click on the [**run page**]({{< relref "/guides/models/track/runs/#view-logged-runs" >}}) link above to see your live results. ## Sweep 101 @@ -239,7 +239,7 @@ See examples of projects tracked and visualized with W&B in our gallery of examp 5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects. ## Advanced Setup -1. [Environment variables](/guides/hosting/env-vars): Set API keys in environment variables so you can run training on a managed cluster. -2. [Offline mode](../support/run_wandb_offline.md) -3. [On-prem](/guides/hosting/hosting-options/self-managed): Install W&B in a private cloud or air-gapped servers in your own infrastructure. We have local installations for everyone from academics to enterprise teams. -4. [Artifacts](/guides/artifacts): Track and version models and datasets in a streamlined way that automatically picks up your pipeline steps as you train models. \ No newline at end of file +1. [Environment variables]({{< relref "/guides/hosting/env-vars/" >}}): Set API keys in environment variables so you can run training on a managed cluster. +2. [Offline mode]({{< relref "/support/run_wandb_offline.md" >}}) +3. [On-prem]({{< relref "/guides/hosting/hosting-options/self-managed" >}}): Install W&B in a private cloud or air-gapped servers in your own infrastructure. We have local installations for everyone from academics to enterprise teams. +4. [Artifacts]({{< relref "/guides/core/artifacts/" >}}): Track and version models and datasets in a streamlined way that automatically picks up your pipeline steps as you train models. \ No newline at end of file diff --git a/content/tutorials/integration-tutorials/tensorflow_sweeps.md b/content/tutorials/integration-tutorials/tensorflow_sweeps.md index d77fe380d..728ae79c8 100644 --- a/content/tutorials/integration-tutorials/tensorflow_sweeps.md +++ b/content/tutorials/integration-tutorials/tensorflow_sweeps.md @@ -23,7 +23,7 @@ Use Weights & Biases Sweeps to automate hyperparameter optimization and explore * **Transparent**: The project cites all algorithms used, and the [code is open source](https://github.com/wandb/wandb/blob/main/wandb/apis/public/sweeps.py). * **Powerful**: Sweeps are completely customizable and configurable. You can launch a sweep across dozens of machines, and it's just as easy as starting a sweep on your laptop. -**[Check out the official documentation](/guides/sweeps)** +**[Check out the official documentation]({{< relref "/guides/models/sweeps/" >}})** ## What this notebook covers @@ -190,7 +190,7 @@ This is where you will: * Provide an objective and a `metric` if using `bayes`, for example to `minimize` the `val_loss`. * Use `hyperband` for early termination of poorly performing runs. -#### [Check out more on Sweep Configs](/guides/sweeps/define-sweep-configuration) +#### [Check out more on Sweep Configs]({{< relref "/guides/models/sweeps/define-sweep-configuration" >}}) ```python @@ -296,6 +296,6 @@ See examples of projects tracked and visualized with W&B in the [Gallery →](ht 5. **Reports**: Take quick notes on progress to share with colleagues and make dashboards and snapshots of your ML projects. ## Advanced Setup -1. [Environment variables](/guides/hosting/env-vars): Set API keys in environment variables so you can run training on a managed cluster. -2. [Offline mode](../support/run_wandb_offline.md) -3. [On-prem](/guides/hosting/hosting-options/self-managed): Install W&B in a private cloud or air-gapped servers in your own infrastructure. Everyone from academics to enterprise teams use local installations. \ No newline at end of file +1. [Environment variables]({{< relref "/guides/hosting/env-vars/" >}}): Set API keys in environment variables so you can run training on a managed cluster. +2. [Offline mode]({{< relref "/support/run_wandb_offline.md" >}}) +3. [On-prem]({{< relref "/guides/hosting/hosting-options/self-managed" >}}): Install W&B in a private cloud or air-gapped servers in your own infrastructure. Everyone from academics to enterprise teams use local installations. \ No newline at end of file diff --git a/content/tutorials/integration-tutorials/xgboost_sweeps.md b/content/tutorials/integration-tutorials/xgboost_sweeps.md index 5dc0ab8b7..9a51c260e 100644 --- a/content/tutorials/integration-tutorials/xgboost_sweeps.md +++ b/content/tutorials/integration-tutorials/xgboost_sweeps.md @@ -44,7 +44,7 @@ In the notebook below, we'll walk through these 3 steps in more detail. We highly encourage you to fork this notebook, tweak the parameters, or try the model with your own dataset. ### Resources -- [Sweeps docs →](/guides/sweeps) +- [Sweeps docs →]({{< relref "/guides/models/sweeps/" >}}) - [Launching from the command line →](https://www.wandb.com/articles/hyperparameter-tuning-as-easy-as-1-2-3) @@ -63,7 +63,7 @@ wandb.login() ## 1. Define the Sweep Weights & Biases sweeps give you powerful levers to configure your sweeps exactly how you want them, with just a few lines of code. The sweeps config can be defined as -[a dictionary or a YAML file](/guides/sweeps/define-sweep-configuration). +[a dictionary or a YAML file]({{< relref "/guides/models/sweeps/define-sweep-configuration" >}}). Let's walk through some of them together: * **Metric**: This is the metric the sweeps are attempting to optimize. Metrics can take a `name` (this metric should be logged by your training script) and a `goal` (`maximize` or `minimize`). @@ -73,7 +73,7 @@ Let's walk through some of them together: * **Bayesian Search**: Creates a probabilistic model that maps hyperparameters to probability of a metric score, and chooses parameters with high probability of improving the metric. The objective of Bayesian optimization is to spend more time in picking the hyperparameter values, but in doing so trying out fewer hyperparameter values. * **Parameters**: A dictionary containing the hyperparameter names, and discrete values, a range, or distributions from which to pull their values on each iteration. -For details, see the [list of all sweep configuration options](/guides/sweeps/define-sweep-configuration). +For details, see the [list of all sweep configuration options]({{< relref "/guides/models/sweeps/define-sweep-configuration" >}}). ```python @@ -187,7 +187,7 @@ and that machine will join the sweep. > _Note_: a `random` sweep will by defauly run forever, trying new parameter combinations until the cows come home -- -or until you [turn the sweep off from the app UI](/guides/sweeps/sweeps-ui). +or until you [turn the sweep off from the app UI]({{< relref "/guides/models/sweeps/sweeps-ui" >}}). You can prevent this by providing the total `count` of runs you'd like the `agent` to complete. diff --git a/content/tutorials/sweeps.md b/content/tutorials/sweeps.md index 080ce1c7e..502551010 100644 --- a/content/tutorials/sweeps.md +++ b/content/tutorials/sweeps.md @@ -21,7 +21,7 @@ In this tutorial you will create a hyperparameter search with W&B PyTorch integr Running a hyperparameter sweep with Weights & Biases is very easy. There are just 3 simple steps: -1. **Define the sweep:** we do this by creating a dictionary or a [YAML file](/guides/sweeps/define-sweep-configuration) that specifies the parameters to search through, the search strategy, the optimization metric et all. +1. **Define the sweep:** we do this by creating a dictionary or a [YAML file]({{< relref "/guides/models/sweeps/define-sweep-configuration" >}}) that specifies the parameters to search through, the search strategy, the optimization metric et all. 2. **Initialize the sweep:** with one line of code we initialize the sweep and pass in the dictionary of sweep configurations: `sweep_id = wandb.sweep(sweep_config)` @@ -64,12 +64,12 @@ Before you start a sweep, you must define your sweep strategy with a _sweep conf {{% alert %}} The sweep configuration you create for a sweep must be in a nested dictionary if you start a sweep in a Jupyter Notebook. -If you run a sweep within the command line, you must specify your sweep config with a [YAML file](/guides/sweeps/define-sweep-configuration). +If you run a sweep within the command line, you must specify your sweep config with a [YAML file]({{< relref "/guides/models/sweeps/define-sweep-configuration" >}}). {{% /alert %}} ### Pick a search method -First, specify a hyperparameter search method within your configuration dictionary. [There are three hyperparameter search strategies to choose from: grid, random, and Bayesian search](/guides/sweeps/sweep-config-keys#method). +First, specify a hyperparameter search method within your configuration dictionary. [There are three hyperparameter search strategies to choose from: grid, random, and Bayesian search]({{< relref "/guides/models/sweeps/define-sweep-configuration/sweep-config-keys/#method" >}}). For this tutorial, you will use a random search. Within your notebook, create a dictionary and specify `random` for the `method` key. @@ -169,7 +169,7 @@ import pprint pprint.pprint(sweep_config) ``` -For a full list of configuration options, see [Sweep configuration options](/guides/sweeps/sweep-config-keys). +For a full list of configuration options, see [Sweep configuration options]({{< relref "/guides/models/sweeps/define-sweep-configuration/sweep-config-keys/" >}}). {{% alert %}} For hyperparameters that have potentially infinite options, @@ -207,7 +207,7 @@ wandb sweep config.yaml ``` {{% /alert %}} -For more information on how to create W&B Sweeps in a terminal, see the [W&B Sweep walkthrough](/guides/sweeps/walkthrough). +For more information on how to create W&B Sweeps in a terminal, see the [W&B Sweep walkthrough]({{< relref "/guides/models/sweeps/walkthrough" >}}). ## Step 3: Define your machine learning code @@ -246,9 +246,9 @@ def train(config=None): ``` Within the `train` function, you will notice the following W&B Python SDK methods: -* [`wandb.init()`](/ref/python/init): Initialize a new W&B run. Each run is a single execution of the training function. -* [`wandb.config`](/guides/track/config): Pass sweep configuration with the hyperparameters you want to experiment with. -* [`wandb.log()`](/ref/python/log): Log the training loss for each epoch. +* [`wandb.init()`]({{< relref "/ref/python/init" >}}): Initialize a new W&B run. Each run is a single execution of the training function. +* [`wandb.config`]({{< relref "/guides/models/track/config" >}}): Pass sweep configuration with the hyperparameters you want to experiment with. +* [`wandb.log()`]({{< relref "/ref/python/log" >}}): Log the training loss for each epoch. The proceeding cell defines four functions: @@ -340,7 +340,7 @@ wandb.agent(sweep_id, train, count=5) Since the `random` search method was specified in the sweep configuration, the sweep controller provides randomly generated hyperparameter values. {{% /alert %}} -For more information on how to create W&B Sweeps in a terminal, see the [W&B Sweep walkthrough](/guides/sweeps/walkthrough). +For more information on how to create W&B Sweeps in a terminal, see the [W&B Sweep walkthrough]({{< relref "/guides/models/sweeps/walkthrough" >}}). ## Visualize Sweep Results diff --git a/content/tutorials/tables.md b/content/tutorials/tables.md index 47c206b55..855e3a6fa 100644 --- a/content/tutorials/tables.md +++ b/content/tutorials/tables.md @@ -250,4 +250,4 @@ wandb.finish() ## What's next? The next tutorial, you will learn how to optimize hyperparameters using W&B Sweeps: -## 👉 [Optimize Hyperparameters](sweeps.md) +## 👉 [Optimize Hyperparameters]({{< relref "sweeps.md" >}}) diff --git a/content/tutorials/workspaces.md b/content/tutorials/workspaces.md index 31ceddc48..0ba693861 100644 --- a/content/tutorials/workspaces.md +++ b/content/tutorials/workspaces.md @@ -10,7 +10,7 @@ weight: 5 {{< cta-button colabLink="https://colab.research.google.com/github/wandb/wandb-workspaces/blob/Update-wandb-workspaces-tuturial/Workspace_tutorial.ipynb" >}} Organize and visualize your machine learning experiments more effectively by programmatically creating, managing, and customizing workspaces. You can define configurations, set panel layouts, and organize sections with the [`wandb-workspaces`](https://github.com/wandb/wandb-workspaces/tree/main) W&B library. You can load and modify workspaces by URL, use expressions to filter and group runs, and customize the appearances of runs. -`wandb-workspaces` is a Python library for programmatically creating and customizing W&B [Workspaces](/guides/track/workspaces) and [Reports](/guides/reports). +`wandb-workspaces` is a Python library for programmatically creating and customizing W&B [Workspaces]({{< relref "/guides/models/track/workspaces/" >}}) and [Reports]({{< relref "/guides/core/reports/" >}}). In this tutorial you will see how to use `wandb-workspaces` to create and customize workspaces by defining configurations, set panel layouts, and organize sections. @@ -20,7 +20,7 @@ In this tutorial you will see how to use `wandb-workspaces` to create and custom {{% alert %}} -Programmatic interaction with workspaces is currently supported for [**Saved workspaces views**](/guides/track/workspaces#saved-workspace-views). Saved workspaces views are collaborative snapshots of a workspace. Anyone on your team can view, edit, and save changes to saved workspace views. +Programmatic interaction with workspaces is currently supported for [**Saved workspaces views**]({{< relref "/guides/models/track/workspaces#saved-workspace-views" >}}). Saved workspaces views are collaborative snapshots of a workspace. Anyone on your team can view, edit, and save changes to saved workspace views. {{% /alert %}} ## 1. Install and import dependencies