Skip to content

Commit

Permalink
Update artifacts doc page (#15093)
Browse files Browse the repository at this point in the history
  • Loading branch information
discdiver authored Aug 27, 2024
1 parent 05b6406 commit 8291ab5
Showing 1 changed file with 46 additions and 62 deletions.
108 changes: 46 additions & 62 deletions docs/3.0rc/develop/artifacts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,25 @@ description: Prefect artifacts are persisted outputs designed for human consumpt
Prefect artifacts:

- are visually rich annotations on flow and task runs
- are human-readable visual metadata produced in code
- come in standardized formats such as tables, progress indicators, images, Markdown, or links
- are human-readable visual metadata defined in code
- come in standardized formats such as tables, progress indicators, images, Markdown, and links
- are stored in Prefect Cloud or Prefect server and rendered in the Prefect UI
- make it easy to visualize outputs or side effects that your runs produce, and capture updates over time

![Markdown artifact sales report screenshot](/3.0rc/img/ui/md-artifact-info.png)

Common use cases for artifacts include:

- **Progress** indicators: Publish progress indicators for long-running tasks. This helps monitor the
progress of your tasks and flows and ensure they are running as expected.
- **Debugging**: By publishing data that you care about in the UI, you can easily see when and where
your results were written. If an artifact doesn't look the way you expect, you can find out which flow
run last updated it, and you can click through a link in the artifact to a storage location (such as an S3 bucket).
- **Data quality checks**: Publish data quality checks from in-progress tasks.
This helps ensure that data quality is maintained throughout the pipeline. Artifacts make for
great performance graphs, such as for long-running tasks like ML model training. You can also track artifact versions,
making it easier to identify changes in your data.
- **Progress** indicators: Publish progress indicators for long-running tasks. This helps monitor the progress of your tasks and flows and ensure they are running as expected.
- **Debugging**: Publish data that you care about in the UI to easily see when and where your results were written. If an artifact doesn't look the way you expect, you can find out which flow run last updated it, and you can click through a link in the artifact to a storage location (such as an S3 bucket).
- **Data quality checks**: Publish data quality checks from in-progress tasks to ensure that data quality is maintained throughout a pipeline. Artifacts make for great performance graphs. For example, you can visualize a long-running machine learning model training run. You can also track artifact versions, making it easier to identify changes in your data.
- **Documentation**: Publish documentation and sample data to help you keep track of your work and share information.
For example, add a description to signify why a piece of data is important.

## Create artifacts

Creating artifacts allows you to publish data from task and flow runs or outside of a flow run context.
Creating artifacts allows you to publish data from task or flow runs.

There are five artifact types:

- links
Expand All @@ -42,24 +37,25 @@ There are five artifact types:
Each artifact created within a task is displayed individually in the Prefect UI.
This means that each call to `create_link_artifact()` or `create_markdown_artifact()` generates a distinct artifact.

Unlike the `print()` command (where you can concatenate multiple calls to include additional items in a report),
these commands must be used multiple times if necessary within a task.
Unlike the Python `print()` function (where you can concatenate multiple calls to include additional items in a report),
these artifact creation functions must be called multiple times, if necessary.

To create artifacts like reports or summaries using `create_markdown_artifact()`, compile your message string
separately and then pass it to `create_markdown_artifact()` to create the complete artifact.
To create artifacts such as reports or summaries using `create_markdown_artifact()`, define your message string
and then pass it to `create_markdown_artifact()` to create the artifact.
</Note>

### Create link artifacts

To create a link artifact, use the `create_link_artifact()` function.
To create multiple versions of the same artifact and/or view them on the Artifacts page of the Prefect UI,
provide a `key` argument to the `create_link_artifact()` function to track an artifact's history over time.
provide a `key` argument to the `create_link_artifact()` function to track the artifact's history over time.
Without a `key`, the artifact is only visible in the Artifacts tab of the associated flow run or task run.

```python
from prefect import flow, task
from prefect.artifacts import create_link_artifact


@task
def my_first_task():
create_link_artifact(
Expand All @@ -68,6 +64,7 @@ def my_first_task():
description="## Highly variable data",
)


@task
def my_second_task():
create_link_artifact(
Expand All @@ -76,42 +73,43 @@ def my_second_task():
description="# Low prediction accuracy",
)


@flow
def my_flow():
my_first_task()
my_second_task()


if __name__ == "__main__":
my_flow()
```

<Tip>
**Specify multiple artifacts with the same key for artifact lineage**

You can specify multiple artifacts with the same key to easily track something very specific, such as irregularities in your data pipeline.
</Tip>
You can specify multiple artifacts with the same key to easily track something very
specific, such as irregularities in your data pipeline.

After running the above flows, you can find your new artifacts in the Artifacts page of the UI.
Click into the "irregular-data" artifact and see all versions of it, along with custom descriptions and
links to the relevant data.
After running flows that create artifacts, view the artifacts in the **Artifacts** page of the UI.
Click into the "irregular-data" artifact to see its versions, along with custom descriptions and links to the relevant data.

![Link artifact details with multiple versions](/3.0rc/img/ui/link-artifact-info.png)

You can also view information about your artifact such as:
You can also view information about the artifact such as:

- its associated flow run or task run id
- previous and future versions of the artifact (multiple artifacts can have the same key to show lineage)
- the data you've stored (in this case a Markdown-rendered link)
- data (in this case a Markdown-rendered link)
- an optional Markdown description
- when the artifact was created or updated

To make the links more readable for you and your collaborators, you can pass in a `link_text` argument for your
link artifacts:
To make the links more readable for you and your collaborators, you can pass a `link_text` argument:

```python
from prefect import flow
from prefect.artifacts import create_link_artifact


@flow
def my_flow():
create_link_artifact(
Expand All @@ -120,23 +118,20 @@ def my_flow():
link_text="Prefect",
)


if __name__ == "__main__":
my_flow()
```

In the above example, the `create_link_artifact` method is used within a flow to create a link artifact with a
key of `my-important-link`.
The `link` parameter specifies the external resource to link to, and `link_text` specifies
the text to display for the link.
You can also add an optional `description` for context.
In the above example, the `create_link_artifact` method is used within a flow to create a link artifact with a key of `my-important-link`.
The `link` parameter specifies the external resource to link to, and `link_text` specifies the text to display for the link.
Add an optional `description` for context.

### Create progress artifacts

Progress artifacts render dynamically on the flow run graph in the Prefect UI, indicating the progress of
long-running tasks.
Progress artifacts render dynamically on the flow run graph in the Prefect UI, indicating the progress of long-running tasks.

To create a progress artifact, use the `create_progress_artifact()` function. To update a progress artifact,
use the `update_progress_artifact()` function.
To create a progress artifact, use the `create_progress_artifact()` function. To update a progress artifact, use the `update_progress_artifact()` function.

![Progress artifact example](/3.0rc/img/ui/progress-artifact-example.png)

Expand Down Expand Up @@ -173,18 +168,14 @@ def etl():

if __name__ == "__main__":
etl()

```

Progress artifacts are updated using a separate `update_progress_artifact()` function. Prefect updates a
progress artifact in place, rather than versioning it.

Progress artifacts are updated with the `update_progress_artifact()` function. Prefect updates a progress artifact in place, rather than versioning it.

### Create Markdown artifacts

To create a Markdown artifact, you can use the `create_markdown_artifact()` function.
To create multiple versions of the same artifact and/or view them on the Artifacts page of the Prefect UI,
provide a `key` argument to the `create_markdown_artifact()` function to track an artifact's history over time.
To create multiple versions of the same artifact and/or view them on the Artifacts page of the Prefect UI, provide a `key` argument to the `create_markdown_artifact()` function to track an artifact's history over time.
Without a `key`, the artifact is only visible in the Artifacts tab of the associated flow run or task run.

<Warning>
Expand All @@ -197,6 +188,7 @@ Don't indent Markdown in mult-line strings. Otherwise it will be interpreted inc
from prefect import flow, task
from prefect.artifacts import create_markdown_artifact


@task
def markdown_task():
na_revenue = 500000
Expand Down Expand Up @@ -235,6 +227,7 @@ the coming quarter.
description="Quarterly Sales Report",
)


@flow()
def my_flow():
markdown_task()
Expand All @@ -253,18 +246,17 @@ You can view the associated flow run id or task run id, previous versions of the
### Create table artifacts

Create a table artifact by calling `create_table_artifact()`.
To create multiple versions of the same artifact and/or view them on the Artifacts page of the Prefect UI,
provide a `key` argument to the `create_table_artifact()` function to track an artifact's history over time.
To create multiple versions of the same artifact and/or view them on the Artifacts page of the Prefect UI, provide a `key` argument to the `create_table_artifact()` function to track an artifact's history over time.
Without a `key`, the artifact is only visible in the artifacts tab of the associated flow run or task run.

<Note>
The `create_table_artifact()` function accepts a `table` argument. Pass this as a
list of lists, a list of dictionaries, or a dictionary of lists.
The `create_table_artifact()` function accepts a `table` argument. Pass this as a list of lists, a list of dictionaries, or a dictionary of lists.
</Note>

```python
from prefect.artifacts import create_table_artifact


def my_fn():
highest_churn_possibility = [
{'customer_id':'12345', 'name': 'John Smith', 'churn_probability': 0.85 },
Expand All @@ -277,18 +269,16 @@ def my_fn():
description= "# Marvin, please reach out to these customers today!"
)


if __name__ == "__main__":
my_fn()
```

![Table artifact with customer info](/3.0rc/img/ui/table-artifact-info.png)
You don't need to create an artifact in a flow run context.
You can create one anywhere in a Python script and see it in the Prefect UI.

### Create image artifacts

Image artifacts render publicly available images in the Prefect UI. To create an image artifact,
use the `create_image_artifact()` function.
Image artifacts render publicly available images in the Prefect UI. To create an image artifact, use the `create_image_artifact()` function.

![Image artifact example](/3.0rc/img/ui/image-artifact-example.png)

Expand All @@ -315,7 +305,6 @@ def my_flow():
if __name__ == "__main__":
image_url = my_flow()
print(f"Image URL: {image_url}")

```

To create an artifact that links to a private image, use the `create_link_artifact()` function instead.
Expand All @@ -324,8 +313,7 @@ To create an artifact that links to a private image, use the `create_link_artifa

### Reading artifacts

In the Prefect UI, you can view all of the latest versions of your artifacts and click into a specific artifact
to see its lineage over time.
In the Prefect UI, you can view all of the latest versions of your artifacts and click into a specific artifact to see its lineage over time.
Additionally, you can inspect all versions of an artifact with a given key from the CLI by running:

```bash
Expand All @@ -348,13 +336,13 @@ In Python code, you can retrieve an existing artifact with the `Artifact.get` cl
```python
from prefect.artifacts import Artifact

my_retrieved_artifact = Artifact.get("my_artifact_key")

my_retrieved_artifact = Artifact.get("my_artifact_key")
```

### Delete artifacts

You can delete an artifact directly using the CLI to delete specific artifacts with a given key or id:
Delete an artifact in the CLI by providing a key or id:

```bash
prefect artifact delete <my_key>
Expand All @@ -364,19 +352,17 @@ prefect artifact delete <my_key>
prefect artifact delete --id <my_id>
```

Alternatively, you can delete artifacts using the [Prefect REST API](https://docs.prefect.io/latest/api-ref/rest-api-reference/#tag/Artifacts/operation/delete_artifact_api_accounts__account_id__workspaces__workspace_id__artifacts__id__delete).

## Artifacts API

Prefect provides the [Prefect REST API](https://docs.prefect.io/latest/api-ref/rest-api-reference/#tag/Artifacts)
to allow you to create, read, and delete artifacts programmatically.
Create, read, or delete artifacts programmatically through the [Prefect REST API](/3.0rc/api-ref/rest-api/).
With the Artifacts API, you can automate the creation and management of artifacts as part of your workflow.

For example, to read the five most recently created Markdown, table, and link artifacts, you can run the following:

```python
import requests


PREFECT_API_URL="https://api.prefect.cloud/api/accounts/abc/workspaces/xyz"
PREFECT_API_KEY="pnu_ghijk"
data = {
Expand All @@ -398,8 +384,6 @@ for artifact in response.json():
print(artifact)
```

If you don't specify a key or that a key must exist, you will also return results
(which are a type of key-less artifact).
If you don't specify a key or that a key must exist, you will also return results, which are a type of key-less artifact.

See the rest of the [Prefect REST API documentation](https://app.prefect.cloud/api/docs#tag/Artifacts)
on artifacts for more information.
See the [Prefect REST API documentation](/3.0rc/api-ref/rest-api/) on artifacts for more information.

0 comments on commit 8291ab5

Please sign in to comment.