Skip to content

Commit

Permalink
Update install instructions for prefect-azure for 3.0GA (#15197)
Browse files Browse the repository at this point in the history
  • Loading branch information
discdiver authored Sep 3, 2024
1 parent 6dc401d commit 10143f8
Show file tree
Hide file tree
Showing 17 changed files with 293 additions and 359 deletions.
30 changes: 13 additions & 17 deletions docs/integrations/prefect-aws/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,30 +9,26 @@ For example, you can retrieve secrets using AWS Secrets Manager, read and write

### Prerequisites

- [Prefect installed](/3.0/get-started/install/).
- An [AWS account](https://aws.amazon.com/account/) and the necessary permissions to access desired services.

### Install prefect-aws
### Install `prefect-aws`

Install or update to the latest version of the prefect-aws library and dependencies.
The following command will install a version of `prefect-aws` compatible with your installed version of `prefect`.
If you don't already have `prefect` installed, it will install the newest version of `prefect` as well.

<Tabs>
<Tab title="Prefect 2">
```bash
pip install -U prefect-aws
```
</Tab>
<Tab title="Prefect 3">
```bash
pip install -U -pre prefect-aws
```
</Tab>
</Tabs>
```bash
pip install "prefect[aws]"
```

Upgrade to the latest versions of `prefect` and `prefect-aws`:

```bash
pip install -U "prefect[aws]"
```

### Register newly installed block types

Register the block types in the prefect-aws module to make them available for use.
Register the block types in the `prefect-aws` module to make them available for use.

```bash
prefect block register -m prefect_aws
Expand Down Expand Up @@ -152,4 +148,4 @@ if __name__ == "__main__":

For assistance using AWS, consult the [AWS documentation](https://docs.aws.amazon.com/) and, in particular, the [Boto3 documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html).

Refer to the prefect-aws API documentation linked in the sidebar to explore all the capabilities of the prefect-aws library.
Refer to the `prefect-aws` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-aws` library.
68 changes: 27 additions & 41 deletions docs/integrations/prefect-azure/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,57 +9,39 @@ For example, you can retrieve secrets, read and write Blob Storage objects, and

### Prerequisites

- [Prefect installed](3.0/get-started/install) in a virtual environment.
- An [Azure account](https://azure.microsoft.com/) and the necessary permissions to access desired services.

### Install prefect-azure
### Install `prefect-azure`

Install or update to the latest version of the prefect-azure library and dependencies.
The following command will install a version of `prefect-azure` compatible with your installed version of `prefect`.
If you don't already have `prefect` installed, it will install the newest version of `prefect` as well.

<Tabs>
<Tab title="Prefect 2">
```bash
pip install -U prefect-azure
```
</Tab>
<Tab title="Prefect 3">
```bash
pip install -U -pre prefect-azure
```
</Tab>
</Tabs>
```bash
pip install "prefect[azure]"
```

Upgrade to the latest versions of `prefect` and `prefect-azure`:

If necessary, see [additional installation options for Blob Storage, Cosmos DB, and ML Datastore](#additional-installation-options).

To install with all additional capabilities, use the following command:
```bash
pip install -U "prefect[azure]"
```

Install or update to the latest version of the prefect-aws library and dependencies.
If necessary, see [additional installation options for Blob Storage, Cosmos DB, and ML Datastore](#additional-installation-options).

<Tabs>
<Tab title="Prefect 2">
```bash
pip install -U prefect-azure[all_extras]
```
</Tab>
<Tab title="Prefect 3">
```bash
pip install -U -pre prefect-azure[all_extras]
```
</Tab>
</Tabs>
To install prefect-azure with all additional capabilities, run the install command above and then run the following command:

```bash
pip install "prefect-azure[all_extras]"
```

### Register newly installed block types

Register the block types in the module to make them available for use.


```bash
prefect block register -m prefect_azure
```


## Examples

### Download a blob
Expand Down Expand Up @@ -108,28 +90,32 @@ If you're using Prefect Cloud, [ACI push work pools](/3.0/deploy/infrastructure-

For assistance using Azure, consult the [Azure documentation](https://learn.microsoft.com/en-us/azure).

Refer to the prefect-azure API documentation linked in the sidebar to explore all the capabilities of the prefect-azure library.
Refer to the `prefect-azure` API documentation linked in the sidebar to explore all the capabilities of the `prefect-azure` library.

### Additional installation options

To use Blob Storage:
First install the main library compatible with your `prefect` version:

```bash
pip install -U "prefect-azure[blob_storage]"
pip install "prefect[azure]"
```

Then install the additional capabilities you need.

To use Cosmos DB:
To use Blob Storage:

```bash
pip install -U "prefect-azure[cosmos_db]"
pip install "prefect-azure[blob_storage]"
```

To use Cosmos DB:

```bash
pip install "prefect-azure[cosmos_db]"
```

To use ML Datastore:


```bash
pip install -U "prefect-azure[ml_datastore]"
pip install "prefect-azure[ml_datastore]"
```

33 changes: 13 additions & 20 deletions docs/integrations/prefect-bitbucket/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,44 +2,37 @@
title: prefect-bitbucket
---

The prefect-bitbucket library makes it easy to interact with Bitbucket repositories and credentials.
The `prefect-bitbucket` library makes it easy to interact with Bitbucket repositories and credentials.

## Getting started

### Prerequisites

- [Prefect installed](/3.0/get-started/install/).
- A [Bitbucket account](https://bitbucket.org/product).

### Install prefect-bitbucket
### Install `prefect-bitbucket`

The following command will install a version of `prefect-bitbucket` compatible with your installed version of `prefect`.
If you don't already have `prefect` installed, it will install the newest version of `prefect` as well.

Install or update to the latest version of the prefect-bitbucket library and dependencies.
```bash
pip install "prefect[bitbucket]"
```

<Tabs>
<Tab title="Prefect 2">
```bash
pip install -U prefect-bitbucket
```
</Tab>
<Tab title="Prefect 3">
```bash
pip install -U -pre prefect-bitbucket
```
</Tab>
</Tabs>
Upgrade to the latest versions of `prefect` and `prefect-bitbucket`:

```bash
pip install -U "prefect[bitbucket]"
```

### Register newly installed block types

Register the block types in the prefect-bitbucket module to make them available for use.

Register the block types in the `prefect-bitbucket` module to make them available for use.

```bash
prefect block register -m prefect_bitbucket
```


## Examples

In the examples below, you create blocks with Python code.
Expand Down Expand Up @@ -145,4 +138,4 @@ if __name__ == "__main__":

For assistance using Bitbucket, consult the [Bitbucket documentation](https://bitbucket.org/product/guides).

Refer to the prefect-bitbucket API documentation linked in the sidebar to explore all the capabilities of the prefect-bitbucket library.
Refer to the `prefect-bitbucket` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-bitbucket` library.
34 changes: 16 additions & 18 deletions docs/integrations/prefect-dask/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,26 +8,20 @@ The `prefect-dask` integration makes it easy to accelerate your flow runs with D

## Getting started

### Prerequisites
### Install `prefect-dask`

- [Prefect installed](/3.0/get-started/install) in a virtual environment.
The following command will install a version of `prefect-dask` compatible with your installed version of `prefect`.
If you don't already have `prefect` installed, it will install the newest version of `prefect` as well.

### Install `prefect-dask`
```bash
pip install "prefect[dask]"
```

Install or update to the latest version of the prefect-dask library and dependencies.
Upgrade to the latest versions of `prefect` and `prefect-dask`:

<Tabs>
<Tab title="Prefect 2">
```bash
pip install -U prefect-dask
```
</Tab>
<Tab title="Prefect 3">
```bash
pip install -U -pre prefect-dask
```
</Tab>
</Tabs>
```bash
pip install -U "prefect[dask]"
```

## Why use Dask?

Expand Down Expand Up @@ -253,6 +247,7 @@ def dask_pipeline():
df_yearly_average = process_data(df)
return df_yearly_average


if __name__ == "__main__":
dask_pipeline()
```
Expand All @@ -266,6 +261,7 @@ import dask.distributed
from prefect import flow, task
from prefect_dask import DaskTaskRunner, get_dask_client


client = dask.distributed.Client()

@task
Expand All @@ -285,6 +281,7 @@ def dask_pipeline():
df_yearly_average = process_data.submit(df)
return df_yearly_average


if __name__ == "__main__":
dask_pipeline()
```
Expand Down Expand Up @@ -322,6 +319,7 @@ import dask
from prefect import flow, task
from prefect_dask.task_runners import DaskTaskRunner


@task
def show(x):
print(x)
Expand All @@ -343,6 +341,7 @@ import dask
from prefect import flow, task
from prefect_dask.task_runners import DaskTaskRunner


@task
def show(x):
print(x)
Expand Down Expand Up @@ -373,9 +372,8 @@ if __name__ == "__main__":
my_flow()
```


## Additional Resources

Refer to the prefect-dask API documentation linked in the sidebar to explore all the capabilities of the prefect-dask library.
Refer to the `prefect-dask` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-dask` library.

For assistance using Dask, consult the [Dask documentation](https://docs.dask.org/en/stable/)
37 changes: 18 additions & 19 deletions docs/integrations/prefect-databricks/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,23 @@ Prefect integrations for interacting with Databricks.

### Prerequisites

- [Prefect installed](/3.0/get-started/install) in a virtual environment.
- A [Databricks account](https://databricks.com/) and the necessary permissions to access desired services.

### Install `prefect-databricks`

Install or update to the latest version of the prefect-databrickslibrary and dependencies.

<Tabs>
<Tab title="Prefect 2">
```bash
pip install -U prefect-databricks
```
</Tab>
<Tab title="Prefect 3">
```bash
pip install -U -pre prefect-databricks
```
</Tab>
</Tabs>

The following command will install a version of `prefect-bitbucket` compatible with your installed version of `prefect`.
If you don't already have `prefect` installed, it will install the newest version of `prefect` as well.

```bash
pip install "prefect[databricks]"
```

Upgrade to the latest versions of `prefect` and `prefect-databricks`:

```bash
pip install -U "prefect[databricks]"
```

### List jobs on the Databricks instance

Expand All @@ -45,7 +43,8 @@ def example_execute_endpoint_flow():
)
return jobs

example_execute_endpoint_flow()
if __name__ == "__main__":
example_execute_endpoint_flow()
```

### Use `with_options` to customize options on any existing task or flow
Expand Down Expand Up @@ -125,8 +124,8 @@ def jobs_runs_submit_flow(notebook_path, **base_parameters):

return run


jobs_runs_submit_flow("/Users/[email protected]/example.ipynb", name="Marvin")
if __name__ == "__main__":
jobs_runs_submit_flow("/Users/[email protected]/example.ipynb", name="Marvin")
```

Note, instead of using the built-in models, you may also input valid JSON. For example, `AutoScale(min_workers=1, max_workers=2)` is equivalent to `{"min_workers": 1, "max_workers": 2}`.
Expand All @@ -135,4 +134,4 @@ Note, instead of using the built-in models, you may also input valid JSON. For e

For assistance using Databricks, consult the [Databricks documentation](https://www.databricks.com/databricks-documentation).

Refer to the prefect-databricks API documentation linked in the sidebar to explore all the capabilities of the prefect-databricks library.
Refer to the `prefect-databricks` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-databricks` library.
Loading

0 comments on commit 10143f8

Please sign in to comment.