diff --git a/docs/integrations/prefect-aws/index.mdx b/docs/integrations/prefect-aws/index.mdx index ef8e12948264..7291d159c19e 100644 --- a/docs/integrations/prefect-aws/index.mdx +++ b/docs/integrations/prefect-aws/index.mdx @@ -9,30 +9,26 @@ For example, you can retrieve secrets using AWS Secrets Manager, read and write ### Prerequisites -- [Prefect installed](/3.0/get-started/install/). - An [AWS account](https://aws.amazon.com/account/) and the necessary permissions to access desired services. -### Install prefect-aws +### Install `prefect-aws` -Install or update to the latest version of the prefect-aws library and dependencies. +The following command will install a version of `prefect-aws` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. - - - ```bash - pip install -U prefect-aws - ``` - - - ```bash - pip install -U -pre prefect-aws - ``` - - +```bash +pip install "prefect[aws]" +``` + +Upgrade to the latest versions of `prefect` and `prefect-aws`: +```bash +pip install -U "prefect[aws]" +``` ### Register newly installed block types -Register the block types in the prefect-aws module to make them available for use. +Register the block types in the `prefect-aws` module to make them available for use. ```bash prefect block register -m prefect_aws @@ -152,4 +148,4 @@ if __name__ == "__main__": For assistance using AWS, consult the [AWS documentation](https://docs.aws.amazon.com/) and, in particular, the [Boto3 documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html). -Refer to the prefect-aws API documentation linked in the sidebar to explore all the capabilities of the prefect-aws library. +Refer to the `prefect-aws` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-aws` library. diff --git a/docs/integrations/prefect-azure/index.mdx b/docs/integrations/prefect-azure/index.mdx index 3932f46d3095..d2c8f0fa88e9 100644 --- a/docs/integrations/prefect-azure/index.mdx +++ b/docs/integrations/prefect-azure/index.mdx @@ -9,57 +9,39 @@ For example, you can retrieve secrets, read and write Blob Storage objects, and ### Prerequisites -- [Prefect installed](3.0/get-started/install) in a virtual environment. - An [Azure account](https://azure.microsoft.com/) and the necessary permissions to access desired services. -### Install prefect-azure +### Install `prefect-azure` -Install or update to the latest version of the prefect-azure library and dependencies. +The following command will install a version of `prefect-azure` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. - - - ```bash - pip install -U prefect-azure - ``` - - - ```bash - pip install -U -pre prefect-azure - ``` - - +```bash +pip install "prefect[azure]" +``` +Upgrade to the latest versions of `prefect` and `prefect-azure`: -If necessary, see [additional installation options for Blob Storage, Cosmos DB, and ML Datastore](#additional-installation-options). - -To install with all additional capabilities, use the following command: +```bash +pip install -U "prefect[azure]" +``` -Install or update to the latest version of the prefect-aws library and dependencies. +If necessary, see [additional installation options for Blob Storage, Cosmos DB, and ML Datastore](#additional-installation-options). - - - ```bash - pip install -U prefect-azure[all_extras] - ``` - - - ```bash - pip install -U -pre prefect-azure[all_extras] - ``` - - +To install prefect-azure with all additional capabilities, run the install command above and then run the following command: +```bash +pip install "prefect-azure[all_extras]" +``` ### Register newly installed block types Register the block types in the module to make them available for use. - ```bash prefect block register -m prefect_azure ``` - ## Examples ### Download a blob @@ -108,28 +90,32 @@ If you're using Prefect Cloud, [ACI push work pools](/3.0/deploy/infrastructure- For assistance using Azure, consult the [Azure documentation](https://learn.microsoft.com/en-us/azure). -Refer to the prefect-azure API documentation linked in the sidebar to explore all the capabilities of the prefect-azure library. +Refer to the `prefect-azure` API documentation linked in the sidebar to explore all the capabilities of the `prefect-azure` library. ### Additional installation options -To use Blob Storage: +First install the main library compatible with your `prefect` version: ```bash -pip install -U "prefect-azure[blob_storage]" +pip install "prefect[azure]" ``` +Then install the additional capabilities you need. -To use Cosmos DB: +To use Blob Storage: ```bash -pip install -U "prefect-azure[cosmos_db]" +pip install "prefect-azure[blob_storage]" ``` +To use Cosmos DB: + +```bash +pip install "prefect-azure[cosmos_db]" +``` To use ML Datastore: - ```bash -pip install -U "prefect-azure[ml_datastore]" +pip install "prefect-azure[ml_datastore]" ``` - diff --git a/docs/integrations/prefect-bitbucket/index.mdx b/docs/integrations/prefect-bitbucket/index.mdx index de3f4e749d42..7450648390b1 100644 --- a/docs/integrations/prefect-bitbucket/index.mdx +++ b/docs/integrations/prefect-bitbucket/index.mdx @@ -2,44 +2,37 @@ title: prefect-bitbucket --- -The prefect-bitbucket library makes it easy to interact with Bitbucket repositories and credentials. +The `prefect-bitbucket` library makes it easy to interact with Bitbucket repositories and credentials. ## Getting started ### Prerequisites -- [Prefect installed](/3.0/get-started/install/). - A [Bitbucket account](https://bitbucket.org/product). -### Install prefect-bitbucket +### Install `prefect-bitbucket` +The following command will install a version of `prefect-bitbucket` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -Install or update to the latest version of the prefect-bitbucket library and dependencies. +```bash +pip install "prefect[bitbucket]" +``` - - - ```bash - pip install -U prefect-bitbucket - ``` - - - ```bash - pip install -U -pre prefect-bitbucket - ``` - - +Upgrade to the latest versions of `prefect` and `prefect-bitbucket`: +```bash +pip install -U "prefect[bitbucket]" +``` ### Register newly installed block types -Register the block types in the prefect-bitbucket module to make them available for use. - +Register the block types in the `prefect-bitbucket` module to make them available for use. ```bash prefect block register -m prefect_bitbucket ``` - ## Examples In the examples below, you create blocks with Python code. @@ -145,4 +138,4 @@ if __name__ == "__main__": For assistance using Bitbucket, consult the [Bitbucket documentation](https://bitbucket.org/product/guides). -Refer to the prefect-bitbucket API documentation linked in the sidebar to explore all the capabilities of the prefect-bitbucket library. +Refer to the `prefect-bitbucket` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-bitbucket` library. diff --git a/docs/integrations/prefect-dask/index.mdx b/docs/integrations/prefect-dask/index.mdx index d8066a012342..afe5c4677b91 100644 --- a/docs/integrations/prefect-dask/index.mdx +++ b/docs/integrations/prefect-dask/index.mdx @@ -8,26 +8,20 @@ The `prefect-dask` integration makes it easy to accelerate your flow runs with D ## Getting started -### Prerequisites +### Install `prefect-dask` -- [Prefect installed](/3.0/get-started/install) in a virtual environment. +The following command will install a version of `prefect-dask` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -### Install `prefect-dask` +```bash +pip install "prefect[dask]" +``` -Install or update to the latest version of the prefect-dask library and dependencies. +Upgrade to the latest versions of `prefect` and `prefect-dask`: - - - ```bash - pip install -U prefect-dask - ``` - - - ```bash - pip install -U -pre prefect-dask - ``` - - +```bash +pip install -U "prefect[dask]" +``` ## Why use Dask? @@ -253,6 +247,7 @@ def dask_pipeline(): df_yearly_average = process_data(df) return df_yearly_average + if __name__ == "__main__": dask_pipeline() ``` @@ -266,6 +261,7 @@ import dask.distributed from prefect import flow, task from prefect_dask import DaskTaskRunner, get_dask_client + client = dask.distributed.Client() @task @@ -285,6 +281,7 @@ def dask_pipeline(): df_yearly_average = process_data.submit(df) return df_yearly_average + if __name__ == "__main__": dask_pipeline() ``` @@ -322,6 +319,7 @@ import dask from prefect import flow, task from prefect_dask.task_runners import DaskTaskRunner + @task def show(x): print(x) @@ -343,6 +341,7 @@ import dask from prefect import flow, task from prefect_dask.task_runners import DaskTaskRunner + @task def show(x): print(x) @@ -373,9 +372,8 @@ if __name__ == "__main__": my_flow() ``` - ## Additional Resources -Refer to the prefect-dask API documentation linked in the sidebar to explore all the capabilities of the prefect-dask library. +Refer to the `prefect-dask` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-dask` library. For assistance using Dask, consult the [Dask documentation](https://docs.dask.org/en/stable/) \ No newline at end of file diff --git a/docs/integrations/prefect-databricks/index.mdx b/docs/integrations/prefect-databricks/index.mdx index baf8f6060938..fd5484dc697c 100644 --- a/docs/integrations/prefect-databricks/index.mdx +++ b/docs/integrations/prefect-databricks/index.mdx @@ -8,25 +8,23 @@ Prefect integrations for interacting with Databricks. ### Prerequisites -- [Prefect installed](/3.0/get-started/install) in a virtual environment. - A [Databricks account](https://databricks.com/) and the necessary permissions to access desired services. ### Install `prefect-databricks` -Install or update to the latest version of the prefect-databrickslibrary and dependencies. - - - - ```bash - pip install -U prefect-databricks - ``` - - - ```bash - pip install -U -pre prefect-databricks - ``` - - + +The following command will install a version of `prefect-bitbucket` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. + +```bash +pip install "prefect[databricks]" +``` + +Upgrade to the latest versions of `prefect` and `prefect-databricks`: + +```bash +pip install -U "prefect[databricks]" +``` ### List jobs on the Databricks instance @@ -45,7 +43,8 @@ def example_execute_endpoint_flow(): ) return jobs -example_execute_endpoint_flow() +if __name__ == "__main__": + example_execute_endpoint_flow() ``` ### Use `with_options` to customize options on any existing task or flow @@ -125,8 +124,8 @@ def jobs_runs_submit_flow(notebook_path, **base_parameters): return run - -jobs_runs_submit_flow("/Users/username@gmail.com/example.ipynb", name="Marvin") +if __name__ == "__main__": + jobs_runs_submit_flow("/Users/username@gmail.com/example.ipynb", name="Marvin") ``` Note, instead of using the built-in models, you may also input valid JSON. For example, `AutoScale(min_workers=1, max_workers=2)` is equivalent to `{"min_workers": 1, "max_workers": 2}`. @@ -135,4 +134,4 @@ Note, instead of using the built-in models, you may also input valid JSON. For e For assistance using Databricks, consult the [Databricks documentation](https://www.databricks.com/databricks-documentation). -Refer to the prefect-databricks API documentation linked in the sidebar to explore all the capabilities of the prefect-databricks library. +Refer to the `prefect-databricks` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-databricks` library. diff --git a/docs/integrations/prefect-dbt/index.mdx b/docs/integrations/prefect-dbt/index.mdx index eaf5fe253292..5c78b9313441 100644 --- a/docs/integrations/prefect-dbt/index.mdx +++ b/docs/integrations/prefect-dbt/index.mdx @@ -9,27 +9,22 @@ Prefect provides a global view of the state of your workflows and allows you to ### Prerequisites -- [Prefect installed](3.0/get-started/install). - A [dbt Cloud account](https://cloud.getdbt.com/signup) if using dbt Cloud. -### Install prefect-dbt +### Install `prefect-dbt` +The following command will install a version of `prefect-dbt` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -Install or update to the latest version of the prefect-dbt library and dependencies. +```bash +pip install "prefect[dbt]" +``` - - - ```bash - pip install -U prefect-dbt - ``` - - - ```bash - pip install -U -pre prefect-dbt - ``` - - +Upgrade to the latest versions of `prefect` and `prefect-dbt`: +```bash +pip install -U "prefect[dbt]" +``` If necessary, see [additional installation options for dbt Core with BigQuery, Snowflake, and Postgres](#additional-installation-options). @@ -40,17 +35,14 @@ To install with all additional capabilities, use the following command: pip install -U "prefect-dbt[all_extras]" ``` - ### Register newly installed blocks types Register the block types in the prefect-dbt module to make them available for use. - ```bash prefect block register -m prefect_dbt ``` - ## dbt Cloud If you have an existing dbt Cloud job, use the pre-built flow `run_dbt_cloud_job` to trigger a job run and wait until the job run is finished. @@ -80,7 +72,6 @@ run_dbt_job_flow() Blocks can be [created through code](/3.0/develop/blocks/#saving-blocks) or through the UI. - To create a dbt Cloud Credentials block: 1. Go to your [dbt Cloud profile](https://cloud.getdbt.com/settings/profile). @@ -123,10 +114,10 @@ Load the saved block, which can access your credentials: ```python from prefect_dbt.cloud import DbtCloudJob + DbtCloudJob.load("JOB-BLOCK-NAME-PLACEHOLDER") ``` - ## dbt Core Prefect-dbt supports execution of dbt Core CLI commands. @@ -179,6 +170,7 @@ os.environ["DBT_PASSWORD"] = DBT_PASSWORD ``` This example `profiles.yml` file could then access that variable. + ```yaml profile: target: prod @@ -194,7 +186,9 @@ profile: ### Programmatic Invocation -`prefect-dbt` has some pre-built tasks that use dbt's [programmatic invocation](https://docs.getdbt.com/reference/programmatic-invocations). For example: +`prefect-dbt` has some pre-built tasks that use dbt's [programmatic invocation](https://docs.getdbt.com/reference/programmatic-invocations). +For example: + ```python from prefect import flow from prefect_dbt.cli.tasks import from prefect import flow @@ -250,6 +244,7 @@ def trigger_dbt_flow(): result = dbt_process.fetch_result() return result + if __name__ == "__main__": trigger_dbt_flow() ``` @@ -276,6 +271,7 @@ To create dbt Core target config and profile blocks for BigQuery: from prefect_gcp.credentials import GcpCredentials from prefect_dbt.cli import BigQueryTargetConfigs, DbtCliProfile + credentials = GcpCredentials.load("CREDENTIALS-BLOCK-NAME-PLACEHOLDER") target_configs = BigQueryTargetConfigs( schema="SCHEMA-NAME-PLACEHOLDER", # also known as dataset @@ -299,6 +295,7 @@ To create a dbt Core operation block: ```python from prefect_dbt.cli import DbtCliProfile, DbtCoreOperation + dbt_cli_profile = DbtCliProfile.load("DBT-CLI-PROFILE-BLOCK-NAME-PLACEHOLDER") dbt_core_operation = DbtCoreOperation( commands=["DBT-CLI-COMMANDS-PLACEHOLDER"], @@ -313,6 +310,7 @@ Load the saved block that holds your credentials: ```python from prefect_dbt.cloud import DbtCoreOperation + DbtCoreOperation.load("DBT-CORE-OPERATION-BLOCK-NAME-PLACEHOLDER") ``` @@ -320,7 +318,7 @@ DbtCoreOperation.load("DBT-CORE-OPERATION-BLOCK-NAME-PLACEHOLDER") For assistance using dbt, consult the [dbt documentation](https://docs.getdbt.com/docs/building-a-dbt-project/documentation). -Refer to the `prefect-dbt` API documentation linked in the sidebar to explore all the capabilities of the prefect-dbt library. +Refer to the `prefect-dbt` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-dbt` library. ### Additional installation options @@ -328,24 +326,25 @@ Additional installation options for dbt Core with BigQuery, Snowflake, and Postg #### Additional capabilities for dbt Core and Snowflake profiles +First install the main library compatible with your Prefect version: ```bash -pip install -U "prefect-dbt[snowflake]" +pip install "prefect[dbt]" ``` +Then install the additional capabilities you need. +```bash +pip install "prefect-dbt[snowflake]" +``` #### Additional capabilities for dbt Core and BigQuery profiles - ```bash -pip install -U "prefect-dbt[bigquery]" +pip install "prefect-dbt[bigquery]" ``` - #### Additional capabilities for dbt Core and Postgres profiles - ```bash -pip install -U "prefect-dbt[postgres]" +pip install "prefect-dbt[postgres]" ``` - diff --git a/docs/integrations/prefect-docker/index.mdx b/docs/integrations/prefect-docker/index.mdx index 5b7e379fc19b..139b6dc588c3 100644 --- a/docs/integrations/prefect-docker/index.mdx +++ b/docs/integrations/prefect-docker/index.mdx @@ -2,34 +2,28 @@ title: prefect-docker --- -The prefect-docker library is required to create deployments that will submit runs to most Prefect work pool infrastructure types. - +The `prefect-docker` library is required to create deployments that will submit runs to most Prefect work pool infrastructure types. ## Getting started ### Prerequisites -- [Prefect installed](3.0/get-started/install) in a virtual environment. - [Docker installed](https://www.docker.com/) and running. -### Install prefect-docker +### Install `prefect-docker` +The following command will install a version of `prefect-docker` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -Install or update to the latest version of the prefect-docker library and dependencies. +```bash +pip install "prefect[docker]" +``` - - - ```bash - pip install -U prefect-docker - ``` - - - ```bash - pip install -U -pre prefect-docker - ``` - - +Upgrade to the latest versions of `prefect` and `prefect-docker`: +```bash +pip install -U "prefect[docker]" +``` ### Examples See the Prefect [Workers docs](/3.0/deploy/infrastructure-examples/docker) to learn how to create and run deployments that use Docker. @@ -37,4 +31,4 @@ See the Prefect [Workers docs](/3.0/deploy/infrastructure-examples/docker) to le For assistance using Docker, consult the [Docker documentation](https://docs.docker.com/). -Refer to the prefect-docker SDK documentation linked in the sidebar to explore all the capabilities of the prefect-docker library. +Refer to the `prefect-docker` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-docker` library. diff --git a/docs/integrations/prefect-email/index.mdx b/docs/integrations/prefect-email/index.mdx index d3628e4fe4b8..46d4e038fcb9 100644 --- a/docs/integrations/prefect-email/index.mdx +++ b/docs/integrations/prefect-email/index.mdx @@ -8,27 +8,23 @@ The `prefect-email` library helps you send emails from your Prefect flows. ### Prerequisites -- [Prefect installed](/3.0/get-started/install/) in a virtual environment. - Many email services, such as Gmail, require an [App Password](https://support.google.com/accounts/answer/185833) to successfully send emails. If you encounter an error similar to `smtplib.SMTPAuthenticationError: (535, b'5.7.8 Username and Password not accepted...`, it's likely you are not using an App Password. -### Install prefect-email +### Install `prefect-email` +The following command will install a version of prefect-email compatible with your installed version of Prefect. +If you don't already have Prefect installed, it will install the newest version of Prefect as well. -Install or update to the latest version of the prefect-email library and dependencies. +```bash +pip install "prefect[email]" +``` + +Upgrade to the latest versions of Prefect and prefect-email: - - - ```bash - pip install -U prefect-email - ``` - - - ```bash - pip install -U -pre prefect-email - ``` - - +```bash +pip install -U "prefect[email]" +``` ### Register newly installed block types @@ -46,6 +42,7 @@ Replace the placeholders with your email address and password. ```python from prefect_email import EmailServerCredentials + credentials = EmailServerCredentials( username="EMAIL-ADDRESS-PLACEHOLDER", password="PASSWORD-PLACEHOLDER", # must be an application password @@ -63,6 +60,7 @@ The code below shows how to send an email using the pre-built `email_send_messag from prefect import flow from prefect_email import EmailServerCredentials, email_send_message + @flow def example_email_send_message_flow(email_addresses): email_server_credentials = EmailServerCredentials.load("BLOCK-NAME-PLACEHOLDER") @@ -74,6 +72,7 @@ def example_email_send_message_flow(email_addresses): email_to=email_address, ) + if __name__ == "__main__": example_email_send_message_flow(["EMAIL-ADDRESS-PLACEHOLDER"]) ``` @@ -89,6 +88,7 @@ from prefect import flow from prefect.context import get_run_context from prefect_email import EmailServerCredentials, email_send_message + def notify_exc_by_email(exc): context = get_run_context() flow_run_name = context.flow_run.name @@ -108,10 +108,11 @@ def example_flow(): notify_exc_by_email(exc) raise + if __name__ == "__main__": example_flow() ``` ## Resources -Refer to the prefect-email API documentation linked in the sidebar to explore all the capabilities of the prefect-email library. +Refer to the `prefect-email` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-email` library. diff --git a/docs/integrations/prefect-gcp/index.mdx b/docs/integrations/prefect-gcp/index.mdx index 54994e97077b..c1667fb97d1a 100644 --- a/docs/integrations/prefect-gcp/index.mdx +++ b/docs/integrations/prefect-gcp/index.mdx @@ -9,38 +9,31 @@ For example, you can run flow on Vertex AI or Cloud Run, read and write data to ### Prerequisites -- [Prefect installed](3.0/get-started/install) in a virtual environment. - A [GCP account](https://cloud.google.com/) and the necessary permissions to access desired services. -### Install prefect-gcp +### Install `prefect-gcp` +The following command will install a version of `prefect-gcp` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -Install or update to the latest version of the prefect-gcp library and dependencies. +```bash +pip install "prefect[gcp]" +``` - - - ```bash - pip install -U prefect-gcp - ``` - - - ```bash - pip install -U -pre prefect-gcp - ``` - - +Upgrade to the latest versions of `prefect` and `prefect-gcp`: +```bash +pip install -U "prefect[gcp]" +``` If using BigQuery, Cloud Storage, Secret Manager, or Vertex AI, see [additional installation options](#additional-installation-options). -To install with all additional capabilities, use the following command: - +To install `prefect-gcp` with all additional capabilities, run the install command above and then run the following command: ```bash -pip install -U "prefect-gcp[all_extras]" +pip install "prefect-gcp[all_extras]" ``` - ### Register newly installed block types Register the block types in the module to make them available for use. @@ -49,7 +42,6 @@ Register the block types in the module to make them available for use. prefect block register -m prefect_gcp ``` - ## Authenticate using a GCP Credentials block Authenticate with a service account to use `prefect-gcp` services. @@ -226,6 +218,7 @@ import google.cloud.bigquery from prefect import flow from prefect_gcp import GcpCredentials + @flow def create_bigquery_client(): gcp_credentials = GcpCredentials.load("BLOCK-NAME-PLACEHOLDER") @@ -239,6 +232,7 @@ To access the underlying client, use the `get_client` method from `GcpCredential from prefect import flow from prefect_gcp import GcpCredentials + @flow def create_bigquery_client(): gcp_credentials = GcpCredentials.load("BLOCK-NAME-PLACEHOLDER") @@ -249,39 +243,38 @@ def create_bigquery_client(): For assistance using GCP, consult the [Google Cloud documentation](https://cloud.google.com/docs). -Refer to the prefect-gcp API documentation linked in the sidebar to explore all the capabilities of the prefect-gcp library. +Refer to the `prefect-gcp` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-gcp` library. ### Additional installation options -Additional installation options for GCP services are shown below. +First install the main library compatible with your `prefect` version: -#### To use Cloud Storage +```bash +pip install "prefect[gcp]" +``` +Then install the additional capabilities you need. + +#### To use Cloud Storage ```bash pip install "prefect-gcp[cloud_storage]" ``` - #### To use BigQuery - ```bash pip install "prefect-gcp[bigquery]" ``` - #### To use Secret Manager - ```bash pip install "prefect-gcp[secret_manager]" ``` - ### To use Vertex AI - ```bash pip install "prefect-gcp[aiplatform]" ``` diff --git a/docs/integrations/prefect-github/index.mdx b/docs/integrations/prefect-github/index.mdx index e3c2138091b6..c3fb12f4022b 100644 --- a/docs/integrations/prefect-github/index.mdx +++ b/docs/integrations/prefect-github/index.mdx @@ -8,37 +8,31 @@ Prefect-github makes it easy to interact with GitHub repositories and use GitHub ### Prerequisites -- [Prefect installed](/3.0/get-started/install/). - A [GitHub account](https://github.com/). -### Install prefect-github +### Install `prefect-github` +The following command will install a version of `prefect-github` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -Install or update to the latest version of the prefect-github library and dependencies. +```bash +pip install "prefect[github]" +``` - - - ```bash - pip install -U prefect-github - ``` - - - ```bash - pip install -U -pre prefect-github - ``` - - +Upgrade to the latest versions of `prefect` and `prefect-github`: +```bash +pip install -U "prefect[github]" +``` ### Register newly installed block types -Register the block types in the prefect-github module to make them available for use. +Register the block types in the `prefect-github` module to make them available for use. ```bash prefect block register -m prefect_github ``` - ## Examples In the examples below, you create blocks with Python code. @@ -129,4 +123,4 @@ if __name__ == "__main__": For assistance using GitHub, consult the [GitHub documentation](https://docs.github.com). -Refer to the prefect-github API documentation linked in the sidebar to explore all the capabilities of the prefect-github library. +Refer to the `prefect-github` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-github` library. diff --git a/docs/integrations/prefect-gitlab/index.mdx b/docs/integrations/prefect-gitlab/index.mdx index d9aa1c6cc33c..3d2ffcddba21 100644 --- a/docs/integrations/prefect-gitlab/index.mdx +++ b/docs/integrations/prefect-gitlab/index.mdx @@ -8,32 +8,26 @@ The prefect-gitlab library makes it easy to interact with GitLab repositories an ### Prerequisites -- [Prefect installed](/3.0/get-started/install/). - A [GitLab account](https://gitlab.com/). -### Install prefect-gitlab +### Install `prefect-gitlab` +The following command will install a version of `prefect-gitlab` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -Install or update to the latest version of the prefect-gitlab library and dependencies. +```bash +pip install "prefect[gitlab]" +``` - - - ```bash - pip install -U prefect-gitlab - ``` - - - ```bash - pip install -U -pre prefect-gitlab - ``` - - +Upgrade to the latest versions of `prefect` and `prefect-gitlab`: +```bash +pip install -U "prefect[gitlab]" +``` ### Register newly installed block types -Register the block types in the prefect-gitlab module to make them available for use. - +Register the block types in the `prefect-gitlab` module to make them available for use. ```bash prefect block register -m prefect_gitlab @@ -126,10 +120,12 @@ For example, download the repository contents with the `.get_directory()` method ```python from prefect_gitlab.repositories import GitLabRepository + def fetch_repo(): private_gitlab_block = GitLabRepository.load("my-gitlab-block") private_gitlab_block.get_directory() + if __name__ == "__main__": fetch_repo() ``` @@ -138,4 +134,4 @@ if __name__ == "__main__": For assistance using GitLab, consult the [GitLab documentation](https://gitlab.com). -Refer to the prefect-gitlab API documentation linked in the sidebar to explore all the capabilities of the prefect-gitlab library. +Refer to the `prefect-gitlab` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-gitlab` library. diff --git a/docs/integrations/prefect-kubernetes/index.mdx b/docs/integrations/prefect-kubernetes/index.mdx index aab0776037ac..b6b7e4c17687 100644 --- a/docs/integrations/prefect-kubernetes/index.mdx +++ b/docs/integrations/prefect-kubernetes/index.mdx @@ -12,26 +12,30 @@ Prefect provides a Helm chart for deploying a worker, a self-hosted Prefect serv ### Prerequisites -- [Prefect installed](3.0/get-started/install) in a virtual environment. - [Kubernetes installed](https://kubernetes.io/). ### Install `prefect-kubernetes` -
+The following command will install a version of `prefect-kubernetes` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. + ```bash - pip install -U prefect-kubernetes - ``` -
+pip install "prefect[kubernetes]" +``` + +Upgrade to the latest versions of `prefect` and `prefect-kubernetes`: + +```bash +pip install -U "prefect[kubernetes]" +``` ### Register newly installed block types -Register the block types in the prefect-aws module to make them available for use. +Register the block types in the `prefect-kubernetes` module to make them available for use. -
```bash prefect block register -m prefect_kubernetes ``` -
## Examples @@ -63,6 +67,7 @@ job = KubernetesJob.from_yaml_file( # or create in the UI with a dict manifest job.save("my-k8s-job", overwrite=True) + if __name__ == "__main__": # run the flow run_namespaced_job(job) @@ -90,6 +95,7 @@ from prefect import flow from prefect_kubernetes.credentials import KubernetesCredentials from prefect_kubernetes.jobs import list_namespaced_job + @flow def kubernetes_orchestrator(): v1_job_list = list_namespaced_job( @@ -108,6 +114,7 @@ from prefect_kubernetes.credentials import KubernetesCredentials from prefect_kubernetes.deployments import patch_namespaced_deployment from prefect_kubernetes.utilities import convert_manifest_to_model + @flow def kubernetes_orchestrator(): @@ -126,4 +133,4 @@ def kubernetes_orchestrator(): For assistance using Kubernetes, consult the [Kubernetes documentation](https://kubernetes.io/). -Refer to the prefect-kubernetes API documentation linked in the sidebar to explore all the capabilities of the prefect-kubernetes library. +Refer to the `prefect-kubernetes` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-kubernetes` library. diff --git a/docs/integrations/prefect-ray/index.mdx b/docs/integrations/prefect-ray/index.mdx index 03b5186cae74..8c1fc3d16997 100644 --- a/docs/integrations/prefect-ray/index.mdx +++ b/docs/integrations/prefect-ray/index.mdx @@ -8,28 +8,25 @@ The `prefect-ray` integration makes it easy to accelerate your flow runs with Ra ## Install `prefect-ray` +The following command will install a version of `prefect-ray` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -Install or update to the latest version of the prefect-ray library and dependencies. - - - - ```bash - pip install -U prefect-ray - ``` - - - ```bash - pip install -U -pre prefect-ray - ``` - - +```bash +pip install "prefect[ray]" +``` + +Upgrade to the latest versions of `prefect` and `prefect-ray`: + +```bash +pip install -U "prefect[ray]" +``` **Ray limitations** There are a few limitations with Ray: -- Ray support for Python 3.11 is [experimental](https://docs.ray.io/en/latest/ray-overview/installation.html#install-nightlies). +- Ray support for Python 3.12 is [experimental](https://docs.ray.io/en/latest/ray-overview/installation.html#install-nightlies). - Ray support for non-x86/64 architectures such as ARM/M1 processors with installation from `pip` alone and will be skipped during installation of Prefect. It is possible to manually install the blocking component with `conda`. See the [Ray documentation](https://docs.ray.io/en/latest/ray-overview/installation.html#m1-mac-apple-silicon-support) for instructions. - Ray support for Windows is currently in beta. @@ -192,6 +189,7 @@ from prefect import flow, task from prefect_ray.task_runners import RayTaskRunner from prefect_ray.context import remote_options + @task def process(x): return x + 1 @@ -206,6 +204,6 @@ def my_flow(): ## Resources -Refer to the `prefect-ray` API documentation to explore all the capabilities of the prefect-ray library. +Refer to the `prefect-ray` SDK documentation to explore all the capabilities of the `prefect-ray` library. For further assistance using Ray, consult the [Ray documentation](https://docs.ray.io/en/latest/index.html). diff --git a/docs/integrations/prefect-shell/index.mdx b/docs/integrations/prefect-shell/index.mdx index 8be1bc59007b..9dff152b6604 100644 --- a/docs/integrations/prefect-shell/index.mdx +++ b/docs/integrations/prefect-shell/index.mdx @@ -6,48 +6,35 @@ Execute shell commands from within Prefect flows. ## Getting started -### Prerequisites - -Requires an installation of Python 3.9+ - -We recommend using a Python virtual environment manager such as pipenv, conda, or virtualenv. - -These tasks are designed to work with Prefect 3. - ### Install `prefect-shell` +The following command will install a version of `prefect-shell` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -Install or update to the latest version of the prefect-shell library and dependencies. +```bash +pip install "prefect[shell]" +``` - - - ```bash - pip install -U prefect-shell - ``` - - - ```bash - pip install -U -pre prefect-shell - ``` - - +Upgrade to the latest versions of `prefect` and `prefect-shell`: -### Register newly installed block types +```bash +pip install -U "prefect[shell]" +``` -Register the block types in the prefect-shell module to make them available for use. +### Register newly installed block types +Register the block types in the `prefect-shell` module to make them available for use. ```bash prefect block register -m prefect_shell ``` - ## Examples ### Integrate shell commands with Prefect flows -With prefect-shell, you can use shell commands (and/or scripts) in Prefect flows to provide observability and resiliency. -Prefect-shell can be a useful tool if you're transitioning your orchestration from shell scripts to Prefect. +With `prefect-shell`, you can use shell commands (and/or scripts) in Prefect flows to provide observability and resiliency. +`prefect-shell` can be a useful tool if you're transitioning your orchestration from shell scripts to Prefect. Let's get the shell-abration started! @@ -143,4 +130,4 @@ ping_op = ShellOperation.load("block-name") ## Resources -Refer to the prefect-shell SDK documentation linked in the sidebar to explore all the capabilities of the prefect-shell library. +Refer to the `prefect-shell` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-shell` library. diff --git a/docs/integrations/prefect-slack/index.mdx b/docs/integrations/prefect-slack/index.mdx index 6e6178bee87e..ba9195ff76e9 100644 --- a/docs/integrations/prefect-slack/index.mdx +++ b/docs/integrations/prefect-slack/index.mdx @@ -8,33 +8,24 @@ title: prefect-slack ## Getting started -### Python setup +### Prerequisites -Requires an installation of Python 3.9+ - -We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv. - -These tasks are designed to work with Prefect 3. +A Slack account with permissions to create a Slack app and install it in your workspace. ### Installation -Install `prefect-slack` +The following command will install a version of `prefect-slack` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. +```bash +pip install "prefect[slack]" +``` -Install or update to the latest version of the prefect-slack library and dependencies. +Upgrade to the latest versions of `prefect` and `prefect-slack`: - - - ```bash - pip install -U prefect-slack - ``` - - - ```bash - pip install -U -pre prefect-slack - ``` - - +```bash +pip install -U "prefect[slack]" +``` ### Slack setup @@ -71,3 +62,9 @@ def example_send_message_flow(): if __name__ == "__main__": example_send_message_flow() ``` + +## Resources + +Refer to the `prefect-slack` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-slack` library. + +For further assistance developing with Slack, consult the [Slack documentation](https://api.slack.com/). \ No newline at end of file diff --git a/docs/integrations/prefect-snowflake/index.mdx b/docs/integrations/prefect-snowflake/index.mdx index 24db05eed645..8123084c985c 100644 --- a/docs/integrations/prefect-snowflake/index.mdx +++ b/docs/integrations/prefect-snowflake/index.mdx @@ -2,7 +2,6 @@ title: prefect-snowflake --- - ## Welcome! The prefect-snowflake collection makes it easy to connect to a Snowflake database in your Prefect flows. Check out the examples below to get started! @@ -15,22 +14,18 @@ The prefect-snowflake collection makes it easy to connect to a Snowflake databas ### Install prefect-snowflake +The following command will install a version of `prefect-snowflake` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -Install or update to the latest version of the prefect-snowflake library and dependencies. +```bash +pip install "prefect[snowflake]" +``` - - - ```bash - pip install -U prefect-snowflake - ``` - - - ```bash - pip install -U -pre prefect-snowflake - ``` - - +Upgrade to the latest versions of `prefect` and `prefect-snowflake`: +```bash +pip install -U "prefect[snowflake]" +``` ### Integrate with Prefect flows @@ -97,6 +92,7 @@ Be sure to [install prefect-snowflake](#installation), [register the blocks](#re from prefect_snowflake import SnowflakeConnector import asyncio + @task async def setup_table(block_name: str) -> None: with await SnowflakeConnector.load(block_name) as connector: @@ -148,6 +144,7 @@ from prefect import flow from prefect_snowflake.database import SnowflakeConnector from snowflake.connector.pandas_tools import write_pandas + @flow def snowflake_write_pandas_flow(): connector = SnowflakeConnector.load("my-block") @@ -179,17 +176,14 @@ Install `prefect-snowflake` with `pip`: pip install prefect-snowflake ``` - ### Registering blocks Register blocks in this module to make them available for use. - ```bash prefect block register -m prefect_snowflake ``` - -### Saving credentials to block +### Saving credentials to a block Note, to use the `load` method on a block, you must already have a block saved through code or saved through the UI. @@ -202,6 +196,7 @@ Below is a walkthrough on saving a `SnowflakeCredentials` block through code. ```python from prefect_snowflake import SnowflakeCredentials + credentials = SnowflakeCredentials( account="ACCOUNT-PLACEHOLDER", # resembles nh12345.us-east-2.snowflake user="USER-PLACEHOLDER", @@ -231,11 +226,12 @@ connector = SnowflakeConnector( connector.save("CONNECTOR-BLOCK-NAME-PLACEHOLDER") ``` -Congrats! You can now easily load the saved block, which holds your credentials and connection info: +You can now easily load the saved block, which holds your credentials and connection info: ```python from prefect_snowflake import SnowflakeCredentials, SnowflakeConnector + SnowflakeCredentials.load("CREDENTIALS-BLOCK-NAME-PLACEHOLDER") SnowflakeConnector.load("CONNECTOR-BLOCK-NAME-PLACEHOLDER") ``` @@ -252,8 +248,11 @@ prefect block register -m prefect_snowflake A list of available blocks in `prefect-snowflake` and their setup instructions can be found [here](https://PrefectHQ.github.io/prefect-snowflake/blocks_catalog). -### Feedback +## Resources + +Refer to the `prefect-snowflake` SDK documentation to explore all the capabilities of the `prefect-snowflake` library. + +For further assistance using Snowflake, consult the [Snowflake documentation](https://docs.snowflake.com/). + -If you encounter any bugs while using `prefect-snowflake`, feel free to open an issue in the [prefect](https://github.com/PrefectHQ/prefect) repository. -If you have any questions or issues while using `prefect-snowflake`, you can find help in the [Prefect Slack community](https://prefect.io/slack). diff --git a/docs/integrations/prefect-sqlalchemy/index.mdx b/docs/integrations/prefect-sqlalchemy/index.mdx index 7a1785061527..f75d11d73120 100644 --- a/docs/integrations/prefect-sqlalchemy/index.mdx +++ b/docs/integrations/prefect-sqlalchemy/index.mdx @@ -1,34 +1,31 @@ -# prefect-sqlalchemy +--- +title: prefect-sqlalchemy +--- -The prefect-sqlalchemy helps you connect to a database in your Prefect flows. +# Welcome! -## Getting started - -### Prerequisites +`prefect-sqlalchemy` helps you connect to a database in your Prefect flows. -- [Prefect installed](/3.0/get-started/install/). +## Getting started ### Install `prefect-sqlalchemy` +The following command will install a version of `prefect-sqlalchemy` compatible with your installed version of `prefect`. +If you don't already have `prefect` installed, it will install the newest version of `prefect` as well. -Install or update to the latest version of the prefect-sqlalchemy library and dependencies. +```bash +pip install "prefect[sqlalchemy]" +``` - - - ```bash - pip install -U prefect-sqlalchemy - ``` - - - ```bash - pip install -U -pre prefect-sqlalchemy - ``` - - +Upgrade to the latest versions of `prefect` and `prefect-sqlalchemy`: + +```bash +pip install -U "prefect[sqlalchemy]" +``` ### Register newly installed block types -Register the block types in the prefect-sqlalchemy module to make them available for use. +Register the block types in the `prefect-sqlalchemy` module to make them available for use. ```bash @@ -41,7 +38,6 @@ prefect block register -m prefect_sqlalchemy To use the `load` method on Blocks, you must have a block saved through code or saved through the UI. - ```python from prefect_sqlalchemy import SqlAlchemyConnector, ConnectionComponents, SyncDriver @@ -157,6 +153,7 @@ def sqlalchemy_flow(block_name: str) -> list: all_rows = fetch_data(block_name) return all_rows + if __name__ == "__main__": sqlalchemy_flow("BLOCK-NAME-PLACEHOLDER") ``` @@ -219,6 +216,6 @@ if __name__ == "__main__": ## Resources -Refer to the prefect-sqlalchemy SDK documentation linked in the sidebar to explore all the capabilities of the prefect-sqlalchemy library. +Refer to the `prefect-sqlalchemy` SDK documentation linked in the sidebar to explore all the capabilities of the `prefect-sqlalchemy` library. For assistance using SQLAlchemy, consult the [SQLAlchemy documentation](https://www.sqlalchemy.org/).