diff --git a/docs/3.0/deploy/index.mdx b/docs/3.0/deploy/index.mdx index 397d19b8cf8d..46662ab1a368 100644 --- a/docs/3.0/deploy/index.mdx +++ b/docs/3.0/deploy/index.mdx @@ -6,7 +6,7 @@ description: Learn how to use deployments to trigger flow runs remotely. Deployments allow you to run flows on a [schedule](/3.0/automate/add-schedules/) and trigger runs based on [events](/3.0/automate/events/automations-triggers/). -[Deployments](/3.0/deploy/infrastructure-examples/docker/) are server-side representations of flows. +Deployments are server-side representations of flows. They store the crucial metadata for remote orchestration including when, where, and how a workflow should run. In addition to manually triggering and managing flow runs, deploying a flow exposes an API and UI that allow you to: @@ -421,5 +421,3 @@ configuration, and prepares the runtime environment for workflow execution. Pull steps allow users to highly decouple their workflow architecture. For example, a common use of pull steps is to dynamically pull code from remote filesystems such as GitHub with each run of their deployment. - -- **`worker_filter_status`**: this field allows filtering workers based on their status using the `WorkerFilterStatus` class. It supports `any_` and `not_any_` fields for including or excluding specific worker statuses, providing flexibility in worker management. \ No newline at end of file diff --git a/docs/3.0/deploy/infrastructure-concepts/deploy-ci-cd.mdx b/docs/3.0/deploy/infrastructure-concepts/deploy-ci-cd.mdx index b9050f678f94..dc87049c5580 100644 --- a/docs/3.0/deploy/infrastructure-concepts/deploy-ci-cd.mdx +++ b/docs/3.0/deploy/infrastructure-concepts/deploy-ci-cd.mdx @@ -199,32 +199,6 @@ For reference, the examples below live in their respective branches of deploy: name: Deploy runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Log in to Docker Hub - uses: docker/login-action@v3 - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKER_PASSWORD }} - - - name: Setup Python - uses: actions/setup-python@v5 - with: - python-version: "3.12" - - - name: Prefect Deploy - env: - PREFECT_API_KEY: ${{ secrets.PREFECT_API_KEY }} - PREFECT_API_URL: ${{ secrets.PREFECT_API_URL }} - run: | - pip install -r requirements.txt - python flow.py - ``` - - - steps: - name: Checkout diff --git a/docs/3.0/deploy/infrastructure-concepts/workers.mdx b/docs/3.0/deploy/infrastructure-concepts/workers.mdx index 76620b1b2889..d46424755d92 100644 --- a/docs/3.0/deploy/infrastructure-concepts/workers.mdx +++ b/docs/3.0/deploy/infrastructure-concepts/workers.mdx @@ -102,6 +102,23 @@ Workers have two statuses: `ONLINE` and `OFFLINE`. A worker is online if it send If a worker misses three heartbeats, it is considered offline. By default, a worker is considered offline a maximum of 90 seconds after it stopped sending heartbeats, but you can configure the threshold with the `PREFECT_WORKER_HEARTBEAT_SECONDS` setting. +### Worker logs + Workers send logs to the Prefect Cloud API if you're connected to Prefect Cloud. + +- All worker logs are automatically sent to the Prefect Cloud API +- Logs are accessible through both the Prefect Cloud UI and API +- Each flow run will include a link to its associated worker's logs + +### Worker details + The **Worker Details** page shows you three key areas of information: + +- Worker status +- Installed Prefect version +- Installed Prefect integrations (e.g., `prefect-aws`, `prefect-gcp`) +- Live worker logs (if worker logging is enabled) + +Access a worker's details by clicking on the worker's name in the Work Pool list. + ### Start a worker Use the `prefect worker start` CLI command to start a worker. You must pass at least the work pool name. @@ -143,13 +160,12 @@ For example, to limit a worker to five concurrent flow runs: prefect worker start --pool "my-pool" --limit 5 ``` -Note: If there are already active workers in the specified work pool, the deployment process will not suggest starting a new worker. ### Configure prefetch -By default, the worker submits flow runs a short time (10 seconds) before they are scheduled to run. +By default, the worker submits flow runs 10 seconds before they are scheduled to run. This allows time for the infrastructure to be created so the flow run can start on time. -In some cases, infrastructure takes longer than 10 seconds to start the flow run. You can increase the prefetch with the +In some cases, infrastructure takes longer than 10 seconds to start the flow run. You can increase the prefetch time with the `--prefetch-seconds` option or the `PREFECT_WORKER_PREFETCH_SECONDS` setting. If this value is _more_ than the amount of time it takes for the infrastructure to start, the flow run will _wait_ until its @@ -180,4 +196,5 @@ See how to [daemonize a Prefect worker](/3.0/resources/daemonize-processes/). See more information on [overriding a work pool's job variables](/3.0/deploy/infrastructure-concepts/customize). + --------- diff --git a/docs/3.0/resources/upgrade-agents-to-workers.mdx b/docs/3.0/resources/upgrade-agents-to-workers.mdx index f811814d33cb..4ad1d5738595 100644 --- a/docs/3.0/resources/upgrade-agents-to-workers.mdx +++ b/docs/3.0/resources/upgrade-agents-to-workers.mdx @@ -25,7 +25,6 @@ specifying an infrastructure block. Instead, infrastructure configuration is spe [work pool](/3.0/deploy/infrastructure-concepts/work-pools/) and passed to each worker that polls work from that pool. -When deploying, the system now checks for active workers in the work pool and omits setup instructions if active workers are already present, streamlining the deployment process. ## Upgrade enhancements ### Workers @@ -33,7 +32,6 @@ When deploying, the system now checks for active workers in the work pool and om - Improved visibility into the status of each worker, including when a worker was started and when it last polled. - Better handling of race conditions for high availability use cases. -- New feature to check for active workers in a work pool before suggesting the start of a new worker during deployment, reducing unnecessary instructions for users with existing active workers. ### Work pools @@ -90,10 +88,6 @@ to enable [dryer deployment definitions](/3.0/deploy/infrastructure-examples/doc file or use the [`deploy`](/3.0/deploy/infrastructure-examples/docker) function. -5. **Worker Status Filtering and Deployment Logic:** - - The deployment process now includes a check for active workers in a work pool. If active workers are detected, the deployment instructions will omit the setup of new workers, streamlining the process. Additionally, the `WorkerFilterStatus` class can be used to filter workers by their status, using the `any_` and `not_any_` fields for more flexible worker management. - ## What's similar - You can set storage blocks as the pull action in a `prefect.yaml` file. @@ -165,8 +159,6 @@ This worker replaces your agent and polls your new work pool for flow runs to ex prefect worker start -p ``` -Note: If there are already active workers in your work pool, you may not need to start a new worker. The system will automatically check for active workers and omit unnecessary instructions. - 3. **Deploy your flows to the new work pool** To deploy your flows to the new work pool, use `flow.deploy` for a Pythonic deployment