Skip to content

Commit

Permalink
Update unreleased documentation (#572)
Browse files Browse the repository at this point in the history
* Update versions.json

* Deployed 61e203c to unreleased in versions with MkDocs 1.5.3 and mike 2.0.0

* Sort docs versions

---------

Co-authored-by: GitHub Actions Bot <[email protected]>
  • Loading branch information
github-actions[bot] and GitHub Actions Bot authored Jan 18, 2024
1 parent 0a282db commit e00b6d5
Show file tree
Hide file tree
Showing 3 changed files with 41 additions and 8 deletions.
47 changes: 40 additions & 7 deletions versions/unreleased/guides/deployment/push-work-pools/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -8984,10 +8984,10 @@

<h1 id="push-work-to-serverless-computing-infrastructure">Push Work to Serverless Computing Infrastructure <span class="badge cloud"></span><a class="headerlink" href="#push-work-to-serverless-computing-infrastructure" title="Permanent link">&para;</a></h1>
<p>Push <a href="/concepts/work-pools/#work-pool-overview">work pools</a> are a special type of work pool that allows Prefect Cloud to submit flow runs for execution to serverless computing infrastructure without running a worker.
Push work pools currently support execution in GCP Cloud Run Jobs, Azure Container Instances, and AWS ECS Tasks.</p>
Push work pools currently support execution in AWS ECS tasks, Azure Container Instances, Google Cloud Run jobs, and Modal.</p>
<p>In this guide you will:</p>
<ul>
<li>Create a push work pool that sends work to Google Cloud Run, Amazon Elastic Container Service (AWS ECS), or Azure Container Instances (ACI)</li>
<li>Create a push work pool that sends work to Amazon Elastic Container Service (AWS ECS), Azure Container Instances (ACI), Google Cloud Run, or Modal</li>
<li>Deploy a flow to that work pool</li>
<li>Execute a flow without having to run a worker or agent process to poll for flow runs</li>
</ul>
Expand All @@ -9002,7 +9002,7 @@ <h2 id="automatic-infrastructure-provisioning">Automatic infrastructure provisio
<p>We'll use the <code>prefect work-pool create</code> CLI command with the <code>--provision-infra</code> flag to automatically provision your serverless cloud resources and set up your Prefect workspace to use a new push pool.</p>
<h3 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">&para;</a></h3>
<p>To use automatic infrastructure provisioning, you'll need to have the relevant cloud CLI library installed and to have authenticated with your cloud provider.</p>
<div class="tabbed-set tabbed-alternate" data-tabs="1:3"><input checked="checked" id="__tabbed_1_1" name="__tabbed_1" type="radio" /><input id="__tabbed_1_2" name="__tabbed_1" type="radio" /><input id="__tabbed_1_3" name="__tabbed_1" type="radio" /><div class="tabbed-labels"><label for="__tabbed_1_1">AWS ECS</label><label for="__tabbed_1_2">Azure Container Instances</label><label for="__tabbed_1_3">Google Cloud Run</label></div>
<div class="tabbed-set tabbed-alternate" data-tabs="1:4"><input checked="checked" id="__tabbed_1_1" name="__tabbed_1" type="radio" /><input id="__tabbed_1_2" name="__tabbed_1" type="radio" /><input id="__tabbed_1_3" name="__tabbed_1" type="radio" /><input id="__tabbed_1_4" name="__tabbed_1" type="radio" /><div class="tabbed-labels"><label for="__tabbed_1_1">AWS ECS</label><label for="__tabbed_1_2">Azure Container Instances</label><label for="__tabbed_1_3">Google Cloud Run</label><label for="__tabbed_1_4">Modal</label></div>
<div class="tabbed-content">
<div class="tabbed-block">
<p>Install the <a href="https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html">AWS CLI</a>, <a href="https://docs.aws.amazon.com/signin/latest/userguide/command-line-sign-in.html">authenticate with your AWS account</a>, and <a href="https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html#cli-configure-files-methods">set a default region</a>.</p>
Expand Down Expand Up @@ -9085,11 +9085,23 @@ <h3 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites"
</ul>
<p>Docker is also required to build and push images to your registry. You can install Docker <a href="https://docs.docker.com/get-docker/">here</a>.</p>
</div>
<div class="tabbed-block">
<p>Install <code>modal</code> by running:</p>
<p><div class="terminal">
<div class="highlight"><pre><span></span><code>pip<span class="w"> </span>install<span class="w"> </span>modal
</code></pre></div>
</div></p>
<p>Create a Modal API token by running:</p>
<p><div class="terminal">
<div class="highlight"><pre><span></span><code>modal<span class="w"> </span>token<span class="w"> </span>new
</code></pre></div>
</div></p>
</div>
</div>
</div>
<h3 id="automatically-creating-a-new-push-work-pool-and-provisioning-infrastructure">Automatically creating a new push work pool and provisioning infrastructure<a class="headerlink" href="#automatically-creating-a-new-push-work-pool-and-provisioning-infrastructure" title="Permanent link">&para;</a></h3>
<p>Here's the command to create a new push work pool and configure the necessary infrastructure.</p>
<div class="tabbed-set tabbed-alternate" data-tabs="2:3"><input checked="checked" id="__tabbed_2_1" name="__tabbed_2" type="radio" /><input id="__tabbed_2_2" name="__tabbed_2" type="radio" /><input id="__tabbed_2_3" name="__tabbed_2" type="radio" /><div class="tabbed-labels"><label for="__tabbed_2_1">AWS ECS</label><label for="__tabbed_2_2">Azure Container Instances</label><label for="__tabbed_2_3">Google Cloud Run</label></div>
<div class="tabbed-set tabbed-alternate" data-tabs="2:4"><input checked="checked" id="__tabbed_2_1" name="__tabbed_2" type="radio" /><input id="__tabbed_2_2" name="__tabbed_2" type="radio" /><input id="__tabbed_2_3" name="__tabbed_2" type="radio" /><input id="__tabbed_2_4" name="__tabbed_2" type="radio" /><div class="tabbed-labels"><label for="__tabbed_2_1">AWS ECS</label><label for="__tabbed_2_2">Azure Container Instances</label><label for="__tabbed_2_3">Google Cloud Run</label><label for="__tabbed_2_4">Modal</label></div>
<div class="tabbed-content">
<div class="tabbed-block">
<div class="terminal">
Expand Down Expand Up @@ -9298,6 +9310,13 @@ <h3 id="automatically-creating-a-new-push-work-pool-and-provisioning-infrastruct
<p>This will build an image with the tag <code>&lt;region&gt;-docker.pkg.dev/&lt;project&gt;/&lt;repository-name&gt;/my-image:latest</code> and push it to the repository.</p>
</div>
</div>
<div class="tabbed-block">
<div class="terminal">
<div class="highlight"><pre><span></span><code>prefect<span class="w"> </span>work-pool<span class="w"> </span>create<span class="w"> </span>--type<span class="w"> </span>modal:push<span class="w"> </span>--provision-infra<span class="w"> </span>my-modal-pool<span class="w"> </span>
</code></pre></div>
</div>
<p>Using the <code>--provision-infra</code> flag will trigger the creation of a <code>ModalCredentials</code> block in your Prefect Cloud workspace. This block will store your Modal API token, which is used to authenticate with Modal's API. By default, the token for your current Modal profile will be used for the new <code>ModalCredentials</code> block. If Prefect is unable to discover a Modal API token for your current profile, you will be prompted to create a new one.</p>
</div>
</div>
</div>
<p>That's it!
Expand Down Expand Up @@ -9369,7 +9388,7 @@ <h2 id="provisioning-infrastructure-for-an-existing-push-work-pool">Provisioning
<h2 id="manual-infrastructure-provisioning">Manual infrastructure provisioning<a class="headerlink" href="#manual-infrastructure-provisioning" title="Permanent link">&para;</a></h2>
<p>If you prefer to set up your infrastructure manually, don't include the <code>--provision-infra</code> flag in the CLI command.
In the examples below, we'll create a push work pool via the Prefect Cloud UI.</p>
<div class="tabbed-set tabbed-alternate" data-tabs="3:3"><input checked="checked" id="__tabbed_3_1" name="__tabbed_3" type="radio" /><input id="__tabbed_3_2" name="__tabbed_3" type="radio" /><input id="__tabbed_3_3" name="__tabbed_3" type="radio" /><div class="tabbed-labels"><label for="__tabbed_3_1">AWS ECS</label><label for="__tabbed_3_2">Azure Container Instances</label><label for="__tabbed_3_3">Google Cloud Run</label></div>
<div class="tabbed-set tabbed-alternate" data-tabs="3:4"><input checked="checked" id="__tabbed_3_1" name="__tabbed_3" type="radio" /><input id="__tabbed_3_2" name="__tabbed_3" type="radio" /><input id="__tabbed_3_3" name="__tabbed_3" type="radio" /><input id="__tabbed_3_4" name="__tabbed_3" type="radio" /><div class="tabbed-labels"><label for="__tabbed_3_1">AWS ECS</label><label for="__tabbed_3_2">Azure Container Instances</label><label for="__tabbed_3_3">Google Cloud Run</label><label for="__tabbed_3_4">Modal</label></div>
<div class="tabbed-content">
<div class="tabbed-block">
<p>To push work to ECS, AWS credentials are required.</p>
Expand Down Expand Up @@ -9404,12 +9423,17 @@ <h2 id="manual-infrastructure-provisioning">Manual infrastructure provisioning<a
<p><img alt="Configuring service account permissions in GCP" src="/img/guides/gcr-service-account-setup.png" /></p>
<p>Once the Service account is created, navigate to its <em>Keys</em> page to add an API key. Create a JSON type key, download it, and store it somewhere safe for use in the next section.</p>
</div>
<div class="tabbed-block">
<p>A Modal API token is required to push work to Modal.</p>
<p>Create a Modal API token by navigating to <strong>Settings</strong> in the Modal UI. In the <strong>API Tokens</strong> section of the Settings page, click <strong>New Token</strong>.</p>
<p>Copy the token ID and token secret and store them somewhere safe for use in the next section.</p>
</div>
</div>
</div>
<h3 id="work-pool-configuration">Work pool configuration<a class="headerlink" href="#work-pool-configuration" title="Permanent link">&para;</a></h3>
<p>Our push work pool will store information about what type of infrastructure our flow will run on, what default values to provide to compute jobs, and other important execution environment parameters. Because our push work pool needs to integrate securely with your serverless infrastructure, we need to start by storing our credentials in Prefect Cloud, which we'll do by making a block.</p>
<h3 id="creating-a-credentials-block">Creating a Credentials block<a class="headerlink" href="#creating-a-credentials-block" title="Permanent link">&para;</a></h3>
<div class="tabbed-set tabbed-alternate" data-tabs="4:3"><input checked="checked" id="__tabbed_4_1" name="__tabbed_4" type="radio" /><input id="__tabbed_4_2" name="__tabbed_4" type="radio" /><input id="__tabbed_4_3" name="__tabbed_4" type="radio" /><div class="tabbed-labels"><label for="__tabbed_4_1">AWS ECS</label><label for="__tabbed_4_2">Azure Container Instances</label><label for="__tabbed_4_3">Google Cloud Run</label></div>
<div class="tabbed-set tabbed-alternate" data-tabs="4:4"><input checked="checked" id="__tabbed_4_1" name="__tabbed_4" type="radio" /><input id="__tabbed_4_2" name="__tabbed_4" type="radio" /><input id="__tabbed_4_3" name="__tabbed_4" type="radio" /><input id="__tabbed_4_4" name="__tabbed_4" type="radio" /><div class="tabbed-labels"><label for="__tabbed_4_1">AWS ECS</label><label for="__tabbed_4_2">Azure Container Instances</label><label for="__tabbed_4_3">Google Cloud Run</label><label for="__tabbed_4_4">Modal</label></div>
<div class="tabbed-content">
<div class="tabbed-block">
<p>Navigate to the blocks page, click create new block, and select AWS Credentials for the type.</p>
Expand All @@ -9427,12 +9451,16 @@ <h3 id="creating-a-credentials-block">Creating a Credentials block<a class="head
<p><img alt="Configuring GCP Credentials block for use in cloud run push work pools" src="/img/guides/gcp-creds-block-setup.png" /></p>
<p>Provide any other optional information and create your block.</p>
</div>
<div class="tabbed-block">
<p>Navigate to the blocks page, click create new block, and select Modal Credentials for the type.</p>
<p>For use in a push work pool, this block must have the token ID and token secret stored in the Token ID and Token Secret fields, respectively.</p>
</div>
</div>
</div>
<h3 id="creating-a-push-work-pool">Creating a push work pool<a class="headerlink" href="#creating-a-push-work-pool" title="Permanent link">&para;</a></h3>
<p>Now navigate to the work pools page.
Click <strong>Create</strong> to start configuring your push work pool by selecting a push option in the infrastructure type step.</p>
<div class="tabbed-set tabbed-alternate" data-tabs="5:3"><input checked="checked" id="__tabbed_5_1" name="__tabbed_5" type="radio" /><input id="__tabbed_5_2" name="__tabbed_5" type="radio" /><input id="__tabbed_5_3" name="__tabbed_5" type="radio" /><div class="tabbed-labels"><label for="__tabbed_5_1">AWS ECS</label><label for="__tabbed_5_2">Azure Container Instances</label><label for="__tabbed_5_3">Google Cloud Run</label></div>
<div class="tabbed-set tabbed-alternate" data-tabs="5:4"><input checked="checked" id="__tabbed_5_1" name="__tabbed_5" type="radio" /><input id="__tabbed_5_2" name="__tabbed_5" type="radio" /><input id="__tabbed_5_3" name="__tabbed_5" type="radio" /><input id="__tabbed_5_4" name="__tabbed_5" type="radio" /><div class="tabbed-labels"><label for="__tabbed_5_1">AWS ECS</label><label for="__tabbed_5_2">Azure Container Instances</label><label for="__tabbed_5_3">Google Cloud Run</label><label for="__tabbed_5_4">Modal</label></div>
<div class="tabbed-content">
<div class="tabbed-block">
<p>Each step has several optional fields that are detailed in the <a href="/concepts/work-pools/">work pools documentation</a>.
Expand All @@ -9448,6 +9476,11 @@ <h3 id="creating-a-push-work-pool">Creating a push work pool<a class="headerlink
Select the block you created under the GCP Credentials field.
This will allow Prefect Cloud to securely interact with your GCP project.</p>
</div>
<div class="tabbed-block">
<p>Each step has several optional fields that are detailed in the <a href="/concepts/work-pools/">work pools documentation</a>.
Select the block you created under the Modal Credentials field.
This will allow Prefect Cloud to securely interact with your Modal account.</p>
</div>
</div>
</div>
<p>Create your pool and you are ready to deploy flows to your Push work pool.</p>
Expand Down
2 changes: 1 addition & 1 deletion versions/unreleased/search/search_index.json

Large diffs are not rendered by default.

Binary file modified versions/unreleased/sitemap.xml.gz
Binary file not shown.

0 comments on commit e00b6d5

Please sign in to comment.