Skip to content

Commit

Permalink
Merge branch 'main' into print-func-for-run-namepsaced-job
Browse files Browse the repository at this point in the history
  • Loading branch information
netanel246 authored Jun 19, 2024
2 parents 23f9c78 + f4fc69b commit 156235d
Show file tree
Hide file tree
Showing 14 changed files with 306 additions and 197 deletions.
75 changes: 75 additions & 0 deletions .github/workflows/nightly-release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Nightly Release Candidate

permissions:
contents: write

on:
schedule:
- cron: '0 8 * * *' # Run at midnight PST (08:00 UTC)
workflow_dispatch:

jobs:
nightly-release:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
persist-credentials: false

- name: Get latest tag
id: get_latest_tag
run: |
latest_tag=$(git tag -l | grep -E '^3\.[0-9]+\.[0-9]+(rc[0-9]+)?$' | sort -V | tail -n 1)
if [ -z "$latest_tag" ]; then
echo "No matching tags found."
exit 1
fi
echo "latest_tag=$latest_tag" >> $GITHUB_ENV
- name: Check for changes since latest tag
id: check_changes
run: |
git fetch --prune --unshallow
latest_tag=${{ env.latest_tag }}
if git diff --exit-code $latest_tag HEAD -- '*.py' 'requirements.txt'; then
echo "SHOULD_CREATE_RELEASE=false" >> $GITHUB_ENV
echo "No changes in .py files or dependencies since the latest tag."
else
echo "SHOULD_CREATE_RELEASE=true" >> $GITHUB_ENV
echo "Changes detected since the latest tag."
fi
- name: Determine next release candidate
id: next_rc
if: env.SHOULD_CREATE_RELEASE == 'true'
run: |
latest_tag=${{ env.latest_tag }}
base_version=$(echo $latest_tag | sed -E 's/(.*rc)[0-9]+/\1/')
rc_number=$(echo $latest_tag | sed -E 's/.*rc([0-9]+)/\1/')
next_rc_number=$((rc_number + 1))
next_tag="${base_version}${next_rc_number}"
echo "next_tag=$next_tag" >> $GITHUB_ENV
echo "Next tag is $next_tag"
- name: Generate release notes
id: generate_release_notes
if: env.SHOULD_CREATE_RELEASE == 'true'
uses: mikepenz/release-changelog-builder-action@v2
with:
tag: ${{ env.next_tag }}

- name: Create release
id: create_release
if: env.SHOULD_CREATE_RELEASE == 'true'
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ env.next_tag }}
release_name: "Nightly Release Candidate ${{ env.next_tag }}"
draft: false
prerelease: true
body: ${{ steps.generate_release_notes.outputs.changelog }}
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,6 @@ repos:
files: |
(?x)^(
.pre-commit-config.yaml|
src/prefect/server/|
src/prefect/server/.*|
scripts/generate_mintlify_openapi_docs.py
)$
37 changes: 14 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,30 +20,30 @@

# Prefect

Prefect is an orchestration and observability platform for building, observing, and triaging workflows.
Prefect is an orchestration and observation platform for building and reacting to workflows.
It's the simplest way to transform Python code into an interactive workflow application.

Prefect allows you to expose your workflows through an API so teams dependent on you can programmatically access your pipelines, business logic, and more.
Prefect also allows you to standardize workflow development and deployment across your organization.
With Prefect you can standardize workflow deployment across your organization.

You can build resilient, dynamic workflows that react to the world around them and recover from unexpected changes.

With Prefect, you can build resilient, dynamic workflows that react to the world around them and recover from unexpected changes.
With just a few decorators, Prefect supercharges your code with features like automatic retries, distributed execution, scheduling, caching, and much more.

Every activity is tracked and can be monitored with a self-hosted [Prefect server](https://docs.prefect.io/latest/guides/host/) instance or managed [Prefect Cloud](https://www.prefect.io/cloud-vs-oss?utm_source=oss&utm_medium=oss&utm_campaign=oss_gh_repo&utm_term=none&utm_content=none) dashboard.
Workflow activity is tracked and can be monitored with a self-hosted [Prefect server](https://docs.prefect.io/latest/guides/host/) instance or managed [Prefect Cloud](https://www.prefect.io/cloud-vs-oss?utm_source=oss&utm_medium=oss&utm_campaign=oss_gh_repo&utm_term=none&utm_content=none) dashboard.

## Getting started

Prefect requires Python 3.9 or later. To [install Prefect](https://docs.prefect.io/getting-started/installation/), run the following command:
Prefect requires Python 3.9 or later. To [install the latest or upgrade to the latest version of Prefect](https://docs.prefect.io/getting-started/installation/), run the following command:

```bash
pip install prefect
pip install -U prefect
```

Then create and run a Python file that uses Prefect `flow` and `task` decorators to orchestrate and observe your workflow - in this case, a simple script that fetches the number of GitHub stars from a repository:

```python
from prefect import flow, task
from typing import List
from typing import list
import httpx


Expand All @@ -55,7 +55,7 @@ def get_stars(repo: str):


@flow(name="GitHub Stars")
def github_stars(repos: List[str]):
def github_stars(repos: list[str]):
for repo in repos:
get_stars(repo)

Expand All @@ -76,6 +76,7 @@ prefect server start
To run your workflow on a schedule, turn it into a deployment and schedule it to run every minute by changing the last line of your script to the following:

```python
if __name__ == "__main__":
github_stars.serve(name="first-deployment", cron="* * * * *")
```

Expand All @@ -97,20 +98,10 @@ Read more about Prefect Cloud [here](https://www.prefect.io/cloud-vs-oss?utm_sou
## prefect-client

If your use case is geared towards communicating with Prefect Cloud or a remote Prefect server, check out our
[prefect-client](https://pypi.org/project/prefect-client/). It was designed to be a lighter-weight option for accessing
client-side functionality in the Prefect SDK and is ideal for use in ephemeral execution environments.
[prefect-client](https://pypi.org/project/prefect-client/). It is a lighter-weight option for accessing client-side functionality in the Prefect SDK and is ideal for use in ephemeral execution environments.

## Next steps

There's lots more you can do to orchestrate and observe your workflows with Prefect!
Start with our [friendly tutorial](https://docs.prefect.io/tutorials) or explore the [core concepts of Prefect workflows](https://docs.prefect.io/concepts/).

## Join the community

Prefect is made possible by the fastest growing community of thousands of friendly data engineers. Join us in building a new kind of workflow system. The [Prefect Slack community](https://prefect.io/slack) is a fantastic place to learn more about Prefect, ask questions, or get help with workflow design. All community forums, including code contributions, issue discussions, and slack messages are subject to our [Code of Conduct](https://discourse.prefect.io/faq).

## Contribute

See our [documentation on contributing to Prefect](https://docs.prefect.io/contributing/overview/).

Thanks for being part of the mission to build a new kind of workflow system and, of course, <i>**happy engineering!**</i>
- Check out the [Docs](https://docs.prefect.io/).
- Join the {Prefect Slack community](https://prefect.io/slack).
- Learn how to [contribute to Prefect](https://docs.prefect.io/contributing/overview/).
2 changes: 1 addition & 1 deletion docs/3.0rc/api-ref/server/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -22524,7 +22524,7 @@
"type": "string",
"format": "duration",
"title": "Prefect Task Scheduling Pending Task Timeout",
"default": "PT30S"
"default": "PT0S"
},
"PREFECT_EXPERIMENTAL_ENABLE_EXTRA_RUNNER_ENDPOINTS": {
"type": "boolean",
Expand Down
Loading

0 comments on commit 156235d

Please sign in to comment.