Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: Migrate miscellaneous SDK tests to GHA Workflow #11032

Merged
merged 3 commits into from
Jul 26, 2024
Merged

test: Migrate miscellaneous SDK tests to GHA Workflow #11032

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
55fece5
test: Migrate SDK isort tests to GHA
DharmitD Jul 24, 2024
16c91da
test: Migrate SDK docformatter tests to GHA
DharmitD Jul 24, 2024
d510af8
test: Migrate SDK Component YAML tests to GHA
DharmitD Jul 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .github/workflows/sdk-component-yaml.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: KFP Component YAML Test

on:
push:
branches: [master]

pull_request:
paths:
- 'components/**/*.yaml'
- 'test/presubmit-component-yaml.sh'
- 'sdk/python/**'
- 'api/v2alpha1/**'
- '.github/workflows/sdk-component-yaml.yml'

jobs:
test-component-yaml-kfp:
runs-on: ubuntu-24.04
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.8

- name: apt-get update
run: sudo apt-get update

- name: Install protobuf-compiler
run: sudo apt-get install protobuf-compiler -y

- name: Install setuptools
run: |
pip3 install setuptools
pip3 freeze

- name: Install Wheel
run: pip3 install wheel==0.42.0

- name: Install protobuf
run: pip3 install protobuf==4.25.3

- name: Generate API proto files
working-directory: ./api
run: make clean python

- name: Install kfp-pipeline-spec from source
run: |
python3 -m pip install api/v2alpha1/python

- name: Install requirements
run: pip install -r ./test/sdk-execution-tests/requirements.txt

- name: Run component YAML tests
run: ./test/presubmit-component-yaml.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused. You removed kubeflow-pipelines-sdk-yapf in https://github.com/GoogleCloudPlatform/oss-test-infra/pull/2327/files#diff-30095b445aa0ad17f1bc23514587283b5ad03959f815fb48e42bdeee2b39c066L186-L196, but in this PR you're adding a test for ./test/presubmit-component-yaml.sh.
Was that a mistake?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't get your question. Looks like the ./test/presubmit-component-yaml.sh runs a file: https://github.com/kubeflow/pipelines/blob/master/test/presubmit-component-yaml.sh#L27

that loads training and testing data: https://github.com/kubeflow/pipelines/blob/master/components/test_load_all_components.sh
Is this not required? If not we could take the GHA out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me clarify...
We have these two tests in Prow:

In this PR, you've added kubeflow-pipelines-component-yaml, but you're not removing it from Prow. Instead, you're removing kubeflow-pipelines-sdk-yapf, which you're not adding in this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops! My bad, deleted the yapf one by mistake instead of the component yaml. Updated now: https://github.com/GoogleCloudPlatform/oss-test-infra/pull/2327/files#diff-30095b445aa0ad17f1bc23514587283b5ad03959f815fb48e42bdeee2b39c066L122

Great catch!

26 changes: 26 additions & 0 deletions .github/workflows/sdk-docformatter.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: KFP SDK Docformatter Test

on:
push:
branches: [master]

pull_request:
paths:
- 'sdk/python/**'
- 'test/presubmit-docformatter-sdk.sh'
- '.github/workflows/sdk-docformatter.yml'

jobs:
test-docformatter-kfp-sdk:
runs-on: ubuntu-24.04
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.8

- name: Run docformatter tests
run: ./test/presubmit-docformatter-sdk.sh
26 changes: 26 additions & 0 deletions .github/workflows/sdk-isort.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: KFP SDK Isort Test

on:
push:
branches: [master]

pull_request:
paths:
- 'sdk/python/**'
- 'test/presubmit-isort-sdk.sh'
- '.github/workflows/sdk-isort.yml'

jobs:
test-isort-kfp-sdk:
runs-on: ubuntu-24.04
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.8

- name: Run isort tests
run: ./test/presubmit-isort-sdk.sh
Loading