test: Migrate KFP SDK Tests to GHA #11037

DharmitD · 2024-07-25T01:12:36Z

Description of your changes:

Resolves #10987

Converts the KFP SDK tests to a GH Action Workflow.

Here's working GH Action run on my fork: https://github.com/DharmitD/data-science-pipelines-argo/actions/runs/10080983836

PR to remove the KFP Execution test from prow config: GoogleCloudPlatform/oss-test-infra#2329

Checklist:

The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

DharmitD · 2024-07-25T02:36:23Z

created an issue to track the one failing SDK test: #11038

DharmitD · 2024-07-25T02:43:11Z

Ready for review. cc: @hbelmiro @rimolive

hbelmiro · 2024-07-25T16:56:06Z

.github/workflows/kfp-sdk-tests.yml

+  pull_request:
+    paths:
+      - 'sdk/**'
+      - 'test/presubmit-tests-sdk.sh'


test/presubmit-tests-sdk.sh is no longer used.
We can remove this line and also the file.

Removed this line and file.

I still see this line. The file was removed though.

Not sure how this change didn't get pushed, but pushed it again, should be corrected now.

@hbelmiro shall I make a separate PR for removal of test/presubmit-tests-sdk.sh file? Removing the file causes Prow runs to fail and that blocks merging of this PR.

Added the script back based on Diego's suggestion here: #11037 (comment)

hbelmiro · 2024-07-25T17:04:21Z

sdk/python/kfp/cli/component_test.py


+# Commenting this test out as it's failing.


Instead of commenting out the test, can we use @unittest.skip? This approach preserves the code, helping us avoid breaking it and maintaining easy access to references.

More info about @unittest.skip: https://docs.python.org/3/library/unittest.html#skipping-tests-and-expected-failures

Can you also add a comment with a link to #11038?

Done, added @unittest.skip statement instead of commenting out the test, and added a comment with a link to #11038

hbelmiro · 2024-07-25T17:05:06Z

.github/workflows/kfp-sdk-tests.yml

+    paths:
+      - 'sdk/**'
+      - 'test/presubmit-tests-sdk.sh'


I think this can be removed.

Done, removed this.

I still see it.

Pushed again, changes should be visible now.

hbelmiro · 2024-07-25T17:06:55Z

.github/workflows/kfp-sdk-tests.yml

+    paths:
+      - 'sdk/**'
+      - 'test/presubmit-tests-sdk.sh'


We should also add .github/workflows/kfp-sdk-tests.yml here.

Done, added.

I don't see this change.

Pushed again, changes should be visible now.

hbelmiro

@DharmitD thank you for addressing my suggestions.
But it seems like you forgot to push some commits or something.

hbelmiro · 2024-07-26T11:49:54Z

.github/workflows/kfp-sdk-tests.yml

+    paths:
+      - 'sdk/**'
+      - 'test/presubmit-tests-sdk.sh'


I still see it.

hbelmiro · 2024-07-26T11:50:03Z

.github/workflows/kfp-sdk-tests.yml

+  pull_request:
+    paths:
+      - 'sdk/**'
+      - 'test/presubmit-tests-sdk.sh'


I still see this line. The file was removed though.

hbelmiro · 2024-07-26T11:50:53Z

.github/workflows/kfp-sdk-tests.yml

+    paths:
+      - 'sdk/**'
+      - 'test/presubmit-tests-sdk.sh'


I don't see this change.

DharmitD · 2024-07-26T12:47:46Z

@DharmitD thank you for addressing my suggestions. But it seems like you forgot to push some commits or something.

@hbelmiro Pushed again, changes should be visible now.

diegolovison · 2024-07-26T13:52:47Z

.github/workflows/kfp-sdk-tests.yml

+        python-version: ${{ matrix.python }}
+
+    - name: Install dependencies
+      run: |


May you keep the bash script? In this case, we can call it locally in order to have it working.

Done, I added the script back.

Yes. Now, delegate the yaml to that script. In this case, locally and GHA will be on sync

Done, updated to use the script here.

Signed-off-by: ddalvi <[email protected]>

hbelmiro

/lgtm

rimolive · 2024-07-26T18:48:30Z

/lgtm

diegolovison · 2024-07-26T20:29:56Z

/lgtm

DharmitD · 2024-07-26T20:47:15Z

@chensun could you please approve? Thanks.

google-oss-prow · 2024-07-26T23:41:14Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chensun, hbelmiro

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~.github/OWNERS~~ [chensun,hbelmiro]
~~sdk/OWNERS~~ [chensun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Signed-off-by: ddalvi <[email protected]>

google-oss-prow bot requested review from connor-mccarthy and rimolive July 25, 2024 01:12

google-oss-prow bot added the size/M label Jul 25, 2024

DharmitD force-pushed the kfp-sdk-tests branch from 41b4f2f to 4714d0f Compare July 25, 2024 01:13

DharmitD mentioned this pull request Jul 25, 2024

chore: Remove KFP presubmit SDK tests prow config GoogleCloudPlatform/oss-test-infra#2329

Merged

DharmitD force-pushed the kfp-sdk-tests branch from 4714d0f to a31e1bc Compare July 25, 2024 01:39

google-oss-prow bot added size/L and removed size/M labels Jul 25, 2024

DharmitD force-pushed the kfp-sdk-tests branch 2 times, most recently from 73d131a to 9f0aaa3 Compare July 25, 2024 02:01

hbelmiro reviewed Jul 25, 2024

View reviewed changes

DharmitD force-pushed the kfp-sdk-tests branch from 9f0aaa3 to 1fcc1ab Compare July 25, 2024 22:02

google-oss-prow bot added size/M and removed size/L labels Jul 25, 2024

hbelmiro reviewed Jul 26, 2024

View reviewed changes

DharmitD force-pushed the kfp-sdk-tests branch from 1fcc1ab to 100fd27 Compare July 26, 2024 12:45

diegolovison reviewed Jul 26, 2024

View reviewed changes

DharmitD force-pushed the kfp-sdk-tests branch 6 times, most recently from b0b61a2 to 1a974c9 Compare July 26, 2024 16:56

test: Migrate KFP SDK Tests to GHA

ff3e034

Signed-off-by: ddalvi <[email protected]>

DharmitD force-pushed the kfp-sdk-tests branch from 1a974c9 to ff3e034 Compare July 26, 2024 16:57

hbelmiro reviewed Jul 26, 2024

View reviewed changes

google-oss-prow bot assigned hbelmiro Jul 26, 2024

google-oss-prow bot added the lgtm label Jul 26, 2024

google-oss-prow bot assigned rimolive Jul 26, 2024

google-oss-prow bot assigned diegolovison Jul 26, 2024

hbelmiro approved these changes Jul 26, 2024

View reviewed changes

chensun approved these changes Jul 26, 2024

View reviewed changes

google-oss-prow bot added the approved label Jul 26, 2024

google-oss-prow bot merged commit 643c923 into kubeflow:master Jul 27, 2024
29 checks passed

R3hankhan123 pushed a commit to R3hankhan123/pipelines that referenced this pull request Sep 20, 2024

test: Migrate KFP SDK Tests to GHA (kubeflow#11037)

2843ce8

Signed-off-by: ddalvi <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: Migrate KFP SDK Tests to GHA #11037

test: Migrate KFP SDK Tests to GHA #11037

DharmitD commented Jul 25, 2024 •

edited

Loading

DharmitD commented Jul 25, 2024

DharmitD commented Jul 25, 2024

hbelmiro Jul 25, 2024

DharmitD Jul 25, 2024

hbelmiro Jul 26, 2024

DharmitD Jul 26, 2024

DharmitD Jul 26, 2024

DharmitD Jul 26, 2024

hbelmiro Jul 25, 2024

hbelmiro Jul 25, 2024

DharmitD Jul 25, 2024

hbelmiro Jul 25, 2024

DharmitD Jul 25, 2024

hbelmiro Jul 26, 2024

DharmitD Jul 26, 2024

hbelmiro Jul 25, 2024

DharmitD Jul 25, 2024

hbelmiro Jul 26, 2024

DharmitD Jul 26, 2024

hbelmiro left a comment

hbelmiro Jul 26, 2024

hbelmiro Jul 26, 2024

hbelmiro Jul 26, 2024

DharmitD commented Jul 26, 2024

diegolovison Jul 26, 2024 •

edited

Loading

DharmitD Jul 26, 2024

diegolovison Jul 26, 2024

DharmitD Jul 26, 2024

hbelmiro left a comment

rimolive commented Jul 26, 2024

diegolovison commented Jul 26, 2024

DharmitD commented Jul 26, 2024

google-oss-prow bot commented Jul 26, 2024

test: Migrate KFP SDK Tests to GHA #11037

test: Migrate KFP SDK Tests to GHA #11037

Conversation

DharmitD commented Jul 25, 2024 • edited Loading

DharmitD commented Jul 25, 2024

DharmitD commented Jul 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hbelmiro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DharmitD commented Jul 26, 2024

diegolovison Jul 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hbelmiro left a comment

Choose a reason for hiding this comment

rimolive commented Jul 26, 2024

diegolovison commented Jul 26, 2024

DharmitD commented Jul 26, 2024

google-oss-prow bot commented Jul 26, 2024

DharmitD commented Jul 25, 2024 •

edited

Loading

diegolovison Jul 26, 2024 •

edited

Loading