[CI, enhancement] add sklearnex CI job with nightly oneDAL DPCPP build in github actions #1845
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
icfaust
requested review from
napetrov,
Alexsandruss,
homksei,
ahuber21 and
ethanglaser
as code owners
icfaust
changed the title
|
/azp run CI |
|
I'm running into issues with the necessary secret. This PR is on hold until this has been updated. |
|
/intelci: run |
|
Issues with secrets have been resolved, and the use of a PAT is not necessary in order to access oneDAL's artifacts. |
|
/intelci: run |
Alexsandruss
approved these changes
Aug 29, 2024
ethanglaser
approved these changes
Aug 29, 2024
Comment on lines
-1
to
3
| pytest==7.4.4 ; python_version <= '3.9' | ||
| pytest==8.3.2 ; python_version >= '3.10' | ||
| pytest==7.4.4 ; python_version <= '3.10' | ||
| pytest==8.3.2 ; python_version >= '3.11' | ||
| numpy>=1.19.5 ; python_version <= '3.9' |
Comment on lines
+23
to
+30
| # This is necessary to prevent import errors from sklearnex and by extension | ||
| # sklearn caused by initial scipy and/or numpy versions that are installed. | ||
| # It is written in a way to be `packaging` independent. This branch occurs | ||
| # when a sklearn version is given to the script externally. | ||
| def sklearn_check_version(ver): | ||
| ver = [int(i) if i != "" else 0 for i in ver.split(".")[:2]] | ||
| sk_ver = [int(i) if i != "" else 0 for i in str(argv[1]).split(".")[:2]] | ||
| return sk_ver[0] > ver[0] or (sk_ver[0] == ver[0] and sk_ver[1] >= ver[1]) |
There was a problem hiding this comment.
To me this is not obvious. In which case this should work better?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR expands sklearnex testing to work with latest oneDAL builds. It uses a custom Visual Studio / Intel DPC++ compiler (AVX2) windows build for Windows testing. It uses a Intel DPC++ Linux build (AVX2). This uses only publicly available components (which would match external contributor capabilities), and follows what an external contributor would have to do to hand-build oneDAL/sklearnex.
It has:
onedal.__init__anddaal4py.__init__Regressions:
Note this PR removes github runner introduced in #1844 because the github.token secret has sufficient permissions (read) able to access artifacts from oneDAL (and other open-source github repos).
Current Setup tests the following for Windows and Linux (matching current CI for verification):
Further development would be required to change these due to the fragility of some of the ci setup scripts and related pandas/scipy/numpy versions (e.g. some work will be required after new values are determined).