Use istio prowgen to generate Prow jobs #3175
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: chizhg The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
knative-prow-robot
added
approved
chizhg
force-pushed
the
migrate-to-prowgen
branch
7 times, most recently
from
b3782e9
to
ef0121e
Compare
|
EDIT: Does not seem like a more lightweight tool. |
chizhg
force-pushed
the
migrate-to-prowgen
branch
from
ef0121e
to
f519ee9
Compare
Most of the files were go vendored packages since I added I just moved it to a separate Go module and removed the vendored packages, now this PR should be much smaller. |
chizhg
force-pushed
the
migrate-to-prowgen
branch
from
f519ee9
to
fa5793b
Compare
|
Also created knative/infra#134 for tracking the potential future work to get rid of |
|
lgtm |
We can build it into a Docker image and publish it to GCR with a postsubmit Prow job, then this tool can be used elsewhere, but I don't think it simplifies things unless we enable in-repo configs for every Knative repo, see #2311, since the only place we need this tool is knative/test-infra (or knative/infra after we move the config files). |
|
Why don't you consume the code at https://github.com/istio/test-infra/tree/master/tools/prowgen directly? Something like this: |
The major reason is for Knative Prow config generation, there is some specific logic that is not generalized and is only for Knative, e.g. automatically calculating and generating cron schedules for periodic Prow jobs. To inject this logic, we need custom preprocessor to the meta config files which needs to be implemented by importing the structs defined in https://github.com/istio/test-infra/blob/master/tools/prowgen/pkg/spec/spec.go. Since for the above reason, we already imports the istio.io/test-infra repo, it would be better to call the |
chizhg
force-pushed
the
migrate-to-prowgen
branch
from
fa5793b
to
2951863
Compare
|
/lgtm |
|
@chizhg: Updated the
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
krsna-m
added a commit
to krsna-m/test-infra
that referenced
this pull request
Mar 23, 2022
This reverts commit 0656566.
knative-prow-robot
pushed a commit
that referenced
this pull request
Mar 23, 2022
chizhg
added a commit
that referenced
this pull request
Mar 23, 2022
chizhg
added a commit
that referenced
this pull request
Mar 23, 2022
knative-prow-robot
pushed a commit
that referenced
this pull request
Mar 24, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue(s) this PR fixes:
Fixes #2986
This PR is a huge overhaul of all the Prow job and TestGrid config files, as well as the config generation process. The most important change is replacing the gigantic bespoke config-generator tool with a more lightweight tool, and the README is at https://github.com/chizhg/test-infra/blob/e96e34f0816b9d4c58d0f832656233e9f0ad6f2f/tools/configgen/README.md
This PR is probably too large to be feasible for reviewing, but I don't have a good approach to break it down. Since I'm confident the major flow is not broken, and I'm pretty sure it's much much better than the previous tool, I would suggest us merging it for now and iterating on it with follow-up PRs.
/cc @kvmware @upodroid