feat(sdk/backend): enable parameterization of container images

[HumairAK]

Resolves #11391

Description of your changes:
This change allows component base images to be parameterized using runtime pipeline parameters. The container images can be specified within an @pipeline decorated function, and takes precedence over the @component(base_image=..) argument.

This change also adds logic to resolve these runtime parameters in the argo driver logic. It also includes resolution steps for resolving the accelerator type which functions the same way but was missing the resolution logic. The resolution logic is a generic workaround solution for any run time pod spec input parameters that cannot be resolved because they cannot be added dynamically in the argo pod spec container template.

Checklist:

• You have signed off your commits
• The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

[jgarciao]

One question: if the user sets the image with set_container_image, is the cache key using that setting?

[HumairAK]

One question: if the user sets the image with set_container_image, is the cache key using that setting?

this is a good point, I think we may want to resolve the container image before creating the fingerprint

[javali-google]

Nit: Is the documentation updated as part of different PR?

[HumairAK]

this is a good point, I think we may want to resolve the container image before creating the fingerprint

Actually we don't need to cover this as it's already covered via the inputs field. Since fingerprint is also generated based on input, every image change via runtime parameter input will result in a new cache, which is what we want.

Example:

For this sample pipeline:

@dsl.component(base_image="docker.io/python:3.9.17")
def empty_component(image: str, otherinput: str):
    print("image=" + image + ", otherinput=" + otherinput)

@dsl.pipeline()
def pipeline_accel(img: str, otherinput: str):
    task = empty_component(image=img, otherinput=otherinput)
    task.set_container_image(img)

We have:

[HumairAK]

@javali-google I belive the docs are auto generated and served via readthedocs

@chensun can confirm

my expectation is the docstring here will be auto surfaced like the other task fields here

[jgarciao]

Actually we don't need to cover this as it's already covered via the inputs field. Since fingerprint is also generated based on input, every image change via runtime parameter input will result in a new cache, which is what we want ...

Thanks for the detailed analysis, @HumairAK !

[jgarciao]

lgtm

[HumairAK]

cc @chensun PTAL when you get the chance it would be good to be able to get this in for 2.11

[leseb]

I believe the CHANGELOG needs to be updated too?

[HumairAK]

I believe the CHANGELOG needs to be updated too?

good call, updated!

[jgarciao]

/lgtm

[google-oss-prow]

@jgarciao: changing LGTM is restricted to collaborators

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[VaniHaripriya]

/lgtm verified the accelerator type and container image by passing them as input parameters. I used the below pipeline to test the functionality -

from kfp import dsl, compiler

@dsl.component(base_image='docker.io/python:3.9.17')
def empty_component():
    pass

@dsl.pipeline()
def simple_pipeline(img: str,atype: str):        
    task = empty_component()    
    task.set_container_image(img)
    task.set_accelerator_type(atype)

if __name__ == '__main__':
    compiler.Compiler().compile(pipeline_accel, __file__.replace('.py', '-run.yaml'))

• The container image in the launcher pod was updated with the input image parameter I provided.
• The accelerator type appeared in the resources section of the launcher pod as show below-

- resources:
       limits:
         gpu-vendor.example/example-gpu: '1'
       requests:
         gpu-vendor.example/example-gpu: '1'

[VaniHaripriya]

/lgtm

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: jgarciao, leseb

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• backend/OWNERS
• sdk/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[zeidsolh]

Hey this is not full working. When I use a base_image that comes from a previous component I get the following error:
│ Warning InspectFailed 64s (x8 over 2m29s) kubelet Failed to apply default image tag "{{$.inputs.parameters['pipelinechannel--get-image-url-Output']}} │
│ ": couldn't parse image name "{{$.inputs.parameters['pipelinechannel--get-image-url-Output']}}": invalid reference format: repository name (library/{{$.inputs.parameters['pipelinechanne │
│ l--get-image-url-Output']}}) must be lowercase

[HumairAK]

@zeidsolh have you deployed from master ? this requires backend changes that are not in the latest kfp release yet, it will be available in 2.4, or you can deploy from the master branch and check there

[zeidsolh]

@HumairAK I did a pip install kfp=2.11.0. So how do you recommend I install it instead? Could you please provide the steps

[zeidsolh]

@zeidsolh have you deployed from master ? this requires backend changes that are not in the latest kfp release yet, it will be available in 2.4, or you can deploy from the master branch and check there

Hey Humair, I just deployed from master and still getting the following issue:
│ Warning InspectFailed 64s (x8 over 2m29s) kubelet Failed to apply default image tag "{{$.inputs.parameters['pipelinechannel--get-image-url-Output']}} │
│ ": couldn't parse image name "{{$.inputs.parameters['pipelinechannel--get-image-url-Output']}}": invalid reference format: repository name (library/{{$.inputs.parameters['pipelinechanne │
│ l--get-image-url-Output']}}) must be lowercase