skaffold dev doesn't work correctly with charts that use CRD

[edmondop]

I am trying to use Skaffold to install two helm charts on a Kubernetes cluster: the ray operator, and a ray cluster. The ray cluster is a CRD that needs to use the custom tag of the image that Skaffold built. This is my yml

apiVersion: skaffold/v4beta11
kind: Config
metadata:
  name: datafusion-ray-pipeline
build:
  artifacts:
    - image: apache/datafusion-ray
      docker:
        dockerfile: k8s/Dockerfile
        buildArgs:
          CARGO_HOME: /usr/local/cargo
        cacheFrom:
          - ghcr.io/apache/datafusion-ray:latest
      context: ../..
  local:
    useBuildkit: true
deploy:
  helm:
    releases:
      - name: kuberay-operator
        repo: "https://ray-project.github.io/kuberay-helm/"
        remoteChart: kuberay-operator
        namespace: default
        version: "1.1.0"
      - name: raycluster
        repo: "https://ray-project.github.io/kuberay-helm/"
        remoteChart: ray-cluster
        namespace: default
        version: "1.1.0"
        setValues:
          image.repository: "apache/datafusion-ray"
          imagePullPolicy: Always
        setValueTemplates:
          image.tag: "{{.IMAGE_TAG}}"
  kubectl: {}

what happens is the following:

[:56](https://cryptocatz.slack.com/archives/C047YHPDV9B/p1732471006623589)
TEST SUITE: None
WARN[0525] image [apache/datafusion-ray:b5f9bddb97383fb1d40306fc8e1008e4cffee61b70cc24d0d35b5d5784e3bb87] is not used.  subtask=-1 task=DevLoop
WARN[0525] See helm documentation on how to replace image names with their actual tags: https://skaffold.dev/docs/pipeline-stages/deployers/helm/#image-configuration  subtask=-1 task=DevLoop
Waiting for deployments to stabilize...
 - deployment/kuberay-operator is ready.
Deployments stabilized in 1.051 second
Cleaning up...
 - No resources found
release "kuberay-operator" uninstalled
release "raycluster" uninstalled
nothing to deploy

Information

Skaffold version v2.13.2

Steps to reproduce the behavior

The repository is https://github.com/edmondop/datafusion-ray/tree/skaffold-experiment , the files can be found in k8s folder

[idsulik]

@edmondop I think the reason is in the IMAGE_TAG, try to use image: "{{.IMAGE_FULLY_QUALIFIED_datafusion_ray}}" (c) https://skaffold.dev/docs/deployers/helm/

[edmondop]

I tried to use "{{ .IMAGE_TAG_datafusion_ray}}"

Error: INSTALLATION FAILED: YAML parse error on ray-cluster/templates/raycluster-cluster.yaml: error converting YAML to JSON: yaml: line 25: mapping values are not allowed in this context
Cleaning up...

• No resources found

[idsulik]

show your yaml

[edmondop]

apiVersion: skaffold/v4beta11
kind: Config
metadata:
  name: datafusion-ray-pipeline
build:
  artifacts:
    - image: apache/datafusion-ray
      docker:
        dockerfile: k8s/Dockerfile
        buildArgs:
          CARGO_HOME: /usr/local/cargo
        cacheFrom:
          - ghcr.io/apache/datafusion-ray:latest
      context: ../..
  local:
    useBuildkit: true
deploy:
  helm:
    releases:
      - name: kuberay-operator
        repo: "https://ray-project.github.io/kuberay-helm/"
        remoteChart: kuberay-operator
        namespace: default
        version: "1.1.0"
      - name: raycluster
        repo: "https://ray-project.github.io/kuberay-helm/"
        remoteChart: ray-cluster
        namespace: default
        version: "1.1.0"
        setValues:
          image.repository: "apache/datafusion-ray"
          imagePullPolicy: Always
        setValueTemplates:
          image.tag: "{{.IMAGE_TAG_datafusion_ray}}"
  kubectl: {}

[rodney-b]

What do you get if you try "{{.IMAGE_TAG_apache_datafusion_ray}}"?