Error spark.kubernetes.executor.podNamePrefix is invalid

[preetsindhal]

Hi
getting below error while job submission.

ERROR ExecutorPodsSnapshotsStoreImpl: Going to stop due to IllegalArgumentException
java.lang.IllegalArgumentException: 'kyuubi-connection-spark-sql-anonymous-c433f017-1c32-489e-b6bf-e1e811ed62ac-df15af838a6cf215' in spark.kubernetes.executor.podNamePrefix is invalid. must conform https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-label-names and the value length <= 47

If i set the spark.kubernetes.executor.podNamePrefix to some string it results in duplicate pod names and fails with error, pod already exists.

Thanks.

[pan3793]

which spark version you are using? there is a known issue in Spark 3.3.0 apache/spark#36999

[fuhaiq]

遇到同样的问题，应该就是 36999 导致的

可以设置 spark.kubernetes.executor.podNamePrefix 这个参数，executor pod 会以这个前缀开头，后面追加数字的。