Set service account for executors

[witten]

To use custom/private Docker images for Spark drivers and executors, one needs to specify an image pull secret. The consensus in #42 was to set the image pull secret onto the default service account in the namespace, and implicitly use that. However, in the case of a cluster that's using RBAC as per this Spark on K8s documentation, Spark will be using a non-default service account, and the image pull secret should be on that service account rather than a global namespace service account.

The difficulty is that, as far as I can tell, there's no way to tell Spark on K8s to spin up executors to use that custom service account, even though there is a way to set the driver's service account with spark.kubernetes.authenticate.driver.serviceAccountName. So the ask in this ticket is to add a comparable mechanism for setting the service account to use for the executors, so that the executors can use the same RBAC configuration and image pull secrets as the driver.

[liyinan926]

So sounds like we should extend the scope of the configuration property to cover both the driver and executors, e.g., something like spark.kubernetes.authenticate.serviceAccountName.

[witten]

That would work great. Or separate settings for each of driver and executor. Either approach would satisfy this use case.

[ku-s-h]

+1