Slurm on GKE - Guide

[danielmarzini]

This guide shows you how to deploy Slurm on a Google Kubernetes Engine (GKE) cluster.

This guide is intended for platform administrators in an enterprise environment who are already managing Kubernetes or GKE clusters, and who need to set up Slurm clusters for AI/ML teams on Kubernetes. This guide is also for AI/ML startups that already use Kubernetes or GKE to run their workloads, such as inference workloads or web apps, and want to use their existing infrastructure to run training workloads with a Slurm interface.

[andrewsykim]

We have a top-level generic module for infrastructure here https://github.com/GoogleCloudPlatform/ai-on-gke/tree/main/infrastructure

Is it possible to reuse that instead of creating another infrastructure module?

[danielmarzini]

unfortunately, the infra part is tied to the described example, I can do it but an a second release. wdyt?

[danielmarzini]

Hello @andrewsykim I implemented the changes with 1 exception still open above, wdyt? Thanks!

[danielmarzini]

@andrewsykim as agreed, can you also add @mwysokin as reviewer? Thanks!

[andrewsykim]

@andrewsykim as agreed, can you also add @mwysokin as reviewer? Thanks!

I'm unable to add him as a reviewer for some reason, but feel free to review anyways

[mwysokin]

Overall LGTM 🖖

I think the module could be improved by making it more flexible when it comes to image versions because most of them are currently hardcoded but it's a nit and a possible room for improvement if people are going to use it.

[andrewsykim]

/gcbrun