@WeichenXu123 please could you review this code and approve the workflow?

Sorry for late response!
One question: I think

SparkSession \
            .builder \
            .appName("JoblibSparkBackend") \
            .getOrCreate()

will reuse existing spark session if there's one, otherwise it will create a spark session.

Did you find any case that one spark session is created but joblib-spark shutdown the session and then start a new session ?

I have run into situations where:

• I wanted to name the spark session that joblib used
• I had spark as an unused variable in my code, and wanted to explicitly show it was used by passing it.
  Both are pretty minor, but having the option to explicitly pass a spark session is nice sometimes, even if the behaviour is unchanged