[FEATURE] Test Spark-Connect Compatibility

[newfront]

Is your feature request related to a problem? Please describe.
In order to continue to work with the Spark ecosystem, we need to ensure we can continue to support the various connection types - including spark-connect.

Describe the solution you'd like
The application will run when using the spark-connect backed SparkSession.

Describe alternatives you've considered
Zero. We need this.

Additional context
This will be a way to easily adapt to the Delta 4.x and Spark 4.x ecosystem.

[stampthecoder]

@newfront I'm interested in this too... This use case is exactly what my company needs. I like the fact nike is keeping it under their umbrella for maintainability ....godspeed.