Unit testing the code with Spark Connect #87
Description
I created a pull request and confirmed that chispa is fully compliant wth Spark Connect: #86
We don't want to make chispa depend on Spark connect because then it'd have to depend on PySpark >= 3.5. We want chispa to support many old Spark versions for users that are still on legacy Spark runtimes.
It would be nice to test chispa+Spark Connect in the CI tho. We don't want to ever add any chispa code that wouldn't work with Spark Connect. We want all Spark Connnect users to always have access to chispa as well.
Any thoughts on how to add Spark Connect to the CI test suite?