Soda-core-spark-df declares a dependency on pyspark>=3.4.0. Looking through the code, it seems to be used only for typing, and should therefore be declared as a dev (eventually optional/extra) dependency.

This makes it unfortunately impossible with databricks-connect: https://docs.databricks.com/aws/en/dev-tools/databricks-connect/python/install#install-the-databricks-connect-client-with-venv