I’m curious about why integration between OPA and Spark is not supported.

[wooarchi]

want to use OPA to control access to specific paths in the data lake that Spark accesses. Is this approach not suitable?

[razvan]

Spark, being mostly a compute engine, is only one from the many ways to access the data your data lake. Therefore a better approach is to implement your data access policies in the storage engine.

For example, you can use Spark to access data in a kerberized Hadoop cluster and implement OPA policies for HDFS paths.

[wooarchi]

Thank you for your kind response. Thanks to you, I fully understand.