You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Nov 14, 2022. It is now read-only.
We try to run K-means algorithm by using PMEM-SPILL on spark-3.0.1 while meet the issue like following picture:
PMEM-SPILL overwrite the source code of spark("src/main/scala/org/apache/spark/internal/config/package.scala") and add some configs for PMEM
While the package.scala of spark-3.0.0 is different with the one of spark-3.0.1 and the issue is caused by the package.scala of OAP-1.1 PMEM-SPILL not contains EXECUTOR_ALLOW_SPARK_CONTEXT.
So how can we decouple package.scala of spark? If we cannot decouple this file, we need to adapt to all some versions of spark.