Skip to content

Commit b6edc89

Browse files
Revert "[GLUTEN-11145][VL] Iceberg test supports Spark3.5 and Spark4.0 (#11146)" (#11374)
This reverts commit 5f6de74.
1 parent ac0611a commit b6edc89

File tree

58 files changed

+21
-126
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

58 files changed

+21
-126
lines changed

.github/workflows/velox_backend_x86.yml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -963,7 +963,7 @@ jobs:
963963
export JAVA_HOME=/usr/lib/jvm/java-17-openjdk
964964
export PATH=$JAVA_HOME/bin:$PATH
965965
java -version
966-
$MVN_CMD clean test -Pspark-3.5 -Pjava-17 -Pbackends-velox -Piceberg -Piceberg-test -Pdelta -Phudi -Ppaimon -Pspark-ut \
966+
$MVN_CMD clean test -Pspark-3.5 -Pjava-17 -Pbackends-velox -Piceberg -Pdelta -Phudi -Ppaimon -Pspark-ut \
967967
-DargLine="-Dspark.test.home=/opt/shims/spark35/spark_home/" \
968968
-DtagsToExclude=org.apache.spark.tags.ExtendedSQLTest,org.apache.gluten.tags.UDFTest,org.apache.gluten.tags.EnhancedFeaturesTest,org.apache.gluten.tags.SkipTest
969969
- name: Upload test report
@@ -1415,7 +1415,6 @@ jobs:
14151415
export PATH=$JAVA_HOME/bin:$PATH
14161416
java -version
14171417
$MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17 -Pbackends-velox \
1418-
-Piceberg -Piceberg-test \
14191418
-Pspark-ut -DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/" \
14201419
-DtagsToExclude=org.apache.spark.tags.ExtendedSQLTest,org.apache.gluten.tags.UDFTest,org.apache.gluten.tags.EnhancedFeaturesTest,org.apache.gluten.tags.SkipTest
14211420
- name: Upload test report

backends-velox/src-iceberg-spark34/test/java/org/apache/gluten/IcebergTestShims.java

Lines changed: 0 additions & 32 deletions
This file was deleted.

backends-velox/src-iceberg10/test/java/org/apache/gluten/execution/TestStoragePartitionedJoins.java renamed to backends-velox/src-iceberg-spark34/test/java/org/apache/gluten/execution/TestStoragePartitionedJoins.java

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,6 @@
1616
*/
1717
package org.apache.gluten.execution;
1818

19-
import org.apache.gluten.IcebergTestShims;
20-
2119
import org.apache.commons.lang3.StringUtils;
2220
import org.apache.iceberg.PlanningMode;
2321
import org.apache.iceberg.Schema;
@@ -657,8 +655,7 @@ private Dataset<Row> randomDataDF(Schema schema, int numRows) {
657655
Iterable<InternalRow> rows = RandomData.generateSpark(schema, numRows, 0);
658656
JavaRDD<InternalRow> rowRDD = sparkContext.parallelize(Lists.newArrayList(rows));
659657
StructType rowSparkType = SparkSchemaUtil.convert(schema);
660-
return IcebergTestShims.internalCreateDataFrame(
661-
spark, JavaRDD.toRDD(rowRDD), rowSparkType, false);
658+
return spark.internalCreateDataFrame(JavaRDD.toRDD(rowRDD), rowSparkType, false);
662659
}
663660

664661
private void append(String table, Dataset<Row> df) throws NoSuchTableException {

backends-velox/src-iceberg10/test/java/org/apache/gluten/execution/TestTPCHStoragePartitionedJoins.java renamed to backends-velox/src-iceberg-spark34/test/java/org/apache/gluten/execution/TestTPCHStoragePartitionedJoins.java

File renamed without changes.

backends-velox/src-iceberg10/test/java/org/apache/gluten/extensions/TestGlutenCopyOnWriteDelete.java renamed to backends-velox/src-iceberg-spark34/test/java/org/apache/gluten/extensions/TestGlutenCopyOnWriteDelete.java

File renamed without changes.

backends-velox/src-iceberg10/test/java/org/apache/gluten/extensions/TestGlutenMergeOnReadDelete.java renamed to backends-velox/src-iceberg-spark34/test/java/org/apache/gluten/extensions/TestGlutenMergeOnReadDelete.java

File renamed without changes.

backends-velox/src-iceberg10/test/java/org/apache/gluten/extensions/TestGlutenMergeOnReadMerge.java renamed to backends-velox/src-iceberg-spark34/test/java/org/apache/gluten/extensions/TestGlutenMergeOnReadMerge.java

File renamed without changes.

backends-velox/src-iceberg10/test/java/org/apache/gluten/extensions/TestGlutenMergeOnReadUpdate.java renamed to backends-velox/src-iceberg-spark34/test/java/org/apache/gluten/extensions/TestGlutenMergeOnReadUpdate.java

File renamed without changes.

backends-velox/src-iceberg10/test/java/org/apache/gluten/extensions/TestGlutenStoragePartitionedJoinsInRowLevelOperations.java renamed to backends-velox/src-iceberg-spark34/test/java/org/apache/gluten/extensions/TestGlutenStoragePartitionedJoinsInRowLevelOperations.java

File renamed without changes.

backends-velox/src-iceberg10/test/java/org/apache/gluten/extensions/TestGlutenSystemFunctionPushDownDQL.java renamed to backends-velox/src-iceberg-spark34/test/java/org/apache/gluten/extensions/TestGlutenSystemFunctionPushDownDQL.java

File renamed without changes.

0 commit comments

Comments
 (0)