Make queries in TPC-DS get distributed (#264)

* Make TPC-DS tests use DataFusion test dataset

* Delete unnecessary scripts and tweak pipelines

* Add TPC-DS plan assertion tests

* Answer to feedback

* Do not use multithreading

Author: gabotechs

.github/workflows/ci.yml

@@ -52,18 +52,12 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-        with:
-          lfs: true
       - uses: ./.github/actions/setup
-      - name: Install DuckDB CLI
-        run: |
-          curl https://install.duckdb.org | sh
-          mkdir -p $HOME/.local/bin
-          mv /home/runner/.duckdb/cli/latest/duckdb $HOME/.local/bin/
-          echo "$HOME/.local/bin" >> $GITHUB_PATH
-      - name: Run TPC-DS test
-        id: test
-        run: cargo test --features tpcds --test tpcds_test
+      - uses: actions/cache@v4
+        with:
+          path: testdata/tpcds/main.zip
+          key: "main.zip"
+      - run: cargo test --features tpcds --test 'tpcds_*'
 
   format-check:
     runs-on: ubuntu-latest

.gitignore

@@ -3,4 +3,6 @@
 /benchmarks/data/
 testdata/tpch/*
 !testdata/tpch/queries
-testdata/tpcds/data/
+testdata/tpcds/*
+!testdata/tpcds/queries
+!testdata/tpcds/README.md