Skip to content

Conversation

@gabotechs
Copy link
Collaborator

@gabotechs gabotechs commented Aug 4, 2025

Adds TPCH benchmarks by bringing a lot of upstream DataFusion code into this repo.

Steps for running benchmarks:

  1. Generate a TPCH dataset (needs docker installed in the system):
./benchmarks/gen-tpch.sh
  1. Run TPCH benchmarks
cargo run -p datafusion-distributed-benchmarks --release -- tpch --path data/tpch_sf1

For now, this benchmarks do not run distributed, they run as in https://github.com/apache/datafusion

@gabotechs gabotechs force-pushed the tpch branch 2 times, most recently from 06085b0 to db7f0b0 Compare August 8, 2025 11:39
@gabotechs gabotechs changed the base branch from main to gabrielmusat/extend-session-builder-trait August 8, 2025 11:40
@gabotechs gabotechs force-pushed the gabrielmusat/extend-session-builder-trait branch from 82a2c82 to b7b62be Compare August 8, 2025 11:46
@gabotechs gabotechs force-pushed the tpch branch 2 times, most recently from 7d264f1 to 12ab500 Compare August 8, 2025 11:49
@gabotechs gabotechs marked this pull request as ready for review August 8, 2025 11:53
@robtandy robtandy merged commit 5a657a4 into gabrielmusat/extend-session-builder-trait Aug 8, 2025
3 checks passed
@robtandy robtandy deleted the tpch branch August 8, 2025 15:38
robtandy pushed a commit that referenced this pull request Aug 8, 2025
…text (#85)

* Move all test utils to src/ and hide them behind an "integration" feature. This way, other crates and integration tests as well can use it

* Extend the `SessionBuilder` trait to be able to operate not only at the `SessionStateBuilder` level, but also on `SessionState` and `SessionContext`

* Add benchmarks crate by copying upstream DataFusion code (#73)

* Add benchmarks crate by copying upstream DataFusion code

* Wire-up distributed execution with tpch benchmarks (#86)
robtandy pushed a commit that referenced this pull request Aug 8, 2025
…ture (#84)

* Move all test utils to src/ and hide them behind an "integration" feature. This way, other crates and integration tests as well can use it

* Extend SessionBuilder trait to operate on SessionState and SessionContext (#85)

* Move all test utils to src/ and hide them behind an "integration" feature. This way, other crates and integration tests as well can use it

* Extend the `SessionBuilder` trait to be able to operate not only at the `SessionStateBuilder` level, but also on `SessionState` and `SessionContext`

* Add benchmarks crate by copying upstream DataFusion code (#73)

* Add benchmarks crate by copying upstream DataFusion code

* Wire-up distributed execution with tpch benchmarks (#86)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants