Curious about the TPC-H(S3) SF=100 Benchmark results #4903
-
|
I see that in almost all benchmarks of TPC-H(S3) category the vortex performs better compared to parquet. But it lags behind in SF=100 benchmark. Any particular reason for this? Is it a vortex performance issue or duckdb? P.S. I am still new to this columnar file formats. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Hello! Sorry for the very late reply, we will do our best in the future to be more on top of these. We think this is a case of bad IO/compute interleaving (scheduling), or in other words the manner in which we perform IO-bound and compute-bound tasks are not scheduled by the runtime appropriately for such a large scale factor. This specific benchmark is something we haven't really looked at yet so it isn't crazy that we perform badly here. Apologies again for the late response! Please let us know if you have other questions. |
Beta Was this translation helpful? Give feedback.
Hello! Sorry for the very late reply, we will do our best in the future to be more on top of these.
We think this is a case of bad IO/compute interleaving (scheduling), or in other words the manner in which we perform IO-bound and compute-bound tasks are not scheduled by the runtime appropriately for such a large scale factor.
This specific benchmark is something we haven't really looked at yet so it isn't crazy that we perform badly here.
Apologies again for the late response! Please let us know if you have other questions.