Add support for TABLESAMPLE #1580

yoavcloud · 2024-12-07T06:57:28Z

This PR adds support for the TABLESAMPLE option in the following dialects:

Collateral work includes expanding the use of a constructor function to create Table structs in unit tests to avoid modifying many files when adding default options to the Table struct.

src/dialect/mod.rs

iffyio · 2024-12-08T08:56:23Z

src/ast/query.rs

+        sample: Option<Box<TableSample>>,
+        /// Position of the table sample modifier in the table factor. Default is after the table alias
+        /// e.g. `SELECT * FROM tbl t TABLESAMPLE (10 ROWS)`. See `Dialect::supports_table_sample_before_alias`.
+        sample_before_alias: bool,


Would it make sense to use an enum here? e.g

enum TableSampleKind { BeforeTableAlias(Box<TableSample>) // ... } sample: Option<TableSampleKind>

thinking that avoids the surplus flag on the table factor and would lend itself to be extensible if required later on

iffyio · 2024-12-08T08:59:12Z

src/parser/mod.rs

+                self.expect_token(&Token::RParen)?;
+                TableSample::Bucket(TableSampleBucket { bucket, total, on })
+            } else {
+                let value = match self.try_parse(|p| p.parse_number_value()) {


Should the try_parse rather be maybe_parse (we seem to be ignoring the returned error otherwise)?

iffyio · 2024-12-08T17:26:10Z

src/parser/mod.rs

+                let value = match self.maybe_parse(|p| p.parse_number_value()) {
+                    Ok(Some(num)) => num,
+                    _ => {


Suggested change

let value = match self.maybe_parse(|p| p.parse_number_value()) {

Ok(Some(num)) => num,

_ => {

let value = match self.maybe_parse(|p| p.parse_number_value())? {

Some(num) => num,

None => {

I think this is usually the recursion limit or similar fatal error we can propagate

iffyio · 2024-12-08T17:27:40Z

src/parser/mod.rs

+                if self.peek_token().token == Token::RParen
+                    && !self.dialect.supports_implicit_table_sample_method()


maybe this could be simplified as if self.dialect.supports_implicit_table_sample_method() && self.consume_token(Token::RParen) it would also let us skip the expect_token that follows as well?

iffyio · 2024-12-08T17:28:18Z

src/parser/mod.rs

+                repeatable: seed,
+            })
+        // Try to parse without an explicit table sample method keyword
+        } else if self.peek_token().token == Token::LParen {


Suggested change

} else if self.peek_token().token == Token::LParen {

} else if self.consume_token(Token::LParen) {

iffyio · 2024-12-08T17:31:39Z

tests/sqlparser_snowflake.rs

+        "SELECT * FROM testtable SAMPLE (10)",
+        "SELECT * FROM testtable TABLESAMPLE BERNOULLI (10)",


for the scenarios that currently rely on one_statement_parse_to, could we represent them faithfully when displaying? e.g this and the ROW vs BERNOULLI variants etc

Not sure if I understand the question, but if you mean whether the variants I added as interchangeable are really interchangeable in the dialect, as far as I understand yes. If not, please elaborate?

Ah yeah so I was rather thinking that ideally we preserve the syntax roundtrip, even though they'r interchangeable in some dialects

snowflake_and_generic().verified_stmt("SELECT * FROM testtable TABLESAMPLE ROW (20.3)"); snowflake_and_generic().verified_stmt("SELECT * FROM testtable SAMPLE BLOCK (3) SEED (82)");

yoavcloud · 2024-12-09T15:23:48Z

@iffyio please see latest commit

iffyio

Thanks @yoavcloud! Left a comment to distinguish sample vs tablesample. Otherwise this looks good to me!

iffyio · 2024-12-09T16:02:06Z

tests/sqlparser_snowflake.rs

+    snowflake_and_generic().one_statement_parses_to(
+        "SELECT * FROM testtable SAMPLE (10)",
+        "SELECT * FROM testtable TABLESAMPLE (10)",
+    );


Oh could we do the same here as well? clickhouse for example doesn't have has SAMPLE but not TABLESAMPLE so that the roundtrip stays correct

Added support for clickhouse as well, with a different approach to how to model the it in the AST

iffyio

LGTM! Thanks @yoavcloud
cc @alamb

alamb · 2024-12-15T21:03:26Z

Thanks @yoavcloud and @iffyio

yoavcloud force-pushed the table_sample branch from 789e1db to 84b58d7 Compare December 7, 2024 06:58

yoavcloud marked this pull request as draft December 7, 2024 07:07

yoavcloud marked this pull request as ready for review December 7, 2024 07:36

iffyio reviewed Dec 8, 2024

View reviewed changes

yoavcloud requested a review from iffyio December 8, 2024 16:49

iffyio reviewed Dec 8, 2024

View reviewed changes

yoavcloud requested a review from iffyio December 9, 2024 08:22

iffyio reviewed Dec 9, 2024

View reviewed changes

yoavcloud added 10 commits December 13, 2024 21:15

Add support for table sample, initial commit

df7b1ba

Model implicit table sample logic in dialect

3f6822f

Add unit tests for common

3a70360

Box TableSample to avoid increasing the size of Table

aa907f1

Box TableSample to avoid increasing the size of Table

92091a9

Code review comments

8113dd6

Code review comments

353ce58

Keep source syntax

464fca3

Remove redundant code now that syntax roundtrip is more accurate

f58f9bb

Add support for table sample in Clickhouse

ef04565

yoavcloud force-pushed the table_sample branch from a0fdfbf to ef04565 Compare December 13, 2024 20:17

yoavcloud requested a review from iffyio December 13, 2024 20:18

iffyio approved these changes Dec 15, 2024

View reviewed changes

iffyio merged commit 316bb14 into apache:main Dec 15, 2024
8 checks passed

yoavcloud mentioned this pull request Jan 12, 2025

feat: Supporting SAMPLE parsing #1566

Closed

1 task

theirix mentioned this pull request Jun 7, 2025

Support data source sampling with TABLESAMPLE apache/datafusion#16325

Open

-                let value = match self.maybe_parse(|p| p.parse_number_value()) {
-                    Ok(Some(num)) => num,
-                    _ => {
+                let value = match self.maybe_parse(|p| p.parse_number_value())? {
+                    Some(num) => num,
+                    None => {

		if self.peek_token().token == Token::RParen
		&& !self.dialect.supports_implicit_table_sample_method()

	} else if self.peek_token().token == Token::LParen {
	} else if self.consume_token(Token::LParen) {

		"SELECT * FROM testtable SAMPLE (10)",
		"SELECT * FROM testtable TABLESAMPLE BERNOULLI (10)",

Add support for TABLESAMPLE #1580

Add support for TABLESAMPLE #1580

Uh oh!

Conversation

yoavcloud commented Dec 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yoavcloud commented Dec 9, 2024

Uh oh!

iffyio left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iffyio left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alamb commented Dec 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

yoavcloud commented Dec 7, 2024 •

edited

Loading