[SQL] add SHOW command #36509

ahmedabu98 · 2025-10-14T16:02:20Z

Addresses #35637

Adds the following commands:

`SHOW CATALOGS [ LIKE regex_pattern ]`

+------+------+
| NAME | TYPE |
+------+------+
| my_very_long_catalog_name | local |
| default | local |
| my_catalog | local |
+------+------+

`SHOW DATABASES [ ( FROM | IN )? catalog_name ] [ LIKE regex_pattern ]`

+------+
| NAME |
+------+
| my_other_db |
| my_db |
| my_database_that_has_a_very_long_name |
| default |
+------+

`SHOW TABLES [ ( FROM | IN )? [ catalog_name '.' ] database_name ] [ LIKE regex_pattern ]`

+------+------+
| NAME | TYPE |
+------+------+
| my_table | text |
| my_other_table | text |
| my_other_table_with_a_long_name | test |
+------+------+

`SHOW CURRENT CATALOG`

+------+------+
| NAME | TYPE |
+------+------+
| my_catalog | local |
+------+------+

`SHOW CURRENT DATABASE`

+------+
| NAME |
+------+
| my_db |
+------+

gemini-code-assist · 2025-10-14T16:02:45Z

Summary of Changes

Hello @ahmedabu98, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the introspection capabilities of the Beam SQL CLI by adding a suite of SHOW commands. These commands provide users with a clear and structured way to explore the available catalogs, databases, and tables within their SQL environment, making it easier to understand and navigate the data landscape. The changes involve updates to the SQL parser, core catalog management, and new classes to handle the execution and formatted output of these commands.

Highlights

New SQL SHOW Commands: Introduced SHOW CATALOGS, SHOW DATABASES, SHOW TABLES, SHOW CURRENT CATALOG, and SHOW CURRENT DATABASE commands to the Beam SQL CLI.
Catalog and Database Introspection: Added functionality to list available catalogs, databases within the current catalog, and tables within the current database, enhancing visibility into the SQL environment.
Parser and Schema Integration: Integrated the new SHOW commands into the SQL parser and extended the Catalog, CatalogSchema, and BeamCalciteSchema interfaces and implementations to support retrieving and displaying this metadata.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

github-actions · 2025-10-14T17:35:35Z

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

ahmedabu98 · 2025-10-20T16:31:07Z

Failing tests are unrelated

ahmedabu98 · 2025-10-20T16:31:08Z

assign set of reviewers

github-actions · 2025-10-20T16:35:07Z

Assigning reviewers:

R: @m-trieu for label java.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

github-actions · 2025-10-28T12:15:56Z

Reminder, please take a look at this pr: @m-trieu

github-actions · 2025-10-30T12:16:11Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

...nsions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlShowCatalogs.java

...sions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlShowDatabases.java

...tensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlShowTables.java

...va/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/catalog/Catalog.java

github-actions · 2025-11-08T12:14:25Z

Reminder, please take a look at this pr: @kennknowles

derrickaw · 2025-11-11T13:29:04Z

waiting on author

ahmedabu98 · 2025-11-11T20:03:20Z

@kennknowles thx for the comments. This is ready for another review

cc @talatuyarer

ahmedabu98 · 2025-11-13T22:54:49Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces several SHOW commands to Beam SQL, enhancing its interactive capabilities. Specifically, it adds SHOW CATALOGS, SHOW DATABASES, SHOW TABLES, and SHOW CURRENT ... commands. The implementation is comprehensive, touching the SQL parser, adding new Sql* node classes for the commands, and integrating them with the existing catalog management infrastructure. The code is well-organized and includes extensive tests.

My review focuses on correctness and maintainability. I've identified a few areas for improvement:

There are a couple of redundant null checks in the new SHOW command implementations.
The string generation for table borders in the output formatting can be made more efficient and readable.
One of the new tests for SHOW CATALOGS doesn't correctly test the LIKE pattern matching.

I've provided specific suggestions for each of these points. Overall, this is a solid contribution that significantly improves the usability of Beam SQL.

...nsions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlShowCatalogs.java

...sions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlShowDatabases.java

...tensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlShowTables.java

...a/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamSqlCliCatalogTest.java

...sions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlShowDatabases.java

...nsions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlShowCatalogs.java

sdks/java/extensions/sql/src/main/codegen/includes/parserImpls.ftl

kennknowles · 2025-11-19T15:44:27Z

sdks/java/extensions/sql/src/main/codegen/includes/parserImpls.ftl

 }

+/**
+ * SHOW TABLES [ ( FROM | IN )? [ catalog_name '.' ] database_name ] [ LIKE regex_pattern ]


Would [catalog_name '.' ] database_name already exist somewhere as a general construction, or could share between grammar rules? It seems like it is a parsing rule for "database reference"

We don't have it as a general construction yet (catalogs and databases are relatively new concepts in Beam SQL).

I could put a comment at the top of the doc, something like databaseRef: [ catalog_name '.' ] database_name, and replace the occurrences with databaseRef

Lmk if that's what you had in mind?

Well.. we've have tables named like bigquery.my_dataset.my_tablename and pubsub.my_topic for a long time. Ideally we can merge these two things soonish since they have identical syntax and meaning.

...sions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlShowDatabases.java

...tensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlShowTables.java

.../extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamSqlCliDatabaseTest.java

…_sql

github-actions · 2025-11-28T12:16:39Z

Reminder, please take a look at this pr: @kennknowles

github-actions · 2025-12-02T12:17:31Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

Abacn · 2025-12-02T16:16:49Z

R: @kennknowles

github-actions · 2025-12-02T16:17:58Z

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

…_sql

kennknowles

My comments are improvement suggestions but we don't have to block on it. Please consider them for followups.

kennknowles · 2025-12-08T23:26:15Z

sdks/java/extensions/sql/src/main/codegen/includes/parserImpls.ftl

 }

+/**
+ * SHOW TABLES [ ( FROM | IN )? [ catalog_name '.' ] database_name ] [ LIKE regex_pattern ]


Well.. we've have tables named like bigquery.my_dataset.my_tablename and pubsub.my_topic for a long time. Ideally we can merge these two things soonish since they have identical syntax and meaning.

kennknowles · 2025-12-08T23:27:42Z

sdks/java/extensions/sql/src/main/codegen/includes/parserImpls.ftl

+    <SHOW> <CURRENT> { s.add(this); }
+    {
+        List<String> path = new ArrayList<String>();
+        path.add("beamsystem");


since this is a weird magic string, maybe also __beamsystem__? But also yea we could leave as beamsystem and that could eventually be something we let users query for more flexible processing of the contents. I know most RDBMSes do that kind of thing, yea?

...tensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/CatalogManagerSchema.java

...sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlCreateExternalTable.java

...nsions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlDropDatabase.java

kennknowles · 2025-12-08T23:33:57Z

...xtensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/parser/SqlDropTable.java

    if (schema instanceof CatalogManagerSchema) {
-      CatalogSchema catalogSchema = ((CatalogManagerSchema) schema).getCatalogSchema(pathOverride);
+      CatalogSchema catalogSchema =
+          pathOverride.catalog() != null


ditto - and I do think doing it in the parse layer is the right way to go. But we should probably be able to make a common "catalog reference" and "table reference" construct.

But we should probably be able to make a common "catalog reference" and "table reference" construct.

Not sure I understand this, but lmk if the new change takes care of it

ahmedabu98 added 2 commits October 14, 2025 10:09

add SHOW command

3d43232

add SHOW CURRENT

ff335dc

github-actions bot added java extensions sql labels Oct 14, 2025

add SHOW CURRENT

50928b8

add LIKE pattern

feb2d81

ahmedabu98 mentioned this pull request Oct 14, 2025

[Task]: Polish Beam SQL user experience #35637

Open

17 tasks

spotless

0e816b0

github-actions bot added the Next Action: Reviewers label Oct 20, 2025

github-actions bot added the slow-review label Oct 28, 2025

github-actions bot added reassigned-reviewers and removed slow-review labels Oct 30, 2025

kennknowles reviewed Oct 31, 2025

View reviewed changes

github-actions bot added the slow-review label Nov 8, 2025

github-actions bot added Next Action: Author and removed Next Action: Reviewers slow-review labels Nov 11, 2025

use SqlCall; remove @nullable

f71cf7e

github-actions bot removed the Next Action: Author label Nov 11, 2025

github-actions bot added the Next Action: Reviewers label Nov 11, 2025

spotless

7e14d95

gemini-code-assist bot reviewed Nov 13, 2025

View reviewed changes

kennknowles requested changes Nov 19, 2025

View reviewed changes

ahmedabu98 added 2 commits November 19, 2025 14:17

Merge branch 'master' of https://github.com/ahmedabu98/beam into show…

64c00a4

…_sql

minor fixes

653cc40

github-actions bot added the slow-review label Nov 28, 2025

github-actions bot removed the slow-review label Dec 2, 2025

use PrintWriter and flush all at once

c97da8e

ahmedabu98 added 2 commits December 2, 2025 11:37

correct PrintWriter usage

6fde9c9

cleanup

bb2baa9

ahmedabu98 force-pushed the show_sql branch from e09946b to bb2baa9 Compare December 8, 2025 19:05

ahmedabu98 added 2 commits December 8, 2025 14:07

Merge branch 'master' of https://github.com/ahmedabu98/beam into show…

5108f8d

…_sql

fix name

4a867e7

ahmedabu98 requested a review from kennknowles December 8, 2025 19:26

kennknowles approved these changes Dec 8, 2025

View reviewed changes

ahmedabu98 added 2 commits December 9, 2025 12:13

address comments

e3b40a0

spotless

c032536

ahmedabu98 merged commit d119b29 into apache:master Dec 9, 2025
20 checks passed

[SQL] add SHOW command #36509

[SQL] add SHOW command #36509

Uh oh!

Conversation

ahmedabu98 commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SHOW CATALOGS [ LIKE regex_pattern ]

SHOW DATABASES [ ( FROM | IN )? catalog_name ] [ LIKE regex_pattern ]

SHOW TABLES [ ( FROM | IN )? [ catalog_name '.' ] database_name ] [ LIKE regex_pattern ]

SHOW CURRENT CATALOG

SHOW CURRENT DATABASE

Uh oh!

gemini-code-assist bot commented Oct 14, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

github-actions bot commented Oct 14, 2025

Uh oh!

ahmedabu98 commented Oct 20, 2025

Uh oh!

ahmedabu98 commented Oct 20, 2025

Uh oh!

github-actions bot commented Oct 20, 2025

Uh oh!

github-actions bot commented Oct 28, 2025

Uh oh!

github-actions bot commented Oct 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 8, 2025

Uh oh!

derrickaw commented Nov 11, 2025

Uh oh!

ahmedabu98 commented Nov 11, 2025

Uh oh!

ahmedabu98 commented Nov 13, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kennknowles Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

ahmedabu98 Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

kennknowles Dec 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 28, 2025

Uh oh!

github-actions bot commented Dec 2, 2025

Uh oh!

Abacn commented Dec 2, 2025

Uh oh!

github-actions bot commented Dec 2, 2025

Uh oh!

kennknowles left a comment

Choose a reason for hiding this comment

Uh oh!

kennknowles Dec 8, 2025

Choose a reason for hiding this comment

Uh oh!

kennknowles Dec 8, 2025

Choose a reason for hiding this comment

ahmedabu98 commented Oct 14, 2025 •

edited

Loading

`SHOW CATALOGS [ LIKE regex_pattern ]`

`SHOW DATABASES [ ( FROM | IN )? catalog_name ] [ LIKE regex_pattern ]`

`SHOW TABLES [ ( FROM | IN )? [ catalog_name '.' ] database_name ] [ LIKE regex_pattern ]`

`SHOW CURRENT CATALOG`

`SHOW CURRENT DATABASE`