feat: support spanner full text search#606
Merged
Conversation
Adds spanddl support for TOKENLIST columns used for full-text search in spanner and the search indexes that accompanies them. https://docs.cloud.google.com/spanner/docs/full-text-search/tokenization https://docs.cloud.google.com/spanner/docs/full-text-search/search-indexes TOKENLIST columns are not valid in SELECT statements, and instead returns errors. They also have no type definition available to use in code generation, and are for those two reasons excluded from the code generator using a new helper method QueryableColumns.
oscarmuhr
force-pushed
the
support-full-text-search
branch
from
9a44f47 to
c55ccd6
Compare
Adds handling of tokenlist columns to the descriptorcodegen. These columns are only included in the top level descriptor of the table, and not in methods that return the full collection of column name, column IDs or column expressions since the columsn are not queryable and this would be the main usecase for those methods. Adds descriptor generation for search indexes. Since these only include TOKENLIST columns, the descriptor is limited to exposing the index ID + Name and not helpers for column access.
alethenorio
reviewed
Mar 24, 2026
alethenorio
approved these changes
Mar 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: support spanner full text search
Adds spanddl support for TOKENLIST columns used for full-text search in spanner and the search indexes that accompanies them.
https://docs.cloud.google.com/spanner/docs/full-text-search/tokenization
https://docs.cloud.google.com/spanner/docs/full-text-search/search-indexes
TOKENLIST columns are not valid in SELECT statements, and instead returns errors. They also have no type definition available to use in code generation, and are for those two reasons excluded from the code generator using a new helper method QueryableColumns.
feat: add tokenlist + search index support to descriptor generation
Adds handling of tokenlist columns to the descriptorcodegen. These columns are only included
in the top level descriptor of the table, and not in methods that return the full collection of
column name, column IDs or column expressions since the columsn are not queryable and this would
be the main usecase for those methods.
Adds descriptor generation for search indexes. Since these only include TOKENLIST columns,
the descriptor is limited to exposing the index ID + Name and not helpers for column access.