feat(bigquery): add maximumBytesBilled source config

[Deeven-Seru]

Fixes #2719
Summary

• add optional maximumBytesBilled to BigQuery source config\n- apply per-query cap to dry runs and query execution- document and test config parsing/initialization

Testing

• go test ./internal/sources/bigquery ./internal/tools/bigquery/...

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the BigQuery source configuration by adding a new "maximumBytesBilled" option. This feature empowers users to define a maximum amount of data (in bytes) that a BigQuery query can process, thereby providing a crucial mechanism for cost control. Queries that are projected to exceed this limit will now fail early, preventing unexpected billing. The change is integrated across query dry runs and execution, and is thoroughly documented and tested to ensure reliability and ease of use.

Highlights

• New Configuration Option: Introduced an optional "maximumBytesBilled" field to the BigQuery source configuration, allowing users to set a per-query byte scanned cap.
• Query Execution Control: Implemented logic to apply the "maximumBytesBilled" cap to both dry runs and actual query executions, ensuring queries exceeding the limit fail before consuming resources.
• Documentation and Testing: Updated documentation to reflect the new configuration and added comprehensive test cases for parsing and initialization of "maximumBytesBilled".

Changelog

• docs/en/resources/sources/bigquery.md
  • Added "maximumBytesBilled" to BigQuery source configuration examples.
  • Included "maximumBytesBilled" in the BigQuery source reference table with its description.
• internal/sources/bigquery/bigquery.go
  • Added "MaximumBytesBilled" field to the Config struct.
  • Initialized "MaximumBytesBilled" in the Initialize method.
  • Added "MaximumBytesBilled" field to the Source struct.
  • Implemented GetMaximumBytesBilled method for the Source struct.
  • Applied MaxBytesBilled to the BigQuery query object if "MaximumBytesBilled" is configured.
• internal/sources/bigquery/bigquery_test.go
  • Added a test case to verify parsing of "maximumBytesBilled" from YAML configuration.
  • Introduced a new test suite, TestInitialize_MaximumBytesBilled, to validate the initialization of the "MaximumBytesBilled" field.
• internal/tools/bigquery/bigqueryanalyzecontribution/bigqueryanalyzecontribution.go
  • Added GetMaximumBytesBilled() to the compatibleSource interface.
  • Passed the maximumBytesBilled value from the source to the bqutil.DryRunQuery function.
• internal/tools/bigquery/bigquerycommon/util.go
  • Updated the DryRunQuery function signature to accept a maximumBytesBilled parameter.
  • Set the MaximumBytesBilled property in the BigQuery Query request during dry runs.
• internal/tools/bigquery/bigqueryexecutesql/bigqueryexecutesql.go
  • Added GetMaximumBytesBilled() to the compatibleSource interface.
  • Passed the maximumBytesBilled value from the source to the bqutil.DryRunQuery function.
• internal/tools/bigquery/bigqueryforecast/bigqueryforecast.go
  • Added GetMaximumBytesBilled() to the compatibleSource interface.
  • Passed the maximumBytesBilled value from the source to the bqutil.DryRunQuery function.
• internal/tools/bigquery/bigquerysql/bigquerysql.go
  • Added GetMaximumBytesBilled() to the compatibleSource interface.
  • Passed the maximumBytesBilled value from the source to the bqutil.DryRunQuery function.

Activity

• No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a maximumBytesBilled configuration for the BigQuery source, which is a valuable addition for managing query costs. The implementation is thorough, with changes applied to the source configuration, query execution logic, tool integrations, documentation, and tests. My review includes one suggestion to enhance the robustness of the new configuration field by adding validation.

[Deeven-Seru]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a new maximumBytesBilled configuration option for the BigQuery source to allow for cost control on queries. The implementation adds the new setting and propagates it to query executions and dry runs in the relevant tools. The changes include validation for the new setting, updated documentation, and new tests for parsing and initialization. The implementation appears correct and complete. I have reviewed the changes and found no issues.