feat(Spanner): Add workflow dispatch options to run specific test suites or individual tests within Spanner PR

[pratickchokhani]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @pratickchokhani, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the CI/CD testing infrastructure by providing more granular control over test execution. It introduces a mechanism to specify and run individual tests or test suites directly from the command line, which is particularly useful for targeted debugging and verification within Spanner PR workflows. This change streamlines the development process by reducing the need to run entire test suites for minor changes, thereby improving efficiency.

Highlights

• New Test Specification Flag: A new command-line flag, --test, has been introduced, allowing users to specify individual tests or test suites to execute within CI/CD workflows.
• Integrated into CI/CD Runners: The new test specification capability has been integrated into the run-it-smoke-tests, run-it-tests, run-load-tests, and run-unit-tests Go programs, enabling targeted test execution.
• Maven Argument Generation: The maven-workflows package has been enhanced to translate the --test flag into the appropriate -Dtest= Maven argument, facilitating specific test execution.

Changelog

• cicd/cmd/run-it-smoke-tests/main.go
  • Added the SpecificTest flag to the runMaven command invocation.
• cicd/cmd/run-it-tests/main.go
  • Added the SpecificTest flag to the runMaven command invocation.
• cicd/cmd/run-load-tests/main.go
  • Added the SpecificTest flag to the runMaven command invocation.
• cicd/cmd/run-unit-tests/main.go
  • Added the SpecificTest flag to the runMaven command invocation.
• cicd/internal/flags/common-flags.go
  • Declared a new package-level variable testToRun.
  • Registered a new command-line flag --test for specifying tests.
  • Added a new public function TestToRun() to retrieve the value of the --test flag.
• cicd/internal/workflows/maven-workflows.go
  • Extended the MavenFlags interface with a SpecificTest method signature.
  • Implemented the SpecificTest method to construct the -Dtest= Maven argument based on the provided test string.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/spanner-pr.yml

Activity

• No specific activity (comments, reviews, or progress updates) has been recorded for this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.69%. Comparing base (18949a0) to head (8499b11).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #3337      +/-   ##
============================================
+ Coverage     51.04%   56.69%   +5.64%     
+ Complexity     5213     1747    -3466     
============================================
  Files           994      474     -520     
  Lines         60672    27066   -33606     
  Branches       6651     2883    -3768     
============================================
- Hits          30972    15345   -15627     
+ Misses        27527    10800   -16727     
+ Partials       2173      921    -1252     

Components	Coverage Δ
spanner-templates	71.94% <ø> (+1.06%)	⬆️
spanner-import-export	∅ <ø> (∅)
spanner-live-forward-migration	79.84% <ø> (+<0.01%)	⬆️
spanner-live-reverse-replication	77.37% <ø> (-0.09%)	⬇️
spanner-bulk-migration	87.99% <ø> (+0.06%)	⬆️
see 552 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[manitgupta]

What is the use-case for this? Curious to know why we would need to run a specific test via the CI/CD flow?

[pratickchokhani]

What is the use-case for this? Curious to know why we would need to run a specific test via the CI/CD flow?

This is only applicable when the workflow is executed manually from the Actions page. While testing integration test, this will reduce the execution time to ~15m from the current time of ~1.5 hr.

Only to be used for debugging purpose.

[manitgupta]

Can't we run the ITs directly using Maven commands?
Something like this -

mvn clean verify -PtemplatesIntegrationTests -pl v2/spanner-to-sourcedb -am -Dtest=SpannerToSourceDbCustomShardIT -DartifactBucket="manit-testing" -Dproject="span-cloud-testing" -DspannerInstanceId="manit-testing"

[pratickchokhani]

Can't we run the ITs directly using Maven commands? Something like this -

mvn clean verify -PtemplatesIntegrationTests -pl v2/spanner-to-sourcedb -am -Dtest=SpannerToSourceDbCustomShardIT -DartifactBucket="manit-testing" -Dproject="span-cloud-testing" -DspannerInstanceId="manit-testing"

We can but, for some tests other resources like Cassandra, Mysql or other instances are required. Thought process was that using this those setup will not be required.

[manitgupta]

Thanks, got it. We should document this class of tests and how to run them somewhere.
CC: @darshan-sj

[pratickchokhani]

Thanks, got it. We should document this class of tests and how to run them somewhere. CC: @darshan-sj

Will be updating the Integration test playbook.

[darshan-sj]

Looks good. Added few comments.