add option yaml files and update manifest file

[derrickaw]

1. Need option yaml files as well to have all details about a blueprint.
2. Update manifest format to be keyed by "blueprints" and "options".

[gemini-code-assist]

Summary of Changes

Hello @derrickaw, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the TemplatesReleaseMojo Maven plugin by extending its capabilities to include the management and uploading of 'option' YAML files alongside the existing 'blueprint' YAML files. The core impact is a more organized and comprehensive release process, as the manifest file now clearly distinguishes between these two types of configuration files, improving clarity and usability for template deployments.

Highlights

• New 'Option' YAML Files Support: The system now supports uploading and managing 'option' YAML files in addition to the existing 'blueprint' YAML files. A new Maven parameter, yamlOptionsPath, has been introduced to specify the directory for these option files.
• Updated Manifest File Format: The manifest file, which lists the uploaded YAML files, has been updated to categorize blueprints and options under distinct keys: 'blueprints' and 'options'. This provides a more structured representation of the available templates and their configurations.
• Refactored GCS Upload Logic: The logic for uploading YAML files to Google Cloud Storage (GCS) has been refactored into reusable private helper methods (uploadYamlFiles and uploadToGcs), improving code organization and maintainability.
• Enhanced Warning Messages: The warning message for missing YAML directories now explicitly checks for and mentions both blueprint and option directories, providing clearer feedback to the user.
• Comprehensive Test Coverage: Existing test cases have been updated and new ones added to ensure the correct functionality of the new option file handling, the updated manifest structure, and scenarios where one or both YAML directories might be missing.

Changelog

• plugins/templates-maven-plugin/src/main/java/com/google/cloud/teleport/plugin/maven/TemplatesReleaseMojo.java
  • Added HashMap and Map imports for new manifest structure.
  • Introduced yamlOptionsPath Maven parameter to specify the path for option YAML files.
  • Modified the execute method to process and upload both blueprint and option YAML files.
  • Refactored GCS upload logic into new private methods: uploadYamlFiles and uploadToGcs.
  • Updated manifest generation to create a JSON object with 'blueprints' and 'options' keys.
  • Adjusted warning message for missing YAML directories to include both blueprint and option paths.
• plugins/templates-maven-plugin/src/test/java/com/google/cloud/teleport/plugin/maven/TemplatesReleaseMojoTest.java
  • Removed unused imports (Comparator, Collectors).
  • Adjusted baseDir initialization in setUp method for broader directory creation.
  • Initialized yamlOptionsPath in setUp for consistent test setup.
  • Renamed test method testExecute_publishesYamlBlueprintsAndCreatesManifest to testExecute_publishesYamlBlueprintsAndOptionsAndCreatesManifest to reflect new functionality.
  • Modified test to create and verify both blueprint and option YAML files in the manifest.
  • Renamed test method testExecute_yamlBlueprintsDirectoryMissing_logsWarning to testExecute_yamlDirectoriesMissing_logsWarning.
  • Updated warning log message assertion to match the combined check for blueprint and option directories.
  • Renamed test method testExecute_uploadsAllBlueprintsWhenSomeExist to testExecute_uploadsOnlyOptionsWhenBlueprintsMissing.
  • Modified test to verify correct behavior when only option files are present and blueprints are missing.
  • Removed ArgumentCaptor for BlobInfo as it's no longer necessary with the refactored upload methods.

Activity

• The author derrickaw initiated this pull request to introduce new functionality for handling 'option' YAML files.
• The changes involve modifications to the core Maven plugin logic and corresponding updates to its test suite.
• The pull request includes refactoring of existing code to improve modularity and maintainability.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[codecov]

Codecov Report

❌ Patch coverage is 90.00000% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 51.05%. Comparing base (ced67a0) to head (6402065).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...ud/teleport/plugin/maven/TemplatesReleaseMojo.java	90.00%	0 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##               main    #3310    +/-   ##
==========================================
  Coverage     51.05%   51.05%            
- Complexity     5173     5557   +384     
==========================================
  Files           978      978            
  Lines         60490    60504    +14     
  Branches       6638     6639     +1     
==========================================
+ Hits          30881    30891    +10     
  Misses        27445    27445            
- Partials       2164     2168     +4     

Components	Coverage Δ
spanner-templates	70.85% <ø> (-0.02%)	⬇️
spanner-import-export	69.05% <ø> (-0.03%)	⬇️
spanner-live-forward-migration	79.82% <ø> (-0.02%)	⬇️
spanner-live-reverse-replication	77.45% <ø> (-0.02%)	⬇️
spanner-bulk-migration	87.92% <ø> (-0.02%)	⬇️

Files with missing lines	Coverage Δ
...ud/teleport/plugin/maven/TemplatesReleaseMojo.java	66.05% <90.00%> (+3.94%)	⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[chamikaramj]

Thanks!

[derrickaw]

Hi @chamikaramj, addressed your comments. Thanks!

[chamikaramj]

LGTM. Thanks.