Add --link_existing_gcs_logs for the Results Uploader

[xianyuanjia]

Allows the invocation to be linked against Mobly logs already uploaded to a given GCS directory outside of the tool

[gemini-code-assist]

Summary of Changes

Hello @xianyuanjia, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the Results Uploader tool by providing a new option to link to existing Mobly logs already present in a Google Cloud Storage (GCS) directory. This avoids redundant uploads when logs are already available, streamlining the process for users who manage their GCS storage independently. The changes involve adding a new command-line flag, refactoring file path retrieval, and updating the Resultstore integration to accommodate this new linking capability.

Highlights

• New Command-Line Argument: Introduced --link_existing_gcs_logs to allow associating an invocation with pre-uploaded Mobly logs in a specified GCS directory, rather than re-uploading them.
• GCS Log Handling Logic: Implemented conditional logic in the main execution flow to either upload logs to GCS or reference existing logs based on the new --link_existing_gcs_logs flag.
• File Path Utility: Added a new helper function _get_file_paths_recursively to efficiently retrieve all file paths within a given directory.
• Resultstore Integration Update: Modified the create_action method in the Resultstore client to support an assign_undeclared_outputs flag, which controls how artifacts are mapped, specifically skipping reserved files like test.log and test.xml.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• src/results_uploader/results_uploader.py
  • Added the --link_existing_gcs_logs command-line argument.
  • Introduced the _get_file_paths_recursively helper function.
  • Refactored _upload_dir_to_gcs to utilize the new file path helper.
  • Modified the _add_resultstore_target function signature and call to include a new assign_undeclared_outputs parameter.
  • Implemented conditional logic in the main function to handle existing GCS logs based on the new argument.
• src/results_uploader/resultstore_client.py
  • Defined _RESULTSTORE_RESERVED_FILES to list files that should not be assigned to undeclared_outputs.
  • Updated the create_action method to accept an assign_undeclared_outputs parameter.
  • Modified the create_action logic to conditionally prepend undeclared_outputs/ to artifact UIDs, respecting reserved files.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a --link_existing_gcs_logs flag to associate an invocation with pre-uploaded Mobly logs in GCS. The implementation involves adding the new command-line argument, refactoring file path collection, and adjusting the logic in both the main uploader script and the Resultstore client to handle this new case. My review focuses on the correctness of the new logic. I've identified a potential issue with handling multiple Mobly directories when the new flag is used, and a platform-dependency issue in path construction. Overall, the changes are well-structured, but these two points should be addressed to ensure robustness.

[gemini-code-assist]

This logic could lead to incorrect behavior if args.link_existing_gcs_logs is true and there are multiple Mobly directories found (len(mobly_dirs) > 1). In that case, all targets will point to the same gcs_base_dir, but will be associated with files from different local mobly_dir directories. This is likely not the intended behavior.

To prevent this, I recommend adding a check earlier in the main function to ensure that if link_existing_gcs_logs is used, there is at most one Mobly directory to process. For example, after mobly_dirs is populated:

if args.link_existing_gcs_logs and len(mobly_dirs) > 1:
    logging.error(
        '--link_existing_gcs_logs can only be used with a single Mobly result directory found within logs_dir.'
    )
    exit(1)

[gemini-code-assist]

The path construction f'{gcs_dir}/{path}' is not platform-independent. On Windows, str(gcs_dir) will use backslashes as separators, which will result in mixed-separator GCS paths (e.g., my\gcs/dir/some/file.txt).

To ensure consistent behavior across platforms, you should use .as_posix() to get a POSIX-style path string, similar to how it's used elsewhere in this file.

Suggested change

	gcs_files += [f'{gcs_dir}/{path}' for path in _get_file_paths_recursively(dir_to_upload)]
	gcs_files += [f'{gcs_dir.as_posix()}/{path}' for path in _get_file_paths_recursively(dir_to_upload)]