Skip to content

Add run_test prompt#9292

Merged
tagboola merged 3 commits intofeature-branch/mcp/mobile-testingfrom
tagboola/add-run-test-prompt
Oct 9, 2025
Merged

Add run_test prompt#9292
tagboola merged 3 commits intofeature-branch/mcp/mobile-testingfrom
tagboola/add-run-test-prompt

Conversation

@tagboola
Copy link
Contributor

@tagboola tagboola commented Oct 9, 2025

Description

Scenarios Tested

Sample Commands

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @tagboola, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant new capability for Firebase App Testing by adding an AI-guided prompt. This prompt, named "run_test", assists developers in setting up and executing automated tests for their Android mobile applications, providing detailed prerequisites, test case generation guidelines, and instructions for running the tests. The feature is currently enabled via the "mcpalpha" experiment flag, signifying its early-stage development.

Highlights

  • AI-Guided App Testing Prompt: Introduced a new "run_test" prompt designed to guide developers through generating and executing tests for mobile applications using the Firebase App Testing agent.
  • Experiment Flag Integration: The new App Testing prompt is integrated under the "mcpalpha" experiment flag, indicating it's an alpha feature.
  • Version Bump: The "firebase-tools" package version has been incremented from "14.19.0" to "14.19.1".
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new run_test prompt for the Firebase App Testing agent, enabled via the mcpalpha experiment flag. The changes include adding the new prompt, its definition with detailed instructions for an AI model, and wiring it into the existing prompt system. My review focuses on the new prompt definition in src/mcp/prompts/apptesting/run_test.ts. I've pointed out several typos in the prompt text and a potential functional issue with an incorrect tool name, which could prevent the feature from working as expected. The changes are otherwise well-structured.

@tagboola tagboola requested a review from jrothfeder October 9, 2025 17:43
@tagboola tagboola changed the title Tagboola/add run test prompt Add run_test prompt Oct 9, 2025
@jrothfeder
Copy link
Contributor

This looks good. One thing I was surprised to see is that Gemini was able to figure a lot out by simply examining the schema of the parameters in the tool. We probably want to take another look at this once the tool is ready.

@tagboola tagboola merged commit 81c1a61 into feature-branch/mcp/mobile-testing Oct 9, 2025
23 of 27 checks passed
@github-project-automation github-project-automation bot moved this from Approved [PR] to Done in [Cloud] Extensions + Functions Oct 9, 2025
@tagboola tagboola deleted the tagboola/add-run-test-prompt branch October 9, 2025 20:00
tagboola added a commit that referenced this pull request Nov 7, 2025
* creation of feature branch

* New MCP tool for running mobile tests (via app distribution). (#9250)

* Scaffolding for new appdistribution MCP tool.

* Refactor business logic out of the appdistribution CLI so that it can be used by an MCP tool.

* Wire new appdistribution tool up to the business logic.

* Fix linting errors.

* Update src/appdistribution/distribution.ts

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: Jamie Rothfeder <rothbutter@google.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Rename appdistribution directory to apptesting (#9268)

* Rename appdistribution directory to apptesting

* Make variables consistent with directory rename.

---------

Co-authored-by: Jamie Rothfeder <rothbutter@google.com>

* Use a datastructure to represent test devices rather than a string. (#9280)

* Use a datastructure to represent test devices rather than a string.

* Update src/mcp/tools/apptesting/tests.ts

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update src/mcp/tools/apptesting/tests.ts

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Pretty

---------

Co-authored-by: Jamie Rothfeder <rothbutter@google.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Add run_test prompt (#9292)

* Add initial MCP prompt for running automated tests

* Fix typos

* MCP tool `apptesting_run_test` can create and run a on-off test. (#9321)

* Create a on-off test and execute.

* Can now create a on-off test.

* Update src/appdistribution/client.ts

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update src/mcp/tools/apptesting/tests.ts

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* PR feedback

* Separate test check in to a different tool so that gemini can orchestrate running and checking for completion.

* Set the devices field to optional

---------

Co-authored-by: Jamie Rothfeder <rothbutter@google.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Use the same default device that's used in the Console (#9320)

* Update prompt to support generating a test case when there is no test description (#9322)

* Use the same default device that's used in the Console

* Update prompt to support generating a test case when there is no test description passed

* Add custom auto-enablement for app testing (#9373)

* Add custom auto-enablement for app testing

* Address gemini code assist comments

* Fix intersection bug

* Fix issues with test

* Add get devices tool (#9387)

* Display link to results in the Firebase Console (#9406)

* Place app testing tools behind an experiment

* Address GCA comments

* Explicitly set default devices

* Address PR comments

* Fix the status URL. (#9438)

Co-authored-by: Jamie Rothfeder <rothbutter@google.com>

---------

Co-authored-by: Jamie Rothfeder <rothbutter@google.com>
Co-authored-by: Jamie Rothfeder <jamie.rothfeder@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants