Add run_test prompt#9292
Conversation
Summary of ChangesHello @tagboola, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant new capability for Firebase App Testing by adding an AI-guided prompt. This prompt, named "run_test", assists developers in setting up and executing automated tests for their Android mobile applications, providing detailed prerequisites, test case generation guidelines, and instructions for running the tests. The feature is currently enabled via the "mcpalpha" experiment flag, signifying its early-stage development. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces a new run_test prompt for the Firebase App Testing agent, enabled via the mcpalpha experiment flag. The changes include adding the new prompt, its definition with detailed instructions for an AI model, and wiring it into the existing prompt system. My review focuses on the new prompt definition in src/mcp/prompts/apptesting/run_test.ts. I've pointed out several typos in the prompt text and a potential functional issue with an incorrect tool name, which could prevent the feature from working as expected. The changes are otherwise well-structured.
|
This looks good. One thing I was surprised to see is that Gemini was able to figure a lot out by simply examining the schema of the parameters in the tool. We probably want to take another look at this once the tool is ready. |
81c1a61
into
feature-branch/mcp/mobile-testing
* creation of feature branch * New MCP tool for running mobile tests (via app distribution). (#9250) * Scaffolding for new appdistribution MCP tool. * Refactor business logic out of the appdistribution CLI so that it can be used by an MCP tool. * Wire new appdistribution tool up to the business logic. * Fix linting errors. * Update src/appdistribution/distribution.ts Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: Jamie Rothfeder <rothbutter@google.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Rename appdistribution directory to apptesting (#9268) * Rename appdistribution directory to apptesting * Make variables consistent with directory rename. --------- Co-authored-by: Jamie Rothfeder <rothbutter@google.com> * Use a datastructure to represent test devices rather than a string. (#9280) * Use a datastructure to represent test devices rather than a string. * Update src/mcp/tools/apptesting/tests.ts Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update src/mcp/tools/apptesting/tests.ts Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Pretty --------- Co-authored-by: Jamie Rothfeder <rothbutter@google.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Add run_test prompt (#9292) * Add initial MCP prompt for running automated tests * Fix typos * MCP tool `apptesting_run_test` can create and run a on-off test. (#9321) * Create a on-off test and execute. * Can now create a on-off test. * Update src/appdistribution/client.ts Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update src/mcp/tools/apptesting/tests.ts Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * PR feedback * Separate test check in to a different tool so that gemini can orchestrate running and checking for completion. * Set the devices field to optional --------- Co-authored-by: Jamie Rothfeder <rothbutter@google.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Use the same default device that's used in the Console (#9320) * Update prompt to support generating a test case when there is no test description (#9322) * Use the same default device that's used in the Console * Update prompt to support generating a test case when there is no test description passed * Add custom auto-enablement for app testing (#9373) * Add custom auto-enablement for app testing * Address gemini code assist comments * Fix intersection bug * Fix issues with test * Add get devices tool (#9387) * Display link to results in the Firebase Console (#9406) * Place app testing tools behind an experiment * Address GCA comments * Explicitly set default devices * Address PR comments * Fix the status URL. (#9438) Co-authored-by: Jamie Rothfeder <rothbutter@google.com> --------- Co-authored-by: Jamie Rothfeder <rothbutter@google.com> Co-authored-by: Jamie Rothfeder <jamie.rothfeder@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Description
Scenarios Tested
Sample Commands