Remove Vertex LLM Inference Integration Test#36172
Remove Vertex LLM Inference Integration Test#36172jrmccluskey merged 9 commits intoapache:masterfrom
Conversation
|
Run Python PostCommit |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @jrmccluskey, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request addresses a critical failure in the Python PostCommit tests for version 3.10 by updating the Vertex AI LLM inference integration test. The change ensures the test utilizes a currently supported LLM model, specifically Gemini Flash 2.5, thereby restoring the stability and reliability of the continuous integration pipeline for machine learning inference components.
Highlights
- LLM Endpoint Update: The
_LLM_ENDPOINT_IDin the Vertex AI inference integration test has been updated to point to a new endpoint. - PostCommit Fix: This change resolves Python PostCommit failures for version 3.10, which were caused by an unsupported old LLM model deployed on GCP.
- Model Migration: The test now routes to a new endpoint that serves the Gemini Flash 2.5 model.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
|
Run Python PostCommit 3.10 |
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
jrmccluskey
changed the title
|
@gemini-code-assist review |
There was a problem hiding this comment.
Code Review
This pull request removes a failing integration test for Vertex AI LLM inference due to a deprecated model, and adds a note to the corresponding example to guide users toward the new recommended approach using Gemini models. The changes are logical and address the issue described. I've provided a couple of suggestions to remove some resulting dead code and to improve the clarity of the new documentation note.
sdks/python/apache_beam/examples/inference/vertex_ai_llm_text_classification.py
Outdated
Show resolved
Hide resolved
sdks/python/apache_beam/ml/inference/vertex_ai_inference_it_test.py
Outdated
Show resolved
Hide resolved
…classification.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
assign set of reviewers |
|
Assigning reviewers: R: @damccorm for label python. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
sdks/python/apache_beam/examples/inference/vertex_ai_llm_text_classification.py
Outdated
Show resolved
Hide resolved
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #36172 +/- ##
=========================================
Coverage 54.90% 54.90%
Complexity 1618 1618
=========================================
Files 1057 1057
Lines 164311 164355 +44
Branches 1165 1165
=========================================
+ Hits 90214 90246 +32
- Misses 71944 71956 +12
Partials 2153 2153
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
failing precommit is unrelated, merging |
2 participants
The Python PostCommit run for 3.10 was failing because the old LLM model deployed on GCP was no longer supported. The suggested migration path is to use a Gemini model; however, Gemini models cannot be accessed through the Vertex AI Predict API, so this use case is no longer supported through the VertexAIModelHandlerJSON class. This PR removes the integration test that was failing and adds a call-out to the corresponding code example routing users to the GeminiModelHandler and its corresponding example.
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.