feat(server): add Vertex AI proxy server and improve product search #141
Conversation
- Add WebSocket proxy server for Vertex AI Gemini Live API - Support service account authentication - Handle bidirectional audio/video streaming - Automatic tool injection and execution - Improve search_products function - Multi-keyword search with AND logic - Search across name, description, category, keywords, features - Use real product data (12 products) instead of mock data - Add dual connection mode support - Direct API key mode (Google AI Studio) - Proxy mode (Vertex AI via backend server) - Security improvements - Update .gitignore to exclude sensitive files - Add .env.example template - Remove hardcoded credentials - Add comprehensive comments and documentation - Tool declaration comments - Function JSDoc comments - Configuration validation helpers
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Summary of ChangesHello @shendongming, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a robust backend proxy server for the Gemini Live API, facilitating secure and flexible integration with Vertex AI. It also refines the product search capabilities within the application, making it more powerful and data-driven. These changes collectively enhance the application's architecture, security posture, and user experience by providing a more versatile and capable AI interaction layer. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a significant feature: a WebSocket proxy server for Vertex AI, enabling more secure and robust connections. It also enhances the product search functionality and adds support for dual connection modes (direct API key and proxy). The code is well-structured, particularly with the new server-side application and the client-side abstractions. I've identified a few areas for improvement, including a logic bug in tool handling, a security enhancement for the CORS policy, and a suggestion to improve maintainability by extracting a large hardcoded string.
| function callCx360Tool(email: string) { | ||
| const info = `The "Pristine Explorer" | ||
| Name: Miles Chen | ||
| Age: 30 | ||
| Gender: Male | ||
| Occupation: UX Designer | ||
| Location: Seattle, WA (Urban resident with easy access to nature) | ||
| Income Level: Middle Income (Budget-conscious but willing to pay for durability) | ||
| Bio & Personality | ||
| Miles is a tech-savvy professional who spends his weekdays in a clean, organized office and his weekends chasing the perfect sunset shot. While he loves the experience of the outdoors—the fresh air, the views, and the exercise—he dislikes the mess associated with it. You will never catch him rolling in the mud or sleeping on the bare ground. He is meticulous, detail-oriented, and values aesthetics just as much as functionality. | ||
| Interests & Hobbies | ||
| Landscape Photography: He carries expensive camera gear and is terrified of getting dust or grit inside his lenses. | ||
| Light Hiking & Trekking: Prefers well-maintained trails over bushwhacking. | ||
| Urban Cycling: Commutes to work occasionally but hates arriving sweaty or splashed with road grime. | ||
| Tech & Gadgets: Loves integrating technology into his outdoor activities (drones, GPS watches). | ||
| Financial Outlook (Moderate Economic Basis) | ||
| Miles is financially stable but not wealthy. He researches purchases extensively before buying. He cannot afford to replace gear constantly, so he looks for "investment pieces"—mid-to-high-range products that promise longevity and versatility. He is susceptible to value bundles or financing options (Buy Now, Pay Later) for more expensive items like high-end jackets or tents. | ||
| The "Clean Freak" Constraint (Dislikes Dirty/Messy Things) | ||
| This is Miles' defining consumer trait. | ||
| The Problem: He loves nature but hates the "grime factor" (mud, bugs, sweat, chaotic packing). | ||
| The Need: He looks for gear that is stain-resistant, waterproof, easy to wipe down, and anti-microbial. | ||
| Organization: He despises a messy backpack. He loves packing cubes, compartmentalized bags, and gear with dedicated pockets for his camera equipment. | ||
| Shopping Preferences & Triggers | ||
| Style: Minimalist, sleek, "Gorpcore" aesthetic (functional but stylish enough for the city). Avoids overly loud colors or rugged "survivalist" looks. | ||
| Keywords that attract him: "Easy-clean," "Water-repellent," "Organized," "Odor-control," "Matte finish." | ||
| Deal Breakers: Materials that attract lint/pet hair, light-colored shoes that stain instantly, or complicated gear that is hard to wash. | ||
| How Nova (The AI) Should Approach Him: | ||
| Tone: Precise, polished, and focusing on specs regarding materials and maintenance. | ||
| Strategy: Highlight products with Nano-tech coatings or stain resistance. When suggesting hiking boots, suggest the ones that are easy to rinse off, not the heavy leather ones that hold mud. Suggest accessories like camera inserts or waterproof dry bags to keep his gear spotless.` | ||
|
|
||
| return { | ||
| info | ||
| } | ||
| } |
There was a problem hiding this comment.
The callCx360Tool function contains a large, multi-line string with customer persona data. Hardcoding this content directly in the source code can make it difficult to update and manage.
For better maintainability, consider moving this text to a separate file (e.g., a .txt or .json file) and loading it at runtime. This separates data from logic, making both easier to handle.
| await fastify.register(fastifyCors, { | ||
| origin: true, // Allow all origins in development | ||
| credentials: true, | ||
| }); |
There was a problem hiding this comment.
The CORS configuration is set to origin: true, which reflects the Origin header of the request. While this is convenient for development, it's too permissive for a production environment as it allows requests from any origin.
For improved security, you should restrict the allowed origins to a specific list of domains that are expected to access this server in production. You can use an environment variable to distinguish between development and production environments.
| await fastify.register(fastifyCors, { | |
| origin: true, // Allow all origins in development | |
| credentials: true, | |
| }); | |
| await fastify.register(fastifyCors, { | |
| origin: process.env.NODE_ENV === 'production' | |
| ? ['https://your-production-domain.com'] // Replace with your actual domain | |
| : true, | |
| credentials: true, | |
| }); |
2 participants
Add WebSocket proxy server for Vertex AI Gemini Live API
Improve search_products function
Add dual connection mode support
Security improvements
Add comprehensive comments and documentation