feat(api): forward llm/stt requests to apps/ai service #2452
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add middleware to forward /chat/completions, /listen, and /transcribe requests from apps/api to apps/ai when AI_SERVICE_URL is configured. This provides backward compatibility for clients that haven't updated yet. Co-Authored-By: yujonglee <[email protected]>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
✅ Deploy Preview for hyprnote ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the Comment |
✅ Deploy Preview for hyprnote-storybook ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
When AI_SERVICE_URL is configured, forward WebSocket connections to the AI service's /stt/listen endpoint with auth header passthrough. Co-Authored-By: yujonglee <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Adds middleware to forward
/chat/completions,/listen, and/transcriberequests fromapps/apitoapps/aiwhenAI_SERVICE_URLenvironment variable is configured. This provides backward compatibility for clients that haven't updated to callapps/aidirectly.Path mappings:
/chat/completions→AI_SERVICE_URL/llm/chat/completions/listen→AI_SERVICE_URL/stt/listen/transcribe→AI_SERVICE_URL/stt/When
AI_SERVICE_URLis not set, requests continue to the original handlers (no change in behavior).Implementation details:
/chat/completions,/transcribe) use fetch-based forwarding with streaming support/listen) uses the existingWsProxyConnectionclass for proper bidirectional message relayReview & Testing Checklist for Human
/llm/chat/completions,/stt/listen,/stt/) match whatapps/aiexpects. This is the highest risk area./listenforwarding - Verify real-time STT streaming works when forwarded through the proxy.AI_SERVICE_URLset - Deploy to staging withAI_SERVICE_URL=https://hyprnote-ai.fly.devand verify all three endpoints work correctly.AI_SERVICE_URL- Verify the original behavior is preserved when the env var is not set.Recommended test plan:
AI_SERVICE_URLin staging environment/chat/completionswith a simple LLM request (both streaming and non-streaming)/listenWebSocket connection for real-time STT/transcribewith a batch audio fileNotes