Desktop: use dev Firebase config for dev builds#5537
Open
beastoin wants to merge 40 commits intocollab/5396-integrationfrom
Open
Desktop: use dev Firebase config for dev builds#5537beastoin wants to merge 40 commits intocollab/5396-integrationfrom
beastoin wants to merge 40 commits intocollab/5396-integrationfrom
Conversation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…5396) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
WebSocket client that connects to /v4/listen with Bearer auth and sends screen_frame JSON messages. Routes focus_result responses back to callers via async continuations with frame_id correlation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
#5396) Replace direct Gemini API calls with backend WebSocket screen_frame messages. Context building (goals, tasks, memories, AI profile) moves server-side. Client becomes thin: encode JPEG→base64, send screen_frame, receive focus_result. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…#5396) Start WS connection when monitoring starts, disconnect on stop. Pass service to FocusAssistant (shared for future assistant types). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…5396) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Vision handlers: analyzeFocus, extractTasks, extractMemories, generateAdvice (send screen_frame with analyze type, receive typed result via frame_id) Text handlers: generateLiveNote, requestProfile, rerankTasks, deduplicateTasks (send typed JSON message, receive result via single-slot continuation) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace GeminiClient tool-calling loop with backendService.extractTasks(). Remove extractTaskSingleStage, refreshContext, vector/keyword search, validateTaskTitle — all LLM logic now server-side. -550 lines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace GeminiClient.sendRequest with backendService.extractMemories(). Remove prompt/schema building — all LLM logic now server-side. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace 2-phase Gemini tool-calling loop (execute_sql + vision) with backendService.generateAdvice(). Remove compressForGemini, getUserLanguage, buildActivitySummary, buildPhase1/2Tools — all LLM logic server-side. -560 lines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace GeminiClient with backendService.deduplicateTasks(). Remove prompt/schema building, local dedup logic — server handles everything. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace GeminiClient with backendService.rerankTasks(). Remove prompt/ schema building, context fetching — server handles reranking. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace 2-stage Gemini profile generation with backendService.requestProfile(). Remove fetchDataSources, buildPrompt, buildConsolidationPrompt — server fetches user data from Firestore and generates profile server-side. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ts (#5396) Pass shared BackendProactiveService to all 4 assistants and 3 text-only services. Remove do/catch since inits no longer throw. Update AdviceTestRunnerWindow fallback creation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace direct GeminiClient usage with BackendProactiveService. Uses configure(backendService:) singleton pattern matching other text-based services. Prompt logic moves server-side. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add configure(backendService:) call for LiveNotesMonitor alongside other singleton text-based services. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update GoogleService-Info-Dev.plist with dev Firebase values: API_KEY, PROJECT_ID, STORAGE_BUCKET, GCM_SENDER_ID, GOOGLE_APP_ID. Fixes #5536 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Dev builds load GoogleService-Info-Dev.plist (via run.sh), prod builds load GoogleService-Info.plist. AuthService now reads API_KEY from whichever plist is in the bundle, with prod key as fallback. Fixes #5536 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Greptile SummaryThis PR bundles two distinct sets of changes: the headline fix (swapping Key changes:
Confidence Score: 3/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant Desktop as Desktop App (Swift)
participant WS as WebSocket /v4/listen
participant Transcribe as transcribe.py
participant LLM as Gemini Flash (LLM)
Note over Desktop,WS: Auth flow uses key from active GoogleService-Info.plist
Desktop->>WS: screen_frame {frame_id, image_b64, analyze: ["focus","tasks","memories","advice"]}
WS->>Transcribe: message handler
par Focus analysis
Transcribe->>LLM: analyze_focus(uid, image_b64, app_name)
LLM-->>Transcribe: FocusResult
Transcribe-->>Desktop: focus_result {frame_id, status, app_or_site, description, message}
and Task extraction
Transcribe->>LLM: extract_tasks(uid, image_b64, app_name)
LLM-->>Transcribe: TaskExtractionResult
Transcribe-->>Desktop: tasks_extracted {frame_id, tasks[]}
and Memory extraction
Transcribe->>LLM: extract_memories(uid, image_b64, app_name)
LLM-->>Transcribe: MemoryExtractionResult
Transcribe-->>Desktop: memories_extracted {frame_id, memories[]}
and Advice generation
Transcribe->>LLM: generate_advice(uid, image_b64, app_name)
LLM-->>Transcribe: AdviceResult
Transcribe-->>Desktop: advice_extracted {frame_id, advice}
end
Desktop->>WS: live_notes_text {text, session_context}
Transcribe->>LLM: generate_live_note(text, session_context)
LLM-->>Transcribe: LiveNote
Transcribe-->>Desktop: live_note {text}
Desktop->>WS: task_rerank
Transcribe->>LLM: rerank_tasks(uid)
LLM-->>Transcribe: RerankResult
Transcribe-->>Desktop: rerank_complete {updated_tasks[]}
Desktop->>WS: task_dedup
Transcribe->>LLM: dedup_tasks(uid)
LLM-->>Transcribe: DedupResult
Transcribe-->>Desktop: dedup_complete {deleted_ids[], reason}
Last reviewed commit: 7796471 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| ], | ||
| required: ["status", "app_or_site", "description"] | ||
| private func analyzeScreenshot(jpegData: Data, appName: String, windowTitle: String?) async throws -> ScreenAnalysis? { | ||
| let base64 = jpegData.base64EncodedString() |
Contributor
There was a problem hiding this comment.
Missing autoreleasepool for screenshot base64 encoding
base64EncodedString() on large JPEG data creates a temporary String (33% larger than the input) that won't be freed until the next autorelease pool drain. The parallel implementations in MemoryAssistant.extractMemories and AdviceAssistant.extractAdvice both wrap this call in autoreleasepool { } — this call should match for consistency and to avoid holding onto a large temporary string longer than necessary.
Suggested change
| let base64 = jpegData.base64EncodedString() | |
| let base64 = autoreleasepool { jpegData.base64EncodedString() } |
Rule Used: Memory management - free large objects immediately... (source)
Comment on lines
+240
to
+246
| Task { [weak self] in | ||
| try? await Task.sleep(nanoseconds: UInt64(timeout * 1_000_000_000)) | ||
| self?.requestLock.lock() | ||
| let cont = remove(frameId) | ||
| self?.requestLock.unlock() | ||
| cont?.resume(throwing: ServiceError.timeout) | ||
| } |
Contributor
There was a problem hiding this comment.
Timeout tasks are never cancelled, accumulating over time
Every call to sendAndTimeout (and sendAndTimeoutSingle) spawns a Task that unconditionally sleeps for the full timeout duration (30 s for vision, 60 s for text) even after the continuation has already been resumed by a successful response. With screen frames arriving every 10–30 s, there will be a steady accumulation of sleeping tasks.
The fix is to return the Task handle and cancel it when the continuation is resolved. For example:
// In sendAndTimeout, return the task handle
let timeoutTask = Task { [weak self] in
try? await Task.sleep(nanoseconds: UInt64(timeout * 1_000_000_000))
guard !Task.isCancelled else { return }
self?.requestLock.lock()
let cont = remove(frameId)
self?.requestLock.unlock()
cont?.resume(throwing: ServiceError.timeout)
}
// Then cancel timeoutTask when the response is received in handleFocusResult, etc.The same pattern applies to sendAndTimeoutSingle at lines 261–267.
| // Fall back to transcription language setting | ||
| let fallback = await MainActor.run { AssistantSettings.shared.transcriptionLanguage } | ||
| return fallback.isEmpty || fallback == "en" ? nil : fallback | ||
| let hasAdvice = adviceDict["has_advice"] as? Bool ?? !adviceDict.isEmpty |
Contributor
There was a problem hiding this comment.
"has_advice" key lookup is always nil — dead code
The Python backend sends the nested advice content dict in the advice_extracted event:
# _handle_advice in transcribe.py
_send_message_event(AdviceExtractedEvent(
frame_id=fid, advice=result.get('advice'), # ← this is {"content":…, "category":…, "confidence":…}
))So adviceDict has keys "content", "category", "confidence" — never "has_advice". The lookup adviceDict["has_advice"] as? Bool is therefore always nil, and the actual logic is always the fallback !adviceDict.isEmpty.
The same dead check appears in testAnalyze at line 416.
Consider replacing with:
Suggested change
| let hasAdvice = adviceDict["has_advice"] as? Bool ?? !adviceDict.isEmpty | |
| let hasAdvice = !adviceDict.isEmpty |
dev.sh builds Omi Dev (com.omi.desktop-dev) but was copying the prod GoogleService-Info.plist. Now uses the same dev plist logic as run.sh. Fixes #5536 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
reset-and-run.sh builds Omi Dev (com.omi.desktop-dev) but was copying the prod GoogleService-Info.plist. Now uses the same dev plist logic as run.sh. Fixes #5536 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CODEx review: dev builds should not silently use prod credentials. Now logs a FATAL warning if GoogleService-Info.plist is missing or has no API_KEY in a dev build (bundle ID ending in -dev). Fixes #5536 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ck to prod CODEx review round 2: logging is not fail-fast. Dev builds now crash with fatalError if GoogleService-Info.plist has no API_KEY, preventing silent use of prod credentials. Prod builds still fall back safely. Fixes #5536 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Collaborator
Author
Independent Verification — PR #5537Verifier: noa (independent) ScopeDev Firebase config: remove hardcoded prod Firebase from dev builds. AuthService reads API_KEY from bundle plist at runtime with prod fallback. Results
New Commits Not Covered4 commits after 7796471:
The crash-on-missing-plist commit (6d8b57e) changes the fallback behavior from what I verified. Re-verification recommended for the new HEAD. Verdict: CONDITIONAL PASS (for SHA 7796471)Verified code works correctly. New commits need re-verification — especially the crash-instead-of-fallback change. |
Collaborator
Author
Re-verification — PR #5537 (updated HEAD)Verifier: noa (independent) New Commits Reviewed
Analysis
Results
Verdict: PASS (upgraded from CONDITIONAL PASS)All 4 new commits reviewed and verified. The crash-on-missing-plist is correctly scoped to dev builds only. Combined with previous E2E results, this PR is clear to merge. |
This was referenced Mar 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GoogleService-Info-Dev.plistfrom prod (based-hardware) to dev (based-hardware-dev) Firebase projectAPI_KEYfrom the active plist at runtime instead of hardcoding the prod key inAuthService.swiftdev.shandreset-and-run.shto also copy the dev plist (they were copying prod)Changes
GoogleService-Info-Dev.plistAuthService.swiftAPI_KEYfrom bundle GoogleService-Info.plist at runtime (fallback: prod key)dev.shrun.shlogic)reset-and-run.shrun.shlogic)How it works
run.sh,dev.sh,reset-and-run.sh(dev builds) copy GoogleService-Info-Dev.plist as GoogleService-Info.plist into the app bundlebuild.sh(prod builds) copies GoogleService-Info.plist (prod) into the bundleCODEx review findings addressed
Test plan
Fixes #5536
by AI for @beastoin