Skip to content

Commit e752032

Browse files
code-yeongyucode-yeongyu
authored
fix(look-at): use direct file passthrough instead of Read tool (#173)
- Embed files directly in message parts using file:// URL format - Remove dependency on Read tool for multimodal-looker agent - Add inferMimeType helper for proper MIME type detection - Disable read tool in agent tools config (no longer needed) - Upgrade multimodal-looker model to gemini-3-flash - Update all README docs to reflect gemini-3-flash change Fixes #126 🤖 GENERATED WITH ASSISTANCE OF [OhMyOpenCode](https://github.com/code-yeongyu/oh-my-opencode)
1 parent 61740e5 commit e752032

File tree

7 files changed

+53
-22
lines changed

7 files changed

+53
-22
lines changed

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ oh-my-opencode/
8787
| explore | opencode/grok-code | Fast codebase exploration, file patterns |
8888
| frontend-ui-ux-engineer | google/gemini-3-pro-preview | UI generation, design-focused |
8989
| document-writer | google/gemini-3-pro-preview | Technical documentation |
90-
| multimodal-looker | google/gemini-2.5-flash | PDF/image/diagram analysis |
90+
| multimodal-looker | google/gemini-3-flash | PDF/image/diagram analysis |
9191

9292
## COMMANDS
9393

‎README.ja.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -317,12 +317,12 @@ opencode auth login
317317
"agents": {
318318
"frontend-ui-ux-engineer": { "model": "google/gemini-3-pro-high" },
319319
"document-writer": { "model": "google/gemini-3-flash" },
320-
"multimodal-looker": { "model": "google/gemini-2.5-flash" }
320+
"multimodal-looker": { "model": "google/gemini-3-flash" }
321321
}
322322
}
323323
```
324324

325-
**利用可能なモデル名**: `google/gemini-3-pro-high`, `google/gemini-3-pro-medium`, `google/gemini-3-pro-low`, `google/gemini-3-flash`, `google/gemini-2.5-flash`, `google/gemini-2.5-flash-lite`, `google/claude-sonnet-4-5`, `google/claude-sonnet-4-5-thinking`, `google/claude-opus-4-5-thinking`, `google/gpt-oss-120b-medium`
325+
**利用可能なモデル名**: `google/gemini-3-pro-high`, `google/gemini-3-pro-medium`, `google/gemini-3-pro-low`, `google/gemini-3-flash`, `google/gemini-3-flash`, `google/gemini-3-flash-lite`, `google/claude-sonnet-4-5`, `google/claude-sonnet-4-5-thinking`, `google/claude-opus-4-5-thinking`, `google/gpt-oss-120b-medium`
326326

327327
その後、認証を行います：
328328

@@ -432,7 +432,7 @@ gh repo star code-yeongyu/oh-my-opencode
432432
- **explore** (`opencode/grok-code`): 高速なコードベース探索、ファイルパターンマッチング。Claude Code は Haiku を使用しますが、私たちは Grok を使います。現在無料であり、極めて高速で、ファイル探索タスクには十分な知能を備えているからです。Claude Code からインスピレーションを得ました。
433433
- **frontend-ui-ux-engineer** (`google/gemini-3-pro-preview`): 開発者に転身したデザイナーという設定です。素晴らしい UI を作ります。美しく独創的な UI コードを生成することに長けた Gemini を使用します。
434434
- **document-writer** (`google/gemini-3-pro-preview`): テクニカルライティングの専門家という設定です。Gemini は文筆家であり、流れるような文章を書きます。
435-
- **multimodal-looker** (`google/gemini-2.5-flash`): 視覚コンテンツ解釈のための専門エージェント。PDF、画像、図表を分析して情報を抽出します。
435+
- **multimodal-looker** (`google/gemini-3-flash`): 視覚コンテンツ解釈のための専門エージェント。PDF、画像、図表を分析して情報を抽出します。
436436

437437
メインエージェントはこれらを自動的に呼び出しますが、明示的に呼び出すことも可能です：
438438

@@ -675,7 +675,7 @@ Oh My OpenCode は以下の場所からフックを読み込んで実行しま
675675
"agents": {
676676
"frontend-ui-ux-engineer": { "model": "google/gemini-3-pro-high" },
677677
"document-writer": { "model": "google/gemini-3-flash" },
678-
"multimodal-looker": { "model": "google/gemini-2.5-flash" }
678+
"multimodal-looker": { "model": "google/gemini-3-flash" }
679679
}
680680
}
681681
```

‎README.ko.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -314,12 +314,12 @@ opencode auth login
314314
"agents": {
315315
"frontend-ui-ux-engineer": { "model": "google/gemini-3-pro-high" },
316316
"document-writer": { "model": "google/gemini-3-flash" },
317-
"multimodal-looker": { "model": "google/gemini-2.5-flash" }
317+
"multimodal-looker": { "model": "google/gemini-3-flash" }
318318
}
319319
}
320320
```
321321

322-
**사용 가능한 모델 이름**: `google/gemini-3-pro-high`, `google/gemini-3-pro-medium`, `google/gemini-3-pro-low`, `google/gemini-3-flash`, `google/gemini-2.5-flash`, `google/gemini-2.5-flash-lite`, `google/claude-sonnet-4-5`, `google/claude-sonnet-4-5-thinking`, `google/claude-opus-4-5-thinking`, `google/gpt-oss-120b-medium`
322+
**사용 가능한 모델 이름**: `google/gemini-3-pro-high`, `google/gemini-3-pro-medium`, `google/gemini-3-pro-low`, `google/gemini-3-flash`, `google/gemini-3-flash`, `google/gemini-3-flash-lite`, `google/claude-sonnet-4-5`, `google/claude-sonnet-4-5-thinking`, `google/claude-opus-4-5-thinking`, `google/gpt-oss-120b-medium`
323323

324324
그 후 인증:
325325

@@ -429,7 +429,7 @@ gh repo star code-yeongyu/oh-my-opencode
429429
- **explore** (`opencode/grok-code`): 빠른 코드베이스 탐색, 파일 패턴 매칭. Claude Code는 Haiku를 쓰지만, 우리는 Grok을 씁니다. 현재 무료이고, 극도로 빠르며, 파일 탐색 작업에 충분한 지능을 갖췄기 때문입니다. Claude Code 에서 영감을 받았습니다.
430430
- **frontend-ui-ux-engineer** (`google/gemini-3-pro-preview`): 개발자로 전향한 디자이너라는 설정을 갖고 있습니다. 멋진 UI를 만듭니다. 아름답고 창의적인 UI 코드를 생성하는 데 탁월한 Gemini를 사용합니다.
431431
- **document-writer** (`google/gemini-3-pro-preview`): 기술 문서 전문가라는 설정을 갖고 있습니다. Gemini 는 문학가입니다. 글을 기가막히게 씁니다.
432-
- **multimodal-looker** (`google/gemini-2.5-flash`): 시각적 콘텐츠 해석을 위한 전문 에이전트. PDF, 이미지, 다이어그램을 분석하여 정보를 추출합니다.
432+
- **multimodal-looker** (`google/gemini-3-flash`): 시각적 콘텐츠 해석을 위한 전문 에이전트. PDF, 이미지, 다이어그램을 분석하여 정보를 추출합니다.
433433

434434
각 에이전트는 메인 에이전트가 알아서 호출하지만, 명시적으로 요청할 수도 있습니다:
435435

@@ -669,7 +669,7 @@ Schema 자동 완성이 지원됩니다:
669669
"agents": {
670670
"frontend-ui-ux-engineer": { "model": "google/gemini-3-pro-high" },
671671
"document-writer": { "model": "google/gemini-3-flash" },
672-
"multimodal-looker": { "model": "google/gemini-2.5-flash" }
672+
"multimodal-looker": { "model": "google/gemini-3-flash" }
673673
}
674674
}
675675
```

‎README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -346,12 +346,12 @@ The `opencode-antigravity-auth` plugin uses different model names than the built
346346
"agents": {
347347
"frontend-ui-ux-engineer": { "model": "google/gemini-3-pro-high" },
348348
"document-writer": { "model": "google/gemini-3-flash" },
349-
"multimodal-looker": { "model": "google/gemini-2.5-flash" }
349+
"multimodal-looker": { "model": "google/gemini-3-flash" }
350350
}
351351
}
352352
```
353353

354-
**Available model names**: `google/gemini-3-pro-high`, `google/gemini-3-pro-medium`, `google/gemini-3-pro-low`, `google/gemini-3-flash`, `google/gemini-2.5-flash`, `google/gemini-2.5-flash-lite`, `google/claude-sonnet-4-5`, `google/claude-sonnet-4-5-thinking`, `google/claude-opus-4-5-thinking`, `google/gpt-oss-120b-medium`
354+
**Available model names**: `google/gemini-3-pro-high`, `google/gemini-3-pro-medium`, `google/gemini-3-pro-low`, `google/gemini-3-flash`, `google/gemini-3-flash`, `google/gemini-3-flash-lite`, `google/claude-sonnet-4-5`, `google/claude-sonnet-4-5-thinking`, `google/claude-opus-4-5-thinking`, `google/gpt-oss-120b-medium`
355355

356356
Then authenticate:
357357

@@ -493,7 +493,7 @@ To remove oh-my-opencode:
493493
- **explore** (`opencode/grok-code`): Fast codebase exploration and pattern matching. Claude Code uses Haiku; we use Grok—it's free, blazing fast, and plenty smart for file traversal. Inspired by Claude Code.
494494
- **frontend-ui-ux-engineer** (`google/gemini-3-pro-preview`): A designer turned developer. Builds gorgeous UIs. Gemini excels at creative, beautiful UI code.
495495
- **document-writer** (`google/gemini-3-pro-preview`): Technical writing expert. Gemini is a wordsmith—writes prose that flows.
496-
- **multimodal-looker** (`google/gemini-2.5-flash`): Visual content specialist. Analyzes PDFs, images, diagrams to extract information.
496+
- **multimodal-looker** (`google/gemini-3-flash`): Visual content specialist. Analyzes PDFs, images, diagrams to extract information.
497497

498498
The main agent invokes these automatically, but you can call them explicitly:
499499

@@ -733,7 +733,7 @@ When using `opencode-antigravity-auth`, disable the built-in auth and override a
733733
"agents": {
734734
"frontend-ui-ux-engineer": { "model": "google/gemini-3-pro-high" },
735735
"document-writer": { "model": "google/gemini-3-flash" },
736-
"multimodal-looker": { "model": "google/gemini-2.5-flash" }
736+
"multimodal-looker": { "model": "google/gemini-3-flash" }
737737
}
738738
}
739739
```

‎README.zh-cn.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -325,12 +325,12 @@ opencode auth login
325325
"agents": {
326326
"frontend-ui-ux-engineer": { "model": "google/gemini-3-pro-high" },
327327
"document-writer": { "model": "google/gemini-3-flash" },
328-
"multimodal-looker": { "model": "google/gemini-2.5-flash" }
328+
"multimodal-looker": { "model": "google/gemini-3-flash" }
329329
}
330330
}
331331
```
332332

333-
**可用模型名**：`google/gemini-3-pro-high`, `google/gemini-3-pro-medium`, `google/gemini-3-pro-low`, `google/gemini-3-flash`, `google/gemini-2.5-flash`, `google/gemini-2.5-flash-lite`, `google/claude-sonnet-4-5`, `google/claude-sonnet-4-5-thinking`, `google/claude-opus-4-5-thinking`, `google/gpt-oss-120b-medium`
333+
**可用模型名**：`google/gemini-3-pro-high`, `google/gemini-3-pro-medium`, `google/gemini-3-pro-low`, `google/gemini-3-flash`, `google/gemini-3-flash`, `google/gemini-3-flash-lite`, `google/claude-sonnet-4-5`, `google/claude-sonnet-4-5-thinking`, `google/claude-opus-4-5-thinking`, `google/gpt-oss-120b-medium`
334334

335335
然后认证：
336336

@@ -440,7 +440,7 @@ gh repo star code-yeongyu/oh-my-opencode
440440
- **explore** (`opencode/grok-code`)：极速代码库扫描、模式匹配。Claude Code 用 Haiku，我们用 Grok——免费、飞快、扫文件够用了。致敬 Claude Code。
441441
- **frontend-ui-ux-engineer** (`google/gemini-3-pro-preview`)：设计师出身的程序员。UI 做得那是真漂亮。Gemini 写这种创意美观的代码是一绝。
442442
- **document-writer** (`google/gemini-3-pro-preview`)：技术写作专家。Gemini 文笔好，写出来的东西读着顺畅。
443-
- **multimodal-looker** (`google/gemini-2.5-flash`)：视觉内容专家。PDF、图片、图表，看一眼就知道里头有啥。
443+
- **multimodal-looker** (`google/gemini-3-flash`)：视觉内容专家。PDF、图片、图表，看一眼就知道里头有啥。
444444

445445
主 Agent 会自动调遣它们，你也可以亲自点名：
446446

@@ -675,7 +675,7 @@ Agent 爽了，你自然也爽。但我还想直接让你爽。
675675
"agents": {
676676
"frontend-ui-ux-engineer": { "model": "google/gemini-3-pro-high" },
677677
"document-writer": { "model": "google/gemini-3-flash" },
678-
"multimodal-looker": { "model": "google/gemini-2.5-flash" }
678+
"multimodal-looker": { "model": "google/gemini-3-flash" }
679679
}
680680
}
681681
```

‎src/agents/multimodal-looker.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ export const multimodalLookerAgent: AgentConfig = {
44
description:
55
"Analyze media files (PDFs, images, diagrams) that require interpretation beyond raw text. Extracts specific information or summaries from documents, describes visual content. Use when you need analyzed/extracted data rather than literal file contents.",
66
mode: "subagent",
7-
model: "google/gemini-2.5-flash",
7+
model: "google/gemini-3-flash",
88
temperature: 0.1,
99
tools: { write: false, edit: false, bash: false, background_task: false },
1010
prompt: `You interpret media files that cannot be read as plain text.

‎src/tools/look-at/tools.ts‎

Lines changed: 35 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,33 @@
1+
import { extname, basename } from "node:path"
12
import { tool, type PluginInput } from "@opencode-ai/plugin"
23
import { LOOK_AT_DESCRIPTION, MULTIMODAL_LOOKER_AGENT } from "./constants"
34
import type { LookAtArgs } from "./types"
45
import { log } from "../../shared/logger"
56

7+
function inferMimeType(filePath: string): string {
8+
const ext = extname(filePath).toLowerCase()
9+
const mimeTypes: Record<string, string> = {
10+
".jpg": "image/jpeg",
11+
".jpeg": "image/jpeg",
12+
".png": "image/png",
13+
".gif": "image/gif",
14+
".webp": "image/webp",
15+
".svg": "image/svg+xml",
16+
".bmp": "image/bmp",
17+
".ico": "image/x-icon",
18+
".pdf": "application/pdf",
19+
".txt": "text/plain",
20+
".md": "text/markdown",
21+
".json": "application/json",
22+
".xml": "application/xml",
23+
".html": "text/html",
24+
".css": "text/css",
25+
".js": "text/javascript",
26+
".ts": "text/typescript",
27+
}
28+
return mimeTypes[ext] || "application/octet-stream"
29+
}
30+
631
export function createLookAt(ctx: PluginInput) {
732
return tool({
833
description: LOOK_AT_DESCRIPTION,
@@ -13,12 +38,14 @@ export function createLookAt(ctx: PluginInput) {
1338
async execute(args: LookAtArgs, toolContext) {
1439
log(`[look_at] Analyzing file: ${args.file_path}, goal: ${args.goal}`)
1540

41+
const mimeType = inferMimeType(args.file_path)
42+
const filename = basename(args.file_path)
43+
1644
const prompt = `Analyze this file and extract the requested information.
1745
18-
File path: ${args.file_path}
1946
Goal: ${args.goal}
2047
21-
Read the file using the Read tool, then provide ONLY the extracted information that matches the goal.
48+
Provide ONLY the extracted information that matches the goal.
2249
Be thorough on what was requested, concise on everything else.
2350
If the requested information is not found, clearly state what is missing.`
2451

@@ -38,7 +65,7 @@ If the requested information is not found, clearly state what is missing.`
3865
const sessionID = createResult.data.id
3966
log(`[look_at] Created session: ${sessionID}`)
4067

41-
log(`[look_at] Sending prompt to session ${sessionID}`)
68+
log(`[look_at] Sending prompt with file passthrough to session ${sessionID}`)
4269
await ctx.client.session.prompt({
4370
path: { id: sessionID },
4471
body: {
@@ -47,8 +74,12 @@ If the requested information is not found, clearly state what is missing.`
4774
task: false,
4875
call_omo_agent: false,
4976
look_at: false,
77+
read: false,
5078
},
51-
parts: [{ type: "text", text: prompt }],
79+
parts: [
80+
{ type: "text", text: prompt },
81+
{ type: "file", mime: mimeType, url: `file://${args.file_path}`, filename },
82+
],
5283
},
5384
})
5485

0 commit comments

Comments
 (0)