Update Supabase prompt to use execute_sql tool for database operations#2829
Update Supabase prompt to use execute_sql tool for database operations#2829wwwillchen wants to merge 1 commit intodyad-sh:mainfrom
Conversation
- Change 'profiles table migration' to 'profiles table using the execute_sql tool' - Replace SQL code example with emphasis on using execute_sql tool - Add warning against manually creating/modifying migration files in supabase/migrations/ - Clarify that database schema setup should use the execute SQL tool
|
@BugBot run |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request refines the guidance provided within the Supabase prompt, ensuring that database interactions are performed exclusively through the Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| \`\`\` | ||
| You will need to setup the database schema. | ||
| You will need to setup the database schema using the execute SQL tool. |
There was a problem hiding this comment.
Inconsistent tool name in LLM system prompt
Medium Severity
Lines 106 and 108 refer to the tool as "execute SQL tool" (informal, no backticks), while line 30 correctly uses `execute_sql`. The actual tool name registered in the system is execute_sql (with underscore). Since this is an LLM system prompt, inconsistent naming risks the model not recognizing or correctly invoking the execute_sql tool when following instructions from the Database section.
Additional Locations (1)
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
The pull request successfully updates the Supabase prompt to emphasize the use of the execute_sql tool for all database operations. It clearly instructs users not to manually create or modify migration files within supabase/migrations/, reinforcing that these are automatically generated. These changes significantly improve the guidance for interacting with the Supabase database, promoting correct tool usage and preventing potential issues from manual migration file management.
Greptile SummaryThis PR updates the Supabase prompt documentation to clarify the proper workflow for database operations. The changes emphasize using the
Confidence Score: 5/5
Important Files Changed
Last reviewed commit: 4dc8321 |
🔍 Dyadbot Code Review SummaryVerdict: ⛔ NO - Do NOT merge Reviewed by 3 independent agents: Correctness Expert, Code Health Expert, UX Wizard. Issues Summary
🟢 Low Priority Notes (1 item)
🚫 Dropped False Positives (1 item)
Generated by Dyadbot multi-agent code review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4dc832161d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ## Database | ||
| If the user wants to use the database, use SQL queries like this: | ||
| **IMPORTANT: Always use the execute SQL tool to run SQL queries against the Supabase database. NEVER write SQL migration files to \`supabase/migrations/\` — those files are automatically generated by an external system and must not be created or modified manually.** |
There was a problem hiding this comment.
execute_sql tool name in prompt
The updated database guidance tells the model to use an "execute SQL" tool, but the actual local-agent tool is registered as execute_sql (src/pro/main/ipc/handlers/local_agent/tools/execute_sql.ts, name: "execute_sql"). In tool-calling flows, name matching is exact, so this wording can cause the model to emit plain SQL/manual file edits instead of calling the executable tool, which means database changes may not be applied.
Useful? React with 👍 / 👎.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| ## Database | ||
| If the user wants to use the database, use SQL queries like this: | ||
| **IMPORTANT: Always use the execute SQL tool to run SQL queries against the Supabase database. NEVER write SQL migration files to \`supabase/migrations/\` — those files are automatically generated by an external system and must not be created or modified manually.** |
There was a problem hiding this comment.
🔴 HIGH | logic
execute_sql tool instruction applies to all chat modes but tool only exists in local-agent mode
The prompt now tells the model to "Always use the execute SQL tool" for all database operations, but execute_sql is a tool only registered in local-agent (Pro) mode. In build mode, SQL is executed via <dyad-execute-sql> XML tags parsed by response_processor.ts — there is no execute_sql tool in that mode.
getSupabaseAvailableSystemPrompt is used in both modes (see chat_stream_handlers.ts), so models running in build mode will be instructed to call a tool that does not exist. This will cause the model to either hallucinate a tool call or fail to execute any SQL, breaking database setup for non-Pro users.
💡 Suggestion: Either conditionally inject different database instructions based on chat mode, or make the instruction mode-agnostic by documenting both the XML tag approach (build mode) and the execute_sql tool (local-agent mode).
| ## Database | ||
| If the user wants to use the database, use SQL queries like this: | ||
| **IMPORTANT: Always use the execute SQL tool to run SQL queries against the Supabase database. NEVER write SQL migration files to \`supabase/migrations/\` — those files are automatically generated by an external system and must not be created or modified manually.** |
There was a problem hiding this comment.
🟡 MEDIUM | accuracy
Inaccurate claim that migration files are generated by an "external system"
The warning states migration files "are automatically generated by an external system," but they are actually written by Dyad's own execute_sql tool when enableSupabaseWriteSqlMigration is enabled (see execute_sql.ts lines 46-53). This is a first-party Dyad feature, not an external system.
This misleading framing could confuse users who see migration files appear after the model says it will never create them, or lead the model to give incorrect explanations.
💡 Suggestion: Revise to: "Migration files in supabase/migrations/ are automatically managed by Dyad when the Write SQL Migration setting is enabled. Do not create or modify these files directly."
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
2 issues found across 1 file
Confidence score: 3/5
- There’s a concrete risk of incorrect tool usage:
src/prompts/supabase_prompt.tstells the model to always use the execute SQL tool even in build mode whereexecute_sqlisn’t registered, which can lead to failed SQL execution paths. - The prompt also inconsistently names the tool (“execute SQL tool” vs
execute_sql), which could cause the model to call the wrong tool identifier. - Pay close attention to
src/prompts/supabase_prompt.ts- prompt/tooling instructions may not match registered tools across modes.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/prompts/supabase_prompt.ts">
<violation number="1" location="src/prompts/supabase_prompt.ts:106">
P1: This prompt unconditionally instructs the model to "Always use the execute SQL tool", but `execute_sql` is only registered as a tool in local-agent (Pro) mode. In build mode, SQL is executed via `<dyad-execute-sql>` XML tags — no `execute_sql` tool exists. Since `getSupabaseAvailableSystemPrompt` is used in both modes, models in build mode will be told to call a tool that doesn't exist, causing them to either hallucinate a tool call or fail to execute SQL entirely. Consider conditionally injecting different instructions based on the chat mode, or making the guidance mode-agnostic.</violation>
<violation number="2" location="src/prompts/supabase_prompt.ts:108">
P2: The tool is referred to here as "the execute SQL tool" (informal, no backticks), but the actual registered tool name is `execute_sql` (with underscore). Line 30 of this same file correctly uses the backtick-quoted `` `execute_sql` `` form. Since this is an LLM system prompt and tool-calling flows rely on exact name matching, using the informal name risks the model not recognizing or correctly invoking the tool. Use the canonical `` `execute_sql` `` name consistently throughout the prompt.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| ## Database | ||
| If the user wants to use the database, use SQL queries like this: | ||
| **IMPORTANT: Always use the execute SQL tool to run SQL queries against the Supabase database. NEVER write SQL migration files to \`supabase/migrations/\` — those files are automatically generated by an external system and must not be created or modified manually.** |
There was a problem hiding this comment.
P1: This prompt unconditionally instructs the model to "Always use the execute SQL tool", but execute_sql is only registered as a tool in local-agent (Pro) mode. In build mode, SQL is executed via <dyad-execute-sql> XML tags — no execute_sql tool exists. Since getSupabaseAvailableSystemPrompt is used in both modes, models in build mode will be told to call a tool that doesn't exist, causing them to either hallucinate a tool call or fail to execute SQL entirely. Consider conditionally injecting different instructions based on the chat mode, or making the guidance mode-agnostic.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/prompts/supabase_prompt.ts, line 106:
<comment>This prompt unconditionally instructs the model to "Always use the execute SQL tool", but `execute_sql` is only registered as a tool in local-agent (Pro) mode. In build mode, SQL is executed via `<dyad-execute-sql>` XML tags — no `execute_sql` tool exists. Since `getSupabaseAvailableSystemPrompt` is used in both modes, models in build mode will be told to call a tool that doesn't exist, causing them to either hallucinate a tool call or fail to execute SQL entirely. Consider conditionally injecting different instructions based on the chat mode, or making the guidance mode-agnostic.</comment>
<file context>
@@ -103,13 +103,9 @@ function Login() {
## Database
-If the user wants to use the database, use SQL queries like this:
+**IMPORTANT: Always use the execute SQL tool to run SQL queries against the Supabase database. NEVER write SQL migration files to \`supabase/migrations/\` — those files are automatically generated by an external system and must not be created or modified manually.**
-\`\`\`sql
</file context>
| \`\`\` | ||
| You will need to setup the database schema. | ||
| You will need to setup the database schema using the execute SQL tool. |
There was a problem hiding this comment.
P2: The tool is referred to here as "the execute SQL tool" (informal, no backticks), but the actual registered tool name is execute_sql (with underscore). Line 30 of this same file correctly uses the backtick-quoted `execute_sql` form. Since this is an LLM system prompt and tool-calling flows rely on exact name matching, using the informal name risks the model not recognizing or correctly invoking the tool. Use the canonical `execute_sql` name consistently throughout the prompt.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/prompts/supabase_prompt.ts, line 108:
<comment>The tool is referred to here as "the execute SQL tool" (informal, no backticks), but the actual registered tool name is `execute_sql` (with underscore). Line 30 of this same file correctly uses the backtick-quoted `` `execute_sql` `` form. Since this is an LLM system prompt and tool-calling flows rely on exact name matching, using the informal name risks the model not recognizing or correctly invoking the tool. Use the canonical `` `execute_sql` `` name consistently throughout the prompt.</comment>
<file context>
@@ -103,13 +103,9 @@ function Login() {
-\`\`\`
-
-You will need to setup the database schema.
+You will need to setup the database schema using the execute SQL tool.
### Row Level Security (RLS)
</file context>
| You will need to setup the database schema using the execute SQL tool. | |
| You will need to setup the database schema using the \`execute_sql\` tool. |
github-actions
bot
added
the
needs-human:review-issue
🎭 Playwright Test Results❌ Some tests failed
Summary: 235 passed, 1 failed, 6 flaky, 6 skipped Failed Tests🍎 macOS
📋 Re-run Failing Tests (macOS)Copy and paste to re-run all failing spec files locally: npm run e2e \
e2e-tests/debugging_logs.spec.ts
|
1 participant
Summary
execute_sqltool for database operationssupabase/migrations/are automatically generatedTest plan
src/prompts/supabase_prompt.ts🤖 Generated with Claude Code