From e437b3a6f918c5fd62a76d10713121041deaec55 Mon Sep 17 00:00:00 2001 From: Evgenii Kniazev Date: Tue, 2 Dec 2025 12:50:29 +0000 Subject: [PATCH 1/6] Fix apps-mcp agent behavior: localhost preference and deployment confirmation Two key issues addressed: 1. Agent was opening deployed URLs instead of localhost when user asked to "open a dev copy" or see the app during development. 2. Agent was deploying too early without user confirmation. Changes: - apps.tmpl: Add localhost guidance and deployment confirmation requirement - CLAUDE.md: Add "Deployment Procedures" section with pre-deployment checklist - CLAUDE.md: Add "Local Development vs Deployed Apps" section with decision tree The agent will now: - Prefer localhost:8000 during active development - Ask "Ready to deploy to [environment]?" before deploying - Only use deployed URLs after deployment with user approval Addresses feedback from PR #4045 discussion. --- experimental/apps-mcp/lib/prompts/apps.tmpl | 29 ++++++++- .../template/{{.project_name}}/CLAUDE.md | 65 +++++++++++++++++++ 2 files changed, 92 insertions(+), 2 deletions(-) diff --git a/experimental/apps-mcp/lib/prompts/apps.tmpl b/experimental/apps-mcp/lib/prompts/apps.tmpl index 9f9b710488..6657e2d991 100644 --- a/experimental/apps-mcp/lib/prompts/apps.tmpl +++ b/experimental/apps-mcp/lib/prompts/apps.tmpl @@ -15,18 +15,43 @@ DATABRICKS APPS DEVELOPMENT invoke_databricks_cli 'experimental apps-mcp tools init-template --name my-app-name --description "My app description"' +# Local Development + +⚠️ PRIMARY development workflow - run app locally: + +1. Navigate to app directory: cd ./your-app +2. Start dev server: npm run dev +3. Open in browser: http://localhost:8000 + +⚠️ IMPORTANT: When user asks to "open the app", "see the app", or "open dev copy": +- During development → ALWAYS use http://localhost:8000 (requires npm run dev to be running) +- After deployment → Use deployed URL from bundle summary + +DO NOT suggest deployed URLs (staging/production) when user wants to test during development! + # Validation ⚠️ Always validate your app before deploying to production: invoke_databricks_cli 'experimental apps-mcp tools validate ./your-app-location' +This runs: npm install → build → typecheck → tests + # Deployment -⚠️ Use the deploy command which validates, deploys, and runs the app: +⚠️ CRITICAL: NEVER deploy without explicit user confirmation! + +Deployment checklist (confirm with user BEFORE deploying): +1. App has been validated successfully +2. Local testing completed (npm run dev) +3. User explicitly approved deployment + +⚠️ ALWAYS ask user: "Ready to deploy to [environment]?" before running deploy command. + +Once confirmed, use the deploy command: invoke_databricks_cli 'experimental apps-mcp tools deploy' -# View and manage your app: +After deployment, get the app URL: invoke_databricks_cli 'bundle summary' diff --git a/experimental/apps-mcp/templates/appkit/template/{{.project_name}}/CLAUDE.md b/experimental/apps-mcp/templates/appkit/template/{{.project_name}}/CLAUDE.md index 940074fc44..2e180db581 100644 --- a/experimental/apps-mcp/templates/appkit/template/{{.project_name}}/CLAUDE.md +++ b/experimental/apps-mcp/templates/appkit/template/{{.project_name}}/CLAUDE.md @@ -37,6 +37,71 @@ TypeScript full-stack template powered by **Databricks AppKit** with tRPC for ad - Pre-commit: `npm run typecheck && npm run lint:fix && npm run format:fix && npm test` - Pre-deploy: `npm run build && npm start` (test locally) → `npm test` +## Deployment Procedures + +**CRITICAL**: Deployment is a significant action that affects live environments. NEVER deploy without explicit user confirmation. + +### Before Deploying + +**ALWAYS** confirm with the user first by asking: "Ready to deploy to [environment]?" + +Pre-deployment checklist: +1. ✅ App validated successfully (`databricks experimental apps-mcp tools validate .`) +2. ✅ Local testing completed (`npm run dev` and manual verification) +3. ✅ User explicitly approved deployment +4. ✅ Code committed to git (recommended) + +### Deployment Commands + +```bash +databricks bundle deploy # Deploy app infrastructure +databricks bundle run app # Start the app +databricks bundle summary # Get deployed app URL +``` + +Or use the all-in-one command: +```bash +databricks experimental apps-mcp tools deploy # Validates, deploys, and runs +``` + +**DO NOT** run deployment commands unless the user explicitly requests it or confirms when asked. + +## Local Development vs Deployed Apps + +**CRITICAL**: Understand the difference between local development and deployed environments. + +### During Development (Before Deployment) + +When the user asks to "open the app", "see the app", "view the app", or "open a dev copy": + +- ✅ **Use localhost**: `http://localhost:8000` +- ✅ **Requires**: `npm run dev` must be running +- ✅ **Purpose**: Hot reload, fast iteration, debugging + +**DO NOT** suggest deployed URLs (staging/production) during active development! + +### After Deployment + +Only after running `databricks bundle deploy && databricks bundle run app`: + +- ✅ **Use deployed URL**: Get from `databricks bundle summary` +- ✅ **Purpose**: Production/staging environment, sharing with others + +### Decision Tree + +``` +User asks to "open the app" or "see the app" +│ +├─ Has the app been deployed in this session? +│ ├─ No → Use http://localhost:8000 (start with `npm run dev` if needed) +│ └─ Yes → Ask user which environment they want: +│ • localhost (development) +│ • deployed URL (production/staging) +│ +└─ User explicitly says "localhost" or "local" + → Always use http://localhost:8000 +``` + ## App Naming Constraints App names must not exceed 30 characters total (including target prefix). From 1f87210a39e3082b370f68ff543432e98c761068 Mon Sep 17 00:00:00 2001 From: Evgenii Kniazev Date: Tue, 2 Dec 2025 12:50:29 +0000 Subject: [PATCH 2/6] Fix apps-mcp agent behavior: localhost, npm dev, and deployment confirmation Three key issues addressed: 1. Agent was opening deployed URLs instead of localhost when user asked to "open a dev copy" or see the app during development. 2. Agent was deploying too early without user confirmation. 3. Need to clarify npm run dev (development) vs npm start (production). Changes: - apps.tmpl: Add localhost guidance and deployment confirmation requirement - apps.tmpl: Clarify "npm run dev" ALWAYS for development, NEVER "npm start" - CLAUDE.md: Add "Deployment Procedures" section with pre-deployment checklist - CLAUDE.md: Add "Local Development vs Deployed Apps" section with decision tree - CLAUDE.md: Add visual box explaining "npm run dev vs npm start" The agent will now: - ALWAYS use "npm run dev" during development (NEVER npm start) - Prefer localhost:8000 during active development - Ask "Ready to deploy to [environment]?" before deploying - Only use deployed URLs after deployment with user approval Addresses feedback from PR #4045 discussion. --- experimental/apps-mcp/lib/prompts/apps.tmpl | 6 +- .../template/{{.project_name}}/CLAUDE.md | 86 ++++++++++++++++++- 2 files changed, 88 insertions(+), 4 deletions(-) diff --git a/experimental/apps-mcp/lib/prompts/apps.tmpl b/experimental/apps-mcp/lib/prompts/apps.tmpl index 6657e2d991..eaad35b26c 100644 --- a/experimental/apps-mcp/lib/prompts/apps.tmpl +++ b/experimental/apps-mcp/lib/prompts/apps.tmpl @@ -20,9 +20,13 @@ invoke_databricks_cli 'experimental apps-mcp tools init-template --name my-app-n ⚠️ PRIMARY development workflow - run app locally: 1. Navigate to app directory: cd ./your-app -2. Start dev server: npm run dev +2. Start dev server: npm run dev (NEVER use npm start during development!) 3. Open in browser: http://localhost:8000 +CRITICAL: Always use `npm run dev` during development, NEVER `npm start`: +- npm run dev = development server with hot reload +- npm start = production server (deployment only) + ⚠️ IMPORTANT: When user asks to "open the app", "see the app", or "open dev copy": - During development → ALWAYS use http://localhost:8000 (requires npm run dev to be running) - After deployment → Use deployed URL from bundle summary diff --git a/experimental/apps-mcp/templates/appkit/template/{{.project_name}}/CLAUDE.md b/experimental/apps-mcp/templates/appkit/template/{{.project_name}}/CLAUDE.md index 2e180db581..75c7573a8e 100644 --- a/experimental/apps-mcp/templates/appkit/template/{{.project_name}}/CLAUDE.md +++ b/experimental/apps-mcp/templates/appkit/template/{{.project_name}}/CLAUDE.md @@ -8,8 +8,8 @@ TypeScript full-stack template powered by **Databricks AppKit** with tRPC for ad ## NPM Scripts ### Development -- `npm run dev` - Start dev server with hot reload (use during active development) -- `npm start` - Start production server (requires `npm run build` first) +- `npm run dev` - Start dev server with hot reload (**ALWAYS use this during development**) +- `npm start` - Start production server (**NEVER use during development - production only!**) ### Build - `npm run build` - Full build (server + client) - use before deployment @@ -35,7 +35,87 @@ TypeScript full-stack template powered by **Databricks AppKit** with tRPC for ad **Common workflows:** - Development: `npm run dev` → make changes → `npm run typecheck` → `npm run lint:fix` - Pre-commit: `npm run typecheck && npm run lint:fix && npm run format:fix && npm test` -- Pre-deploy: `npm run build && npm start` (test locally) → `npm test` +- Pre-deploy: Validate with `databricks experimental apps-mcp tools validate .` + +**CRITICAL - npm run dev vs npm start:** +``` +┌─────────────────────────────────────────────────────┐ +│ DURING DEVELOPMENT: ALWAYS use `npm run dev` │ +│ - Hot reload enabled │ +│ - Fast iteration │ +│ - Runs on localhost:8000 │ +│ │ +│ NEVER use `npm start` during development! │ +│ - npm start is for production deployment only │ +│ - No hot reload │ +│ - Requires build step first │ +└─────────────────────────────────────────────────────┘ +``` + +## Deployment Procedures + +**CRITICAL**: Deployment is a significant action that affects live environments. NEVER deploy without explicit user confirmation. + +### Before Deploying + +**ALWAYS** confirm with the user first by asking: "Ready to deploy to [environment]?" + +Pre-deployment checklist: +1. ✅ App validated successfully (`databricks experimental apps-mcp tools validate .`) +2. ✅ Local testing completed (`npm run dev` and manual verification) +3. ✅ User explicitly approved deployment +4. ✅ Code committed to git (recommended) + +### Deployment Commands + +```bash +databricks bundle deploy # Deploy app infrastructure +databricks bundle run app # Start the app +databricks bundle summary # Get deployed app URL +``` + +Or use the all-in-one command: +```bash +databricks experimental apps-mcp tools deploy # Validates, deploys, and runs +``` + +**DO NOT** run deployment commands unless the user explicitly requests it or confirms when asked. + +## Local Development vs Deployed Apps + +**CRITICAL**: Understand the difference between local development and deployed environments. + +### During Development (Before Deployment) + +When the user asks to "open the app", "see the app", "view the app", or "open a dev copy": + +- ✅ **Use localhost**: `http://localhost:8000` +- ✅ **Requires**: `npm run dev` must be running +- ✅ **Purpose**: Hot reload, fast iteration, debugging + +**DO NOT** suggest deployed URLs (staging/production) during active development! + +### After Deployment + +Only after running `databricks bundle deploy && databricks bundle run app`: + +- ✅ **Use deployed URL**: Get from `databricks bundle summary` +- ✅ **Purpose**: Production/staging environment, sharing with others + +### Decision Tree + +``` +User asks to "open the app" or "see the app" +│ +├─ Has the app been deployed in this session? +│ ├─ No → Use http://localhost:8000 (start with `npm run dev` if needed) +│ └─ Yes → Ask user which environment they want: +│ • localhost (development) +│ • deployed URL (production/staging) +│ +└─ User explicitly says "localhost" or "local" + → Always use http://localhost:8000 +``` ## Deployment Procedures From dd645374a8a5a0c5acbbbb71a99114ed0302b70d Mon Sep 17 00:00:00 2001 From: Evgenii Kniazev Date: Tue, 2 Dec 2025 14:07:58 +0000 Subject: [PATCH 3/6] Refactor: separate MCP workflow from template-specific details apps.tmpl is now stack-agnostic and focused only on MCP workflow: - init-template command - Validation workflow - Deployment confirmation - References CLAUDE.md for template-specific details CLAUDE.md retains all template-specific guidance: - npm run dev vs npm start - localhost vs deployed URLs - Deployment procedures with checklist - TypeScript, testing, and other stack-specific details This removes duplication and properly separates concerns. --- experimental/apps-mcp/lib/prompts/apps.tmpl | 41 ++++++++------------- 1 file changed, 15 insertions(+), 26 deletions(-) diff --git a/experimental/apps-mcp/lib/prompts/apps.tmpl b/experimental/apps-mcp/lib/prompts/apps.tmpl index 135c618dd8..fd4f9e8eb1 100644 --- a/experimental/apps-mcp/lib/prompts/apps.tmpl +++ b/experimental/apps-mcp/lib/prompts/apps.tmpl @@ -26,51 +26,40 @@ When building or planning a new Databricks app, use the following steps: # Initialization -⚠️ ALWAYS start by scaffolding a new Databricks app using command: +⚠️ ALWAYS start by scaffolding a new Databricks app: invoke_databricks_cli 'experimental apps-mcp tools init-template --name my-app-name --description "My app description"' -# Local Development - -⚠️ PRIMARY development workflow - run app locally: - -1. Navigate to app directory: cd ./your-app -2. Start dev server: npm run dev (NEVER use npm start during development!) -3. Open in browser: http://localhost:8000 +After scaffolding, refer to CLAUDE.md in the app directory for template-specific instructions. -CRITICAL: Always use `npm run dev` during development, NEVER `npm start`: -- npm run dev = development server with hot reload -- npm start = production server (deployment only) - -⚠️ IMPORTANT: When user asks to "open the app", "see the app", or "open dev copy": -- During development → ALWAYS use http://localhost:8000 (requires npm run dev to be running) -- After deployment → Use deployed URL from bundle summary +# Local Development -DO NOT suggest deployed URLs (staging/production) when user wants to test during development! +⚠️ After initialization, check the app's CLAUDE.md for: +- How to run the app locally +- Development commands specific to the template +- Local vs deployed environment guidance # Validation -⚠️ Always validate your app before deploying to production: +⚠️ Before deploying, validate your app: invoke_databricks_cli 'experimental apps-mcp tools validate ./your-app-location' -This runs: npm install → build → typecheck → tests +This ensures the app is production-ready by running template-specific checks. # Deployment ⚠️ CRITICAL: NEVER deploy without explicit user confirmation! -Deployment checklist (confirm with user BEFORE deploying): -1. App has been validated successfully -2. Local testing completed (npm run dev) -3. User explicitly approved deployment - -⚠️ ALWAYS ask user: "Ready to deploy to [environment]?" before running deploy command. +Before deploying: +1. ✅ Validate the app successfully +2. ✅ Test locally per CLAUDE.md instructions +3. ✅ Ask user: "Ready to deploy to [environment]?" -Once confirmed, use the deploy command: +Once confirmed: invoke_databricks_cli 'experimental apps-mcp tools deploy' -After deployment, get the app URL: +After deployment: invoke_databricks_cli 'bundle summary' From 399e6fffb978622074acdc90ae5bdeb9935bd79c Mon Sep 17 00:00:00 2001 From: Evgenii Kniazev Date: Tue, 2 Dec 2025 14:27:55 +0000 Subject: [PATCH 4/6] Clean up apps.tmpl after rebase: maintain stack-agnostic approach MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Remove template-specific details (npm, SQL, tRPC, UI components) that came from merge. Keep only MCP workflow commands and reference CLAUDE.md for template-specific guidance. Maintains: - init-template → cd → read CLAUDE.md workflow - Validation command - Deployment confirmation requirement --- experimental/apps-mcp/lib/prompts/apps.tmpl | 22 +++++---------------- 1 file changed, 5 insertions(+), 17 deletions(-) diff --git a/experimental/apps-mcp/lib/prompts/apps.tmpl b/experimental/apps-mcp/lib/prompts/apps.tmpl index fd4f9e8eb1..8db44fbda8 100644 --- a/experimental/apps-mcp/lib/prompts/apps.tmpl +++ b/experimental/apps-mcp/lib/prompts/apps.tmpl @@ -9,28 +9,16 @@ DATABRICKS APPS DEVELOPMENT ============================ -When building or planning a new Databricks app, use the following steps: - -1. MUST start by scaffolding a new Databricks app using command: invoke_databricks_cli 'experimental apps-mcp tools init-template --name my-app-name --description "My app description"' - MUST run this command before planning the app. You must understand tge generated file structure and the purpose of the files. -2. MUST read the CLAUDE.md file in the app directory. -3. SHOULD run `npm install`, run `npm run dev` in the background, and open http://localhost:8000 in your browser before making changes to the app. -4. MUST run `cd ` to navigate to the app directory. MUST execute all commands from this directory. -4. If applicable, create SQL query files in the ./config/queries directory. -5. If applicable, create tRPC endpoints in the ./server/trpc.ts file. -6. If applicable, build UI components in the ./client/src directory. -7. MUST update the ./client/src/App.tsx file -8. MUST update smoke test in the ./tests/smoke.spec.ts file -9. MUST run validation using command: invoke_databricks_cli 'experimental apps-mcp tools validate ./' -10. USER CONSENT: Only deploy the app with explicit consent from the user. Deployment using command: invoke_databricks_cli 'experimental apps-mcp tools deploy' - # Initialization -⚠️ ALWAYS start by scaffolding a new Databricks app: +⚠️ MUST start by scaffolding a new Databricks app: invoke_databricks_cli 'experimental apps-mcp tools init-template --name my-app-name --description "My app description"' -After scaffolding, refer to CLAUDE.md in the app directory for template-specific instructions. +After scaffolding: +1. MUST read CLAUDE.md in the app directory for template-specific instructions +2. MUST run `cd ` to navigate to the app directory +3. MUST execute all subsequent commands from this directory # Local Development From b984c2e8af812ad25c4c7a343b9f5bb153d47346 Mon Sep 17 00:00:00 2001 From: Evgenii Kniazev Date: Tue, 2 Dec 2025 14:35:20 +0000 Subject: [PATCH 5/6] Restore numbered workflow steps in apps.tmpl Restore the step-by-step guidance that was accidentally removed. This provides clearer structure for agents building apps. --- experimental/apps-mcp/lib/prompts/apps.tmpl | 56 ++++++--------------- 1 file changed, 14 insertions(+), 42 deletions(-) diff --git a/experimental/apps-mcp/lib/prompts/apps.tmpl b/experimental/apps-mcp/lib/prompts/apps.tmpl index 8db44fbda8..399800191b 100644 --- a/experimental/apps-mcp/lib/prompts/apps.tmpl +++ b/experimental/apps-mcp/lib/prompts/apps.tmpl @@ -9,45 +9,17 @@ DATABRICKS APPS DEVELOPMENT ============================ -# Initialization - -⚠️ MUST start by scaffolding a new Databricks app: - -invoke_databricks_cli 'experimental apps-mcp tools init-template --name my-app-name --description "My app description"' - -After scaffolding: -1. MUST read CLAUDE.md in the app directory for template-specific instructions -2. MUST run `cd ` to navigate to the app directory -3. MUST execute all subsequent commands from this directory - -# Local Development - -⚠️ After initialization, check the app's CLAUDE.md for: -- How to run the app locally -- Development commands specific to the template -- Local vs deployed environment guidance - -# Validation - -⚠️ Before deploying, validate your app: - -invoke_databricks_cli 'experimental apps-mcp tools validate ./your-app-location' - -This ensures the app is production-ready by running template-specific checks. - -# Deployment - -⚠️ CRITICAL: NEVER deploy without explicit user confirmation! - -Before deploying: -1. ✅ Validate the app successfully -2. ✅ Test locally per CLAUDE.md instructions -3. ✅ Ask user: "Ready to deploy to [environment]?" - -Once confirmed: - -invoke_databricks_cli 'experimental apps-mcp tools deploy' - -After deployment: - -invoke_databricks_cli 'bundle summary' +When building or planning a new Databricks app, use the following steps: + +1. MUST start by scaffolding a new Databricks app using command: invoke_databricks_cli 'experimental apps-mcp tools init-template --name my-app-name --description "My app description"' + MUST run this command before planning the app. You must understand the generated file structure and the purpose of the files. +2. MUST read the CLAUDE.md file in the app directory. +3. SHOULD run `npm install`, run `npm run dev` in the background, and open http://localhost:8000 in your browser before making changes to the app. +4. MUST run `cd ` to navigate to the app directory. MUST execute all commands from this directory. +5. If applicable, create SQL query files in the ./config/queries directory. +6. If applicable, create tRPC endpoints in the ./server/trpc.ts file. +7. If applicable, build UI components in the ./client/src directory. +8. MUST update the ./client/src/App.tsx file +9. MUST update smoke test in the ./tests/smoke.spec.ts file +10. MUST run validation using command: invoke_databricks_cli 'experimental apps-mcp tools validate ./' +11. USER CONSENT: Only deploy the app with explicit consent from the user. Deployment using command: invoke_databricks_cli 'experimental apps-mcp tools deploy' From 5523cca8602e9245925d6be7495d8368d7c0326f Mon Sep 17 00:00:00 2001 From: Evgenii Kniazev Date: Tue, 2 Dec 2025 15:02:55 +0000 Subject: [PATCH 6/6] Revert apps.tmpl to match main Remove all branch-specific changes to apps.tmpl, keeping only the version from main branch. --- experimental/apps-mcp/lib/prompts/apps.tmpl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/experimental/apps-mcp/lib/prompts/apps.tmpl b/experimental/apps-mcp/lib/prompts/apps.tmpl index a730a23fb4..fbd25d3708 100644 --- a/experimental/apps-mcp/lib/prompts/apps.tmpl +++ b/experimental/apps-mcp/lib/prompts/apps.tmpl @@ -12,7 +12,7 @@ DATABRICKS APPS DEVELOPMENT When building or planning a new Databricks app, use the following steps: 1. MUST start by scaffolding a new Databricks app using command: invoke_databricks_cli 'experimental apps-mcp tools init-template --name my-app-name --description "My app description"' - MUST run this command before planning the app. You must understand the generated file structure and the purpose of the files. + MUST run this command before planning the app. You must understand tge generated file structure and the purpose of the files. 2. MUST read the CLAUDE.md file in the app directory. 3. SHOULD run `npm install`, run `npm run dev` in the background, and open http://localhost:8000 in your browser before making changes to the app. 4. MUST run `cd ` to navigate to the app directory. MUST execute all commands from this directory.