Skip to content

Favor DOM Selectors for Browser Tools Clicks#4

Open
dshaikh-godaddy wants to merge 126 commits intoComposioHQ:masterfrom
gdcorp-engineering:IncludePageContext
Open

Favor DOM Selectors for Browser Tools Clicks#4
dshaikh-godaddy wants to merge 126 commits intoComposioHQ:masterfrom
gdcorp-engineering:IncludePageContext

Conversation

@dshaikh-godaddy
Copy link
Copy Markdown

@dshaikh-godaddy dshaikh-godaddy commented Nov 26, 2025

  • Cache getPageContext results with full bounding-box metadata and route click requests through a selector resolver so the agent clicks DOM elements directly instead of guessing screenshot coordinates.

vyung-godaddy and others added 30 commits November 12, 2025 11:45
@vyung-godaddy @claude
feat: Rebrand to GoDaddy ANS and add multi-provider support
3266d81 
- Rebrand from Atlas to GoDaddy ANS Chat Sidebar
- Add support for Anthropic Claude and OpenAI providers
- Implement browser automation tools for Anthropic (navigate, click, type, screenshot)
- Add custom base URL and model name configuration
- Implement per-tab isolated chat history
- Add automatic page context extraction
- Fix auto-scroll during streaming responses
- Update header to show correct provider/model

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: MCP tool execution using transport.send() workaround
ae9fd3e 
Fixed MCP tool execution by bypassing broken client.callTool() method in AI SDK.
The AI SDK's experimental_createMCPClient has a bug where client.callTool() hangs
forever with StreamableHTTPClientTransport. Implemented workaround using direct
transport.send() with onmessage handler.

Changes:
- mcp-service.ts: Rewrote executeToolCall() to use transport layer directly
- mcp-tool-router.ts: Added formatToolsForAnthropic() to unwrap jsonSchema
- trusted-business-service.ts: Fixed URL extraction from protocolExtensions.mcp.remotes[0].url
- anthropic-browser-tools.ts: Added AbortSignal validation
- anthropic-service.ts: Added AbortSignal validation
- sidepanel.tsx: Added MCP initialization for browser tools, fixed missing onComplete parameter
- types.ts: Added serverUrl to MCPConnection interface
- test-mcp-client.mjs: Test script proving transport.send() works in 33ms

Result: MCP tools now execute successfully, returning "Quack Time - A Duck Blog" in 33ms.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: UI improvements and model display updates
35a6692 
Settings Page Updates:
- Renamed "Custom Base URL" to "GoCode URL" with consistent width
- Renamed "Anthropic API Key" to "GoCode Key" (static label for all providers)
- Made GoCode URL and GoCode Key labels static regardless of provider selection
- Hidden Composio API Key field from UI
- Removed provider-specific help text from GoCode Key field
- Updated service count from "400+" to "115 Million" in Business Services
- Added Claude Sonnet 4.5 (claude-sonnet-4-5-20250929) to model selection
- Fixed field widths: GoCode URL, GoCode Key, and ANS API Token now use same width

Sidepanel Updates:
- Added MODEL_DISPLAY_NAMES mapping for friendly model names
- Added getModelDisplayName() helper function
- Updated chat header to show friendly model names (e.g., "Claude Sonnet 4.5")
- Applied display names to browser tools mode and warning banners

Result: Cleaner UI with consistent naming and proper model display names

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: Add A2A protocol support and fix browser tools execution
1838030 
## A2A Protocol Support
- Add A2A service with direct HTTP POST to /invoke endpoint
- Implement site detection and automatic agent matching
- Add trusted agent badge UI with opt-in/out control
- Filter A2A servers from MCP connection process
- Support domain matching with/without www prefix
- Convert agent names for display (www-godaddy-com -> www.godaddy.com)
- Initialize A2A service on extension load
- Route all messages to site's A2A agent when opted in
- Update trusted badge on URL changes via chrome.tabs.onUpdated

## Browser Tools Fixes
- Fix tool call format: keep content as arrays instead of stringified JSON
- Add smart input detection: automatically find and click search boxes when nothing is focused
- Limit conversation history to 6 recent messages to prevent context overflow
- Reduce page context from 3000 to 800 chars in browser tools mode

## UX Improvements
- Add loading state for input field: show "Loading settings..." until ready
- Disable input/submit until settings are loaded to prevent lost messages
- Fix infinite loop in useEffect by using empty dependency array and refs
- Remove unnecessary settings check from trusted agent detection

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: Prevent context overflow in Anthropic browser tools
a48d566 
## Context Management Improvements
- Reduce initial history to 2 messages (only keep last user message)
- Reduce loop limit to 6 messages during tool execution
- Reduce page context from 500 to 300 chars in browser tools mode
- Add in-loop trimming to prevent context accumulation

## Remove Screenshot Tool
- Remove screenshot tool from Anthropic browser tools
- Screenshots consume 50K-150K tokens each (25-75% of 200K context!)
- Anthropic uses text-based navigation, not vision
- Only Gemini Computer Use needs screenshots for visual element location

## Better Error Messages
- Show clear "Context limit exceeded. Please start a new chat" message
- Replace cryptic API error with user-friendly guidance

This change provides massive context savings and eliminates the "Input is too long"
errors that were blocking multi-step browser automation tasks.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: Add visual feedback for browser automation
9d8651a 
Added persistent blue border overlay and "Take Over Control" button to give
users clear visual indication when AI is controlling the browser.

Key Features:
- Blue border (#007AFF) with subtle glow effect around browser window
- "Take Over Control" button centered at bottom of page
- Overlay appears immediately when ANY browser tool executes
- Overlay persists throughout entire automation session
- Overlay survives page navigations by re-showing after each action
- Overlay disappears only when automation completes or user clicks button

Implementation:
- content.ts: Overlay creation/destruction functions with DOM ready checks
- sidepanel.tsx: Show overlay before every browser tool execution
- sidepanel.tsx: Re-show overlay after every tool (handles navigation)
- sidepanel.tsx: Hide overlay on completion, error, or user abort
- Works with both Anthropic and Gemini browser tools

User Experience:
- Clear visual feedback during automation (blue border)
- Non-intrusive design (pointer-events: none on border)
- Single-click abort button for immediate control takeover
- Overlay maintains visibility across page navigations and all actions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: Improve auto-scroll and click accuracy
5d37d37 
Auto-scroll improvements:
- Fixed messagesEndRef placement - moved inside scrollable container
- Increased scroll threshold from 100px to 200px for more aggressive auto-scroll
- Added block: 'end' parameter for more reliable bottom scrolling
- Added incremental message updates in Gemini Computer Use
- Messages now update after each text chunk and tool execution
- Auto-scroll triggers more frequently during browser automation

Click accuracy improvements:
- Added smart input detection when clicking near search boxes
- Searches for input fields inside clicked containers
- Finds nearby inputs within 100px radius when coordinates are inaccurate
- Auto-adjusts coordinates to center of found input field
- Explicitly focuses input fields after clicking
- Uses distance calculation for fuzzy targeting

User Experience:
- Chat automatically stays scrolled to bottom during automation
- Users no longer need to manually scroll to see latest messages
- Search box clicks are much more accurate (e.g., Amazon search)
- Works even when coordinates are slightly off target

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: Add GitHub Actions workflow with environment-specific builds
b32ced4 
Added manual-trigger workflow for building Dev, Test, and Prod environments:

Workflow features:
- Manual trigger from GitHub Actions tab
- Environment selection dropdown (dev, test, prod)
- Passes BUILD_ENV variable to npm run build
- Uploads artifacts to GitHub for download
- 30-day artifact retention
- Build summary with environment info

Build output structure:
- Dev builds → artifacts/Dev/
- Test builds → artifacts/Test/
- Prod builds → artifacts/Prod/

Vite config updates:
- Reads BUILD_ENV from environment variable (defaults to 'dev')
- Outputs to artifacts/{env}/ based on BUILD_ENV
- Makes BUILD_ENV available in extension via import.meta.env.BUILD_ENV
- Logs build environment and output directory

Local usage:
BUILD_ENV=dev npm run build   → artifacts/Dev/
BUILD_ENV=test npm run build  → artifacts/Test/
BUILD_ENV=prod npm run build  → artifacts/Prod/

GitHub Actions usage:
1. Go to Actions tab on GitHub
2. Select "Build Extension" workflow
3. Click "Run workflow"
4. Select environment (dev/test/prod)
5. Download artifact: extension-{env}-{run_number}.zip

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
docs: Reorganize README with ANS as primary focus
61f60d4 
Restructured README to prioritize GoDaddy ANS use case:

Features section:
- Highlighted browser automation and ANS integration
- Moved Composio to last as optional feature

Getting Started section:
- Split into two clear paths:
  1. For GoDaddy Employees (ANS Extension)
     - VPN connection steps
     - ANS authentication via www.dev-godaddy.com
     - GoCode and ANS credential setup
  2. For Developers (Building from Source)
     - BUILD_ENV environment variable usage
     - Multi-environment build instructions

Composio content:
- Moved to separate section at end
- "For Composio Users (Tool Router Mode)"
- Includes prerequisites, setup, and usage examples
- Composio-specific resources grouped together

References:
- Removed Composio links (now in Composio section)
- Added GoDaddy ANS documentation link

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: Add multi-provider support and Anthropic computer use for Elect…
80002a7 
…ron browser

- Add support for Google Gemini, Anthropic Claude, and OpenAI in chat mode
- Implement Anthropic computer use capabilities for browse mode
- Fix GoCode API endpoint compatibility with custom base URLs
- Update IPC handlers to pass provider, model, and baseURL parameters
- Refactor ComputerUseService to route between Gemini and Anthropic implementations
- Add Anthropic Messages API integration with computer_20241022 tool
- Map Anthropic computer actions (left_click, type, key) to browser automation
- Update preload bridge and renderer services for multi-provider support
- Install @ai-sdk/anthropic and @ai-sdk/openai packages

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: Fix packaged app asar path resolution for renderer and preload
d55fb7c 
- Use app.getAppPath() instead of __dirname for production builds
- Fixes blank screen issue when loading from asar archive
- Properly load renderer HTML and preload script in packaged app
- Add dev tools in production mode for debugging

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: Prevent crash on app quit by checking if objects are destroyed
d64dee4 
- Add isDestroyed() check before cleaning up BrowserView
- Wrap cleanup in try-catch to handle already-destroyed objects
- Fixes "Object has been destroyed" error on app close

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: improve Chrome extension UI/UX for browser automation
2f314af 
- Add magical blue overlay (50% opacity) with sparkle effects when browser actions occur
- Compact tool execution messages with smaller font and reduced spacing
- Add color differentiation for page context in user messages (blue scrollable box)
- Fix message fragmentation - AI responses now flow as unified text
- Compact JSON formatting for tool parameters (single line instead of multi-line)
- Version bumped to 1.1.9

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: enhance GitHub Actions artifact management
d5f74ae 
- Add ZIP packaging for easy distribution
- Create both ZIP and unzipped artifact uploads
- Organize artifacts by environment (dev/test/prod)
- Add comprehensive build summary with:
  - Version and build number info
  - Download instructions
  - Installation steps for Chrome
- Extend artifact retention to 90 days

This makes it easier for users to find and download the correct
build artifacts from the GitHub Actions UI.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: correct artifact packaging paths in GitHub Actions
50c36d3 
- Fix ZIP path from artifacts directory (use ../../ instead of ../../../)
- Add validation to check artifacts directory exists
- Add helpful error messages if build artifacts are missing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
docs: add comprehensive wiki documentation
fd8fb40 
- Add Home page with overview and quick links
- Add Installation guide for end users
- Add Building guide for different environments
- Add Architecture documentation with system diagrams
- Add Artifacts guide for GitHub Actions downloads
- Add Development guide for contributors
- Add README with upload instructions

Wiki content covers:
- Installation from artifacts and source
- Build process for dev/test/prod
- Technical architecture and data flow
- Artifact download and usage
- Development workflow and code style
- Common issues and troubleshooting

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: enable browser tools by default
6dca346 
- Change browserToolsEnabled initial state from false to true
- Browser automation tools (navigate, click, type, scroll) now active on startup
- Users can still disable via the ○/◉ button in sidebar if needed
- Update version to 1.2.0

This improves the default user experience by enabling computer use
capabilities without requiring manual activation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: add environment label to GitHub Actions job name
ee37fde 
- Add dynamic job name showing environment (e.g., "Build Extension - prod")
- Update version in build summary from 1.1.9 to 1.2.0
- Makes it easier to identify build jobs in GitHub Actions UI

Before: All jobs showed "Build Extension"
After: Jobs show "Build Extension - dev", "Build Extension - test", "Build Extension - prod"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: add run-name to show environment in Actions UI
21e379d 
- Add run-name field to display environment in workflow run list
- This makes "Build Extension - dev/test/prod" visible in the Actions UI
- Previous change only affected job name within the run details

The run-name field controls what appears in the GitHub Actions
workflow runs list, making it easy to identify which environment
was built at a glance.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: revert browser tools to disabled by default
21fead3 
- Change browserToolsEnabled initial state from true back to false
- Users must explicitly enable browser tools by clicking the ○ button
- Update version to 1.2.1
- Update workflow build summary to show 1.2.1

This provides a safer default experience, requiring users to opt-in
to browser automation capabilities rather than having them active
immediately on startup.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: abort all pending browser operations on "Take Over"
fefd8e6 
- Add ABORT_ALL_BROWSER_OPERATIONS message type
- Background script sets abort flag when user clicks "Take Over"
- All browser operations (navigate, click, type, scroll) check abort flag
- Operations return "Operation aborted by user" error if flag is set
- Abort flag clears automatically after 1 second
- Update version to 1.2.2

When users click "Take Over Control", the extension now:
1. Aborts the AI API call (existing)
2. Sets abort flag in background script (new)
3. Prevents any queued browser operations from executing (new)
4. Hides the browser overlay (existing)

This gives users immediate control and stops all pending automation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: prevent overlay from reappearing after "Take Over"
8e43ba2 
- Add isUserAborted flag in content script
- Block overlay from showing for 5 seconds after "Take Over" is clicked
- Immediately hide overlay before sending abort message
- Add parentNode check before removing elements
- Enhanced logging for debugging overlay behavior
- Update version to 1.2.3

This fixes the issue where the overlay would remain visible or
reappear after clicking "Take Over Control". The abort flag
prevents any new overlay requests from showing for 5 seconds,
giving the user full control.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
debug: add logging for click coordinate debugging
6167ee4 
- Log received click coordinates (x, y)
- Log viewport dimensions
- Log document scroll position
- Log element found at coordinates
- Version 1.2.4

This will help diagnose why clicks are happening at incorrect
positions. Check browser console (F12) when AI attempts to click
to see the coordinate information.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: smart input field detection for clicks
ec55f3d 
- When AI clicks near an input, find the largest visible input field
- Search all visible inputs/textareas/contenteditable elements
- Select the largest one by area (likely the main chat input)
- Update coordinates to center of found input
- Add enhanced logging for debugging
- Version 1.2.5

This helps when AI is clicking blindly without screenshots by
automatically finding and targeting the most prominent input field
on the page when coordinates miss the actual input.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: add screenshot tool for Anthropic Claude
628b12e 
- Add screenshot tool to Anthropic browser tools
- Format screenshot results as base64 images for Claude vision API
- Enable Claude to see the page before clicking/interacting
- This fixes blind clicking issues where AI guesses coordinates
- Version 1.3.0

With screenshots, Claude can:
- See where UI elements are located
- Click accurately on buttons, inputs, etc.
- Navigate complex interfaces visually
- Provide better context-aware interactions

The AI will now automatically take screenshots when needed
to understand page layout before performing actions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: add explicit screenshot workflow instructions for Claude
deb5974 
- Add detailed system prompt instructing Claude to take screenshots
- Enforce workflow: screenshot → analyze → click → type
- Tell Claude to ALWAYS screenshot before clicking or typing
- Provide step-by-step interaction instructions
- Version 1.3.1

New workflow Claude will follow:
1. Navigate to URL (if needed)
2. Take screenshot to see the page
3. Identify target element coordinates from screenshot
4. Click at those coordinates
5. Type text (if needed)

This ensures Claude uses its vision capabilities proactively
instead of guessing coordinates blindly.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: include viewport dimensions with screenshots for accurate clicking
fda2805 
- Add GET_VIEWPORT_SIZE handler in content script
- Pass viewport dimensions (width, height, devicePixelRatio) with screenshot
- Include viewport info in text description to Claude
- Tell Claude exact coordinate bounds: (0,0) to (width,height)
- Version 1.3.2

This fixes coordinate mismatch issues where Claude sees a screenshot
but doesn't know the actual viewport size, causing clicks to be
off-target. Now Claude knows the exact bounds for coordinates.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
debug: add visual debug marker for click coordinates
56d8999 
- Add red/yellow circle marker at exact click coordinates
- Shows for 3 seconds where Claude is trying to click
- Log devicePixelRatio to diagnose scaling issues
- Version 1.3.3

This helps diagnose coordinate mismatch issues by showing
exactly where the extension thinks Claude is clicking.
If the marker appears in the wrong place, it indicates a
coordinate scaling or interpretation problem.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: significantly improve coordinate instructions for Claude
68667d8 
- Add detailed CRITICAL COORDINATE INSTRUCTIONS in system prompt
- Explain coordinate system with multiple examples
- Tell Claude to measure from top-left and estimate center points
- Add comprehensive coordinate guide with each screenshot
- Include corner coordinates and center point calculations
- Emphasize clicking CENTER of elements, not edges
- Version 1.4.0

System prompt now includes:
- Visual coordinate examples (top-right, center, etc.)
- Step-by-step workflow for measuring in screenshots
- Explicit instruction to retry with adjusted coordinates if needed

Screenshot results now include:
- All corner coordinates labeled
- Center point calculated
- Clear X/Y axis explanations
- Step-by-step clicking instructions

This should dramatically improve Claude's ability to click accurately
by teaching it how to measure and estimate coordinates from screenshots.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy
feat: prioritize DOM-based interactions over screenshots
ff6b751 
BREAKING CHANGE: Changed browser automation workflow to prefer HTML DOM methods

Changes:
- Added new `clickElement` tool that uses CSS selectors or text content to find elements
- Enhanced `click` action in content script to support text-based element searching
- Searches through buttons, links, inputs, and elements with onclick/aria-label
- Updated system prompt to instruct Claude to try DOM methods FIRST before screenshots
- Enhanced `getPageContext` to include interactive elements with their selectors
- Screenshots are now only used as fallback when DOM methods fail

Benefits:
- More reliable clicking (CSS selectors don't depend on viewport coordinates)
- More efficient (no need to process/send large screenshots for every action)
- Lower token costs (screenshots consume ~1,500-2,000 tokens each)
- Works better with dynamic content and responsive layouts

Version: 1.4.0 → 1.5.0
vyung-godaddy and others added 30 commits November 22, 2025 23:37
@vyung-godaddy @claude
fix: simplify to "Click on Filters" instead of navigate + look
5cf99a0 
Simplified instruction to be more direct and action-oriented.

Changed from:
- Look for "Filters" in the left sidebar navigation
- Click on "Filters"

To:
- Click on "Filters" in the left sidebar

More concise and clear.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: add comprehensive general browser interaction framework
686e573 
Completely rewrote the system prompt to include extensive general browser
interaction patterns that work alongside site-specific instructions.

NEW HIERARCHICAL STRUCTURE:
1. MCP tools (highest priority)
2. Site-specific instructions (Jira, Workday, etc.)
3. General browser patterns (new - fallback for any site)

NEW CAPABILITIES:

Standard Workflow:
- STEP 1: Understand (screenshot + getPageContext)
- STEP 2: Plan (break down request, identify elements)
- STEP 3: Execute (one action at a time with verification)

Common Interaction Patterns:
📝 Filling Forms - systematic field-by-field approach
🔍 Searching Content - find inputs, type, extract results
🧭 Navigation - menus, links, breadcrumbs
📊 Extracting Data - tables, lists, structured content
🎯 Clicking Elements - preference order (text > selector > coords)
⌨️ Typing Text - with auto-submit for search boxes
⏱️ Waiting & Timing - page loads, confirmations, dynamic content
❌ Error Handling - element not found, click failed, form errors
🔔 Modals & Popups - wait, interact, close, verify
🗂️ Dropdowns & Menus - open, select, verify
📁 File Uploads - limitations and alternatives

Tool Usage Guidelines:
- Clear descriptions of when to use each tool
- Emphasis on getPageContext first, screenshots when needed
- Preference for DOM methods over coordinates

Benefits:
✓ Agent can now handle unfamiliar websites intelligently
✓ Site-specific instructions still take priority when available
✓ Comprehensive error recovery strategies
✓ Clear "think first, act second" workflow
✓ Covers common web patterns (forms, navigation, data extraction)

The agent is no longer limited to Jira/Workday - it can now interact
with any website using general web interaction patterns.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: disable smart input-finding that was redirecting coordinate clicks
78bfde5 
Fixed issue where coordinate-based clicks were landing in wrong locations.

Problem:
- Agent tried to click "Assigned to me" at (470, 441)
- Code searched 100px radius for input fields
- Found search box nearby and redirected click to (611.2, 87.5)
- Click landed on wrong element (search input instead of link)

Root Cause:
The click handler had "smart" behavior that would:
1. Search for any input within 100px of click coordinates
2. Redirect the click to that input's center
3. Override the AI's intentional coordinate choice

This was meant to be helpful for fuzzy clicking near inputs,
but it was actually breaking precise clicks on links/buttons
that happened to be near search boxes.

Solution:
Disabled the smart input-finding entirely for coordinate clicks.
When AI provides explicit coordinates from a screenshot, we now
trust those coordinates exactly and click precisely where requested.

Impact:
- Coordinate clicks now land exactly where AI intends
- No more unexpected redirects to nearby inputs
- Clicks on links/buttons near search boxes now work correctly

Testing:
Reload extension and try clicking on elements near search inputs.
Clicks should now land precisely on the intended element.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: handle devicePixelRatio scaling for screenshot coordinates
2ab8ba9 
Fixed coordinate mismatch on high-DPI displays (Retina, 4K monitors).

Problem:
- On high-DPI displays, devicePixelRatio can be 2 or more
- Screenshot captures at physical resolution (e.g., 2560×1600px)
- Viewport uses logical resolution (e.g., 1280×800px)
- AI measured coordinates from physical screenshot pixels
- But viewport coordinates need logical pixels
- Result: clicks land at 2x the intended position (way off!)

Example with devicePixelRatio=2:
- AI measures "Assigned to me" at (940, 882) from screenshot
- Should use viewport coordinates (470, 441) for click
- Was using (940, 882) directly → click way off target

Solution:
Updated screenshot tool result message to explicitly tell AI:
"IMPORTANT: This is a high-DPI display (devicePixelRatio=2).
When measuring coordinates from this screenshot, DIVIDE by 2
to get viewport coordinates."

Now provides clear example calculation to guide the AI.

Also updated message format:
- Old: "Screenshot: 1280×800px"
- New: "Screenshot: 1280×800px viewport (image is 2560×1600px)"
- Makes the scaling explicit

Impact:
- Coordinate clicks will now be accurate on Retina/4K displays
- AI will automatically scale measurements correctly
- Works on all devicePixelRatio values (1, 1.5, 2, 3, etc.)

Testing:
Reload extension on high-DPI display and try coordinate clicking.
Coordinates should now match intended targets precisely.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: improve coordinate measurement instructions and Jira workflow
e08e989 
Made two improvements based on debugging:

1. IMPROVED COORDINATE MEASUREMENT GUIDANCE:
   - Added warning: "Be precise - off by even 20-30 pixels can miss"
   - Added: "Double-check your measurement before clicking"
   - Added: "If click misses, take new screenshot and measure again"

   Issue: AI measured (470, 441) from screenshot when correct was (494, 404)
   - Off by 24px horizontally, 37px vertically
   - This is an AI vision measurement error, not coordinate transform bug
   - Clicking at explicit coordinates works perfectly

   These instructions help AI be more careful with coordinate measurements.

2. UPDATED JIRA INSTRUCTIONS WITH TWO OPTIONS:
   OPTION 1 - Try page tabs first (faster):
   - Try clickElement({text: "Assigned to me"})
   - If fails, use coordinates as last resort

   OPTION 2 - Use Filters sidebar (most reliable):
   - Navigate to Filters in sidebar
   - Click existing filter

   Why: The agent's clickElement attempts failed:
   - clickElement({text:"Assigned to me"}) - failed
   - clickElement({selector:"button[data-testid*='assigned']"}) - failed

   Giving two options lets agent try fast method first, then fall back
   to more reliable Filters sidebar approach.

Impact:
- More accurate coordinate clicking when needed
- Better fallback strategy for Jira navigation
- Clearer guidance reduces click misses

Testing:
Reload extension and try "show my open bugs" - should work via
either tab clicking or Filters sidebar.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: add auto-save screenshots to Downloads folder
fa0d2be 
Added new feature to automatically save all screenshots to your Downloads
folder for debugging and coordinate measurement.

Changes:

1. Added "downloads" permission to manifest.json
   - Required for chrome.downloads.download API

2. Added autoSaveScreenshots setting to types.ts
   - Boolean flag to enable/disable auto-save

3. Modified background.ts screenshot handler
   - Checks if autoSaveScreenshots setting is enabled
   - Generates timestamped filename: atlas-screenshot-YYYY-MM-DDTHH-MM-SS-mmmZ.png
   - Saves to Downloads folder without prompting (saveAs: false)
   - Logs success/errors without failing screenshot operation

4. Added UI toggle in settings.tsx
   - Checkbox: "Auto-save Screenshots"
   - Help text: "📸 Automatically save all screenshots to your Downloads folder"
   - Placed below "Enable Business Services" setting

Usage:
1. Open Settings (gear icon)
2. Enable "Auto-save Screenshots" checkbox
3. Click "Save Settings"
4. All screenshots taken by the AI will now be saved to ~/Downloads/

Files saved as: atlas-screenshot-2025-11-23T06-30-15-123Z.png

Benefits:
- Debug coordinate clicking issues
- Review what AI sees in screenshots
- Keep history of automation sessions
- Measure exact coordinates for testing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: improve coordinate measurement guidance and click logging
2f62dbf 
Made coordinate clicking more precise based on user feedback that clicks
should land on the center of visible text, not button borders.

Changes:

1. UPDATED AI INSTRUCTIONS for coordinate clicking:
   - "Measure to the EXACT CENTER of the visible text/icon, not the button border"
   - "For text like 'Assigned to me': measure horizontal center of entire phrase and vertical center of the text height"
   - "Account for padding - don't measure to button edges, measure to actual text center"

   Problem: AI was measuring (470, 441) when correct was (494, 404)
   Root cause: AI likely measured to button/wrapper center instead of text center
   Solution: Explicit instructions to measure to visual text center

2. ADDED DETAILED CONSOLE LOGGING:

   When finding element by text:
   - Log actual text content found
   - Log element bounding box

   When clicking element:
   - Log element tag and class
   - Log full element bounds (left, top, width, height)
   - Log calculated click coordinates with explanation
   - Log element text content
   - Return click coordinates in response for debugging

   Example output:
   🎯 Click position calculated:
      Element: A.tab-link
      Element bounds: left=450, top=380, width=88, height=48
      Click coordinates: (494, 404) - center of element
      Element text: "Assigned to me"

3. PURPOSE:
   - Help users debug coordinate clicking issues
   - Verify clicks are centered on elements correctly
   - Identify if element bounds include excessive padding
   - Compare calculated coordinates with manual measurements

Impact:
- AI will measure coordinates more accurately
- Console logs show exactly what element was found and where click lands
- Easier to debug coordinate vs visual text mismatches

Testing:
After reload, check console when clicking to see detailed coordinate info.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: clarify coordinate measurement to center of entire control
e964266 
Corrected AI instructions based on user clarification about where to measure
coordinates for clicking.

Issue: Previous commit incorrectly told AI to measure "center of visible text"
Correct: Should measure "center of entire clickable control"

Example from Jira:
HTML: <span>Assigned to me<span>11</span></span>
- WRONG: Measure center of "Assigned to me" text only
- CORRECT: Measure center of entire span including text AND badge "11"

Updated instructions:
- "Measure to the CENTER of the ENTIRE clickable control (button/link/tab)"
- "Include all visible parts: text, badges, icons, padding"
- "For 'Assigned to me 11': measure center of the entire tab"

The code already calculates this correctly:
  const rect = element.getBoundingClientRect();  // Full element
  const clickX = rect.left + rect.width / 2;     // Center of control
  const clickY = rect.top + rect.height / 2;     // Center of control

This matches user's measurement of (494, 404) which is the center of the
entire "Assigned to me 11" tab control, not just the text.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: improve DOM-based element finding for clickElement
7249f5b 
Expanded element search to find more types of clickable elements:
- Added [role="tab"] for Jira-style navigation tabs
- Added [role="link"] for ARIA links
- Added span[onclick] and div[onclick] for clickable spans/divs
- Added comprehensive fallback to search ALL elements with matching text

This enables clickElement({text: "..."}) to find elements like:
<span role="tab">Assigned to me<span>11</span></span>

When found, uses pixel-perfect getBoundingClientRect() coordinates
instead of relying on AI vision measurement (~10-40px error).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: re-enable browser automation visual overlay
283f653 
Re-enabled the "AI Automation Active" indicator that shows:
- Floating badge in top-right corner
- Pulsing green dot animation
- "Stop" button to abort operations

This provides visual feedback when the AI is controlling the browser.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: comprehensive conversation context improvements
f9968fc 
Implemented all four requested improvements to prevent context loss:

1. **Increased Message Limits (10x improvement)**
   - Initial history: 1 → 10 messages (default)
   - Loop history: 4 → 15 messages (default)
   - Configurable via user settings

2. **Conversation Persistence**
   - Save conversations to chrome.storage.local per tab
   - Persist across browser restarts and extension reloads
   - Load persisted messages on tab switch
   - Clear persisted messages on "New Chat"
   - Default: enabled

3. **User Settings UI**
   - Added "Conversation Memory Settings" section
   - Toggle: Save Conversations (persist to storage)
   - Input: Initial Message History (1-50, default: 10)
   - Input: Loop Message History (4-50, default: 15)
   - Toggle: Smart Summarization
   - Clear help text explaining token usage implications

4. **Smart Summarization**
   - Automatically summarize old messages when approaching limits
   - Uses Claude 3.5 Haiku for fast, cheap summarization
   - Keeps 50% most recent messages intact
   - Inserts summary as context for AI
   - Only triggers when >8 messages and enabled
   - Default: enabled

**Technical Implementation:**
- Added 4 new settings fields to types.ts
- Created summarizeOldMessages() function using Claude API
- Updated streamAnthropicWithBrowserTools() to accept settings
- Modified sidepanel.tsx to persist/load from chrome.storage
- Enhanced settings.tsx with comprehensive controls

**Benefits:**
- Users can now have meaningful multi-turn conversations
- Context preserved across sessions
- Flexible configuration for power users
- Automatic optimization via summarization
- No more "AI forgets" after a few messages

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: interactive JIRA bug creation with project selection
2495014 
Updated JIRA site instructions to require user input before creating bugs:

**New Interactive Workflow:**
1. AI opens Create dialog and takes screenshot
2. AI lists available projects from dropdown
3. AI asks: "Which project should I create this in?"
4. AI asks: "What is the bug summary/title?"
5. AI optionally asks for description
6. AI creates bug with user-provided information
7. AI reports back with issue key

**Key Changes:**
- Split creation into 3 clear steps: Gather Project → Gather Details → Create
- Added CRITICAL warning: "Do NOT create any issue without first asking"
- AI will show user list of available projects from screenshot
- Ensures no bugs are created without explicit user confirmation

**User Experience:**
- User: "Create a JIRA bug"
- AI: Opens create dialog, screenshots, "Which project? [lists options]"
- User: "MyProject"
- AI: "What is the bug summary?"
- User: "Login button not working"
- AI: Creates bug and reports "Created bug PROJ-123 in MyProject: Login button not working"

This prevents accidental bug creation and gives users full control.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: JIRA search should stop immediately after clicking Assigned to me
c4c3de8 
Fixed AI continuing unnecessarily after finding assigned issues:

**Problem:**
- AI clicked "Assigned to me" ✓
- Then kept scrolling, filtering, using getPageContext ✗
- User just wanted the list, not extra actions

**Solution:**
- Added explicit STOP instructions after clicking
- Step 5: Take ONE final screenshot
- Step 6: STOP HERE - Extract and return ALL visible issues
- Added CRITICAL section:
  - DO NOT scroll
  - DO NOT try to filter further
  - DO NOT use getPageContext or other tools
  - IMMEDIATELY extract what you see and return it

**Changes:**
- Renamed to "bugs/issues assigned to me" (covers all issue types)
- Added "PREFERRED - fastest" label to page tabs option
- Added explicit 2-second wait for results to load
- Added mandatory final screenshot before extraction
- Added clear STOP boundary to prevent over-execution

Now when user asks "show my bugs", AI will:
1. Click "Assigned to me"
2. Wait 2 seconds
3. Take screenshot
4. Extract and return issues
5. DONE

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: prevent tool_use/tool_result pair breakage during message trimming
411dffc 
Fixed critical bug causing API errors during conversation trimming:

**Error:**
```
messages.0.content.0: unexpected tool_use_id found in tool_result blocks:
toolu_bdrk_01PxDwmXWytcDykVswc7FJYq. Each tool_result block must have
a corresponding tool_use block in the previous message.
```

**Root Cause:**
When trimming conversation history with `slice(-MAX_MESSAGES)`, we could:
1. Remove: Assistant message with tool_use (id: ABC)
2. Keep: User message with tool_result (tool_use_id: ABC) ← BROKEN REFERENCE

Anthropic API requires every tool_result to have its corresponding
tool_use in the immediately previous assistant message.

**Solution:**
Implemented "smart trimming" that preserves tool pairs:

1. **Initial Trim** (lines 216-237):
   - Trim to MAX_HISTORY_MESSAGES
   - Check if first message is user with tool_result
   - If yes, include one more message to get the assistant's tool_use
   - Log: "📎 Preserved tool_use/tool_result pair during initial trim"

2. **Loop Trim** (lines 617-638):
   - Trim to MAX_LOOP_MESSAGES
   - Same pair-preserving logic
   - Log: "↳ Included previous assistant message to preserve tool_use/tool_result pair"

**How It Works:**
```typescript
// Before: Simple trim (BREAKS PAIRS)
conversationMessages.slice(-15)

// After: Smart trim (PRESERVES PAIRS)
let trimmed = conversationMessages.slice(-15);
if (firstMsg.role === 'user' && hasToolResult) {
  trimmed = conversationMessages.slice(-16); // Include one more
}
```

**Testing:**
This fix ensures multi-turn tool-using conversations won't fail with
the tool_use_id error, especially during long JIRA automation sessions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: handle HTML error responses instead of crashing on JSON.parse
312580c 
Fixed crash when API returns HTML instead of JSON:

**Error:**
```
SyntaxError: Unexpected token '<', "<html> <h"... is not valid JSON
```

**Root Cause:**
When the Anthropic API (or any intermediate proxy/CDN) returns an error
as HTML instead of JSON, we tried to parse it directly:

```typescript
if (!response.ok) {
  const error = await response.json();  // ❌ Crashes if HTML
  ...
}
```

**Common scenarios returning HTML:**
- 401/403 Authentication failures → redirect to login page
- 500 Internal Server Error → HTML error page
- CORS/Network errors → browser error page
- Wrong API endpoint → 404 HTML page
- Proxy/CDN issues → intermediate error page

**Solution:**
Added try-catch around ALL JSON parsing with graceful fallbacks:

1. **Error responses** (lines 506-520):
   ```typescript
   try {
     const error = await response.json();
     errorMsg = error.error?.message || errorMsg;
   } catch (parseError) {
     const text = await response.text();
     console.error('❌ Non-JSON API Error:', text.substring(0, 200));
     errorMsg = `API Error (${response.status} ${response.statusText})`;
   }
   ```

2. **Successful responses** (lines 532-539):
   Protected against redirect/proxy edge cases

3. **Summarization** (lines 169-175):
   Gracefully falls back to original messages

**Result:**
- User sees clear error: "API Error (401 Unauthorized)"
- Instead of cryptic: "Unexpected token '<'"
- Logs first 200 chars of HTML for debugging
- No crash, conversation continues

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: scroll not working on custom scrollable containers
09a92c1 
Fixed scrolling failure on pages with custom scroll containers:

**Problem:**
Scroll commands would fail silently on many modern web apps (JIRA,
Slack, Notion, etc.) that use custom scrollable divs instead of
window scroll.

**Root Cause:**
The code used `element.scrollBy()` method which is NOT supported
on all DOM elements:

```typescript
// Before (content.ts:1054, 1062)
scrollableElement.scrollBy({ top: -scrollAmount, behavior: 'smooth' });
scrollableElement.scrollBy({ top: scrollAmount, behavior: 'smooth' });
```

**Browser Compatibility:**
- `scrollBy()` works on: Window object
- `scrollBy()` FAILS on: Most custom DIV scrollable containers
- `scrollTop` property works on: ALL scrollable elements

**Solution:**
Use different scroll methods based on element type:

```typescript
// After (content.ts:1055-1062, 1072-1080)
if (isWindow) {
  window.scrollBy({ top: scrollAmount, behavior: 'smooth' });
} else {
  // Use scrollTop for universal compatibility
  const maxScroll = scrollableElement.scrollHeight - scrollableElement.clientHeight;
  const targetScroll = Math.min(maxScroll, scrollableElement.scrollTop + scrollAmount);
  scrollableElement.scrollTop = targetScroll;
}
```

**Benefits:**
- ✅ Works on window scroll
- ✅ Works on custom div containers
- ✅ Works on overflow:auto/scroll elements
- ✅ Works on Jira, Slack, Notion, etc.
- ✅ Proper boundary checking (doesn't scroll past end)

**Testing:**
Scroll commands should now work on all pages, including SPAs with
custom scrollable containers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: add SharePoint/Office Online scroll support and comprehensive lo…
cfcf078 
…gging

Fixed scrolling not working on SharePoint and Office Online documents.

**Problem:**
Scrolling failed on SharePoint/Office Online documents because the
scroll detection couldn't find Office's custom document canvases.

From user screenshot: AI was falling back to keyboard arrow keys
instead of scroll commands, indicating scroll detection failure.

**Root Cause:**
Office Online documents use special containers that weren't detected:
- Word Online: #WACViewPanel_EditingElement
- Excel Online: #m_excelWebRenderer_ewaCtl_scrollableContainer
- Generic: [role="document"], .CanvasElement

**Solution:**

1. **Added Office Online Detection** (lines 968-989):
   ```typescript
   const officeCanvases = [
     document.querySelector('#WACViewPanel_EditingElement'),  // Word Online
     document.querySelector('.CanvasElement'),                // Office canvas
     document.querySelector('[role="document"]'),             // Generic document
     document.querySelector('#m_excelWebRenderer_ewaCtl_scrollableContainer'), // Excel
     document.querySelector('.ewaCtl_table')                  // Excel table
   ].filter(el => el !== null);
   ```

2. **Comprehensive Logging** (lines 1050-1074):
   - Log scroll target selection
   - Log current position and dimensions
   - Log actual scroll delta after scrolling
   - Warning if scroll doesn't move (detection issue)

3. **Better Error Detection**:
   ```typescript
   if (actualDelta === 0 && beforeScroll < maxScroll) {
     console.warn('⚠️ Scroll didn't move! This might indicate a scrolling issue');
   }
   ```

**Debugging Info Now Available:**
- 📍 Final scroll target details
- 🔽/🔼 Scroll attempt with before/after positions
- ⚠️ Warnings when scroll fails
- Element type, class, dimensions logged

**Testing:**
1. Reload extension
2. Open SharePoint document
3. Open DevTools Console
4. Ask to scroll down
5. Check console logs for detailed scroll info

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: add SharePoint/Office Online site-specific instructions
874dc3b 
Added comprehensive SharePoint instructions to fix AI using wrong
scroll methods.

**Problem:**
AI was avoiding the scroll command on SharePoint and instead:
- Clicking on document
- Using arrow keys repeatedly
- None of these worked

**Root Cause:**
No site-specific instructions for SharePoint, so AI had no guidance
on how to handle Office Online documents. AI likely learned from
past failures to avoid scroll and use keyboard shortcuts instead.

**Solution:**
Added SharePoint-specific instructions (*.sharepoint.com):

1. **Explicit Scroll Guidance:**
   - ✅ CORRECT: scroll({direction: "down", amount: 500})
   - ❌ WRONG: Click document, then pressKey({key: "ArrowDown"})
   - "ALWAYS use the scroll tool, not arrow keys"
   - "DO NOT click first - just use scroll directly"

2. **Error Handling:**
   - Try scroll max 2 times
   - If fails, report to user clearly
   - Don't repeatedly try arrow keys as fallback

3. **Common SharePoint Workflows:**
   - Reading documents (scroll-based navigation)
   - Navigating SharePoint sites
   - Searching documents (Ctrl+F)

**Why Arrow Keys Don't Work:**
SharePoint/Office Online documents use:
- Custom event handlers
- Virtual scroll containers
- Canvas rendering
- Arrow keys often captured by UI, not document

**Testing:**
1. Reload extension
2. Open SharePoint document
3. Ask "scroll down"
4. AI should now use scroll() directly without clicking
5. Check console for scroll detection logs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: PageDown and navigation keys not working due to wrong key codes
53d9356 
Fixed critical bug where PageDown, ArrowDown, and other navigation
keys were completely ignored by browsers.

**Problem:**
User reported: "why is PageDown keypress event not working"
All navigation keys (PageDown, ArrowDown, ArrowUp, etc.) were failing
silently with no effect.

**Root Cause:**
Line 1312 was generating WRONG key codes for special keys:

```typescript
// BEFORE (BROKEN)
code: keyToPress === 'Tab' ? 'Tab' : keyToPress === 'Escape' ? 'Escape' : `Key${keyToPress}`,
```

This generated:
- PageDown → code: 'KeyPageDown' ❌ (should be 'PageDown')
- ArrowDown → code: 'KeyArrowDown' ❌ (should be 'ArrowDown')
- Home → code: 'KeyHome' ❌ (should be 'Home')

The `Key` prefix is ONLY for letter keys (KeyA, KeyB, etc.), not
navigation keys. Browsers ignore events with malformed codes.

**Solution:**

1. **Added getKeyCode() function** (lines 1311-1345):
   - Navigation keys: ArrowUp, ArrowDown, PageUp, PageDown, Home, End
   - Special keys: Tab, Escape, Enter, Backspace, Delete, Space
   - Function keys: F1-F12
   - Letter keys: KeyA-KeyZ (with Key prefix)
   - Digit keys: Digit0-Digit9

2. **Added getKeyCodeValue() function** (lines 1348-1375):
   - Proper keyCode values for compatibility with older apps
   - PageDown: 34, ArrowDown: 40, etc.

3. **Added logging** (line 1380):
   ```
   ⌨️  Pressing key: "PageDown" (code: PageDown, keyCode: 34)
   ```

**Before/After:**

| Key | Before | After |
|-----|--------|-------|
| PageDown | code: 'KeyPageDown' ❌ | code: 'PageDown', keyCode: 34 ✅ |
| ArrowDown | code: 'KeyArrowDown' ❌ | code: 'ArrowDown', keyCode: 40 ✅ |
| Home | code: 'KeyHome' ❌ | code: 'Home', keyCode: 36 ✅ |
| A | code: 'KeyA' ✅ | code: 'KeyA', keyCode: 65 ✅ |

**Testing:**
1. Reload extension
2. Press PageDown, ArrowDown, etc.
3. Check console: "⌨️  Pressing key: ..."
4. Keys should now work properly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: filter out empty messages before sending to Anthropic API
eaa4eaf 
Filter out messages with empty content in both anthropic-service.ts and
anthropic-browser-tools.ts to prevent API errors. Anthropic API requires
all messages to have non-empty content except for the optional final
assistant message.

Handles both string and array content types, logging filtered messages
for debugging.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: add API key validation and improve error messages
c31a216 
Added validation to check for empty/missing API keys before making
API requests in both anthropic-service.ts and anthropic-browser-tools.ts.

Also improved error handling in anthropic-service.ts to handle non-JSON
error responses (HTML error pages) with better error messages.

Changes:
- Validate API key is not empty before making requests
- Improved error message: "Anthropic API key is not configured. Please add it in Settings"
- Added try-catch for JSON parsing of error responses
- Show helpful message when API returns HTML instead of JSON

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: update error message to mention GoCode Key
84d5511 
Updated API key validation error messages to mention both GoCode Key
and Anthropic API key, since the extension supports both direct Anthropic
API usage and GoDaddy's internal GoCode API.

Error message changed from:
"Anthropic API key is not configured..."

To:
"API key is not configured. Please add your GoCode Key or Anthropic API key..."

This makes it clearer for internal GoDaddy users who use GoCode.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: enforce GoCode-only API access, remove direct Anthropic option
f0760fc 
All API requests now route through GoCode endpoint only. No direct
Anthropic API access is allowed.

Changes:
- Default baseUrl to GoCode production endpoint in both anthropic-service.ts
  and anthropic-browser-tools.ts
- Updated error messages to say "GoCode Key" instead of "API key" or
  "Anthropic API key"
- Updated settings UI:
  - Placeholder: "Enter your GoCode key"
  - Added help text with link to get GoCode API key
- Updated code comments to reference GoCode instead of Anthropic

Default endpoint: https://caas-gocode-prod.caas-prod.prod.onkatana.net

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: add manual scroll fallback for synthetic keyboard events
d8c9b63 
Synthetic keyboard events have isTrusted=false and browsers block them
from triggering default actions (scrolling, navigation) for security.

Added manual scroll implementation for navigation keys:
- PageDown: Scroll one viewport height down
- PageUp: Scroll one viewport height up
- ArrowDown: Scroll 40px down
- ArrowUp: Scroll 40px up
- Home: Scroll to top
- End: Scroll to bottom

This ensures keyboard navigation works even though synthetic events
cannot trigger browser default scroll behavior.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
fix: add friendly error message for VPN connection issues
4d3dffb 
When GoCode API endpoint is unreachable (network error), show a
user-friendly message instead of generic "Failed to fetch" error.

New error message explains:
🔌 Cannot reach GoCode API endpoint.

This usually means:
• You are not connected to GoDaddy VPN
• The GoCode service is temporarily unavailable

Please connect to VPN and try again.

Changes:
- Wrapped fetch calls in try-catch for network errors
- anthropic-service.ts: Main API calls
- anthropic-browser-tools.ts: Both main API and summarization calls
- Summarization silently falls back to original messages on network error

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: update ANS API endpoint to OTE environment
caecb22 
Changed ANS discovery service URL from internal dev endpoint to
OTE (Operational Test Environment) endpoint for production readiness.

Changes:
- trusted-business-service.ts: API_URL constant updated
- settings.tsx: Updated endpoint display and sign-in URL
- README.md: Updated documentation links (2 locations)
- test-url-extraction.mjs: Updated mock data URLs (3 locations)

Old endpoint: https://ra.int.dev-godaddy.com
New endpoint: https://api.ote-godaddy.com

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: update MCP protocol extension to mcp1
3a6d459 
Changed MCP protocol discovery from protocolExtensions.mcp to
protocolExtensions.mcp1 to match updated API format.

Changes:
- trusted-business-service.ts: Look for mcp1 instead of mcp
- test-url-extraction.mjs: Updated test to match new format
- Updated diagnostic logging to show mcp1RemotesUrl

Old path: protocolExtensions.mcp.remotes[0].url
New path: protocolExtensions.mcp1.remotes[0].url

Example format:
{
  "mcp1": {
    "url": "",
    "remotes": [
      {
        "url": "https://lyrics.agenth.godaddy.com/mcp"
      }
    ]
  }
}

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vyung-godaddy @claude
feat: display MCP server URL in Available Tools panel
9f4439b 
Added serverUrl to MCP tool metadata and updated UI to show the
actual MCP server URL instead of empty parentheses.

Changes:
- types.ts: Added serverUrl field to MCPToolWithOrigin interface
- mcp-service.ts: Include serverUrl when collecting tools with origin
- sidepanel.tsx: Display tool.serverUrl instead of non-existent tool.origin

UI now shows:
🔌 MCP Tools (1)
tool_name (https://lyrics.agenth.godaddy.com/mcp)

Instead of:
tool_name ()

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@dshaikh-godaddy
include page context
520ddf0
@dshaikh-godaddy
disable screenshot processing and reprocess dom context after every e…
6620ad3 
…vent
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dshaikh-godaddy @vyung-godaddy