Add integrated web application preview with element selection for AI context

[SannidhyaSah]

What specific problem does this solve?

Web developers using Roo Code face significant productivity barriers when working on web applications due to constant context-switching and communication difficulties with the AI assistant.

Who is affected: All web developers using Roo Code for frontend development, UI/UX work, and responsive design testing

When this happens: Every time developers need to:

• Test visual changes or debug UI issues
• Describe specific UI elements or problems to the AI
• Test responsive design across different screen sizes
• Iterate on visual components

Current behavior: Developers must constantly switch between the IDE and external browsers, manually describe UI elements through text or screenshots, and manage multiple browser windows for responsive testing

Expected behavior: Developers should be able to preview their web applications directly within the IDE and click on any UI element to automatically provide that element's context to the AI assistant

Impact:

• Time waste: 5-10 minutes per development cycle lost to context-switching between IDE and browser
• Communication friction: Difficulty accurately describing visual bugs or desired changes to AI, leading to misunderstandings and multiple iteration cycles
• Responsive testing overhead: Manual window resizing and management across multiple browser instances
• Reduced development flow: Constant interruption of the coding mindset due to external tool switching

Additional context

This feature is inspired by Windsurf IDE's "Windsurf Previews" functionality, which provides:

• Live web application preview embedded directly in the IDE
• Click-to-select functionality that automatically sends the clicked element's context to the AI assistant
• Seamless integration with the development workflow

Research source: https://docs.codeium.com/windsurf/previews

---

🛠️ Contributing & Technical Analysis

✅ I'm interested in implementing this feature
✅ I understand this needs approval before implementation begins

How should this be solved?

Create an integrated web preview panel within Roo Code that displays live web applications and enables element selection for AI context. The solution should include:

Core Components:

1. Preview Panel: An embedded webview/iframe that displays the running web application
2. Element Inspector: Click-to-select functionality that captures element context (HTML, CSS, position, etc.)
3. AI Integration: Automatic context injection when elements are selected
4. Responsive Controls: Built-in device simulation and viewport controls

User Interaction Flow:

1. Developer starts local development server
2. Preview panel automatically detects and displays the application
3. Developer clicks on any UI element in the preview
4. Element context is automatically captured and sent to AI assistant
5. Developer can provide instructions referencing the selected element

Technical Implementation:

• Embed a webview component within the VSCode extension
• Implement element selection overlay with DOM inspection
• Create context extraction system for clicked elements
• Integrate with existing AI prompt system

How will we know it works? (Acceptance Criteria)

`
Given I have a web application running locally
When I open the preview panel in Roo Code
Then I see my application displayed within the IDE
And the preview updates automatically when I save changes

Given I see my application in the preview panel
When I click on any UI element (button, div, text, etc.)
Then that element is highlighted with a selection indicator
And the element's context (HTML, CSS, position) is captured
And this context is automatically added to my next AI prompt

Given I have selected an element in the preview
When I type a message to the AI assistant
Then my message includes the selected element's context automatically
And I can reference "this element" or "the selected button" in my instructions
And the AI understands exactly which element I'm referring to

Given I'm testing responsive design
When I use the preview panel's device controls
Then I can simulate different screen sizes and devices
And element selection works consistently across all viewport sizes
But the preview doesn't interfere with my code editing workflow
`

Technical considerations

• VSCode Extension Architecture: Will need to extend the webview capabilities and integrate with the existing AI prompt system
• Performance: Preview should not significantly impact IDE performance, especially with large applications
• Security: Ensure safe handling of local development servers and user code execution
• Cross-platform Compatibility: Must work consistently across Windows, macOS, and Linux
• Framework Agnostic: Should work with React, Vue, Angular, vanilla HTML, and other web frameworks

Trade-offs and risks

Alternative Approaches:

• External browser integration: Could integrate with existing browsers via browser extensions, but this maintains the context-switching problem
• Screenshot-based selection: Could use image analysis instead of DOM inspection, but this would be less accurate and provide limited context

Potential Risks:

• Performance impact: Embedding a full browser engine could slow down the IDE
• Security concerns: Running user code within the IDE context requires careful sandboxing
• Compatibility issues: Different local development setups might not work consistently

Breaking Changes:

• None expected - this is a new feature addition
• Existing workflows remain unchanged

Edge Cases:

• Applications with complex authentication flows
• Single Page Applications with dynamic routing
• Applications using WebSockets or real-time features
• Development servers with custom configurations or HTTPS requirements

[roomote]

I am implementing this integrated web preview feature with element selection for AI context. The implementation will include:

• Embedded webview panel for live application preview within VSCode
• Click-to-select element functionality with DOM inspection
• Automatic context extraction and injection into AI prompts
• Responsive design controls with device simulation
• Framework-agnostic support for all web technologies

I will analyze the existing codebase structure and create a comprehensive implementation following the acceptance criteria outlined in the issue.

[roomote]

I am working on implementing the integrated web preview feature with element selection for AI context. This implementation will include:

• Embedded Webview Panel: Live preview of web applications directly within VSCode
• Element Selection: Click-to-select functionality with DOM inspection and context extraction
• AI Context Integration: Automatic injection of selected element details into AI prompts
• Responsive Testing: Built-in viewport controls and device simulation capabilities

I will analyze the existing codebase architecture, particularly the webview infrastructure and AI integration points, to ensure a seamless implementation that follows the acceptance criteria.