fix(browser-use): Enhanced error classification and anti-bot detection

- Add JavaScript execution error patterns for better retry logic
- Add dynamic content loading detection patterns
- Add comprehensive anti-bot detection patterns (Cloudflare, CAPTCHA, etc.)
- Enhance navigation watchdog with improved text-based challenge detection
- Remove duplicate challenge detection logic for better maintainability

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

Author: chrisschnabl

.agent/analysis_20250916.md

@@ -0,0 +1,64 @@
+# Browser-Use RSI Analysis - September 16, 2025
+
+## Overview
+Analysis of recent evaluation runs shows consistent 0% success rates across all runs, despite some runs having decent comprehensive scores (52-62) and self-report success rates (63-76%). This suggests fundamental issues in either the evaluation criteria or core browser automation functionality.
+
+## Key Issues Identified
+
+### 1. Evaluation System Issues
+- All recent runs show 0% success rate regardless of comprehensive scores
+- API parsing errors preventing detailed failure analysis
+- Suggests success criteria may be too strict or broken
+
+### 2. Error Handling & Classification
+Current implementation in `browser_use/tools/error_classifier.py`:
+- Good error categorization system (RETRYABLE_NETWORK, RETRYABLE_TIMING, etc.)
+- Pattern-based classification with retry strategies
+- However, may not be catching all failure patterns effectively
+
+### 3. Element Detection & Staleness
+From code analysis, potential issues:
+- Stale element references in DOM interactions
+- Element detection timeouts
+- Lack of intelligent element recovery strategies
+
+### 4. Navigation & Page Load Detection
+- Navigation timeout issues
+- Incomplete page load detection
+- Missing robust document ready monitoring
+
+## Critical Areas for Improvement
+
+### High Priority Fixes:
+1. **Element Staleness Recovery**: Implement intelligent element re-detection when elements become stale
+2. **Navigation Reliability**: Improve page load detection with multiple validation strategies
+3. **Error Recovery**: Enhance error classification to catch more edge cases
+
+### Current Error Classification Gaps:
+- Missing patterns for JavaScript execution failures
+- Limited handling of anti-bot detection
+- Insufficient Cloudflare/CAPTCHA handling
+- Missing patterns for dynamic content loading failures
+
+## Recommended Fixes
+
+### Fix 1: ElementStalenessWatchdog
+- Add watchdog to automatically re-detect stale elements
+- Implement intelligent element recovery strategies
+- Add element reference caching with refresh mechanisms
+
+### Fix 2: Enhanced Page Load Detection
+- Multi-layered page load verification (document.readyState, network idle, DOM stable)
+- Better handling of Single Page Applications (SPAs)
+- Dynamic content detection and waiting
+
+### Fix 3: Anti-Bot & Security Handling
+- Better detection of anti-bot measures
+- Cloudflare challenge detection and handling
+- CAPTCHA detection with appropriate user feedback
+
+## Implementation Strategy
+1. Focus on most common failure patterns first
+2. Implement minimal viable fixes, not overengineered solutions
+3. Ensure backward compatibility
+4. Test each fix with representative tasks before committing

browser_use/browser/watchdogs/enhanced_navigation_watchdog.py

@@ -189,6 +189,7 @@ async def _detect_and_handle_antibot(self, target_id: str, url: str) -> None:
 						let challengeFound = false;
 						let challengeType = '';
 
+						// Check by selectors
 						for (const selector of allSelectors) {
 							const element = document.querySelector(selector);
 							if (element) {
@@ -201,27 +202,31 @@ async def _detect_and_handle_antibot(self, target_id: str, url: str) -> None:
 							}
 						}
 
-						// Check for challenge text content
-						const pageText = document.body.textContent.toLowerCase();
-						const challengeTexts = [
-							'cloudflare',
-							'checking your browser',
-							'human verification',
-							'please wait while we verify',
-							'security check',
-							'press & hold',
-							'click to verify',
-							'i am human'
-						];
-
-						let textChallenge = '';
-						for (const text of challengeTexts) {
-							if (pageText.includes(text)) {
-								challengeFound = true;
-								textChallenge = text;
-								break;
+						// Additional text-based detection
+						if (!challengeFound) {
+							const bodyText = document.body.innerText.toLowerCase();
+							const challengeTexts = [
+								'checking your browser',
+								'verify you are human',
+								'complete the security check',
+								'please wait while we verify',
+								'cloudflare security challenge',
+								'ddos protection by cloudflare',
+								'ray id:',
+								'just a moment',
+								'please solve the captcha',
+								'prove you are not a robot'
+							];
+
+							for (const text of challengeTexts) {
+								if (bodyText.includes(text)) {
+									challengeFound = true;
+									challengeType = 'text:' + text;
+									break;
+								}
 							}
 						}
+						}
 
 						return {
 							challengeFound,

browser_use/tools/error_classifier.py

@@ -133,6 +133,43 @@ def __init__(self):
 			(r'incompatible.*version', "Incompatible version")
 		]
 
+		# JavaScript execution error patterns (retryable with DOM refresh)
+		self.javascript_patterns = [
+			(r'javascript.*error', "JavaScript execution error"),
+			(r'script.*error', "Script execution failed"),
+			(r'uncaught.*exception', "Uncaught JavaScript exception"),
+			(r'reference.*error', "JavaScript reference error"),
+			(r'type.*error.*javascript', "JavaScript type error"),
+			(r'cannot.*read.*propert', "JavaScript property access error"),
+			(r'function.*not.*defined', "JavaScript function not defined")
+		]
+
+		# Anti-bot and security challenge patterns
+		self.antibot_patterns = [
+			(r'cloudflare.*challenge', "Cloudflare security challenge"),
+			(r'captcha.*required', "CAPTCHA challenge detected"),
+			(r'recaptcha.*challenge', "reCAPTCHA challenge detected"),
+			(r'bot.*detect', "Bot detection triggered"),
+			(r'rate.*limit.*exceed', "Rate limiting activated"),
+			(r'suspicious.*activity', "Suspicious activity detected"),
+			(r'please.*verify.*human', "Human verification required"),
+			(r'security.*check.*required', "Security verification required"),
+			(r'access.*denied.*bot', "Bot access denied"),
+			(r'automated.*traffic.*detect', "Automated traffic detected")
+		]
+
+		# Dynamic content loading patterns
+		self.dynamic_content_patterns = [
+			(r'content.*still.*loading', "Dynamic content still loading"),
+			(r'ajax.*request.*pending', "AJAX request in progress"),
+			(r'react.*component.*mounting', "React component mounting"),
+			(r'spa.*router.*navigating', "SPA navigation in progress"),
+			(r'virtual.*dom.*updating', "Virtual DOM update in progress"),
+			(r'lazy.*load.*pending', "Lazy loading in progress"),
+			(r'infinite.*scroll.*loading', "Infinite scroll loading"),
+			(r'skeleton.*loader.*active', "Skeleton loader active")
+		]
+
 	def classify_error(
 		self,
 		error: Exception,
@@ -196,6 +233,30 @@ def classify_error(
 					technical_details=error_str
 				)
 
+		# JavaScript execution errors - retry with DOM refresh
+		for pattern, description in self.javascript_patterns:
+			if re.search(pattern, error_str, re.IGNORECASE):
+				return ErrorClassificationResult(
+					category=ErrorCategory.RETRYABLE_TIMING,
+					should_retry=True,
+					retry_delay=1.0,
+					max_retries=2,
+					user_message=f"JavaScript error: {description}. Waiting for page to stabilize and retrying...",
+					technical_details=error_str
+				)
+
+		# Dynamic content loading - retry with longer wait
+		for pattern, description in self.dynamic_content_patterns:
+			if re.search(pattern, error_str, re.IGNORECASE):
+				return ErrorClassificationResult(
+					category=ErrorCategory.RETRYABLE_TIMING,
+					should_retry=True,
+					retry_delay=3.0,
+					max_retries=2,
+					user_message=f"Dynamic content loading: {description}. Waiting for content to load...",
+					technical_details=error_str
+				)
+
 		# Check for permanent error patterns
 
 		# Invalid input - don't retry
@@ -238,6 +299,16 @@ def classify_error(
 					technical_details=error_str
 				)
 
+		# Anti-bot detection - don't retry (needs human intervention)
+		for pattern, description in self.antibot_patterns:
+			if re.search(pattern, error_str, re.IGNORECASE):
+				return ErrorClassificationResult(
+					category=ErrorCategory.PERMANENT_ACCESS_DENIED,
+					should_retry=False,
+					user_message=f"Anti-bot protection: {description}. Manual intervention required.",
+					technical_details=error_str
+				)
+
 		# Special handling for specific exception types
 		if isinstance(error, TimeoutError):
 			return ErrorClassificationResult(