AutoGPT Recursive Self-Analysis Loop Detection (#149)

MAVRICK-1 · web-flow · commit c1fe8191431c · 2025-09-03T10:45:39.000-05:00
* added cre

* added cre

* added cre
diff --git a/rules/cre-2025-0200/autogpt-recursive-self-analysis-loop.yaml b/rules/cre-2025-0200/autogpt-recursive-self-analysis-loop.yaml
@@ -0,0 +1,68 @@
+rules:
+- cre:
+    id: CRE-2025-0200
+    severity: 0
+    title: AutoGPT Recursive Self-Analysis Loop Leading to Token Exhaustion and System Crash
+    category: infinite-loop-problem
+    author: prequel
+    description: |
+      - AutoGPT enters an infinite recursive loop when attempting to analyze and fix its own execution errors
+      - The agent repeatedly tries to debug its own code, spawning new analysis tasks for each failure
+      - Each iteration consumes API tokens and memory, eventually exhausting resources
+      - The loop accelerates as error messages grow longer, consuming tokens exponentially
+      - System becomes unresponsive and crashes with out-of-memory errors or API rate limit failures
+    cause: |
+      - AutoGPT's autonomous reasoning incorrectly identifies its own execution as a problem to solve
+      - Lack of loop detection mechanisms allows unlimited recursive task spawning
+      - Error context accumulation causes exponential growth in prompt size
+      - Missing safeguards for self-referential task creation
+      - Insufficient resource monitoring and circuit breakers for runaway processes
+    tags:
+      - autogpt
+      - infinite-loop
+      - token-exhaustion
+      - autonomous-agents
+      - llm
+      - openai
+      - recursive-analysis
+      - critical-failure
+      - memory-exhaustion
+      - crash-loop
+      - rate-limiting
+    mitigation: |
+      - Implement loop detection to identify and break recursive self-analysis patterns
+      - Add resource consumption thresholds (tokens, memory, API calls) with automatic shutdown
+      - Create task depth limits to prevent unlimited recursion
+      - Implement circuit breakers that trigger after repeated similar failures
+      - Add explicit blacklist for self-referential task creation
+      - Monitor token usage rate and implement exponential backoff
+      - Use separate monitoring process to detect and kill runaway AutoGPT instances
+      - Implement task deduplication to prevent identical recursive operations
+    references:
+      - https://github.com/Significant-Gravitas/AutoGPT/issues/1994
+      - https://github.com/Significant-Gravitas/AutoGPT/issues/3766
+      - https://github.com/Significant-Gravitas/AutoGPT/issues/1543
+      - https://jina.ai/news/auto-gpt-unmasked-hype-hard-truths-production-pitfalls/
+    applications:
+      - name: autogpt
+        version: ">=0.3.0"
+      - name: openai
+        version: ">=0.27.0"
+    impact: Complete system failure with resource exhaustion, potential financial losses from API overconsumption
+    impactScore: 9
+    mitigationScore: 3
+    reports: 15
+  metadata:
+    kind: prequel
+    id: 8qy5Et9NbNGgGxhBP7umKa
+    gen: 1
+  rule:
+    set:
+      window: 30s
+      event:
+        source: cre.log.autogpt
+      match:
+        - value: 'Entering recursive analysis loop'
+        - value: 'COMMAND = analyze_code'
+        - value: 'recursion depth'
+        - value: 'RecursionError: maximum recursion depth exceeded'
diff --git a/rules/cre-2025-0200/test.log b/rules/cre-2025-0200/test.log
@@ -0,0 +1,35 @@
+2025-08-31 14:23:45.234 [INFO] [autogpt.main] Starting AutoGPT v0.5.1 with goal: "Optimize my Python code for better performance"
+2025-08-31 14:23:45.567 [INFO] [autogpt.llm] Initializing OpenAI API client with model gpt-4
+2025-08-31 14:23:46.102 [INFO] [autogpt.agent] Agent initialized with memory backend: LocalCache
+2025-08-31 14:23:47.234 [INFO] [autogpt.agent] COMMAND = analyze_code args: {"code": "def slow_function():\\n    result = []\\n    for i in range(1000000):\\n        result.append(i**2)\\n    return result"}
+2025-08-31 14:23:48.567 [ERROR] [autogpt.commands] Error executing analyze_code: JSONDecodeError in response
+2025-08-31 14:23:48.890 [WARN] [autogpt.agent] Entering recursive analysis loop to debug previous error
+2025-08-31 14:23:49.234 [INFO] [autogpt.agent] THOUGHTS: Previous command failed, need to analyze what went wrong
+2025-08-31 14:23:49.567 [INFO] [autogpt.agent] NEXT ACTION: COMMAND = analyze_code args: {"code": "analyze_code function from autogpt/commands/analyze_code.py", "recursion depth": 1}
+2025-08-31 14:23:50.123 [DEBUG] [autogpt.memory] Storing error context, current size: 2.3MB
+2025-08-31 14:23:50.890 [ERROR] [autogpt.commands] Error executing analyze_code: Cannot analyze own execution context
+2025-08-31 14:23:51.234 [WARN] [autogpt.agent] Thinking... need to fix my own error handling
+2025-08-31 14:23:51.678 [INFO] [autogpt.agent] COMMAND = analyze_code args: {"code": "autogpt error handling module", "recursion depth": 2}
+2025-08-31 14:23:52.345 [DEBUG] [autogpt.memory] Memory usage increasing: 5.7MB, token count: 8234
+2025-08-31 14:23:52.890 [ERROR] [autogpt.llm] API request failed: context length exceeded
+2025-08-31 14:23:53.234 [INFO] [autogpt.agent] SYSTEM: Potential loop detected but continuing to resolve errors
+2025-08-31 14:23:53.567 [INFO] [autogpt.agent] THOUGHTS: Error analyzing previous attempt, need deeper analysis
+2025-08-31 14:23:54.012 [INFO] [autogpt.agent] NEXT ACTION: COMMAND = analyze_code args: {"code": "full autogpt execution trace", "recursion depth": 3}
+2025-08-31 14:23:54.678 [WARN] [autogpt.monitor] Task queue growing: 12 pending tasks
+2025-08-31 14:23:55.234 [DEBUG] [autogpt.memory] Memory usage: 12.4MB, token count: 15672
+2025-08-31 14:23:55.890 [ERROR] [autogpt.commands] RecursionError: maximum recursion depth exceeded while calling analyze_code
+2025-08-31 14:23:56.345 [CRITICAL] [autogpt.agent] Task queue overflow: 47 pending recursive tasks
+2025-08-31 14:23:56.789 [INFO] [autogpt.agent] COMMAND = analyze_code args: {"code": "recursion error in analyze_code", "recursion depth": 4}
+2025-08-31 14:23:57.234 [ERROR] [autogpt.llm] openai.error.RateLimitError: Rate limit reached for gpt-4 in organization
+2025-08-31 14:23:57.567 [WARN] [autogpt.monitor] Token consumption rate: 2341 tokens/second
+2025-08-31 14:23:58.012 [DEBUG] [autogpt.memory] Memory usage critical: 45.8MB, token count: 42318
+2025-08-31 14:23:58.456 [ERROR] [autogpt.agent] Too many pending tasks: 89 in queue
+2025-08-31 14:23:58.890 [INFO] [autogpt.agent] THOUGHTS: Still analyzing previous errors, must understand the recursion
+2025-08-31 14:23:59.234 [INFO] [autogpt.agent] NEXT ACTION: COMMAND = analyze_code args: {"code": "entire autogpt error stack", "recursion depth": 5}
+2025-08-31 14:23:59.678 [CRITICAL] [autogpt.monitor] JavaScript heap out of memory
+2025-08-31 14:24:00.123 [ERROR] [autogpt.memory] MemoryError: Cannot allocate memory for context storage
+2025-08-31 14:24:00.456 [CRITICAL] [autogpt.agent] Task buffer exceeded: 156 recursive analyze_code calls pending
+2025-08-31 14:24:00.789 [ERROR] [autogpt.llm] API rate limit exceeded: 429 Too Many Requests
+2025-08-31 14:24:01.123 [FATAL] [autogpt.main] AutoGPT crashed: Unrecoverable recursive loop detected
+2025-08-31 14:24:01.234 [INFO] [autogpt.cleanup] Emergency shutdown initiated
+2025-08-31 14:24:01.345 [ERROR] [autogpt.cleanup] Failed to save state: Out of memory
diff --git a/rules/tags/categories.yaml b/rules/tags/categories.yaml
@@ -244,6 +244,12 @@ categories:
     description: |
       Failures that prevent MongoDB from starting successfully due to corrupted metadata, invalid configurations,
       or unrecoverable internal errors (e.g., WiredTiger metadata corruption). These failures often require manual repair or backup restoration.
+  - name: infinite-loop-problem
+    displayName: Infinite Loop Problems
+    description: |
+      Problems where applications enter infinite loops causing resource exhaustion, system hangs, or crashes.
+      This includes recursive loops, circular dependencies, and runaway processes that consume CPU, memory,
+      or API resources without termination conditions.
   - name: workflow-automation-problem
     displayName: Workflow Automation Problems
     description: Problems related to workflow automation platforms including silent data loss, execution failures, and data integrity issues in workflow processing systems
diff --git a/rules/tags/tags.yaml b/rules/tags/tags.yaml
@@ -845,6 +845,27 @@ tags:
   - name: cluster-scaling
     displayName: Cluster Scaling
     description: Problems related to Kubernetes cluster scaling operations and capacity management
+  - name: autogpt
+    displayName: AutoGPT
+    description: Problems related to AutoGPT autonomous AI agent framework
+  - name: infinite-loop
+    displayName: Infinite Loop
+    description: Problems where code enters infinite loops causing resource exhaustion or system hangs
+  - name: token-exhaustion
+    displayName: Token Exhaustion
+    description: Problems where LLM API token limits are exceeded causing service failures
+  - name: autonomous-agents
+    displayName: Autonomous Agents
+    description: Problems related to autonomous AI agents that chain LLM reasoning with real-world actions
+  - name: llm
+    displayName: LLM
+    description: Problems related to Large Language Models and their API integrations
+  - name: openai
+    displayName: OpenAI
+    description: Problems related to OpenAI API services including GPT models
+  - name: recursive-analysis
+    displayName: Recursive Analysis
+    description: Problems where systems enter recursive self-analysis loops leading to resource exhaustion
   - name: n8n
     displayName: N8N
     description: Problems related to n8n workflow automation platform
@@ -859,4 +880,4 @@ tags:
     description: Issues that have severe impact on production systems and require immediate attention
   - name: data-integrity
     displayName: Data Integrity
-    description: Problems that affect the completeness, accuracy, or consistency of data
+    description: Problems that affect the completeness, accuracy, or consistency of data
