Skip to content

Add rule for prompt injection memory exhaustion vulnerabilities #145

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add rule for prompt injection memory exhaustion vulnerabilities #145

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b3b9d32
Add new category for prompt injection memory exhaustion vulnerabilities
dhvll Aug 31, 2025
435c0e5
fix tags
dhvll Aug 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions rules/cre-2025-0150/autogpt-prompt-injection-memory-exhaustion.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
rules:
- metadata:
kind: prequel
id: 5K8mNpQr8vTzHJsEXwGcM
version: "1.0.0"
cre:
id: CRE-2025-0100
severity: 0
title: "AutoGPT Prompt Injection + Memory Exhaustion Cascade"
category: "prompt-injection-memory-exhaustion"
author: "Security Researcher"
description: |
AutoGPT is vulnerable to a sophisticated prompt injection attack that triggers a memory exhaustion cascade,
leading to complete system failure. This vulnerability exploits the interaction between prompt compression logic,
LLM retry mechanisms, and task execution loops.

The vulnerability stems from three interconnected failure modes:
1. Prompt Compression Infinite Loop: The compress_prompt function can enter an infinite loop during middle-out deletion
2. LLM Retry Memory Accumulation: Retry mechanisms accumulate memory during failed API calls without proper cleanup
3. Task Execution Loop Exploitation: Main execution loop can be exploited to create recursive task generation

A malicious user can craft a prompt that triggers the prompt compression algorithm to enter an infinite loop,
causes LLM API failures that accumulate memory during retries, and exploits the task execution loop to generate
recursive tasks, resulting in memory exhaustion and system crash (SIGKILL).
cause: |
ROOT CAUSES:
- Prompt compression algorithm in backend/util/prompt.py lacks proper loop detection and timeout mechanisms
- LLM retry mechanism in backend/blocks/llm.py accumulates memory during failed API calls without cleanup
- Task execution loop in backend/executor/manager.py can be exploited for recursive task generation
- Insufficient input validation for prompt size and structure
- Missing circuit breakers for resource exhaustion scenarios
impact: |
BUSINESS IMPACT:
- CRITICAL: Complete system failure with AutoGPT process crash (exit code 137)
- Memory exhaustion causing OOM killer termination
- Service outage affecting all running agents and tasks
- Data loss of in-progress tasks and agent states
- Resource exhaustion consuming CPU and memory until system failure
- Potential cascading failures across integrated systems
impactScore: 10
mitigation: |
IMMEDIATE ACTIONS:
- Set strict memory limits on AutoGPT containers (max 2GB per container)
- Implement input validation for prompt size and structure
- Reduce LLM retry attempts and implement exponential backoff
- Deploy real-time resource monitoring and alerting

RECOVERY ACTIONS (15-60 minutes):
- Restart AutoGPT services with memory limits
- Implement prompt compression timeout mechanisms
- Add proper memory cleanup in retry mechanisms
- Deploy circuit breakers for resource exhaustion scenarios

PREVENTION STRATEGIES:
- Implement loop detection and timeout mechanisms in prompt compression
- Add task structure validation to prevent recursive loops
- Deploy comprehensive monitoring for memory usage patterns
- Implement rate limiting and request size validation
mitigationScore: 7
references:
- "https://github.com/Significant-Gravitas/AutoGPT/blob/main/autogpt_platform/backend/backend/util/prompt.py"
- "https://github.com/Significant-Gravitas/AutoGPT/blob/main/autogpt_platform/backend/backend/blocks/llm.py"
- "https://github.com/Significant-Gravitas/AutoGPT/blob/main/autogpt_platform/backend/backend/executor/manager.py"
applications:
- name: "AutoGPT"
version: ">=0.4.0"
containerName: "autogpt-backend"
tags:
- autogpt
- prompt-injection
- memory-exhaustion
- crash
- sigkill
- oom-killer
- recursive-tasks
- prompt-compression
- llm-retry
- critical-failure
reports: 1
rule:
sequence:
window: "300s"
event:
source: application-logs
origin: true
order:
- prompt_compression_errors
- memory_pressure_indicators
- llm_retry_failures
- system_crash
negate:
- normal_operation
- graceful_shutdown

terms:
prompt_compression_errors:
field: message
regex: "(?i)(compress_prompt.*exceeds budget|prompt.*compression.*triggered|token.*count.*exceeded)"
count: 2

memory_pressure_indicators:
field: message
regex: "(?i)(memory.*usage.*exceeded|memory.*accumulation.*retry|memory.*exhaustion)"
count: 3

llm_retry_failures:
field: message
regex: "(?i)(LLM.*API.*call.*failed.*retrying|maximum.*retry.*attempts.*exceeded)"
count: 2

system_crash:
field: message
regex: "(?i)(process.*killed.*signal.*9|SIGKILL|OOM.*killer.*activated|container.*crashed)"
count: 1

normal_operation:
field: message
regex: "(?i)(started.*successfully|ready.*accept.*connections|health.*check.*passed)"

graceful_shutdown:
field: message
regex: "(?i)(graceful.*shutdown|shutting.*down.*normally|clean.*exit)"
100 changes: 100 additions & 0 deletions rules/cre-2025-0150/test.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
2025-08-31T10:17:12Z [INFO] * Serving Flask app 'vulnerable_server'
2025-08-31T10:17:12Z [INFO] * Debug mode: on
2025-08-31T10:17:12Z [INFO] * Tip: There are .env files present. Install python-dotenv to use them.
2025-08-31T10:17:12Z [WARNING] INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
2025-08-31T10:17:12Z [INFO] * Running on all addresses (0.0.0.0)
2025-08-31T10:17:12Z [INFO] * Running on http://127.0.0.1:8000
2025-08-31T10:17:12Z [INFO] * Running on http://172.18.0.7:8000
2025-08-31T10:17:12Z [INFO] INFO:werkzeug:Press CTRL+C to quit
2025-08-31T10:17:12Z [INFO] INFO:werkzeug: * Restarting with stat
2025-08-31T10:17:12Z [INFO] * Tip: There are .env files present. Install python-dotenv to use them.
2025-08-31T10:17:12Z [WARNING] WARNING:werkzeug: * Debugger is active!
2025-08-31T10:17:12Z [INFO] INFO:werkzeug: * Debugger PIN: 312-761-273
2025-08-31T10:17:12Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:17:12] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:17:17Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:17:17] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:17:22Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:17:22] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:17:27Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:17:27] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:17:32Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:17:32] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:17:37Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:17:37] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:17:42Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:17:42] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:17:47Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:17:47] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:17:52Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:17:52] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:17:57Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:17:57] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:18:02Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:18:02] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:18:07Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:18:07] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:18:08Z [INFO] INFO:werkzeug:172.18.0.1 - - [31/Aug/2025 10:18:08] "GET /health HTTP/1.1" 200 -
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Recursive content detected - vulnerability trigger 1
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Memory exhaustion keywords detected - vulnerability trigger 3
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Prompt compression triggered - attempting to compress tokens
2025-08-31T10:18:08Z [ERROR] ERROR:vulnerable_server:compress_prompt: prompt still exceeds budget (120,000 > 100,000)
2025-08-31T10:18:08Z [ERROR] ERROR:vulnerable_server:LLM API call failed, retrying... (attempt 1/5)
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Memory accumulation during retry cycle - current usage: 200MB
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Allocated 209715200 bytes, total chunks: 1
2025-08-31T10:18:08Z [ERROR] ERROR:vulnerable_server:LLM API call failed, retrying... (attempt 2/5)
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Memory accumulation during retry cycle - current usage: 400MB
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Allocated 209715200 bytes, total chunks: 2
2025-08-31T10:18:08Z [ERROR] ERROR:vulnerable_server:LLM API call failed, retrying... (attempt 3/5)
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Memory accumulation during retry cycle - current usage: 600MB
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Allocated 209715200 bytes, total chunks: 3
2025-08-31T10:18:08Z [ERROR] ERROR:vulnerable_server:LLM API call failed, retrying... (attempt 4/5)
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Memory accumulation during retry cycle - current usage: 800MB
2025-08-31T10:18:08Z [WARNING] WARNING:vulnerable_server:Allocated 209715200 bytes, total chunks: 4
2025-08-31T10:18:12Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:18:12] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:18:12Z [ERROR] ERROR:vulnerable_server:LLM API call failed, retrying... (attempt 5/5)
2025-08-31T10:18:12Z [WARNING] WARNING:vulnerable_server:Memory accumulation during retry cycle - current usage: 1000MB
2025-08-31T10:18:12Z [WARNING] WARNING:vulnerable_server:Allocated 209715200 bytes, total chunks: 5
2025-08-31T10:18:12Z [CRITICAL] CRITICAL:vulnerable_server:Memory usage exceeded threshold: 95%
2025-08-31T10:18:12Z [CRITICAL] CRITICAL:vulnerable_server:OOM killer activated - terminating AutoGPT process
2025-08-31T10:18:12Z [ERROR] ERROR:vulnerable_server:Process killed by signal 9 (SIGKILL)
2025-08-31T10:18:12Z [INFO] * Tip: There are .env files present. Install python-dotenv to use them.
2025-08-31T10:18:12Z [INFO] * Serving Flask app 'vulnerable_server'
2025-08-31T10:18:12Z [INFO] * Debug mode: on
2025-08-31T10:18:12Z [WARNING] INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
2025-08-31T10:18:12Z [INFO] * Running on all addresses (0.0.0.0)
2025-08-31T10:18:12Z [INFO] * Running on http://127.0.0.1:8000
2025-08-31T10:18:12Z [INFO] * Running on http://172.18.0.7:8000
2025-08-31T10:18:12Z [INFO] INFO:werkzeug:Press CTRL+C to quit
2025-08-31T10:18:12Z [INFO] INFO:werkzeug: * Restarting with stat
2025-08-31T10:18:12Z [INFO] * Tip: There are .env files present. Install python-dotenv to use them.
2025-08-31T10:18:12Z [WARNING] WARNING:werkzeug: * Debugger is active!
2025-08-31T10:18:12Z [INFO] INFO:werkzeug: * Debugger PIN: 803-379-536
2025-08-31T10:18:17Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:18:17] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:18:17Z [WARNING] WARNING:vulnerable_server:Recursive content detected - vulnerability trigger 1
2025-08-31T10:18:17Z [WARNING] WARNING:vulnerable_server:Memory exhaustion keywords detected - vulnerability trigger 3
2025-08-31T10:18:17Z [WARNING] WARNING:vulnerable_server:Prompt compression triggered - attempting to compress tokens
2025-08-31T10:18:17Z [ERROR] ERROR:vulnerable_server:compress_prompt: prompt still exceeds budget (120,000 > 100,000)
2025-08-31T10:18:17Z [ERROR] ERROR:vulnerable_server:LLM API call failed, retrying... (attempt 1/5)
2025-08-31T10:18:17Z [WARNING] WARNING:vulnerable_server:Memory accumulation during retry cycle - current usage: 200MB
2025-08-31T10:18:17Z [WARNING] WARNING:vulnerable_server:Allocated 209715200 bytes, total chunks: 1
2025-08-31T10:18:17Z [ERROR] ERROR:vulnerable_server:LLM API call failed, retrying... (attempt 2/5)
2025-08-31T10:18:17Z [WARNING] WARNING:vulnerable_server:Memory accumulation during retry cycle - current usage: 400MB
2025-08-31T10:18:17Z [WARNING] WARNING:vulnerable_server:Allocated 209715200 bytes, total chunks: 2
2025-08-31T10:18:22Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:18:22] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:18:22Z [ERROR] ERROR:vulnerable_server:LLM API call failed, retrying... (attempt 3/5)
2025-08-31T10:18:22Z [WARNING] WARNING:vulnerable_server:Memory accumulation during retry cycle - current usage: 600MB
2025-08-31T10:18:22Z [WARNING] WARNING:vulnerable_server:Allocated 209715200 bytes, total chunks: 3
2025-08-31T10:18:22Z [ERROR] ERROR:vulnerable_server:LLM API call failed, retrying... (attempt 4/5)
2025-08-31T10:18:22Z [WARNING] WARNING:vulnerable_server:Memory accumulation during retry cycle - current usage: 800MB
2025-08-31T10:18:22Z [WARNING] WARNING:vulnerable_server:Allocated 209715200 bytes, total chunks: 4
2025-08-31T10:18:25Z [INFO] INFO:werkzeug:172.18.0.1 - - [31/Aug/2025 10:18:25] "GET /health HTTP/1.1" 200 -
2025-08-31T10:18:25Z [ERROR] ERROR:vulnerable_server:LLM API call failed, retrying... (attempt 5/5)
2025-08-31T10:18:25Z [WARNING] WARNING:vulnerable_server:Memory accumulation during retry cycle - current usage: 1000MB
2025-08-31T10:18:25Z [WARNING] WARNING:vulnerable_server:Allocated 209715200 bytes, total chunks: 5
2025-08-31T10:18:25Z [CRITICAL] CRITICAL:vulnerable_server:Memory usage exceeded threshold: 95%
2025-08-31T10:18:25Z [CRITICAL] CRITICAL:vulnerable_server:OOM killer activated - terminating AutoGPT process
2025-08-31T10:18:25Z [ERROR] ERROR:vulnerable_server:Process killed by signal 9 (SIGKILL)
2025-08-31T10:18:25Z [INFO] * Tip: There are .env files present. Install python-dotenv to use them.
2025-08-31T10:18:25Z [INFO] * Serving Flask app 'vulnerable_server'
2025-08-31T10:18:25Z [INFO] * Debug mode: on
2025-08-31T10:18:25Z [WARNING] INFO:werkzeug:WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
2025-08-31T10:18:25Z [INFO] * Running on all addresses (0.0.0.0)
2025-08-31T10:18:25Z [INFO] * Running on http://127.0.0.1:8000
2025-08-31T10:18:25Z [INFO] * Running on http://172.18.0.7:8000
2025-08-31T10:18:25Z [INFO] INFO:werkzeug:Press CTRL+C to quit
2025-08-31T10:18:25Z [INFO] INFO:werkzeug: * Restarting with stat
2025-08-31T10:18:25Z [INFO] * Tip: There are .env files present. Install python-dotenv to use them.
2025-08-31T10:18:25Z [WARNING] WARNING:werkzeug: * Debugger is active!
2025-08-31T10:18:25Z [INFO] INFO:werkzeug: * Debugger PIN: 142-012-072
2025-08-31T10:18:32Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:18:32] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:18:37Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:18:37] "GET /metrics HTTP/1.1" 404 -
2025-08-31T10:18:42Z [INFO] INFO:werkzeug:172.18.0.4 - - [31/Aug/2025 10:18:42] "GET /metrics HTTP/1.1" 404 -
3 changes: 3 additions & 0 deletions rules/tags/categories.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,9 @@ categories:
- name: configuration-problem
displayName: Configuration Problem
description: Problems related to system or application configurations
- name: prompt-injection-memory-exhaustion
displayName: Prompt Injection Memory Exhaustion
description: Security vulnerabilities related to prompt injection attacks causing memory exhaustion and system failures
- name: monitoring-problem
displayName: Monitoring Problem
description: Problems related to system or application monitoring
Expand Down
40 changes: 30 additions & 10 deletions rules/tags/tags.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,34 @@ tags:
description: Failures involving the OVSDB (Open vSwitch Database) protocol or schema.
- name: panic
displayName: Panic
description: Crashes due to unrecoverable errors, especially in Go or Rust applications.
description: Application panics and unrecoverable runtime errors
- name: autogpt
displayName: AutoGPT
description: Problems with AutoGPT autonomous agent framework
- name: prompt-injection
displayName: Prompt Injection
description: Security vulnerabilities related to prompt injection attacks
- name: memory-exhaustion
displayName: Memory Exhaustion
description: Problems with memory exhaustion and resource depletion
- name: sigkill
displayName: SIGKILL
description: Process termination by SIGKILL signal
- name: oom-killer
displayName: OOM Killer
description: Out of Memory killer termination
- name: recursive-tasks
displayName: Recursive Tasks
description: Problems with recursive task generation
- name: prompt-compression
displayName: Prompt Compression
description: Issues with prompt compression algorithms
- name: llm-retry
displayName: LLM Retry
description: Problems with LLM retry mechanisms
- name: critical-failure
displayName: Critical Failure
description: Critical system failures requiring immediate attention, including crashes due to unrecoverable errors
- name: password
displayName: Password
description: Problems with password policies, validation, or storage.
Expand Down Expand Up @@ -666,9 +693,6 @@ tags:
- name: container-crash
displayName: Container Crash
description: Failures causing container crashes or unexpected terminations.
- name: memory-exhaustion
displayName: Memory Exhaustion
description: Failures due to running out of memory or excessive memory consumption.
- name: configuration-failure
displayName: Configuration Failure
description: Problems caused by incorrect or invalid configuration settings.
Expand All @@ -681,9 +705,7 @@ tags:
- name: permission-denied
displayName: Permission Denied
description: Failures caused by insufficient access rights or permission errors.
- name: sigkill
displayName: SIGKILL
description: Failures caused by processes being terminated with a SIGKILL signal.

- name: jetstream
displayName: JetStream
description: NATS JetStream persistence & streaming subsystem issues.
Expand Down Expand Up @@ -827,9 +849,7 @@ tags:
- name: cache-eviction
displayName: Cache Eviction
description: Problems related to cache eviction policies and performance
- name: critical-failure
displayName: Critical Failure
description: Failures that cause immediate service termination or data loss

- name: ssl-certificate
displayName: SSL Certificate
description: Problems related to SSL/TLS certificate validation, expiration, trust chain issues, or handshake failures
Expand Down
Loading