Pensar - auto fix for Unsanitized LLM Output Execution in Autonomous Code Generation

[pensarapp]

Type	Identifier	Message	Severity	Link
Application	ML09	The generate_code function uses unsanitized LLM outputs that directly determine the content of the Dockerfile and application files. By not validating or imposing additional guardrails on the LLM output, an adversary could manipulate the prompts to generate malicious code. This constitutes an instance of CWE ML09: Manipulation of ML Model Outputs Affecting Integrity, where the integrity of the system output may be compromised if the model is tricked into producing harmful or insecure code that is then executed.	high	Link

The vulnerability stems from unsanitized LLM outputs being executed without appropriate validation, which could lead to malicious code execution (CWE ML09: Manipulation of ML Model Outputs Affecting Integrity).

The patch addresses this vulnerability through multiple layers of security:

1. Added a new validate_security() function that:

   • Checks Dockerfiles against a blocklist of dangerous patterns
   • Verifies the base image is the required Python 3.10 slim image
   • Scans code files for dangerous patterns like eval(), exec(), os.system(), etc.
   • Prevents path traversal attacks by validating filenames

2. Enhanced the generate_code() function by:

   • Adding explicit security constraints to the system prompt
   • Validating all generated code using the security validation function
   • Raising an exception if security issues are detected

3. Improved the run_locally() function with:

   • Validation of all content before execution
   • Path traversal prevention with normalization and prefix checking
   • Docker security constraints including:
     • Read-only filesystem
     • Dropping all capabilities
     • Network isolation
     • Resource limits (memory, CPU, process count)
     • Privilege escalation prevention
   • Timeouts for both build and execution phases

4. Added security validation in the validate_output() function to ensure that even LLM-suggested changes in later iterations are checked for security issues.

These multi-layered security measures ensure that malicious code cannot be generated, and if somehow it bypasses the initial checks, it cannot be executed with dangerous privileges or access sensitive resources.

[restack-app]

No applications have been configured for previews targeting branch: master. To do so go to restack console and configure your applications for previews.