Pensar - auto fix for Unsanitized LLM Prompt Injection Vulnerability in Code Generation System

[pensarapp]

Type	Identifier	Message	Severity	Link
Application	ML01	The endpoint for updating prompts allows users to supply 'generate_code_prompt' and 'validate_output_prompt' without any sanitization or validation. This unsanitized input can be exploited to manipulate the behavior of the downstream LLM used for code generation, potentially leading to adversarial input manipulation attacks (ML01:2023 Adversarial Input Manipulation in ML Models). An attacker could craft prompts that bypass or skew internal guardrails, leading to harmful code generation or system compromise.	high	Link

The vulnerability exists in the /prompts endpoint where user-provided inputs (generate_code_prompt and validate_output_prompt) are accepted without any validation or sanitization before being passed to the set_prompts function. This could allow an attacker to manipulate the downstream LLM behavior through adversarial prompts.

Changes made:

1. Added a validate_prompt function that:

   • Checks if the prompt exceeds a maximum length (8192 characters) to prevent resource exhaustion
   • Checks for prohibited patterns commonly used in prompt injection attacks
   • Returns False if any validation fails, True otherwise

2. Modified the update_prompts endpoint to:

   • Validate both input prompts before passing them to set_prompts
   • Raise an HTTP 400 error with a descriptive message if validation fails

This approach provides several security benefits:

• Prevents excessively long prompts that could cause resource exhaustion
• Blocks common adversarial patterns that might attempt to bypass model guardrails
• Provides clear error messages to aid in debugging and understanding of validation failures

The solution is minimally invasive, requiring no new dependencies while maintaining compatibility with the existing codebase. The prohibited patterns list can be expanded based on specific application threats and requirements.

[restack-app]

No applications have been configured for previews targeting branch: master. To do so go to restack console and configure your applications for previews.