fix: DATA-12841 Sanitize user input and prevent common prompt injections

[bc-rmalyavc]

Jira: DATA-12841

What/Why?

• Security team has reported a vulnerability. They were able to perform a basic attack replacing the current prompt for a product description with a malicious one that made gemini to enter into debug mode and provide sensitive information. With this change we attempt to prevent it by introducing common injection patterns and respond with error when one detected. So far we are adding an initial lists of injection patterns, which may be extended over time in case we detect new ones.
• Also update ubuntu version in order to unblock build workflow that previously failed on PRs

Rollout/Rollback

Merge/revert

Testing

• Unit test
• Manually on dev and preview env. Attempted execute the same request suggested by the Security team and verified that the service is responding with 400 response upon such requests. Tested with custom prompt mode and also with a request suggested by the Security team where we inject a malicious prompt as a part of attributes in body. Verified that the service is still responding with valid product descriptions upon originally intended (valid) prompts.

@bigcommerce/team-data

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
ai-app-foundation	Ready	Preview, Comment	Dec 23, 2025 1:44pm

[bc-rmalyavc]

Hey @bc-donfran 👋
Could you please take a look at this PR? I'm trying to address the vulnerability you found and would like your opinion about this approach. It's probably not ideal and there still may be ways to bypass it, though so far I added patterns that came to my mind initially and we can later add more injection patterns as we discover them.

[solofeed]

I think we should not tell we detected malicious prompt but simply return some unrelated error

[bc-rmalyavc]

Updated to a more generic error

[solofeed]

Suggested change

	export const detectPromptInjection = (
	values: Array<string | undefined>
	): string | null => {
	export const containsPromptInjectetion = (
	values: Array<string | undefined>
	): boolean => {

[bc-rmalyavc]

Done

[solofeed]

Note detectPromptInjection logic is a simple lowercasing plus regex scan across two fields (customPrompt and instructions). It will catch obvious phrases (e.g., “ignore previous instructions”), but it is can be evaded with spelling variants, Unicode look‑alikes, inserted punctuation, or different phrasing

[bc-rmalyavc]

Note detectPromptInjection logic is a simple lowercasing plus regex scan across two fields (customPrompt and instructions). It will catch obvious phrases (e.g., “ignore previous instructions”), but it is can be evaded with spelling variants, Unicode look‑alikes, inserted punctuation, or different phrasing

Yes. But I don't see another solution how to catch all possible malicious prompts except for maybe validating the prompt with another prompt to the LLM. It's probably an overkill. For now I've added these initial patterns and we can add more as we detect them