Add more about asking LLM to be an expert from Dilgren (#963)

Signed-off-by: David A. Wheeler <[email protected]>

Author: david-a-wheeler

docs/Security-Focused-Guide-for-AI-Code-Assistant-Instructions.md

@@ -105,7 +105,7 @@ One of the first sections in your instructions should reinforce general secure c
 * **Testing for Security:** Encourage the AI to produce or suggest tests for critical code paths including negative tests that verify that what shouldn't happen, doesn't happen. In your instructions, add: *"When applicable, generate unit tests for security-critical functions (including negative tests to ensure the code fails safely)"*. [[anssibsi2024c]](#anssibsi2024c) [[markvero2025b]](#markvero2025b)
 * **Data Protection:** When generating code, always prioritize data minimization and avoid storing or processing confidential or otherwise sensitive information (like personal data - PII) unless absolutely necessary. For that case, suggest strong encryption at rest and in transit, and recommend techniques like anonymization. For example: *"Generate a function that securely handles user input for a registration form, asking for necessary fields to avoid logging sensitive PII. Ensure that no sensitive or PII is stored in plaintext"*. [[swaroopdora2025f]](#swaroopdora2025f)
 
-Note that we are *not* currently recommending in the general case that the AI be told to respond from a particular viewpoint (e.g., a role or persona) or character a.k.a. "persona pattern/memetic proxy". An example of this approach would be the instruction "Act as a software security expert. Provide outputs that a security expert would give". One set of experiments found that this approach performs poorly, producing the worst number of security weaknesses compared to other approaches. [[catherinetony2024b]](#catherinetony2024b) However, we encourage continued experimentation, and may change our recommendations based on future information.
+Note that we are *not* currently recommending in the general case that the AI be told to respond from a particular viewpoint (e.g., a role or persona) or character a.k.a. "persona pattern/memetic proxy". An example of this approach would be the instruction "Act as a software security expert. Provide outputs that a security expert would give". Some experiments found that telling the system it is an expert often makes it perform poorly or worse on these tasks [[catherinetony2024b]](#catherinetony2024b) [[connordilgren2015b](#connordilgren2025b)]. However, we encourage continued experimentation, and may change our recommendations based on future information.
 
 ---
 
@@ -266,6 +266,10 @@ mechanisms, session management..." (Swaroop Dora, Deven Lunkad, Naziya Aslam, S.
 
 <a id="catherinetony2024b">[catherinetony2024b]</a> "Across all the examined LLMs, the persona/memetic proxy approach has led to the highest average number of security weaknesses among all the evaluated prompting techniques excluding the baseline prompt that does not include any security specifications." (Catherine Tony, Nicolás E. Díaz Ferreyra, Markus Mutas, Salem Dhiff, Riccardo Scandariato - [Prompting Techniques for Secure Code Generation: A Systematic Investigation](https://arxiv.org/abs/2407.07064v2))
 
+<a id="connordilgren2025b">[connordilgren2025b]</a> "The sec-generic and sec-specific prompts [which tell the system to act as an expert] ...  do not result in a consistent increase in secure-pass@1 scores [on a more realistic benchmark] The security-policy prompt provides the LLM with a stronger hint on how to generate secure code...  This prompt also works better than sec-generic and sec-specific on SecRepoBench, but with a much smaller improvement of 1.6 percentage points on average [for full C/C++ programs, with larger increases on larger LLM models]" (Connor Dilgren, Purva Chiniya, Luke Griffith, Yu Ding, Yizheng Chen - [Benchmarking LLMs for Secure Code Generation in Real-World Repositories](https://arxiv.org/abs/2504.21205))
+
+Across all the examined LLMs, the persona/memetic proxy approach has led to the highest average number of security weaknesses among all the evaluated prompting techniques excluding the baseline prompt that does not include any security specifications." (Catherine Tony, Nicolás E. Díaz Ferreyra, Markus Mutas, Salem Dhiff, Riccardo Scandariato - [Prompting Techniques for Secure Code Generation: A Systematic Investigation](https://arxiv.org/abs/2407.07064v2))
+
 ## Credit
 
 This work's development was led by Avishay Balter.