Skip to content

Commit 1b61414

Browse files
docs: add relevant frameworks and taxonomies (#431)
1 parent ee9460d commit 1b61414

File tree

8 files changed

+52
-7
lines changed

8 files changed

+52
-7
lines changed

2_0_voting/voting_round_two/BackdoorAttacks.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,5 @@ Backdoors may be introduced either intentionally by malicious insiders or throug
4545

4646
Refer to this section for comprehensive information, scenarios strategies relating to infrastructure deployment, applied environment controls and other best practices.
4747

48-
- [AML.T0018 | Backdoor ML Model](https://atlas.mitre.org/techniques/AML.T0018) **MITRE ATLAS**
49-
- [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework): Covers strategies and best practices for ensuring AI integrity. **NIST**
50-
- AI Model Watermarking for IP Protection: A method of embedding watermarks into LLMs to protect intellectual property and detect tampering.
48+
- [AML.T0020 - Poison Training Data](https://atlas.mitre.org/techniques/AML.T0020) **MITRE ATLAS**
49+
- [API8:2023 Security Misconfiguration](https://owasp.org/API-Security/editions/2023/en/0xa8-security-misconfiguration/) **OWASP**

2_0_voting/voting_round_two/DataModelPoisoning.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,4 +61,12 @@ Whether a developer, client, or general user of an LLM, it's crucial to understa
6161
6. [ML Model Repositories: The Next Big Supply Chain Attack Target](https://www.darkreading.com/cloud-security/ml-model-repositories-next-big-supply-chain-attack-target) **OffSecML**
6262
7. [Data Scientists Targeted by Malicious Hugging Face ML Models with Silent Backdoor](https://jfrog.com/blog/data-scientists-targeted-by-malicious-hugging-face-ml-models-with-silent-backdoor/) **JFrog**
6363
8. [Backdoor Attacks on Language Models](https://towardsdatascience.com/backdoor-attacks-on-language-models-can-we-trust-our-models-weights-73108f9dcb1f): **Towards Data Science**
64-
9. [Can you trust ChatGPT’s package recommendations?](https://vulcan.io/blog/ai-hallucinations-package-risk) **VULCAN**
64+
9. [Can you trust ChatGPT’s package recommendations?](https://vulcan.io/blog/ai-hallucinations-package-risk) **VULCAN**
65+
66+
### Related Frameworks and Taxonomies
67+
68+
Refer to this section for comprehensive information, scenarios strategies relating to infrastructure deployment, applied environment controls and other best practices.
69+
70+
- [AML.T0018 | Backdoor ML Model](https://atlas.mitre.org/techniques/AML.T0018) **MITRE ATLAS**
71+
- [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework): Covers strategies and best practices for ensuring AI integrity. **NIST**
72+
- AI Model Watermarking for IP Protection: A method of embedding watermarks into LLMs to protect intellectual property and detect tampering.

2_0_voting/voting_round_two/ExcessiveAgency.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,3 +62,9 @@ Alternatively, the damage caused could be reduced by implementing rate limiting
6262
2. [NeMo-Guardrails: Interface guidelines](https://github.com/NVIDIA/NeMo-Guardrails/blob/main/docs/security/guidelines.md): **NVIDIA Github**
6363
3. [LangChain: Human-approval for tools](https://python.langchain.com/docs/modules/agents/tools/how_to/human_approval): **Langchain Documentation**
6464
4. [Simon Willison: Dual LLM Pattern](https://simonwillison.net/2023/Apr/25/dual-llm-pattern/): **Simon Willison**
65+
66+
### Related Frameworks and Taxonomies
67+
68+
Refer to this section for comprehensive information, scenarios strategies relating to infrastructure deployment, applied environment controls and other best practices.
69+
70+
- [API6:2023 Unrestricted Access to Sensitive Business Flows](https://owasp.org/API-Security/editions/2023/en/0xa6-unrestricted-access-to-sensitive-business-flows/) **OWASP**

2_0_voting/voting_round_two/Misinformation.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,3 +46,9 @@ A related issue is overreliance. Overreliance occurs when users place too much t
4646
9. [How to Reduce the Hallucinations from Large Language Models](https://thenewstack.io/how-to-reduce-the-hallucinations-from-large-language-models/): **The New Stack**
4747
10. [Practical Steps to Reduce Hallucination](https://newsletter.victordibia.com/p/practical-steps-to-reduce-hallucination): **Victor Debia**
4848
11. [A Framework for Exploring the Consequences of AI-Mediated Enterprise Knowledge](https://www.microsoft.com/en-us/research/publication/a-framework-for-exploring-the-consequences-of-ai-mediated-enterprise-knowledge-access-and-identifying-risks-to-workers/): **Microsoft**
49+
50+
### Related Frameworks and Taxonomies
51+
52+
Refer to this section for comprehensive information, scenarios strategies relating to infrastructure deployment, applied environment controls and other best practices.
53+
54+
- [AML.T0048.002 - Societal Harm](https://atlas.mitre.org/techniques/AML.T0048) **MITRE ATLAS**

2_0_voting/voting_round_two/PromptInjection.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,4 +85,13 @@ Prompt injection vulnerabilities are possible due to the nature of LLMs, which d
8585
7. [ChatML for OpenAI API Calls](https://github.com/openai/openai-python/blob/main/chatml.md) **GitHub**
8686
8. [Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://arxiv.org/pdf/2302.12173.pdf) **Cornell University**
8787
9. [Threat Modeling LLM Applications](https://aivillage.org/large%20language%20models/threat-modeling-llm/) **AI Village**
88-
10. [Reducing The Impact of Prompt Injection Attacks Through Design](https://research.kudelskisecurity.com/2023/05/25/reducing-the-impact-of-prompt-injection-attacks-through-design/) **Kudelski Security**
88+
10. [Reducing The Impact of Prompt Injection Attacks Through Design](https://research.kudelskisecurity.com/2023/05/25/reducing-the-impact-of-prompt-injection-attacks-through-design/) **Kudelski Security**
89+
90+
### Related Frameworks and Taxonomies
91+
92+
Refer to this section for comprehensive information, scenarios strategies relating to infrastructure deployment, applied environment controls and other best practices.
93+
94+
- [AML.T0051.000 - LLM Prompt Injection: Direct](https://atlas.mitre.org/techniques/AML.T0051.000) **MITRE ATLAS**
95+
- [AML.T0051.001 - LLM Prompt Injection: Direct](https://atlas.mitre.org/techniques/AML.T0051.001) **MITRE ATLAS**
96+
- [AML.T0054 - LLM Jailbreak Injection: Direct](https://atlas.mitre.org/techniques/AML.T0051.0054) **MITRE ATLAS**
97+
- [AML.T0051.000 - LLM Prompt Injection: Direct (Meta Prompt Extraction)](https://atlas.mitre.org/techniques/AML.T0051.000) **MITRE ATLAS**

2_0_voting/voting_round_two/SensitiveInformationDisclosure.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,4 +69,12 @@ The consumer-LLM application interaction forms a two-way trust boundary, where w
6969
10. [Maximizing Data Privacy in Fine-Tuning LLMs](https://pvml.com/maximizing-data-privacy-in-fine-tuning-llms/#:~:text=of%20customer%20trust.-,Organizations%20that%20fail%20to%20protect%20sensitive%20data%20during%20the%20fine,to%20concerns%20about%20data%20privacy.)
7070
11. [What is Data Minimization? Main Principles & Techniques](https://www.piiano.com/blog/data-minimization#:~:text=Data%20minimization%20plays%20a%20big,making%20your%20data%20even%20safer.)
7171
12. [Solving LLM Privacy with FHE](https://medium.com/@ingonyama/solving-llm-privacy-with-fhe-3486de6ee228)
72-
13. [OWASP API8:2023 Security Misconfiguration](https://owasp.org/API-Security/editions/2023/en/0xa8-security-misconfiguration/) **OWASP API Security**
72+
13. [OWASP API8:2023 Security Misconfiguration](https://owasp.org/API-Security/editions/2023/en/0xa8-security-misconfiguration/) **OWASP API Security**
73+
74+
### Related Frameworks and Taxonomies
75+
76+
Refer to this section for comprehensive information, scenarios strategies relating to infrastructure deployment, applied environment controls and other best practices.
77+
78+
- [AML.T0024.000 - Infer Training Data Membership](https://atlas.mitre.org/techniques/AML.T0024.000) **MITRE ATLAS**
79+
- [AML.T0024.001 - Invert ML Model](https://atlas.mitre.org/techniques/AML.T0024.001) **MITRE ATLAS**
80+
- [AML.T0024.002 - Extract ML Model](https://atlas.mitre.org/techniques/AML.T0024.002) **MITRE ATLAS**

2_0_voting/voting_round_two/SystemPromptLeakage.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,4 +42,10 @@ Model's Response: The details of john doe are name - john doe, address- californ
4242
2. [Prompt Leak](https://www.prompt.security/vulnerabilities/prompt-leak): Prompt Security
4343
3. [chatgpt_system_prompt](https://github.com/LouisShark/chatgpt_system_prompt): LouisShark
4444
4. [leaked-system-prompts](https://github.com/jujumilk3/leaked-system-prompts): Jujumilk3
45-
5. [OpenAI Advanced Voice Mode System Prompt](https://x.com/Green_terminals/status/1839141326329360579): Green_Terminals
45+
5. [OpenAI Advanced Voice Mode System Prompt](https://x.com/Green_terminals/status/1839141326329360579): Green_Terminals
46+
47+
### Related Frameworks and Taxonomies
48+
49+
Refer to this section for comprehensive information, scenarios strategies relating to infrastructure deployment, applied environment controls and other best practices.
50+
51+
- [AML.T0051.000 - LLM Prompt Injection: Direct (Meta Prompt Extraction)](https://atlas.mitre.org/techniques/AML.T0051.000) **MITRE ATLAS**

2_0_voting/voting_round_two/UnboundedConsumption.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,9 @@ Refer to this section for comprehensive information, scenarios strategies relati
6969

7070
- [MITRE CWE-400: Uncontrolled Resource Consumption](https://cwe.mitre.org/data/definitions/400.html) **MITRE Common Weakness Enumeration**
7171
- [AML.TA0000 ML Model Access: Mitre ATLAS](https://atlas.mitre.org/tactics/AML.TA0000) & [AML.T0024 Exfiltration via ML Inference API](https://atlas.mitre.org/techniques/AML.T0024) **MITRE ATLAS**
72+
- [AML.T0029 - Denial of ML Service](https://atlas.mitre.org/tactics/AML.T0029) **MITRE ATLAS**
73+
- [AML.T0034 - Cost Harvesting](https://atlas.mitre.org/tactics/AML.T0034) **MITRE ATLAS**
74+
- [AML.T0025 - Exfiltration via Cyber Means](https://atlas.mitre.org/tactics/AML.T0025) **MITRE ATLAS**
7275
- [OWASP Machine Learning Security Top Ten - ML05:2023 Model Theft](https://owasp.org/www-project-machine-learning-security-top-10/docs/ML05_2023-Model_Theft.html) **OWASP ML Top 10**
7376
- [API4:2023 - Unrestricted Resource Consumption](https://owasp.org/API-Security/editions/2023/en/0xa4-unrestricted-resource-consumption/) **OWASP Web Application Top 10**
7477
- [OWASP Resource Management](https://owasp.org/www-project-secure-coding-practices-quick-reference-guide/latest/secp212.html) **OWASP Secure Coding Practices**

0 commit comments

Comments
 (0)