Merge branch 'main' into allie/add-agentic-code-samples

Author: alliehowe29

2_0_vulns/LLM03_SupplyChain.md

@@ -28,7 +28,7 @@ A simple threat model can be found [here](https://github.com/jsotiro/ThreatModel
 
 #### 4. Vulnerable Pre-Trained Model
 
-  Models are binary black boxes and unlike open source, static inspection can offer little to security assurances. Vulnerable pre-trained models can contain hidden biases, backdoors, or other malicious features that have not been identified through the safety evaluations of model repositories. Vulnerable models can be created by both poisoned datasets and direct model tampering using techniques such as ROME also known as lobotomisation.
+  Models are binary black boxes and unlike open source, static inspection can offer little to no security assurances. Vulnerable pre-trained models can contain hidden biases, backdoors, or other malicious features that have not been identified through the safety evaluations of model repositories. Vulnerable models can be created by both poisoned datasets and direct model tampering using techniques such as ROME also known as lobotomisation.
 
 #### 5. Weak Model Provenance
 
@@ -40,7 +40,7 @@ A simple threat model can be found [here](https://github.com/jsotiro/ThreatModel
 
 #### 7. Exploit Collaborative Development Processes
 
-  Collaborative model merge and model handling services (e.g. conversions) hosted in shared environments can be exploited to introduce vulnerabilities in shared models. Model merging is very popular on Hugging Face with model-merged models topping the OpenLLM leaderboard and can be exploited to bypass reviews. Similarly, services such as conversation bot have been proved to be vulnerable to manipulation and introduce malicious code in models.
+  Collaborative model merge and model handling services (e.g. conversions) hosted in shared environments can be exploited to introduce vulnerabilities in shared models. Model merging is very popular on Hugging Face with model-merged models topping the OpenLLM leaderboard and can be exploited to bypass reviews. Similarly, services such as a conversation bot have been proved to be vulnerable to manipulation and introduce malicious code in models.
 
 #### 8. LLM Model on Device supply-chain vulnerabilities
 

2_0_vulns/LLM04_DataModelPoisoning.md

@@ -11,12 +11,12 @@ Moreover, models distributed through shared repositories or open-source platform
 ### Common Examples of Vulnerability
 
 1. Malicious actors introduce harmful data during training, leading to biased outputs. Techniques like "Split-View Data Poisoning" or "Frontrunning Poisoning" exploit model training dynamics to achieve this.
-  (Ref. link: [Split-View Data Poisoning](https://github.com/GangGreenTemperTatum/speaking/blob/main/dc604/hacker-summer-camp-23/Ads%20_%20Poisoning%20Web%20Training%20Datasets%20_%20Flow%20Diagram%20-%20Exploit%201%20Split-View%20Data%20Poisoning.jpeg))
-  (Ref. link: [Frontrunning Poisoning](https://github.com/GangGreenTemperTatum/speaking/blob/main/dc604/hacker-summer-camp-23/Ads%20_%20Poisoning%20Web%20Training%20Datasets%20_%20Flow%20Diagram%20-%20Exploit%202%20Frontrunning%20Data%20Poisoning.jpeg))
-2. Attackers can inject harmful content directly into the training process, compromising the model’s output quality.
-3. Users unknowingly inject sensitive or proprietary information during interactions, which could be exposed in subsequent outputs.
-4. Unverified training data increases the risk of biased or erroneous outputs.
-5. Lack of resource access restrictions may allow the ingestion of unsafe data, resulting in biased outputs.
+  (Ref. link: [Split-View Data Poisoning](https://github.com/GangGreenTemperTatum/speaking/blob/aad68f8521119596abb567d94fbd10bdd652ac82/docs/conferences/dc604/hacker-summer-camp-23/Ads%20_%20Poisoning%20Web%20Training%20Datasets%20_%20Flow%20Diagram%20-%20Exploit%201%20Split-View%20Data%20Poisoning.jpeg))
+  (Ref. link: [Frontrunning Poisoning](https://github.com/GangGreenTemperTatum/speaking/blob/aad68f8521119596abb567d94fbd10bdd652ac82/docs/conferences/dc604/hacker-summer-camp-23/Ads%20_%20Poisoning%20Web%20Training%20Datasets%20_%20Flow%20Diagram%20-%20Exploit%202%20Frontrunning%20Data%20Poisoning.jpeg))
+1. Attackers can inject harmful content directly into the training process, compromising the model’s output quality.
+2. Users unknowingly inject sensitive or proprietary information during interactions, which could be exposed in subsequent outputs.
+3. Unverified training data increases the risk of biased or erroneous outputs.
+4. Lack of resource access restrictions may allow the ingestion of unsafe data, resulting in biased outputs.
 
 ### Prevention and Mitigation Strategies
 

2_0_vulns/LLM08_VectorAndEmbeddingWeaknesses.md

@@ -4,7 +4,7 @@
 
 Vectors and embeddings vulnerabilities present significant security risks in systems utilizing Retrieval Augmented Generation (RAG) with Large Language Models (LLMs). Weaknesses in how vectors and embeddings are generated, stored, or retrieved can be exploited by malicious actions (intentional or unintentional) to inject harmful content, manipulate model outputs, or access sensitive information.
 
-Retrieval Augmented Generation (RAG) is a model adaptation technique that enhances the performance and contextual relevance of responses from LLM Applications, by combining pre-trained language models with external knowledge sources.Retrieval Augmentation uses vector mechanisms and embedding. (Ref #1)
+Retrieval Augmented Generation (RAG) is a model adaptation technique that enhances the performance and contextual relevance of responses from LLM Applications, by combining pre-trained language models with external knowledge sources. Retrieval Augmentation uses vector mechanisms and embedding. (Ref #1)
 
 ### Common Examples of Risks
 

CODEOWNERS

@@ -5,7 +5,7 @@
 # TBC
 
 ## Either Ads or Steve can approve changes to CODEOWNERS or github:
-CODEOWNERS @GangGreenTemperTatum @virtualsteve-star
+CODEOWNERS @GangGreenTemperTatum @virtualsteve-star @rossja
 .github/ @GangGreenTemperTatum @virtualsteve-star
 
 ## Data Gathering
@@ -43,4 +43,6 @@ _template.md @rossja
 # Sub-projects of the OWASP Top 10 for LLM Apps and Gen AI Project
 
 ## Agentic Security Initiative (ASI) under the Top 10 for LLM and Gen AI Project
-initiatives/agent_security_initiative/ @guerilla7 @allie-secvendors @hoeg @itskerenkatz
+initiatives/agent_security_initiative/ @guerilla7 @allie-secvendors @hoeg @itskerenkatz
+
+initiatives/genai_red_team_handbook/ @rossja