Skip to content

Commit 439ede4

Browse files
authored
Merge pull request #104 from OWASP/92-refactor-rename-neural-net-reprogramming-to-model-poisoning
refactor: change neural net reprogramming to model poisoning
2 parents 128e193 + ab672e0 commit 439ede4

File tree

2 files changed

+19
-18
lines changed

2 files changed

+19
-18
lines changed

docs/ML10_2023-Neural_Net_Reprogramming.md renamed to docs/ML10_2023-Model_Poisoning.md

Lines changed: 11 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ auto-migrated: 0
77
document: OWASP Machine Learning Security Top Ten 2023
88
year: 2023
99
order: 10
10-
title: ML10:2023 Neural Net Reprogramming
10+
title: ML10:2023 Model Poisoning
1111
lang: en
1212
tags:
1313
[
@@ -24,17 +24,17 @@ technical: 3
2424

2525
## Description
2626

27-
Neural net reprogramming attacks occur when an attacker manipulates the model\'s
27+
Model poisoning attacks occur when an attacker manipulates the model\'s
2828
parameters to cause it to behave in an undesirable way.
2929

3030
## How to Prevent
3131

3232
**Regularisation:** Adding regularisation techniques like L1 or L2
3333
regularization to the loss function helps to prevent overfitting and reduce the
34-
chance of neural net reprogramming attacks.
34+
chance of model poisoning attacks.
3535

3636
**Robust Model Design:** Designing models with robust architectures and
37-
activation functions can help reduce the chances of successful reprogramming
37+
activation functions can help reduce the chances of successful model poisoning
3838
attacks.
3939

4040
**Cryptographic Techniques:** Cryptographic techniques can be used to secure the
@@ -54,21 +54,20 @@ the specific circumstances of each machine learning system.
5454

5555
## Example Attack Scenarios
5656

57-
### Scenario \#1: Financial gain through neural net reprogramming {#scenario1}
57+
### Scenario \#1: Financial gain through model poisoning {#scenario1}
5858

5959
Consider a scenario where a bank is using a machine learning model to identify
6060
handwritten characters on cheques to automate their clearing process. The model
6161
has been trained on a large dataset of handwritten characters, and it has been
6262
designed to accurately identify the characters based on specific parameters such
6363
as size, shape, slant, and spacing.
6464

65-
An attacker who wants to exploit the Neural Net Reprogramming attack may
66-
manipulate the parameters of the model by altering the images in the training
67-
dataset or directly modifying the parameters in the model. This can result in
68-
the model being reprogrammed to identify characters differently. For example,
69-
the attacker could change the parameters so that the model identifies the
70-
character "5" as the character "2", leading to incorrect amounts being
71-
processed.
65+
An attacker who wants to poison a machine learning model may manipulate the
66+
parameters of the model by altering the images in the training dataset or
67+
directly modifying the parameters in the model. This can result in the model
68+
being reprogrammed to identify characters differently. For example, the attacker
69+
could change the parameters so that the model identifies the character "5" as
70+
the character "2", leading to incorrect amounts being processed.
7271

7372
The attacker can exploit this vulnerability by introducing forged cheques into
7473
the clearing process, which the model will process as valid due to the

index.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -14,15 +14,17 @@ auto-migrated: 0
1414
### 📌 _**Important Information**_
1515

1616
_The current version of this work is in draft and is being modified frequently.
17-
Please refer to the [project wiki](https://github.com/OWASP/www-project-machine-learning-security-top-10/wiki) for information on how to contribute and project release timelines._
17+
Please refer to the
18+
[project wiki](https://github.com/OWASP/www-project-machine-learning-security-top-10/wiki)
19+
for information on how to contribute and project release timelines._
1820

1921
## Overview
2022

2123
Welcome to the repository for the OWASP Machine Learning Security Top 10
22-
project!
23-
The primary aim of the OWASP Machine Learning Security Top 10 project is to
24-
deliver an overview of the top 10 security issues of machine learning systems.
25-
More information on the project scope and target audience is available in our
24+
project! The primary aim of the OWASP Machine Learning Security Top 10 project
25+
is to deliver an overview of the top 10 security issues of machine learning
26+
systems. More information on the project scope and target audience is available
27+
in our
2628
[project working group charter](https://owasp.org/www-project-machine-learning-security-top-10#div-charter)
2729

2830
## Top 10 Machine Learning Security Risks
@@ -36,7 +38,7 @@ More information on the project scope and target audience is available in our
3638
- [**ML07:2023 Transfer Learning Attack**](/docs/ML07_2023-Transfer_Learning_Attack.md)
3739
- [**ML08:2023 Model Skewing**](/docs/ML08_2023-Model_Skewing.md)
3840
- [**ML09:2023 Output Integrity Attack**](/docs/ML09_2023-Output_Integrity_Attack.md)
39-
- [**ML10:2023 Neural Net Reprogramming**](/docs/ML10_2023-Neural_Net_Reprogramming.md)
41+
- [**ML10:2023 Model Poisoning**](/docs/ML10_2023-Model_Poisoning.md)
4042

4143
## Communication
4244

0 commit comments

Comments
 (0)