You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
| Exploitability: 5 (Easy to exploit) ML Application Specific: 4 ML Operations Specific: 3 | Detectability: 3 (The adversarial image may not be noticeable to the naked eye, making it difficult to detect the attack) | Technical: 5 (The attack requires technical knowledge of deep learning and image processing techniques) |
28
-
| Threat Agent: Attacker with knowledge of deep learning and image processing techniques Attack Vector: Deliberately crafted adversarial image that is similar to a legitimate image | Vulnerability in the deep learning model's ability to classify images accurately | Misclassification of the image, leading to security bypass or harm to the system |
27
+
Adversarial attacks are a type of attack in which an attacker deliberately
28
+
alters input data to mislead the model.
29
29
30
-
It is important to note that this chart is only a sample based on
31
-
scenario below, and the actual risk assessment will depend on the
32
-
specific circumstances of each machine learning system.
30
+
## How to Prevent
31
+
32
+
**Adversarial training:** One approach to defending against adversarial attacks
33
+
is to train the model on adversarial examples. This can help the model become
34
+
more robust to attacks and reduce its susceptibility to being misled.
33
35
34
-
**Description**:
35
-
Adversarial attacks are a type of attack in which an attacker
36
-
deliberately alters input data to mislead the model.
36
+
**Robust models:** Another approach is to use models that are designed to be
37
+
robust against adversarial attacks, such as adversarial training or models that
38
+
incorporate defense mechanisms.
37
39
38
-
**Example Attack Scenario:**
40
+
**Input validation:** Input validation is another important defense mechanism
41
+
that can be used to detect and prevent adversarial attacks. This involves
42
+
checking the input data for anomalies, such as unexpected values or patterns,
43
+
and rejecting inputs that are likely to be malicious.
39
44
40
-
Scenario 1: Image classification
45
+
## Risk Factors
41
46
42
-
A deep learning model is trained to classify images into different
43
-
categories, such as dogs and cats. An attacker creates an adversarial
44
-
image that is very similar to a legitimate image of a cat, but with
45
-
small, carefully crafted perturbations that cause the model to
46
-
misclassify it as a dog. When the model is deployed in a real-world
47
-
setting, the attacker can use the adversarial image to bypass security
| Exploitability: 5 (Easy) <br><br> _ML Application Specific: 4_ <br> _ML Operations Specific: 3_| Detectability: 3 (Moderate) <br><br> _The adversarial image may not be noticeable to the naked eye, making it difficult to detect the attack._| Technical: 5 (Difficult) <br><br> _The attack requires technical knowledge of deep learning and image processing techniques._|
50
+
| Threat Agent: Attacker with knowledge of deep learning and image processing techniques. <br><br> Attack Vector: Deliberately crafted adversarial image that is similar to a legitimate image. | Vulnerability in the deep learning model's ability to classify images accurately. | Misclassification of the image, leading to security bypass or harm to the system. |
49
51
50
-
Scenario 2: Network intrusion detection
52
+
It is important to note that this chart is only a sample based on
53
+
[the scenario below](#scenario1) only. The actual risk assessment will depend on
54
+
the specific circumstances of each machine learning system.
51
55
52
-
A deep learning model is trained to detect intrusions in a network. An
53
-
attacker creates adversarial network traffic by carefully crafting
54
-
packets in such a way that they will evade the model\'s intrusion
55
-
detection system. The attacker can manipulate the features of the
56
-
network traffic, such as the source IP address, destination IP address,
57
-
or payload, in such a way that they are not detected by the intrusion
58
-
detection system. For example, the attacker may hide their source IP
59
-
address behind a proxy server or encrypt the payload of their network
60
-
traffic. This type of attack can have serious consequences, as it can
61
-
lead to data theft, system compromise, or other forms of damage.
| Threat Agent: Attacker who has access to the training data used for the model.<br>Attack Vector: The attacker injects malicious data into the training data set. | Lack of data validation and insufficient monitoring of the training data. | The model will make incorrect predictions based on the poisoned data, leading to false decisions and potentially serious consequences. |
25
+
## Description
27
26
28
-
It is important to note that this chart is only a sample based on
29
-
scenario below, and the actual risk assessment will depend on the
30
-
specific circumstances of each machine learning system.
27
+
Data poisoning attacks occur when an attacker manipulates the training data to
28
+
cause the model to behave in an undesirable way.
31
29
32
-
**Description:**
30
+
## How to Prevent
33
31
34
-
Data poisoning attacks occur when an attacker manipulates the training
35
-
data to cause the model to behave in an undesirable way.
32
+
**Data validation and verification:** Ensure that the training data is
33
+
thoroughly validated and verified before it is used to train the model. This can
34
+
be done by implementing data validation checks and employing multiple data
35
+
labelers to validate the accuracy of the data labeling.
36
36
37
-
**Example Attack Scenario:**
37
+
**Secure data storage:** Store the training data in a secure manner, such as
38
+
using encryption, secure data transfer protocols, and firewalls.
38
39
39
-
Scenario 1: Training a spam classifier
40
+
**Data separation:** Separate the training data from the production data to
41
+
reduce the risk of compromising the training data.
40
42
41
-
An attacker poisons the training data for a deep learning model that
42
-
classifies emails as spam or not spam. The attacker executed this attack
43
-
by injecting the maliciously labeled spam emails into the training data
44
-
set. This could be done by compromising the data storage system, for
45
-
example by hacking into the network or exploiting a vulnerability in the
46
-
data storage software. The attacker could also manipulate the data
47
-
labeling process, such as by falsifying the labeling of the emails or by
48
-
bribing the data labelers to provide incorrect labels.
43
+
**Access control:** Implement access controls to limit who can access the
44
+
training data and when they can access it.
49
45
50
-
Scenario 2: Training a network traffic classification system
46
+
**Monitoring and auditing:** Regularly monitor the training data for any
47
+
anomalies and conduct audits to detect any data tampering.
51
48
52
-
An attacker poisons the training data for a deep learning model that is
53
-
used to classify network traffic into different categories, such as
54
-
email, web browsing, and video streaming. They introduce a large number
55
-
of examples of network traffic that are incorrectly labeled as a
56
-
different type of traffic, causing the model to be trained to classify
57
-
this traffic as the incorrect category. As a result, the model may be
58
-
trained to make incorrect traffic classifications when the model is
59
-
deployed, potentially leading to misallocation of network resources or
60
-
degradation of network performance.
49
+
**Model validation:** Validate the model using a separate validation set that
50
+
has not been used during training. This can help to detect any data poisoning
51
+
attacks that may have affected the training data.
61
52
62
-
**How to Prevent:**
53
+
**Model ensembles:** Train multiple models using different subsets of the
54
+
training data and use an ensemble of these models to make predictions. This can
55
+
reduce the impact of data poisoning attacks as the attacker would need to
56
+
compromise multiple models to achieve their goals.
63
57
64
-
Data validation and verification: Ensure that the training data is
65
-
thoroughly validated and verified before it is used to train the model.
66
-
This can be done by implementing data validation checks and employing
67
-
multiple data labelers to validate the accuracy of the data labeling.
58
+
**Anomaly detection:** Use anomaly detection techniques to detect any abnormal
59
+
behavior in the training data, such as sudden changes in the data distribution
60
+
or data labeling. These techniques can be used to detect data poisoning attacks
61
+
early on.
68
62
69
-
Secure data storage: Store the training data in a secure manner, such as
70
-
using encryption, secure data transfer protocols, and firewalls.
63
+
## Risk Factors
71
64
72
-
Data separation: Separate the training data from the production data to
73
-
reduce the risk of compromising the training data.
| Threat Agent: Attacker who has access to the training data used for the model. <br><br> Attack Vector: The attacker injects malicious data into the training data set. | Lack of data validation and insufficient monitoring of the training data. | The model will make incorrect predictions based on the poisoned data, leading to false decisions and potentially serious consequences. |
74
69
75
-
Access control: Implement access controls to limit who can access the
76
-
training data and when they can access it.
70
+
It is important to note that this chart is only a sample based on
71
+
[the scenario below](#scenario1) only. The actual risk assessment will depend on
72
+
the specific circumstances of each machine learning system.
77
73
78
-
Monitoring and auditing: Regularly monitor the training data for any
79
-
anomalies and conduct audits to detect any data tampering.
74
+
## Example Attack Scenarios
75
+
76
+
### Scenario \#1: Training a spam classifier {#scenario1}
80
77
81
-
Model validation: Validate the model using a separate validation set
82
-
that has not been used during training. This can help to detect any data
83
-
poisoning attacks that may have affected the training data.
78
+
An attacker poisons the training data for a deep learning model that classifies
79
+
emails as spam or not spam. The attacker executed this attack by injecting the
80
+
maliciously labeled spam emails into the training data set. This could be done
81
+
by compromising the data storage system, for example by hacking into the network
82
+
or exploiting a vulnerability in the data storage software. The attacker could
83
+
also manipulate the data labeling process, such as by falsifying the labeling of
84
+
the emails or by bribing the data labelers to provide incorrect labels.
84
85
85
-
Model ensembles: Train multiple models using different subsets of the
86
-
training data and use an ensemble of these models to make predictions.
87
-
This can reduce the impact of data poisoning attacks as the attacker
88
-
would need to compromise multiple models to achieve their goals.
86
+
### Scenario \#2: Training a network traffic classification system
89
87
90
-
Anomaly detection: Use anomaly detection techniques to detect any
91
-
abnormal behavior in the training data, such as sudden changes in the
92
-
data distribution or data labeling. These techniques can be used to
93
-
detect data poisoning attacks early on.
88
+
An attacker poisons the training data for a deep learning model that is used to
89
+
classify network traffic into different categories, such as email, web browsing,
90
+
and video streaming. They introduce a large number of examples of network
91
+
traffic that are incorrectly labeled as a different type of traffic, causing the
92
+
model to be trained to classify this traffic as the incorrect category. As a
93
+
result, the model may be trained to make incorrect traffic classifications when
94
+
the model is deployed, potentially leading to misallocation of network resources
0 commit comments