Skip to content

Commit ab3838f

Browse files
Merge pull request #4416 from MicrosoftDocs/main
[AutoPublish] main to live - 07/03 10:31 PDT | 07/03 23:01 IST
2 parents ee9650a + 7d4d079 commit ab3838f

File tree

2 files changed

+11
-91
lines changed

2 files changed

+11
-91
lines changed

defender-office-365/email-authentication-about.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -180,7 +180,7 @@ These values are explained at [Authentication-results message header](message-he
180180
Admins and users can examine the message headers to discover how Microsoft 365 identified the sender as a suspicious spoofed sender or legitimate.
181181

182182
> [!TIP]
183-
> It's important to understand that a composite authentication failure doesn't directly result in a message being blocked. Our system using a holistic evaluation strategy that considers the overall suspicious nature of a message along with composite authentication results. This method is designed to mitigate the risk of incorrectly blocking legitimate email from domains that might not strictly adhere to email authentication protocols. This balanced approach helps distinguish genuinely malicious email from message senders that simply fail to conform to standard email authentication practices.
183+
> It's important to understand that a composite authentication failure doesn't directly result in a message being blocked. Our system uses a holistic evaluation strategy that considers the overall suspicious nature of a message along with composite authentication results. This method is designed to mitigate the risk of incorrectly blocking legitimate email from domains that might not strictly adhere to email authentication protocols. This balanced approach helps distinguish genuinely malicious email from message senders that simply fail to conform to standard email authentication practices.
184184
185185
The following examples focus on the results of email authentication only (the `compauth` value and reason). Other Microsoft 365 protection technologies can identify messages that pass email authentication as spoofed, or identify messages that fail email authentication as legitimate.
186186

defender-office-365/reports-mdo-email-collaboration-dashboard.md

Lines changed: 10 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -46,11 +46,13 @@ By default, the data on the page is shown for the last 30 days. But, you can sho
4646

4747
The information in the **Defender for Office 365** summary at the top of the page is described in the following subsections.
4848

49-
### Efficacy card
49+
<a name='efficacy-card'></a>
50+
51+
### Phish / Malware Efficacy card
5052

5153
<!--- https://go.microsoft.com/fwlink/?linkid=2324012 --->
5254

53-
The graph on the **Efficacy** card visually represents the protection given by Defender for Office 365 against phishing and malware in email messages:
55+
The graph on the **Phish / Malware Efficacy** card visually represents the protection given by Defender for Office 365 against phishing and malware in email messages:
5456

5557
- **Pre-delivery**: Items detected before they reach the recipient's mailbox.
5658
- **Post-delivery**: Items removed after the item was delivered to the recipient's mailbox via [zero-hour auto purge (ZAP)](zero-hour-auto-purge.md).
@@ -252,175 +254,93 @@ The graph on the **Microsoft 365 Secure Email Gateway performance** card compare
252254

253255
## Appendix: Advanced hunting efficacy query in Defender for Office 365 Plan 2
254256

255-
Organizations with Defender for Office 365 Plan 2 can use the following query in [advanced hunting](/defender-xdr/advanced-hunting-overview) to generate the same data on the [**Efficacy** card](#efficacy-card).
257+
Organizations with Defender for Office 365 Plan 2 can use the following query in [advanced hunting](/defender-xdr/advanced-hunting-overview) to generate the same data on the [**Phish / Malware Efficacy** card](#phish--malware-efficacy-card).
256258

257259
> [!NOTE]
258260
> The numbers might differ slightly due to the different refresh rates for advanced hunting vs. reporting data.
259261
260262
```kusto
261263
// This query by default will take the last 30 days of data.
262-
263264
// The query and calculation can be tweaked to meet individual needs, and will update over time to get incrementally more accurate.
264-
265265
// Ben Harris - Microsoft Defender for Office 365 PM.
266-
267266
let _startTime = ago(30d);
268-
269267
let _endTime = now();
270-
271268
// Get all mailflow detected as clean at time of delivery
272-
273269
let EmailEventsClean = materialize(
274-
275270
EmailEvents
276-
277271
| where Timestamp between (_startTime .. _endTime) and EmailDirection == "Inbound"
278-
279272
| where ThreatTypes !contains "Phish" and ThreatTypes !contains "Malware"
280-
281273
| project NetworkMessageId,ThreatTypes
282-
283274
);
284-
285275
// Get all mailflow detected as phish or malware at time of delivery
286-
287276
let EmailEventsThreats = materialize(
288-
289277
EmailEvents
290-
291278
| where Timestamp between (_startTime .. _endTime) and EmailDirection == "Inbound"
292-
293279
| where ThreatTypes contains "Phish" or ThreatTypes contains "Malware"
294-
295280
| extend MDO_detection = parse_json(DetectionMethods)
296-
297281
| extend FirstDetection = iif(isempty(MDO_detection), "Clean", tostring(bag_keys(MDO_detection)[0]))
298-
299282
| extend FirstSubcategory = iif(FirstDetection != "Clean" and array_length(MDO_detection[FirstDetection]) > 0, strcat(FirstDetection, ": ", tostring(MDO_detection[FirstDetection][0])), "No Detection (clean)")
300-
301283
| project NetworkMessageId,FirstDetection,FirstSubcategory,MDO_detection,ThreatTypes
302-
303284
);
304-
305285
// Get all post delivery ZAP / Redelivery events, and arg_max them to ensure we have the latest verdict to work with for each
306-
307286
let EmailPostDeliveryFiltered = materialize(
308-
309287
EmailPostDeliveryEvents
310-
311288
| where Timestamp between (_startTime .. datetime_add('day', 7, _endTime))
312-
313289
| where ActionType in ("Malware ZAP","Phish ZAP","Redelivery")
314-
315290
| extend Key = strcat(NetworkMessageId , "-", RecipientEmailAddress)
316-
317291
| summarize arg_max(Timestamp, *) by Key
318-
319292
| project Action,ActionType,ActionResult,ThreatTypes,NetworkMessageId
320-
321293
);
322-
323294
// Optional - get all admin submissions for malware or phish, so we can also count these in the miss bucket.
324-
325295
let CloudAppEventsFiltered = materialize(
326-
327296
CloudAppEvents
328-
329297
| where Timestamp between (_startTime .. datetime_add('day', 7, _endTime))
330-
331298
| where ActionType == "AdminSubmissionSubmitted"
332-
333299
| extend SubmissionType = tostring(parse_json(RawEventData).SubmissionType)
334-
335300
| extend NetworkMessageId = tostring(parse_json(RawEventData).ObjectId)
336-
337301
| where SubmissionType in ("1", "2")
338-
339302
| project SubmissionType,NetworkMessageId
340-
341303
);
342-
343304
// get the number of threats caught in mailflow
344-
345305
let Mal_Phish_Mailflow = toscalar(
346-
347306
EmailEventsThreats
348-
349-
| summarize count(NetworkMessageId)
350-
307+
| summarize count()
351308
);
352-
353309
// get the number of threats caught in mailflow which turned out to be false positives (FPs) so we can correct the calculation
354-
355310
let FP_ZAP = toscalar(
356-
357311
EmailPostDeliveryFiltered
358-
359312
| where ThreatTypes !contains "Phish" and ThreatTypes !contains "Malware" and ActionType == "Redelivery"
360-
361313
| join kind=leftsemi (EmailEventsThreats) on NetworkMessageId
362-
363-
| summarize count(NetworkMessageId)
364-
314+
| summarize count()
365315
);
366-
367316
// get the number of threats successfully cleaned up post delivery, ignoring where administrative policy stopped action
368-
369317
let FN_ZAP_Successful = toscalar(
370-
371318
EmailPostDeliveryFiltered
372-
373319
| where ActionType in ("Malware ZAP","Phish ZAP") and ActionResult in ("Success","AdminPolicy")
374-
375320
| join kind=leftsemi (EmailEventsClean) on NetworkMessageId
376-
377-
| summarize count(NetworkMessageId)
378-
321+
| summarize count()
379322
);
380-
381323
// get the number of threats unsuccessfully cleaned up post delivery.
382-
383324
let FN_ZAP_Unsuccessful = toscalar(
384-
385325
EmailPostDeliveryFiltered
386-
387326
| where ActionType in ("Malware ZAP","Phish ZAP") and ActionResult !in ("Success","AdminPolicy")
388-
389327
| join kind=leftsemi (EmailEventsClean) on NetworkMessageId
390-
391-
| summarize count(NetworkMessageId)
392-
328+
| summarize count()
393329
);
394-
395330
// join the administrative submissions to clean mailflow to find the additional miss
396-
397331
let FN_Admin_Submissions = toscalar(
398-
399332
CloudAppEventsFiltered
400-
401333
| join kind=rightsemi (EmailEventsClean) on NetworkMessageId
402-
403-
| summarize count(NetworkMessageId)
404-
334+
| summarize count()
405335
);
406-
407336
// print each result, and run the calculation to work out effectiveness at time of delivery and post delivery.
408-
409337
union withsource=Table
410-
411338
(print StatisticName="Mal/Phish Mailflow totals - Minus FPs", Value=toreal(Mal_Phish_Mailflow) - toreal(FP_ZAP)),
412-
413339
(print StatisticName="Admin Mal/Phish FNs Submitted", Value=toreal(FN_Admin_Submissions)),
414-
415340
(print StatisticName="Mal/Phish FPs Reverse Zapped", Value=toreal(FP_ZAP)),
416-
417341
(print StatisticName="Mal / Phish Successfully Zapped", Value=toreal(FN_ZAP_Successful)),
418-
419342
(print StatisticName="Mal / Phish UN-Successfully Zapped", Value=toreal(FN_ZAP_Unsuccessful)),
420-
421343
(print StatisticName="Effectiveness Post Delivery", Value=abs(round(((toreal(FN_Admin_Submissions)+toreal(FN_ZAP_Unsuccessful))/(toreal(Mal_Phish_Mailflow)+toreal(FN_ZAP_Successful)+toreal(FN_ZAP_Unsuccessful)+toreal(FN_Admin_Submissions)-toreal(FP_ZAP))*100-100),2))),
422-
423344
(print StatisticName="Effectiveness Pre-Delivery", Value=abs(round(((toreal(FN_Admin_Submissions)+toreal(FN_ZAP_Unsuccessful)+toreal(FN_ZAP_Successful))/(toreal(Mal_Phish_Mailflow)+toreal(FN_ZAP_Successful)+toreal(FN_ZAP_Unsuccessful)+toreal(FN_Admin_Submissions)-toreal(FP_ZAP))*100-100),2)))
424-
425345
| project StatisticName, Value
426346
```

0 commit comments

Comments
 (0)