|
| 1 | +--- |
| 2 | +title: "Incident response in Azure AI Content Safety" |
| 3 | +titleSuffix: Azure AI services |
| 4 | +description: Learn about content incidents and how you can use Azure AI Content Safety to handle them on your platform. |
| 5 | +#services: cognitive-services |
| 6 | +author: PatrickFarley |
| 7 | +manager: nitinme |
| 8 | +ms.service: azure-ai-content-safety |
| 9 | +ms.custom: build-2024 |
| 10 | +ms.topic: conceptual |
| 11 | +ms.date: 04/11/2024 |
| 12 | +ms.author: pafarley |
| 13 | +--- |
| 14 | + |
| 15 | +# Incident response |
| 16 | + |
| 17 | +In content moderation scenarios, incident response is the process of identifying, analyzing, containing, eradicating, and recovering from cyber incidents that involve inappropriate or harmful content on online platforms. |
| 18 | + |
| 19 | +An incident may involve a set of emerging content patterns (text, image, or other modalities) that violate Microsoft community guidelines or the customers' own policies and expectations. These incidents need to be mitigated quickly and accurately to avoid potential live site issues or harm to users and communities. |
| 20 | + |
| 21 | +## Incident response API features |
| 22 | + |
| 23 | +One way to deal with emerging content incidents is to use [Blocklists](/azure/ai-services/content-safety/how-to/use-blocklist), but that only allows exact text matching and no image matching. The Azure AI Content Safety incident response API offers the following advanced capabilities: |
| 24 | +- semantic text matching using embedding search with a lightweight classifier |
| 25 | +- image matching with a lightweight object-tracking model and embedding search. |
| 26 | + |
| 27 | +## How it works |
| 28 | + |
| 29 | +First, you use the API to create an incident object with a description. Then you add any number of image or text samples to the incident. No training step is needed. |
| 30 | + |
| 31 | +Then, you can include your defined incident in a regular text analysis or image analysis request. The service will indicate whether the submitted content is an instance of your incident. The service can still do other content moderation tasks in the same API call. |
| 32 | + |
| 33 | +## Limitations |
| 34 | + |
| 35 | +### Language availability |
| 36 | + |
| 37 | +The text incident response API supports all languages that are supported by Content Safety text moderation. See [Language support](/azure/ai-services/content-safety/language-support). |
| 38 | + |
| 39 | +### Input limitations |
| 40 | + |
| 41 | +See the following table for the input limitations of the incident response API: |
| 42 | + |
| 43 | +| Object | Limitation | |
| 44 | +| :------------ | :----------- | |
| 45 | +| Maximum length of an incident name | 100 characters | |
| 46 | +| Maximum number of text/image samples per incident | 1000 | |
| 47 | +| Maximum size of each sample | Text: 500 characters<br>Image: 4 MB | |
| 48 | +| Maximum number of text or image incidents per resource| 100 | |
| 49 | +| Supported Image formats | BMP, GIF, JPEG, PNG, TIF, WEBP | |
| 50 | + |
| 51 | +### Region availability |
| 52 | + |
| 53 | +To use this API, you must create your Azure AI Content Safety resource in one of the supported regions: |
| 54 | +- East US |
| 55 | +- Sweden Central |
| 56 | + |
| 57 | +## Next steps |
| 58 | + |
| 59 | +Follow the how-to guide to use the Azure AI Content Safety incident response API. |
| 60 | + |
| 61 | +* [Use the incident response API](../how-to/incident-response.md) |
0 commit comments