Skip to content

Commit f443008

Browse files
Merge pull request #1132 from jinruishao/patch-21
Update custom-categories.md
2 parents cded828 + 2a0de75 commit f443008

File tree

1 file changed

+0
-34
lines changed

1 file changed

+0
-34
lines changed

articles/ai-services/content-safety/concepts/custom-categories.md

Lines changed: 0 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -16,38 +16,6 @@ ms.author: pafarley
1616

1717
Azure AI Content Safety lets you create and manage your own content moderation categories for enhanced moderation and filtering that matches your specific policies or use cases.
1818

19-
## Custom categories Training Pipeline Overview
20-
![image](https://github.com/user-attachments/assets/2e097136-0e37-4b5e-ba59-cafcfd733d72)
21-
22-
### Pipeline Components
23-
The training pipeline is designed to leverage a combination of universal data assets, user-provided inputs, and advanced GPT model fine-tuning techniques to produce high-quality models tailored to specific tasks.
24-
#### Data Assets
25-
Filtered Universal Data: This component gathers datasets from multiple domains to create a comprehensive and diverse dataset collection. The goal is to have a robust data foundation that provides a variety of contexts for model training.
26-
User Inputs
27-
Customer Task Metadata: Metadata provided by customers, which defines the specific requirements and context of the task they wish the model to perform.
28-
Customer Demonstrations: Sample demonstrations provided by customers that illustrate the expected output or behavior for the model. These demonstrations help optimize the model’s response based on real-world expectations.
29-
30-
#### Optimized Customer Prompt
31-
Based on the customer metadata and demonstrations, an optimized prompt is generated. This prompt refines the inputs provided to the model, aligning it closely with customer needs and enhancing the model’s task performance.
32-
33-
#### GPTX Synthetic Task-Specific Dataset
34-
Using the optimized prompt and filtered universal data, a synthetic, task-specific dataset is created. This dataset is tailored to the specific task requirements, enabling the model to understand and learn the desired behaviors and patterns.
35-
### Model Training and Fine-Tuning
36-
37-
#### Model Options: The pipeline supports multiple language models (LM), including Zcode, SLM, or any other language model (LM) suitable for the task.
38-
Task-Specific Fine-Tuned Model: The selected language model is fine-tuned on the synthetic task-specific dataset to produce a model that is highly optimized for the specific task.
39-
User Outputs
40-
41-
#### ONNX Model: The fine-tuned model is converted into an ONNX (Open Neural Network Exchange) model format, ensuring compatibility and efficiency for deployment.
42-
Deployment: The ONNX model is deployed, enabling users to make inference calls and access the model’s predictions. This deployment step ensures that the model is ready for production use in customer applications.
43-
Key Features of the Training Pipeline
44-
45-
#### Task Specificity: The pipeline allows for the creation of models finely tuned to specific customer tasks, thanks to the integration of customer metadata and demonstrations.
46-
- Scalability and Flexibility: The pipeline supports multiple language models, providing flexibility in choosing the model architecture best suited to the task.
47-
- Efficiency in Deployment: The conversion to ONNX format ensures that the final model is lightweight and efficient, optimized for deployment environments.
48-
- Continuous Improvement: By using synthetic datasets generated from diverse universal data sources, the pipeline can continuously improve model quality and applicability across various domains.
49-
50-
5119
## Types of customization
5220

5321
There are multiple ways to define and use custom categories, which are detailed and compared in this section.
@@ -82,8 +50,6 @@ This implementation works on text content and image content.
8250
## How it works
8351

8452
### [Custom categories (standard) API](#tab/standard)
85-
![image](https://github.com/user-attachments/assets/5c377ec4-379b-4b41-884c-13524ca126d0)
86-
8753

8854
The Azure AI Content Safety custom categories feature uses a multi-step process for creating, training, and using custom content classification models. Here's a look at the workflow:
8955

0 commit comments

Comments
 (0)