Merge pull request #4899 from laujan/paul-4894-face

Paul 4894 face

Author: v-regandowner

articles/ai-services/content-understanding/face/overview.md

@@ -0,0 +1,95 @@
+---
+title: Azure AI Content Understanding face overview
+titleSuffix: Azure AI services
+description: Learn about Azure AI Content Understanding face solutions.
+author: laujan
+ms.author: quentinm
+manager: nitinme
+ms.service: azure-ai-content-understanding
+ms.topic: overview
+ms.date: 05/19/2025
+ms.custom: build-2025-understanding-refresh
+---
+
+# Azure AI Content Understanding face solutions (preview)
+
+> [!IMPORTANT]
+>
+> * Azure AI Content Understanding is available in preview. Public preview releases provide early access to features that are in active development.
+> * Features, approaches, and processes can change or have limited capabilities, before General Availability (GA).
+> * For more information, *see* [**Supplemental Terms of Use for Microsoft Azure Previews**](https://azure.microsoft.com/support/legal/preview-supplemental-terms).
+
+Azure AI Content Understanding provides a cloud-based solution for face detection, enrollment, and recognition, enabling secure and intelligent applications. Developers can apply these capabilities to detect faces, organize them into a structured directory, and perform recognition tasks for identity verification and content management.
+
+This service is ideal for building secure access systems, streamlining photo management, or implementing intelligent attendance and check-in solutions. It supports both standalone face records and structured person identity management, offering flexibility for various real-world scenarios.
+
+## Business use cases
+
+Content Understanding enables a wide range of real-world applications, including face detection, verification, identification, and large-scale content processing.
+
+##### Detect faces in images
+
+Automatically identify faces in an image and return their bounding boxes. This capability simplifies tasks like highlighting, blurring, or counting faces without manual review. Common use cases include:
+
+* Cropping detected faces for ID photos, albums, or personalized content.
+* Blurring faces to ensure privacy before sharing images publicly.
+* Counting people in event photos, crowd scenes, or security footage.
+
+##### Verify if two faces match
+
+Compare a face in one image with another face or an enrolled person and determine if they belong to the same individual. This comparison feature is ideal for identity verification scenarios such as photo ID checks or sign-ins. Common use cases include:
+
+* Verifying if a driver's selfie matches their profile photo.
+* Confirming a student's identity before starting an online exam.
+* Comparing a live photo with an uploaded ID for identity confirmation.
+
+##### Identify a person from their face
+
+Match a face in a photo to a saved list of people and identify them. Common use cases include:
+
+* Matching a patient's face to hospital records during check-in.
+* Identifying a student or employee from their face photo.
+* Recognizing someone from a watch list entering a secure area.
+
+##### Save faces for faster future searches
+
+Index faces from images to enable quicker searches later without needing to reprocess the original content. This feature is especially useful for recurring search scenarios. Common use cases include:
+
+* Extracting and saving faces from group photos at events to recognize returning participants in future sessions.
+* Processing images from school activities or sports events to easily identify students or athletes in subsequent searches.
+* Matching a theme park visitor's face to records from previous visits without rescanning years of archived photos.
+
+## Face capabilities
+
+Content Understanding offers robust face capabilities, including detection, quality analysis, identity enrollment, and search. Central to these features is the **person directory**—a scalable, structured repository for managing persons and faces.
+
+### Person Directory
+
+The person directory is a flexible system for organizing face data and identity profiles. Key features include:
+* **Face enrollment**: Add detected faces as standalone entries or associate them with specific persons.
+* **Person enrollment**: Create identity records that link to one or more enrolled faces.
+* **Metadata management**: Update metadata, apply tags, and manage relationships between persons and faces.
+* **Dynamic associations**: Associate or disassociate faces from persons without losing data, ensuring clean and maintainable identity management.
+
+:::image type="content" source="../media/face/person-directory-overview.png" alt-text="Diagram illustrating the flow of the person directory.":::
+
+### Search capabilities
+
+The person directory enables powerful search and matching functionalities:
+* **Identify candidate persons**: Match an input face to candidate persons in the directory.
+* **Find similar faces**: Search for all similar faces across the entire directory.
+
+## Key benefits
+
+Content Understanding provides advanced face recognition capabilities tailored for secure, scalable, and intelligent applications:
+* **Comprehensive face intelligence**: Detect, enroll, and recognize faces seamlessly using a unified cloud-based service. It supports both standalone face records and structured person identity management.
+* **Adaptable and scalable for diverse scenarios**: Enables secure access, streamlined check-ins, customer recognition, and efficient photo management with rapid, accurate face searches across extensive collections.
+
+## Data privacy and security
+
+Azure AI Content Understanding adheres to Microsoft's strict policies on customer data protection and privacy. Developers should review these policies to ensure compliance and understand how data is handled. For more details, visit the [Data protection and privacy](https://www.microsoft.com/trust-center/privacy) page.
+
+## Next steps
+
+* Learn to build a [**person directory**](../tutorial/build-person-directory.md).
+* Review code sample: [**person directory**](https://github.com/Azure-Samples/azure-ai-content-understanding-python/blob/zhizho/face/notebooks/build_person_directory.ipynb).

articles/ai-services/content-understanding/media/face/person-directory-overview.png

@@ -54,6 +54,9 @@ items:
   - name: Video
     href: video/overview.md
     displayName: video, audio, voice, recognition, synthesis, speaker, identification, verification, diarization, transcription, translation, language, understanding, sentiment, analysis, emotion, detection, pronunciation, model
+  - name: Face
+    displayName: face, recognition, detection, analysis, identification, verification
+    href: face/overview.md
   - name: Best practices
     displayName: best practices, analyzers, optimization, fields
     href: concepts/best-practices.md   
@@ -82,6 +85,9 @@ items:
     - name: Build a retrieval-augmented solution
       displayName: RAG, retrieval, augmented, generation, knowledge, base, search, index, vector
       href: tutorial/build-rag-solution.md
+    - name: Build a person directory
+      displayName: person, directory, search, index, vector
+      href: tutorial/build-person-directory.md
 - name: Responsible AI
   items:
     - name: Transparency note

articles/ai-services/content-understanding/media/face/person-directory-processes.png

@@ -0,0 +1,252 @@
+---
+title: Build a person directory with Azure AI Content Understanding Face APIs
+titleSuffix: Azure AI services
+description: Learn to build a person directory with Content Understanding Face APIs
+author: laujan
+ms.author: quentinm
+manager: nitinme
+ms.service: azure-ai-content-understanding
+ms.topic: tutorial
+ms.date: 05/19/2025
+---
+
+# Tutorial: Build a person directory
+
+A person directory provides a structured approach to storing face data for recognition tasks. It allows you to add individual faces, search for visually similar faces, and create person profiles. You can associate faces with these profiles and match new face images to known individuals. This setup supports both flexible face matching and identity recognition across images and videos.
+
+:::image type="content" source="../media/face/person-directory-processes.png" alt-text="Diagram illustrating the processes of enrollment and search in a person directory.":::
+
+## Data storage recommendation
+
+For secure and scalable access, we recommend storing all face images in Azure Blob Storage. When making API calls, ensure that the face URLs reference these stored images in Blob Storage.
+
+## Enrollment
+
+Enrollment involves the following steps:
+
+1. [Create an empty person directory](#create-an-empty-person-directory)
+1. [Add persons](#add-persons)
+1. [Add faces and associate with a person](#add-faces-and-associate-with-a-person)
+  
+### Create an empty person directory
+
+To create a new person directory, send a `PUT` request to the API endpoint. This directory serves as the container for storing faces and associated persons.
+
+```http
+PUT {endpoint}/contentunderstanding/personDirectories/{personDirectoryId}?api-version=2025-05-01-preview
+Content-Type: application/json
+
+{
+  "description": "A brief description of the directory",
+  "tags": {
+    "project": "example-project",
+    "owner": "team-name"
+  }
+}
+```
+
+* `personDirectoryId`: A unique, user-defined identifier for the directory within the resource.
+* `description`: (Optional) A short description of the directory's purpose.
+* `tags`: (Optional) Key-value pairs to help organize and manage the directory.
+
+The API creates the directory and returns a confirmation response.
+
+```json
+200 OK
+
+{
+  "personDirectoryId": "{personDirectoryId}",
+  "description": "A brief description of the directory",
+  "createdAt": "2025-05-01T18:46:36.051Z",
+  "lastModifiedAt": "2025-05-01T18:46:36.051Z",
+  "tags": {
+    "project": "example-project",
+    "owner": "team-name"
+  },
+  "personCount": 0,
+  "faceCount": 0
+}
+```
+
+### Add persons
+
+To recognize or manage individuals, you need to create a person profile. Once created, you can associate faces with this person.
+
+```http
+POST {endpoint}/contentunderstanding/personDirectories/{personDirectoryId}/persons?api-version=2025-05-01-preview
+Content-Type: application/json
+
+{
+  "tags": {
+    "name": "Alice",
+    "age": "20"
+  }
+}
+```
+
+* `personDirectoryId`: The unique identifier of the directory created in Step 1.
+* `tags`: Key-value pairs to describe the person, such as their name or age.
+
+The API returns a `personId` that uniquely identifies the created person.
+
+```json
+200 OK
+
+{
+  "personId": "4f66b612-e57d-4d17-9ef7-b951aea2cf0f",
+  "tags": {
+    "name": "Alice",
+    "age": "20"
+  }
+}
+```
+
+### Add faces and associate with a person
+
+You can add a face to the directory and optionally associate it with an existing person. The API supports both image URLs and base64-encoded image data.
+
+```http
+POST {endpoint}/contentunderstanding/personDirectories/{personDirectoryId}/faces?api-version=2025-05-01-preview
+Content-Type: application/json
+
+{
+  "faceSource": {
+    "url": "https://mystorageaccount.blob.core.windows.net/container/face.jpg",
+    // "data": "<base64 data>",
+    "imageReferenceId": "face.jpg",
+    "targetBoundingBox": {
+      "left": 33,
+      "top": 73,
+      "width": 262,
+      "height": 324
+    }
+  },
+  "qualityThreshold": "medium",
+  "personId": "{personId}"
+}
+```
+
+- `personDirectoryId`: The unique identifier of the person directory created in Step 1.
+- `faceSource`: Specifies the face image.
+  - `url`: The file path of the image stored in Azure Blob Storage.
+  - `data`: Base64-encoded image data as optional alternative to `url`.
+  - `imageReferenceId`: (Optional) A user-defined identifier for the image. This identifier can be helpful for tracking the image's origin or for mapping it to other data.
+  - `targetBoundingBox`: (Optional) Approximate location of the face in the image. If omitted, the API detects and uses the largest face.
+- `qualityThreshold`: (Optional) Filters face quality (`low`, `medium`, or `high`). The default is `medium`, meaning only medium or high-quality faces are stored. Lower quality faces are rejected.
+- `personId`: (Optional) The `personId` of an existing person to associate the face with.
+
+The API returns a `faceId` that uniquely identifies the created face with the detected `boundingBox` of the face.
+
+```json
+{
+  "faceId": "{faceId}",
+  "personId": "{personId}",
+  "imageReferenceId": "face.jpg",
+  "boundingBox": {
+    "left": 30,
+    "top": 78,
+    "width": 251,
+    "height": 309
+  }
+}
+```
+
+## Search
+
+After creating your person directory and adding face images with optional person associations, you can perform two key tasks:
+
+1. **[Identify a person](#identify-a-person)**: Match a face image against enrolled persons in the directory and determine the most likely identity.
+1. **[Find similar faces](#find-similar-faces)**: Search for visually similar faces across all stored face entries in the directory.
+
+These capabilities enable robust face recognition and similarity matching for various applications.
+
+### Identify a person
+
+Identify the most likely person matches by comparing the input face against enrolled persons in the directory.
+
+```http
+POST {endpoint}/contentunderstanding/personDirectory/{personDirectoryId}/persons:identify?api-version=2025-05-01-preview
+Content-Type: application/json
+
+{
+  "faceSource": {
+    "url": "https://mystorageaccount.blob.core.windows.net/container/unknown.jpg",
+    "targetBoundingBox": { ... }
+  },
+  "maxPersonCandidates": 1
+}
+```
+
+- `faceSource.url`: The URL of the input face image stored in Azure Blob Storage.
+- `faceSource.targetBoundingBox`: (Optional) The approximate bounding box of the face in the image. If omitted, the API detects the largest face.
+- `maxPersonCandidates`: (Optional) The maximum number of person candidates to return. Default is 1.
+
+The API returns the detected bounding box of the face along with the top person candidates.
+
+```json
+{
+  "detectedFace": {
+    "boundingBox": { ... }
+  },
+  "personCandidates": [
+    {
+      "personId": "{personId1}",
+      "tags": {
+        "name": "Alice",
+        "age": "20"
+      },
+      "confidence": 0.92
+    }
+  ]
+}
+```
+
+- `detectedFace.boundingBox`: The bounding box of the detected face in the input image.
+- `personCandidates`: A list of potential matches, each with a `personId`, associated `tags`, and a `confidence` score indicating the likelihood of a match.
+
+### Find similar faces
+
+Find visually similar faces from all stored face entries in the directory.
+
+```http
+POST {endpoint}/personDirectory/{personDirectoryId}/faces:find?api-version=2025-05-01-preview
+Content-Type: application/json
+
+{
+  "faceSource": {
+    "url": "https://mystorageaccount.blob.core.windows.net/container/target.jpg",
+    "targetBoundingBox": { ... }
+  },
+  "maxSimilarFaces": 10
+}
+```
+
+- `faceSource.url`: The URL of the input face image stored in Azure Blob Storage.
+- `faceSource.targetBoundingBox`: (Optional) The approximate bounding box of the face in the image. If omitted, the API detects the largest face.
+- `maxSimilarFaces`: (Optional) The maximum number of similar faces to return. Defaults to 1000, with a maximum limit of 1000.
+
+The API returns the detected bounding box of the face along with the most similar faces from the directory.
+
+```json
+{
+  "detectedFace": {
+    "boundingBox": { ... }
+  },
+  "similarFaces": [
+    {
+      "faceId": "{faceId}",
+      "boundingBox": { ... },
+      "confidence": 0.92,
+      "imageReferenceId": "face.jpg"
+    }
+  ]
+}
+```
+
+- `detectedFace.boundingBox`: The bounding box of the detected face in the input image.
+- `similarFaces`: A list of similar faces, each with a `faceId`, `boundingBox`, `confidence` score, and an `imageReferenceId` indicating the source image.
+
+
+## Next steps
+
+Explore how to identify individuals in video content using the [Azure AI Content Understanding video solutions (preview)](../video/overview.md).