Skip to content

Commit 3f003b0

Browse files
committed
custom avatar in foundry
1 parent e46f4e5 commit 3f003b0

File tree

4 files changed

+54
-47
lines changed

4 files changed

+54
-47
lines changed

articles/ai-services/speech-service/includes/how-to/custom-avatar/create-avatar/ai-foundry.md

Lines changed: 48 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -19,23 +19,22 @@ You need a video recording of the talent reading a consent statement acknowledgi
1919

2020
You need video recordings of your avatar talent as training data. You upload these videos when you prepare training data. For more information, see [Add training data](#step-3-add-training-data).
2121

22-
## Step 1: Create a custom avatar project
22+
## Step 1: Start fine-tuning
2323

24-
To create a custom avatar project, follow these steps:
25-
1. Sign in to the [Speech Studio](https://speech.microsoft.com) and select your subscription and Speech resource.
26-
1. Select **Custom avatar (preview)**.
27-
28-
:::image type="content" source="../../../../media/custom-avatar/speech-studio/select-custom-avatar-tile.png" alt-text="Screenshot of the tile to select custom avatar among other tiles." lightbox="../../../../media/custom-avatar/speech-studio/select-custom-avatar-tile.png":::
29-
30-
1. Select **+Create a project**.
31-
1. Follow the instructions provided by the wizard to create your project.
24+
> [!TIP]
25+
> Don't mix data for different avatars in one fine-tuning workspace. Each avatar must have its own fine-tuning workspace.
3226
33-
> [!TIP]
34-
> Don't mix data for different avatars in one project. Always create a new project for a new avatar.
27+
To fine-tune a custom avatar, follow these steps:
3528

36-
1. Select the new project by name. Then you see these menu items in the left panel: **Set up avatar talent**, **Prepare training data**, **Train model**, and **Deploy model**.
29+
1. Go to your Azure AI Foundry project in the [Azure AI Foundry portal](https://ai.azure.com). If you need to create a project, see [Create an Azure AI Foundry project](/azure/ai-foundry/how-to/create-projects).
30+
1. Select **Fine-tuning** from the left pane.
31+
1. Select **AI Service fine-tuning** > **+ Fine-tune**.
3732

38-
:::image type="content" source="../../../../media/custom-avatar/speech-studio/new-empty-project.png" alt-text="Screenshot of the new empty custom avatar project." lightbox="../../../../media/custom-avatar/speech-studio/new-empty-project.png":::
33+
:::image type="content" source="../../../../media/custom-voice/professional-voice/fine-tune-azure-ai-services.png" alt-text="Screenshot of the page to select fine-tuning of Azure AI Services models." lightbox="../../../../media/custom-voice/professional-voice/fine-tune-azure-ai-services.png":::
34+
35+
1. In the wizard, select **Custom avatar (text to speech avatar fine-tuning)**.
36+
1. Select **Next**.
37+
1. Follow the instructions provided by the wizard to create your fine-tuning workspace.
3938

4039
## Step 2: Add avatar talent consent
4140

@@ -46,24 +45,28 @@ You must provide a video file with a recorded statement from your avatar talent,
4645
- If you want to create a voice sync for avatar during avatar training, a custom voice resembling your avatar is created alongside the custom avatar. The voice is used exclusively with the specified avatar. Your consent statement must include both the custom avatar and the voice sync for avatar.
4746
- If you don't create a voice sync for avatar, only the custom avatar is trained, and your consent statement must reflect this scope.
4847

49-
You can find the verbal consent statement in multiple languages via the [Azure-Samples/cognitive-services-speech-sdk](https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/sampledata/customavatar/verbal-statement-all-locales.txt) GitHub repository. The language of the verbal statement must be the same as your recording. See also the [Disclosure for voice talent](/legal/cognitive-services/speech-service/disclosure-voice-talent?context=/azure/ai-services/speech-service/context/context).
48+
You can find the verbal consent statement in multiple languages via the [Azure-Samples/cognitive-services-speech-sdk](https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/sampledata/customavatar/verbal-statement-all-locales.txt) GitHub repository. The language of the verbal statement must be the same as your recording. See also the [Disclosure for avatar talent](/legal/cognitive-services/speech-service/disclosure-voice-talent?context=/azure/ai-services/speech-service/context/context).
5049

5150
For more information about recording the consent video, see [How to record video samples](../../../../text-to-speech-avatar/custom-avatar-record-video-samples.md).
5251

5352
To add an avatar talent profile and upload their consent statement in your project, follow these steps:
54-
1. Sign in to the [Speech Studio](https://speech.microsoft.com).
55-
1. Select **Custom avatar** > Your project name > **Set up avatar talent** > **Upload consent video**.
53+
54+
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com).
55+
1. Select **Fine-tuning** from the left pane and then select **AI Service fine-tuning**.
56+
1. Select the custom avatar fine-tuning task (by model name) that you [started as described in the previous section](#step-1-start-fine-tuning).
57+
1. Select **Set up avatar talent** > **Upload consent video**.
58+
5659
1. On the **Upload consent video** page, follow the instructions to upload the avatar talent consent video you recorded beforehand.
5760
- Select the avatar type to build. Build a voice sync for avatar which sounds like your avatar talent together with the avatar model, or build avatar without the voice sync for avatar. The option to build a voice sync for avatar is only available in the Southeast Asia, West Europe, and West US 2 regions.
5861
- Select the speaking language of the verbal consent statement recorded by the avatar talent.
5962
- Enter the avatar talent name and your company name in the same language as the recorded statement.
6063
- The avatar talent name must be the name of the person who recorded the consent statement.
6164
- The company name must match the company name that was spoken in the recorded statement.
6265
- You can choose to upload your data from local files, or from a shared storage with Azure Blob.
63-
64-
:::image type="content" source="../../../../media/custom-avatar/speech-studio/upload-consent-video.png" alt-text="Screenshot of the dialog where you select the talent consent video to upload." lightbox="../../../../media/custom-avatar/speech-studio/upload-consent-video.png":::
6566

66-
5. Select **Upload**.
67+
1. Select local files from your computer or enter the Azure Blob storage URL where your data is stored.
68+
1. Select **Next**.
69+
1. Review the upload details, and select **Upload**.
6770

6871
After the avatar talent consent upload is successful, you can proceed to train your custom avatar model.
6972

@@ -78,30 +81,29 @@ All data you upload must meet the requirements for the data type that you choose
7881
When you're ready to upload your data, go to the **Prepare training data** tab to add your data.
7982

8083
To upload training data, follow these steps:
81-
1. Sign in to the [Speech Studio](https://speech.microsoft.com).
82-
1. Select **Custom avatar** > Your project name > **Prepare training data** > **Upload data**.
83-
84-
:::image type="content" source="../../../../media/custom-avatar/speech-studio/upload-training-data.png" alt-text="Screenshot of the training data preparation page with the button to upload training data." lightbox="../../../../media/custom-avatar/speech-studio/upload-training-data.png":::
85-
86-
1. In the **Upload data** wizard, choose a data type and then select **Next**. For more information about the data types (including **Naturally speaking**, **Silent**, **Gesture**, and **Status 0 speaking**), see [what video clips to record](../../../../text-to-speech-avatar/custom-avatar-record-video-samples.md#what-video-clips-to-record).
84+
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com).
85+
1. Select **Fine-tuning** from the left pane and then select **AI Service fine-tuning**.
86+
1. Select the custom avatar fine-tuning task (by model name) that you [started as described in the previous section](#step-1-start-fine-tuning).
87+
1. Select **Prepare training data** > **Upload data**.
88+
1. In the **Upload data** wizard, choose a data type and then select **Next**. For more information about the data types (including **Naturally Speaking**, **Silent Status**, **Gesture**, and **Status 0 speaking**), see [what video clips to record](../../../../text-to-speech-avatar/custom-avatar-record-video-samples.md#what-video-clips-to-record).
8789
1. Select local files from your computer or enter the Azure Blob storage URL where your data is stored.
8890
1. Select **Next**.
89-
1. Review the upload details, and select **Submit**.
90-
91-
Data files are automatically validated when you select **Submit**. Data validation includes series of checks on the video files to verify their file format, size, and total volume. If there are any errors, fix them and submit again.
91+
1. Review the upload details, and select **Upload**.
9292

93-
After you upload the data, you can check the data overview which indicates whether you provided enough data to start training. This screenshot shows an example of enough data added for training an avatar without other gestures.
93+
Data files are automatically validated when you select **Upload**. Data validation includes series of checks on the video files to verify their file format, size, and total volume. If there are any errors, fix them and submit again.
9494

95-
:::image type="content" source="../../../../media/custom-avatar/speech-studio/review-training-data.png" alt-text="Screenshot of enough data added for training an avatar without other gestures." lightbox="../../../../media/custom-avatar/speech-studio/review-training-data.png":::
95+
After you upload the data, you can check the data overview which indicates whether you provided enough data to start training.
9696

9797
## Step 4: Train your avatar model
9898

9999
> [!IMPORTANT]
100100
> All the training data in the project is included in the training. The model quality is highly dependent on the data you provided, and you're responsible for the video quality. Make sure you record the training videos according to the [how to record video samples guide](../../../../text-to-speech-avatar/custom-avatar-record-video-samples.md).
101101
102-
To create a custom avatar in Speech Studio, follow these steps for one of the following methods:
103-
1. Sign in to the [Speech Studio](https://speech.microsoft.com).
104-
1. Select **Custom avatar** > Your project name > **Train model** > **Train model**.
102+
To create a custom avatar in the Azure AI Foundry portal, follow these steps for one of the following methods:
103+
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com).
104+
1. Select **Fine-tuning** from the left pane and then select **AI Service fine-tuning**.
105+
1. Select the custom avatar fine-tuning task (by model name) that you [started as described in the previous section](#step-1-start-fine-tuning).
106+
1. Select **Train model** > **+ Train model**.
105107
1. Enter a Name to help you identify the model. Choose a name carefully. The model name is used as the avatar name in your synthesis request by the SDK and SSML input. Only letters, numbers, hyphens, and underscores are allowed. Use a unique name for each model.
106108

107109
> [!IMPORTANT]
@@ -123,7 +125,7 @@ To copy your custom avatar model to another project:
123125
1. Select the subscription, region, AI Services resource for Speech, and project where you want to copy the model to. You must have an AI Services resource for Speech and project in the target region, otherwise you need to create them first.
124126
1. Select **Submit** to copy the model.
125127

126-
Once the model is copied, you see a notification in the Speech Studio.
128+
Once the model is copied, you see a notification in the Azure AI Foundry portal.
127129

128130
Navigate to the project where you copied the model to deploy the model copy.
129131

@@ -132,23 +134,27 @@ Navigate to the project where you copied the model to deploy the model copy.
132134
After you successfully created and trained your avatar model, you deploy it to your endpoint.
133135

134136
To deploy your avatar:
135-
1. Sign in to the [Speech Studio](https://speech.microsoft.com).
136-
1. Select **Custom avatar** > Your project name > **Deploy model**.
137-
1. Select **Deploy model** and select a model that you want to deploy.
137+
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com).
138+
1. Select **Fine-tuning** from the left pane and then select **AI Service fine-tuning**.
139+
1. Select the custom avatar fine-tuning task (by model name) that you [started as described in the previous section](#step-1-start-fine-tuning).
140+
1. Select **Deploy model** > **Deploy model**.
141+
1. Select a model that you want to deploy.
138142
1. Select **Deploy** to start the deployment.
143+
139144
> [!IMPORTANT]
140145
> When a model is deployed, you pay for continuous up time of the endpoint regardless of your interaction with that endpoint. Check the pricing note on how model deployment is charged. You can delete a deployment when the model isn't in use to reduce spending and conserve resources.
141146
142-
After you deploy your custom avatar, it's available to use in Speech Studio or via API:
143-
- The avatar appears in the avatar list of [text to speech avatar on Speech Studio](https://speech.microsoft.com/portal/talkingavatar).
144-
- The avatar appears in the avatar list of [live chat avatars via Speech Studio](https://speech.microsoft.com/portal/livechat).
147+
After you deploy your custom avatar, it's available to use in the Azure AI Foundry portal or via API:
148+
- The avatar appears in the avatar list of [text to speech avatar on Azure AI Foundry portal](https://speech.microsoft.com/portal/talkingavatar).
149+
- The avatar appears in the avatar list of [live chat avatars via Azure AI Foundry portal](https://speech.microsoft.com/portal/livechat).
145150
- You can call the avatar from the SDK and SSML input by specifying the avatar model name. For more information, see the [avatar properties](../../../../text-to-speech-avatar/batch-synthesis-avatar-properties.md#avatar-properties).
146151

147152
### Remove a deployment
148153

149154
To remove your deployment, follow these steps:
150-
1. Sign in to the [Speech Studio](https://speech.microsoft.com).
151-
1. Navigate to **Custom Avatar** > Your project name > **Deploy model**.
155+
1. Sign in to the [Azure AI Foundry portal](https://ai.azure.com).
156+
1. Select **Fine-tuning** from the left pane and then select **AI Service fine-tuning**.
157+
1. Select the custom avatar fine-tuning task (by model name) that you [started as described in the previous section](#step-1-start-fine-tuning).
152158
1. Select the deployment on the **Deploy model** page. The model is actively hosted if the status is "Succeeded".
153159
1. You can select the **Delete deployment** button and confirm the deletion to remove the hosting.
154160

articles/ai-services/speech-service/includes/how-to/professional-voice/create-consent/ai-foundry.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ To add a voice talent profile and upload their consent statement, follow these s
3232
1. In the **Add new voice talent** wizard, select the target scenarios for the voice talent. The target scenarios must be consistent with what you provided in the application form. The scenarios are used to help identify the voice talent and to ensure that the voice model is trained for the intended use cases.
3333
1. Optionally in the **Voice characteristics** text box, enter a description of the characteristics of the voice you're going to create.
3434
1. Select **Next**.
35-
1. On the **Upload verbal statement** page, follow the instructions to upload the voice talent statement you've recorded beforehand.
35+
1. On the **Upload verbal statement** page, follow the instructions to upload the voice talent statement you recorded beforehand.
3636

3737
- Enter the voice talent name and company name. The voice talent name must be the name of the person who recorded the consent statement. Enter the name in the same language used in the recorded statement. The company name must match the company name that was spoken in the recorded statement. Ensure the company name is entered in the same language as the recorded statement.
3838
- Make sure the verbal statement was [recorded](../../../../record-custom-voice-samples.md) with the same settings, environment, and speaking style as your training data.

articles/ai-services/speech-service/includes/how-to/professional-voice/create-project/ai-foundry.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,19 +14,20 @@ In the [Azure AI Foundry portal](https://ai.azure.com), you can fine-tune some A
1414

1515
To fine-tune a professional voice model, follow these steps:
1616

17-
1. Go to your project in the [Azure AI Foundry portal](https://ai.azure.com). If you need to create a project, see [Create an Azure AI Foundry project](/azure/ai-foundry/how-to/create-projects).
17+
1. Go to your Azure AI Foundry project in the [Azure AI Foundry portal](https://ai.azure.com). If you need to create a project, see [Create an Azure AI Foundry project](/azure/ai-foundry/how-to/create-projects).
1818
1. Select **Fine-tuning** from the left pane.
1919
1. Select **AI Service fine-tuning** > **+ Fine-tune**.
2020

2121
:::image type="content" source="../../../../media/custom-voice/professional-voice/fine-tune-azure-ai-services.png" alt-text="Screenshot of the page to select fine-tuning of Azure AI Services models." lightbox="../../../../media/custom-voice/professional-voice/fine-tune-azure-ai-services.png":::
2222

23-
1. In the wizard, select **Custom voice (professional voice fine-tuning)** for custom voice. Then select **Next**.
24-
1. Follow the instructions provided by the wizard to create your project.
23+
1. In the wizard, select **Custom voice (professional voice fine-tuning)**.
24+
1. Select **Next**.
25+
1. Follow the instructions provided by the wizard to create your fine-tuning workspace.
2526

2627
## Continue fine-tuning
2728

2829
Go to the Azure AI Speech documentation to learn how to continue fine-tuning your professional voice model:
29-
* [Add voice talent consent to the professional voice project](../../../../professional-voice-create-consent.md)
30+
* [Add voice talent consent](../../../../professional-voice-create-consent.md)
3031
* [Add training datasets](../../../../professional-voice-create-training-set.md)
3132
* [Train your voice model](../../../../professional-voice-train-voice.md)
3233
* [Deploy your professional voice model as an endpoint](../../../../professional-voice-deploy-endpoint.md)
295 KB
Loading

0 commit comments

Comments
 (0)