Skip to content

Commit 7941942

Browse files
committed
create fine-tune refresh in foundry portal
1 parent 2823aea commit 7941942

27 files changed

+36
-34
lines changed

articles/ai-services/speech-service/how-to-custom-speech-create-project.md

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,11 @@ zone_pivot_groups: foundry-speech-studio
1414

1515
# Customize speech models with fine-tuning
1616

17-
With custom speech, you can enhance speech recognition accuracy for your applications by using a custom model for real-time speech to text, speech translation, and batch transcription. The base model, trained with Microsoft-owned data, handles common spoken language well, but a custom model can improve domain-specific vocabulary and audio conditions by providing text and audio data for training. Additionally, you can train the model with structured text for custom pronunciations, display text formatting, and profanity filtering.
17+
With custom speech, you can enhance speech recognition accuracy for your applications by using a custom model for real-time speech to text, speech translation, and batch transcription.
18+
19+
You create a custom speech model by fine-tuning an Azure AI Speech base model with your own data. You can upload your data, test and train a custom model, compare accuracy between models, and deploy a model to a custom endpoint.
20+
21+
This article shows you how to use fine-tuning to create a custom speech model. For more information about custom speech, see the [custom speech overview](./custom-speech-overview.md) documentation.
1822

1923
> [!TIP]
2024
> You can bring your custom speech models from [Speech Studio](https://speech.microsoft.com) to the [Azure AI Foundry portal](https://ai.azure.com). In Azure AI Foundry, you can pick up where you left off by connecting to your existing Speech resource. For more information about connecting to an existing Speech resource, see [Connect to an existing Speech resource](../../ai-studio/ai-services/how-to/connect-ai-services.md#connect-azure-ai-services-after-you-create-a-project).
@@ -25,31 +29,30 @@ Custom speech fine-tuning includes models, training and testing datasets, and de
2529

2630
::: zone pivot="ai-foundry-portal"
2731

28-
In [Azure AI Foundry portal](https://ai.azure.com), you create a custom speech model by fine-tuning an Azure AI Speech base model with your own data. You can upload your data, test and train a custom model, compare accuracy between models, and deploy a model to a custom endpoint.
29-
30-
This article shows you how to use fine-tuning in Azure AI Foundry portal to create a custom speech model. For more information about custom speech, see the [custom speech overview](./custom-speech-overview.md) documentation.
31-
32-
33-
## Start fine-tuning a model with your data
34-
35-
In Azure AI Foundry, you can fine-tune some Azure AI services models. For example, you can fine-tune a model for custom speech. Each custom model is specific to a [locale](language-support.md?tabs=stt). For example, you might fine-tune a model for English in the United States.
32+
In the [Azure AI Foundry portal](https://ai.azure.com), you can fine-tune some Azure AI services models. For example, you can fine-tune a model for custom speech. Each custom model is specific to a [locale](language-support.md?tabs=stt). For example, you might fine-tune a model for English in the United States.
3633

37-
1. Go to your Azure AI Foundry project. If you need to create a project, see [Create an Azure AI Foundry project](../../ai-studio/how-to/create-projects.md).
34+
1. Go to your project in the [Azure AI Foundry portal](https://ai.azure.com). If you need to create a project, see [Create an Azure AI Foundry project](../../ai-studio/how-to/create-projects.md).
3835
1. Select **Fine-tuning** from the left pane.
3936
1. Select **AI Service fine-tuning** > **+ Fine-tune**.
4037

41-
:::image type="content" source="../../ai-studio/media/ai-services/fine-tune-azure-ai-services.png" alt-text="Screenshot of the page to select fine-tuning of Azure AI Services models." lightbox="../../ai-studio/media/ai-services/fine-tune-azure-ai-services.png":::
38+
:::image type="content" source="./media/ai-foundry/custom-speech/fine-tune-azure-ai-services.png" alt-text="Screenshot of the page to select fine-tuning of Azure AI Services models." lightbox="./media/ai-foundry/custom-speech/fine-tune-azure-ai-services.png":::
4239

4340
1. In the wizard, select **Speech to text (speech recognition)** for custom speech. Then select **Next**.
44-
1. Select the connected service resource that you want to use for fine-tuning. Then select **Next**.
41+
1. Select the connected service resource that you want to use for fine-tuning.
4542

46-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-select-connection.png" alt-text="Screenshot of the page to select the connected service resource that you want to use for fine-tuning." lightbox="./media/ai-studio/custom-speech/new-fine-tune-select-connection.png":::
43+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-select-connection.png" alt-text="Screenshot of the page to select the connected service resource that you want to use for fine-tuning." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-select-connection.png":::
4744

48-
In this example, we can choose from the following options:
45+
You can choose from the following connected resource types:
4946
- **AI Service**: The Azure AI Services multi-service resource that [came with the Azure AI Foundry project](../../ai-studio/ai-services/how-to/connect-ai-services.md#connect-azure-ai-services-when-you-create-a-project-for-the-first-time).
5047
- **Speech Service**: An Azure AI Speech resource that was [connected after the project was created](../../ai-studio/ai-services/how-to/connect-ai-services.md#connect-azure-ai-services-after-you-create-a-project).
5148

52-
1. Enter a name and description for the fine-tuning job. Then select **Next**.
49+
1. Enter the language, name, and description for the fine-tuning job. Then select **Create**.
50+
51+
Go to the Azure AI Speech documentation to continue fine-tuning your model.
52+
* [Upload training and testing datasets](./how-to-custom-speech-upload-data.md)
53+
* [Train a model](how-to-custom-speech-train-model.md)
54+
* [Test model quantitatively](how-to-custom-speech-evaluate-data.md) and [test model qualitatively](how-to-custom-speech-test-model-qualitatively.md)
55+
* [Deploy a model](how-to-custom-speech-deploy-model.md)
5356

5457
## View fine-tuned models
5558

@@ -58,7 +61,7 @@ You can access your custom speech models and deployments from the **Fine-tuning*
5861
1. Select **Fine-tuning** from the left pane.
5962
1. Select **AI Service fine-tuning**.
6063

61-
:::image type="content" source="./media/ai-studio/custom-speech/fine-tune-succeeded-list.png" alt-text="Screenshot of the page where you can view fine-tuned AI services models." lightbox="./media/ai-studio/custom-speech/fine-tune-succeeded-list.png":::
64+
:::image type="content" source="./media/ai-foundry/custom-speech/fine-tune-succeeded-list.png" alt-text="Screenshot of the page where you can view fine-tuned AI services models." lightbox="./media/ai-foundry/custom-speech/fine-tune-succeeded-list.png":::
6265

6366
::: zone-end
6467

@@ -79,7 +82,6 @@ Select the new project by name or select **Go to project**. You'll see these men
7982

8083
::: zone-end
8184

82-
8385
## Choose your model
8486

8587
There are a few approaches to using custom speech models:
@@ -100,5 +102,5 @@ Whether you train your own model or use a snapshot of a base model, you can use
100102
## Related content
101103

102104
* [Training and testing datasets](./how-to-custom-speech-test-and-train.md)
103-
* [Test model quantitatively](how-to-custom-speech-evaluate-data.md)
104105
* [Train a model](how-to-custom-speech-train-model.md)
106+
* [Test model quantitatively](how-to-custom-speech-evaluate-data.md)

articles/ai-services/speech-service/how-to-custom-speech-deploy-model.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,17 +31,17 @@ You can deploy an endpoint for a base or custom model, and then [update](#change
3131

3232
1. After you're satisfied with the test results, select **Deploy models** from the left menu. Then select **+ Deploy model**.
3333

34-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-deploy-model.png" alt-text="Screenshot of the page with an option to deploy the custom speech model." lightbox="./media/ai-studio/custom-speech/new-fine-tune-deploy-model.png":::
34+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-deploy-model.png" alt-text="Screenshot of the page with an option to deploy the custom speech model." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-deploy-model.png":::
3535

3636
1. In the **Deploy a new model** wizard, select the model that you want to deploy.
3737

38-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-deploy-model-select-and-deploy.png" alt-text="Screenshot of the page with an option to select the model that you want to deploy." lightbox="./media/ai-studio/custom-speech/new-fine-tune-deploy-model-select-and-deploy.png":::
38+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-deploy-model-select-and-deploy.png" alt-text="Screenshot of the page with an option to select the model that you want to deploy." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-deploy-model-select-and-deploy.png":::
3939

4040
1. Enter a name and description for the deployment. Select the box to agree to the terms of use. Then select **Deploy**.
4141

4242
1. After the deployment status is **Succeeded**, you can view the deployment details. Select the deployment to view the details like the endpoint ID.
4343

44-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-deploy-model-status-succeeded.png" alt-text="Screenshot of the page with an option to select the deployment to view the details like the endpoint ID." lightbox="./media/ai-studio/custom-speech/new-fine-tune-deploy-model-status-succeeded.png":::
44+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-deploy-model-status-succeeded.png" alt-text="Screenshot of the page with an option to select the deployment to view the details like the endpoint ID." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-deploy-model-status-succeeded.png":::
4545

4646

4747
::: zone-end

articles/ai-services/speech-service/how-to-custom-speech-evaluate-data.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,21 +29,21 @@ You can test the accuracy of your custom model by creating a test. A test requir
2929

3030
1. After the model is trained, select **Test models** from the left menu. Then select **+ Create test**.
3131

32-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-test-model.png" alt-text="Screenshot of the page with an option to test your custom speech model." lightbox="./media/ai-studio/custom-speech/new-fine-tune-test-model.png":::
32+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-test-model.png" alt-text="Screenshot of the page with an option to test your custom speech model." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-test-model.png":::
3333

3434
1. In the **Create a new test** wizard, select the test type. In this example, we select **Evaluate accuracy (Audio + transcript data)**. Then select **Next**.
3535

36-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-test-model-select-type.png" alt-text="Screenshot of the page with an option to select the test type." lightbox="./media/ai-studio/custom-speech/new-fine-tune-test-model-select-type.png":::
36+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-test-model-select-type.png" alt-text="Screenshot of the page with an option to select the test type." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-test-model-select-type.png":::
3737

3838
1. Select the data that you want to use for testing. Then select **Next**.
3939
1. Select up to two models to evaluate and compare accuracy. In this example, we select the model that we trained and the base model. Then select **Next**.
4040

41-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-test-model-select-models.png" alt-text="Screenshot of the page with an option to select up to two models to evaluate and compare accuracy." lightbox="./media/ai-studio/custom-speech/new-fine-tune-test-model-select-models.png":::
41+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-test-model-select-models.png" alt-text="Screenshot of the page with an option to select up to two models to evaluate and compare accuracy." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-test-model-select-models.png":::
4242

4343
1. Enter a name and description for the test. Then select **Next**.
4444
1. Review the settings and select **Create test**. You're taken back to the **Test models** page. The status of the data is **Processing**.
4545

46-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-test-model-status-processing.png" alt-text="Screenshot of the page that shows the status of the test as processing." lightbox="./media/ai-studio/custom-speech/new-fine-tune-test-model-status-processing.png":::
46+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-test-model-status-processing.png" alt-text="Screenshot of the page that shows the status of the test as processing." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-test-model-status-processing.png":::
4747

4848

4949

articles/ai-services/speech-service/how-to-custom-speech-train-model.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -35,17 +35,17 @@ You can use a custom model for a limited time after it was trained. You must per
3535

3636
1. After the data is processed, select **Train model** from the left menu. Then select **+ Train model**.
3737

38-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-train-model.png" alt-text="Screenshot of the page with an option to start training for a custom speech model." lightbox="./media/ai-studio/custom-speech/new-fine-tune-train-model.png":::
38+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-train-model.png" alt-text="Screenshot of the page with an option to start training for a custom speech model." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-train-model.png":::
3939

4040
1. In the **Train a new model** wizard, select the base model that you want to fine-tune. Then select **Next**.
4141

42-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-train-model-select-base.png" alt-text="Screenshot of the page with an option to select the base model that you want to fine-tune." lightbox="./media/ai-studio/custom-speech/new-fine-tune-train-model-select-base.png":::
42+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-train-model-select-base.png" alt-text="Screenshot of the page with an option to select the base model that you want to fine-tune." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-train-model-select-base.png":::
4343

4444
1. Select the data that you want to use for training. Then select **Next**.
4545
1. Enter a name and description for the model. Then select **Next**.
4646
1. Review the settings and select **Train a new model**. You're taken back to the **Train model** page. The status of the data is **Processing**.
4747

48-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-train-model-status-processing.png" alt-text="Screenshot of the page that shows the status of the training as processing." lightbox="./media/ai-studio/custom-speech/new-fine-tune-train-model-status-processing.png":::
48+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-train-model-status-processing.png" alt-text="Screenshot of the page that shows the status of the training as processing." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-train-model-status-processing.png":::
4949

5050
::: zone-end
5151

articles/ai-services/speech-service/how-to-custom-speech-upload-data.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,17 +25,17 @@ You need audio or text data for testing the accuracy of speech recognition or tr
2525

2626
1. From the new left menu in the fine-tuning page, select **Manage data** and then select **Add data**.
2727

28-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-add-data.png" alt-text="Screenshot of the page with an option to add data to the custom speech project." lightbox="./media/ai-studio/custom-speech/new-fine-tune-add-data.png":::
28+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-add-data.png" alt-text="Screenshot of the page with an option to add data to the custom speech project." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-add-data.png":::
2929

3030
1. In the **Add data** wizard, select the type of training data you want to add. In this example, we select **Audio + human-labeled transcript**. Then select **Next**.
3131

32-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-add-data-select-type.png" alt-text="Screenshot of the page with an option to select the type of training data you want to add." lightbox="./media/ai-studio/custom-speech/new-fine-tune-add-data-select-type.png":::
32+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-add-data-select-type.png" alt-text="Screenshot of the page with an option to select the type of training data you want to add." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-add-data-select-type.png":::
3333

3434
1. On the **Upload your data** page, select local files, Azure Blob Storage, or other shared web locations. Then select **Next**.
3535
1. Enter a name and description for the data. Then select **Next**.
3636
1. Review the data and select **Upload**. You're taken back to the **Manage data** page. The status of the data is **Processing**.
3737

38-
:::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-add-data-status-processing.png" alt-text="Screenshot of the page that shows the status of the data as processing." lightbox="./media/ai-studio/custom-speech/new-fine-tune-add-data-status-processing.png":::
38+
:::image type="content" source="./media/ai-foundry/custom-speech/new-fine-tune-add-data-status-processing.png" alt-text="Screenshot of the page that shows the status of the data as processing." lightbox="./media/ai-foundry/custom-speech/new-fine-tune-add-data-status-processing.png":::
3939

4040
::: zone-end
4141

articles/ai-services/speech-service/includes/quickstarts/speech-to-text-basics/ai-studio.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,12 +39,12 @@ In this quickstart, you try real-time speech to text in [Azure AI Foundry](https
3939

4040
1. Select an audio file to upload, or record audio in real-time. In this example, we use the `Call1_separated_16k_health_insurance.wav` file that's available in the [Speech SDK repository on GitHub](https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/scenarios/call-center/sampledata/Call1_separated_16k_health_insurance.wav). You can download the file or use your own audio file.
4141

42-
:::image type="content" source="../../../media/ai-studio/real-time-speech-to-text-audio.png" alt-text="Screenshot of the option to select an audio file or speak into a microphone." lightbox="../../../media/ai-studio/real-time-speech-to-text-audio.png":::
42+
:::image type="content" source="../../../media/ai-foundry/real-time-speech-to-text-audio.png" alt-text="Screenshot of the option to select an audio file or speak into a microphone." lightbox="../../../media/ai-foundry/real-time-speech-to-text-audio.png":::
4343

4444
1. You can view the real-time transcription at the bottom of the page.
4545

46-
:::image type="content" source="../../../media/ai-studio/real-time-speech-to-text-results.png" alt-text="Screenshot of the real-time transcription results in Azure AI Foundry." lightbox="../../../media/ai-studio/real-time-speech-to-text-results.png":::
46+
:::image type="content" source="../../../media/ai-foundry/real-time-speech-to-text-results.png" alt-text="Screenshot of the real-time transcription results in Azure AI Foundry." lightbox="../../../media/ai-foundry/real-time-speech-to-text-results.png":::
4747

4848
1. You can select the **JSON** tab to see the JSON output of the transcription. Properties include `Offset`, `Duration`, `RecognitionStatus`, `Display`, `Lexical`, `ITN`, and more.
4949

50-
:::image type="content" source="../../../media/ai-studio/real-time-speech-to-text-results-json.png" alt-text="Screenshot of the real-time transcription results in JSON format in Azure AI Foundry." lightbox="../../../media/ai-studio/real-time-speech-to-text-results-json.png":::
50+
:::image type="content" source="../../../media/ai-foundry/real-time-speech-to-text-results-json.png" alt-text="Screenshot of the real-time transcription results in JSON format in Azure AI Foundry." lightbox="../../../media/ai-foundry/real-time-speech-to-text-results-json.png":::
694 KB
Loading

0 commit comments

Comments
 (0)