|
| 1 | +--- |
| 2 | +title: Customize a speech model with fine-tuning in AI Studio |
| 3 | +titleSuffix: Azure AI services |
| 4 | +description: Learn about how to customize a speech model with fine-tuning in AI Studio. |
| 5 | +author: eric-urban |
| 6 | +manager: nitinme |
| 7 | +ms.service: azure-ai-speech |
| 8 | +ms.topic: how-to |
| 9 | +ms.date: 11/19/2024 |
| 10 | +ms.author: eur |
| 11 | +#Customer intent: As a developer, I want to learn how to customize a speech model with fine-tuning in AI Studio so that I can deploy a custom model. |
| 12 | +--- |
| 13 | + |
| 14 | +# Customize a speech model with fine-tuning in AI Studio |
| 15 | + |
| 16 | +With custom speech, you can enhance speech recognition accuracy for your applications by using a custom model for real-time speech to text, speech translation, and batch transcription. The base model, trained with Microsoft-owned data, handles common spoken language well, but a custom model can improve domain-specific vocabulary and audio conditions by providing text and audio data for training. Additionally, you can train the model with structured text for custom pronunciations, display text formatting, and profanity filtering. |
| 17 | + |
| 18 | +> [!TIP] |
| 19 | +> You can bring your custom speech models from [Speech Studio](https://speech.microsoft.com) to [AI Studio](https://ai.azure.com). In AI Studio, you can pick up where you left off by connecting to your existing Speech resource. For more information about connecting to an existing Speech resource, see [Connect to an existing Speech resource](../../ai-studio/ai-services/how-to/connect-ai-services.md#connect-azure-ai-services-after-you-create-a-project). |
| 20 | +
|
| 21 | +In [AI Studio](https://ai.azure.com), you create a custom speech model by fine-tuning an Azure AI Speech base model with your own data. You can upload your data, test and train a custom model, compare accuracy between models, and deploy a model to a custom endpoint. |
| 22 | + |
| 23 | +This article shows you how to use fine-tuning in AI Studio to create a custom speech model. For more information about custom speech, see the [custom speech overview](./custom-speech-overview.md) documentation. |
| 24 | + |
| 25 | +## Start fine-tuning a model with your data |
| 26 | + |
| 27 | +In AI Studio, you can fine-tune some Azure AI services models. For example, you can fine-tune a model for custom speech. Each custom model is specific to a [locale](language-support.md?tabs=stt). For example, you might fine-tune a model for English in the United States. |
| 28 | + |
| 29 | +1. Go to your AI Studio project. If you need to create a project, see [Create an AI Studio project](../../ai-studio/how-to/create-projects.md). |
| 30 | +1. Select **Fine-tuning** from the left pane. |
| 31 | +1. Select **AI Service fine-tuning**. |
| 32 | + |
| 33 | + :::image type="content" source="../../ai-studio/media/ai-services/fine-tune-azure-ai-services.png" alt-text="Screenshot of the page to select fine-tuning of Azure AI Services models." lightbox="../../ai-studio/media/ai-services/fine-tune-azure-ai-services.png"::: |
| 34 | + |
| 35 | +1. In the wizard, select **Speech to text (speech recognition)** for custom speech. Then select **Next**. |
| 36 | +1. Select the connected service resource that you want to use for fine-tuning. Then select **Next**. |
| 37 | + |
| 38 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-select-connection.png" alt-text="Screenshot of the page to select the connected service resource that you want to use for fine-tuning." lightbox="./media/ai-studio/custom-speech/new-fine-tune-select-connection.png"::: |
| 39 | + |
| 40 | + In this example, we can choose from the following options: |
| 41 | + - **AI Service**: The Azure AI Services multi-service resource that [came with the AI Studio project](../../ai-studio/ai-services/how-to/connect-ai-services.md#connect-azure-ai-services-when-you-create-a-project-for-the-first-time). |
| 42 | + - **Speech Service**: An Azure AI Speech resource that was [connected after the project was created](../../ai-studio/ai-services/how-to/connect-ai-services.md#connect-azure-ai-services-after-you-create-a-project). |
| 43 | + |
| 44 | +1. Enter a name and description for the fine-tuning job. Then select **Next**. |
| 45 | +1. From the new left menu in the fine-tuning page, select **Manage data** and then select **Add data**. |
| 46 | + |
| 47 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-add-data.png" alt-text="Screenshot of the page with an option to add data to the custom speech project." lightbox="./media/ai-studio/custom-speech/new-fine-tune-add-data.png"::: |
| 48 | + |
| 49 | +1. In the **Add data** wizard, select the type of training data you want to add. In this example, we select **Audio + human-labeled transcript**. Then select **Next**. |
| 50 | + |
| 51 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-add-data-select-type.png" alt-text="Screenshot of the page with an option to select the type of training data you want to add." lightbox="./media/ai-studio/custom-speech/new-fine-tune-add-data-select-type.png"::: |
| 52 | + |
| 53 | +1. On the **Upload your data** page, select local files, Azure Blob Storage, or other shared web locations. Then select **Next**. |
| 54 | +1. Enter a name and description for the data. Then select **Next**. |
| 55 | +1. Review the data and select **Upload**. You're taken back to the **Manage data** page. The status of the data is **Processing**. |
| 56 | + |
| 57 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-add-data-status-processing.png" alt-text="Screenshot of the page that shows the status of the data as processing." lightbox="./media/ai-studio/custom-speech/new-fine-tune-add-data-status-processing.png"::: |
| 58 | + |
| 59 | +## Train a custom model from your data |
| 60 | + |
| 61 | +1. After the data is processed, select **Train model** from the left menu. Then select **+ Train model**. |
| 62 | + |
| 63 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-train-model.png" alt-text="Screenshot of the page with an option to start training for a custom speech model." lightbox="./media/ai-studio/custom-speech/new-fine-tune-train-model.png"::: |
| 64 | + |
| 65 | +1. In the **Train a new model** wizard, select the base model that you want to fine-tune. Then select **Next**. |
| 66 | + |
| 67 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-train-model-select-base.png" alt-text="Screenshot of the page with an option to select the base model that you want to fine-tune." lightbox="./media/ai-studio/custom-speech/new-fine-tune-train-model-select-base.png"::: |
| 68 | + |
| 69 | +1. Select the data that you want to use for training. Then select **Next**. |
| 70 | +1. Enter a name and description for the model. Then select **Next**. |
| 71 | +1. Review the settings and select **Train a new model**. You're taken back to the **Train model** page. The status of the data is **Processing**. |
| 72 | + |
| 73 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-train-model-status-processing.png" alt-text="Screenshot of the page that shows the status of the training as processing." lightbox="./media/ai-studio/custom-speech/new-fine-tune-train-model-status-processing.png"::: |
| 74 | + |
| 75 | +## Test the custom model |
| 76 | + |
| 77 | +1. After the model is trained, select **Test models** from the left menu. Then select **+ Create test**. |
| 78 | + |
| 79 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-test-model.png" alt-text="Screenshot of the page with an option to test your custom speech model." lightbox="./media/ai-studio/custom-speech/new-fine-tune-test-model.png"::: |
| 80 | + |
| 81 | +1. In the **Create a new test** wizard, select the test type. In this example, we select **Evaluate accuracy (Audio + transcript data)**. Then select **Next**. |
| 82 | + |
| 83 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-test-model-select-type.png" alt-text="Screenshot of the page with an option to select the test type." lightbox="./media/ai-studio/custom-speech/new-fine-tune-test-model-select-type.png"::: |
| 84 | + |
| 85 | +1. Select the data that you want to use for testing. Then select **Next**. |
| 86 | +1. Select up to two models to evaluate and compare accuracy. In this example, we select the model that we trained and the base model. Then select **Next**. |
| 87 | + |
| 88 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-test-model-select-models.png" alt-text="Screenshot of the page with an option to select up to two models to evaluate and compare accuracy." lightbox="./media/ai-studio/custom-speech/new-fine-tune-test-model-select-models.png"::: |
| 89 | + |
| 90 | +1. Enter a name and description for the test. Then select **Next**. |
| 91 | +1. Review the settings and select **Create test**. You're taken back to the **Test models** page. The status of the data is **Processing**. |
| 92 | + |
| 93 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-test-model-status-processing.png" alt-text="Screenshot of the page that shows the status of the test as processing." lightbox="./media/ai-studio/custom-speech/new-fine-tune-test-model-status-processing.png"::: |
| 94 | + |
| 95 | +1. When the test status is **Succeeded**, you can view the results. Select the test to view the results. |
| 96 | + |
| 97 | + |
| 98 | +## Deploy the custom model |
| 99 | + |
| 100 | +1. After you're satisfied with the test results, select **Deploy models** from the left menu. Then select **+ Deploy model**. |
| 101 | + |
| 102 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-deploy-model.png" alt-text="Screenshot of the page with an option to deploy the custom speech model." lightbox="./media/ai-studio/custom-speech/new-fine-tune-deploy-model.png"::: |
| 103 | + |
| 104 | +1. In the **Deploy a new model** wizard, select the model that you want to deploy. |
| 105 | + |
| 106 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-deploy-model-select-and-deploy.png" alt-text="Screenshot of the page with an option to select the model that you want to deploy." lightbox="./media/ai-studio/custom-speech/new-fine-tune-deploy-model-select-and-deploy.png"::: |
| 107 | + |
| 108 | +1. Enter a name and description for the deployment. Select the box to agree to the terms of use. Then select **Deploy**. |
| 109 | + |
| 110 | +1. After the deployment status is **Succeeded**, you can view the deployment details. Select the deployment to view the details like the endpoint ID. |
| 111 | + |
| 112 | + :::image type="content" source="./media/ai-studio/custom-speech/new-fine-tune-deploy-model-status-succeeded.png" alt-text="Screenshot of the page with an option to select the deployment to view the details like the endpoint ID." lightbox="./media/ai-studio/custom-speech/new-fine-tune-deploy-model-status-succeeded.png"::: |
| 113 | + |
| 114 | +## View fine-tuned models |
| 115 | + |
| 116 | +You can access your custom speech models and deployments from the **Fine-tuning** page. |
| 117 | + |
| 118 | +1. Select **Fine-tuning** from the left pane. |
| 119 | +1. Select **AI Service fine-tuning**. |
| 120 | + |
| 121 | + :::image type="content" source="./media/ai-studio/custom-speech/fine-tune-succeeded-list.png" alt-text="Screenshot of the page where you can view fine-tuned AI services models." lightbox="./media/ai-studio/custom-speech/fine-tune-succeeded-list.png"::: |
| 122 | + |
| 123 | +## Related content |
| 124 | + |
| 125 | +- [Custom speech overview](./custom-speech-overview.md) |
0 commit comments