|
| 1 | +--- |
| 2 | +title: Fine-tuning in Azure AI Studio |
| 3 | +titleSuffix: Azure AI Studio |
| 4 | +description: This article introduces fine-tuning of models in Azure AI Studio. |
| 5 | +manager: nitinme |
| 6 | +ms.service: azure-ai-studio |
| 7 | +ms.topic: conceptual |
| 8 | +ms.date: 5/13/2024 |
| 9 | +ms.reviewer: eur |
| 10 | +ms.author: eur |
| 11 | +author: eric-urban |
| 12 | +--- |
| 13 | + |
| 14 | +# Fine-tune models in Azure AI Studio |
| 15 | + |
| 16 | +[!INCLUDE [Azure AI Studio preview](../includes/preview-ai-studio.md)] |
| 17 | + |
| 18 | +When we talk about fine-tuning, we really mean *supervised fine-tuning* not continuous pretraining or Reinforcement Learning through Human Feedback (RLHF). Supervised fine-tuning refers to the process of retraining pretrained models on specific datasets, typically to improve model performance on specific tasks or introduce information that wasn't well represented when the base model was originally trained. |
| 19 | + |
| 20 | +In this article, you learn whether or not fine-tuning is the right solution for your given use case and how Azure AI studio can support your fine-tuning needs. |
| 21 | + |
| 22 | +## Getting started with fine-tuning |
| 23 | + |
| 24 | +When deciding whether or not fine-tuning is the right solution to explore for a given use case, there are some key terms that it's helpful to be familiar with: |
| 25 | + |
| 26 | +- [Prompt Engineering](../../ai-services/openai/concepts/prompt-engineering.md) is a technique that involves designing prompts for natural language processing models. This process improves accuracy and relevancy in responses, optimizing the performance of the model. |
| 27 | +- [Retrieval Augmented Generation (RAG)](../concepts/retrieval-augmented-generation.md) improves Large Language Model (LLM) performance by retrieving data from external sources and incorporating it into a prompt. RAG allows businesses to achieve customized solutions while maintaining data relevance and optimizing costs. |
| 28 | +- [Fine-tuning](#why-do-you-want-to-fine-tune-a-model) retrains an existing Large Language Model using example data, resulting in a new "custom" Large Language Model that's optimized using the provided examples. |
| 29 | + |
| 30 | +Fine-tuning is an advanced technique that requires expertise to use appropriately. The questions below can help you evaluate whether you're ready for fine-tuning, and how well you thought through the process. You can use these to guide your next steps or identify other approaches that might be more appropriate. |
| 31 | + |
| 32 | +## Why do you want to fine-tune a model? |
| 33 | + |
| 34 | +You might be ready for fine-tuning if you: |
| 35 | + |
| 36 | +- You should be able to clearly articulate a specific use case for fine-tuning and identify the [model](../how-to/model-catalog.md) you hope to fine-tune. |
| 37 | +- Good use cases for fine-tuning include steering the model to output content in a specific and customized style, tone, or format, or scenarios where the information needed to steer the model is too long or complex to fit into the prompt window. |
| 38 | +- Have clear examples on how you have approached the challenges in alternate approaches and what's been tested as possible resolutions to improve performance. |
| 39 | +- You've identified shortcomings using a base model, such as inconsistent performance on edge cases, inability to fit enough few shot prompts in the context window to steer the model, high latency, etc. |
| 40 | + |
| 41 | +You might not be ready for fine-tuning if: |
| 42 | + |
| 43 | +- Insufficient knowledge from the model or data source. |
| 44 | +- Inability to find the right data to serve the model. |
| 45 | +- No clear use case for fine-tuning, or an inability to articulate more than "I want to make a model better". |
| 46 | +- If you identify cost as your primary motivator, proceed with caution. Fine-tuning might reduce costs for certain use cases by shortening prompts or allowing you to use a smaller model but there's a higher upfront cost to training and you have to pay for hosting your own custom model. Refer to the [pricing page](https://azure.microsoft.com/pricing/details/cognitive-services/openai-service/) for more information on Azure OpenAI fine-tuning costs. |
| 47 | +- If you want to add out of domain knowledge to the model, you should start with retrieval augmented generation (RAG) with features like Azure OpenAI's [on your data](../../ai-services/openai/concepts/use-your-data.md) or [embeddings](../../ai-services/openai/tutorials/embeddings.md). Often, this is a cheaper, more adaptable, and potentially more effective option depending on the use case and data. |
| 48 | + |
| 49 | +## What isn't working with alternate approaches? |
| 50 | + |
| 51 | +Understanding where prompt engineering falls short should provide guidance on going about your fine-tuning. Is the base model failing on edge cases or exceptions? Is the base model not consistently providing output in the right format, and you can't fit enough examples in the context window to fix it? |
| 52 | + |
| 53 | +Examples of failure with the base model and prompt engineering will help you identify the data they need to collect for fine-tuning, and how you should be evaluating your fine-tuned model. |
| 54 | + |
| 55 | +Here's an example: A customer wanted to use GPT-3.5-Turbo to turn natural language questions into queries in a specific, nonstandard query language. They provided guidance in the prompt ("Always return GQL") and used RAG to retrieve the database schema. However, the syntax wasn't always correct and often failed for edge cases. They collected thousands of examples of natural language questions and the equivalent queries for their database, including cases where the model had failed before – and used that data to fine-tune the model. Combining their new fine-tuned model with their engineered prompt and retrieval brought the accuracy of the model outputs up to acceptable standards for use. |
| 56 | + |
| 57 | +## What have you tried so far? |
| 58 | + |
| 59 | +Fine-tuning is an advanced capability, not the starting point for your generative AI journey. You should already be familiar with the basics of using Large Language Models (LLMs). You should start by evaluating the performance of a base model with prompt engineering and/or Retrieval Augmented Generation (RAG) to get a baseline for performance. |
| 60 | + |
| 61 | +Having a baseline for performance without fine-tuning is essential for knowing whether or not fine-tuning has improved model performance. Fine-tuning with bad data makes the base model worse, but without a baseline, it's hard to detect regressions. |
| 62 | + |
| 63 | +**If you are ready for fine-tuning you:** |
| 64 | + |
| 65 | +- Should be able to demonstrate evidence and knowledge of Prompt Engineering and RAG based approaches. |
| 66 | +- Be able to share specific experiences and challenges with techniques other than fine-tuning that were already tried for your use case. |
| 67 | +- Need to have quantitative assessments of baseline performance, whenever possible. |
| 68 | + |
| 69 | +**Common signs you might not be ready for fine-tuning yet:** |
| 70 | + |
| 71 | +- Starting with fine-tuning without having tested any other techniques. |
| 72 | +- Insufficient knowledge or understanding on how fine-tuning applies specifically to Large Language Models (LLMs). |
| 73 | +- No benchmark measurements to assess fine-tuning against. |
| 74 | + |
| 75 | +## What data are you going to use for fine-tuning? |
| 76 | + |
| 77 | +Even with a great use case, fine-tuning is only as good as the quality of the data that you're able to provide. You need to be willing to invest the time and effort to make fine-tuning work. Different models will require different data volumes but you often need to be able to provide fairly large quantities of high-quality curated data. |
| 78 | + |
| 79 | +Another important point is even with high quality data if your data isn't in the necessary format for fine-tuning you'll need to commit engineering resources in order to properly format the data. For more information on how to prepare your data for fine-tuning, refer to the [fine-tuning documentation](../../ai-services/openai/how-to/fine-tuning.md?context=/azure/ai-studio/context/context). |
| 80 | + |
| 81 | +**If you are ready for fine-tuning you:** |
| 82 | + |
| 83 | +- Have identified a dataset for fine-tuning. |
| 84 | +- The dataset is in the appropriate format for training. |
| 85 | +- Some level of curation has been employed to ensure dataset quality. |
| 86 | + |
| 87 | +**Common signs you might not be ready for fine-tuning yet:** |
| 88 | + |
| 89 | +- Dataset hasn't been identified yet. |
| 90 | +- Dataset format doesn't match the model you wish to fine-tune. |
| 91 | + |
| 92 | +## How will you measure the quality of your fine-tuned model? |
| 93 | + |
| 94 | +There isn't a single right answer to this question, but you should have clearly defined goals for what success with fine-tuning looks like. Ideally, this shouldn't just be qualitative but should include quantitative measures of success like utilizing a holdout set of data for validation, as well as user acceptance testing or A/B testing the fine-tuned model against a base model. |
| 95 | + |
| 96 | +## Supported models for fine-tuning in Azure AI Studio |
| 97 | + |
| 98 | +Now that you know when to leverage fine-tuning for your use-case, you can go to Azure AI Studio to find several models available to fine-tune including: |
| 99 | +- Azure OpenAI models |
| 100 | +- Llama 2 family models |
| 101 | + |
| 102 | + |
| 103 | +### Azure OpenAI models |
| 104 | +The following Azure OpenAI models are supported in Azure AI Studio for fine-tuning: |
| 105 | + |
| 106 | +| Model ID | Fine-Tuning Regions | Max Request (tokens) | Training Data (up to) | |
| 107 | +| --- | --- | :---: | :---: | |
| 108 | +| `babbage-002` | North Central US <br> Sweden Central <br> Switzerland West | 16,384 | Sep 2021 | |
| 109 | +| `davinci-002` | North Central US <br> Sweden Central <br> Switzerland West | 16,384 | Sep 2021 | |
| 110 | +| `gpt-35-turbo` (0613) | East US2 <br> North Central US <br> Sweden Central <br> Switzerland West | 4,096 | Sep 2021 | |
| 111 | +| `gpt-35-turbo` (1106) | East US2 <br> North Central US <br> Sweden Central <br> Switzerland West | Input: 16,385<br> Output: 4,096 | Sep 2021| |
| 112 | +| `gpt-35-turbo` (0125) | East US2 <br> North Central US <br> Sweden Central <br> Switzerland West | 16,385 | Sep 2021 | |
| 113 | + |
| 114 | +`babbage-002` and `davinci-002` are not trained to follow instructions. Querying these base models should only be done as a point of reference to a fine-tuned version to evaluate the progress of your training. |
| 115 | + |
| 116 | +`gpt-35-turbo` - fine-tuning of this model is limited to a subset of regions, and is not available in every region the base model is available. |
| 117 | + |
| 118 | +Please note for fine-tuning Azure OpenAI models, you must add a connection to an Azure OpenAI resource with a supported region to your project. |
| 119 | + |
| 120 | +### Llama 2 family models |
| 121 | +The following Llama 2 family models are supported in Azure AI Studio for fine-tuning: |
| 122 | +- `Llama-2-70b` |
| 123 | +- `Llama-2-7b` |
| 124 | +- `Llama-2-13b` |
| 125 | + |
| 126 | +Fine-tuning of Llama 2 models is currently supported in projects located in West US 3. |
| 127 | + |
| 128 | +## Related content |
| 129 | + |
| 130 | +- [Learn how to fine-tune an Azure OpenAI model in Azure AI Studio](../../ai-services/openai/how-to/fine-tuning.md?context=/azure/ai-studio/context/context) |
| 131 | +- [Learn how to fine-tune a Llama 2 model in Azure AI Studio](../how-to/fine-tune-model-llama.md) |
0 commit comments