|
| 1 | +--- |
| 2 | +title: How to use image models in the model catalog |
| 3 | +titleSuffix: Azure AI Foundry |
| 4 | +description: Learn how to use image models from the AI Foundry model catalog. |
| 5 | +manager: scottpolly |
| 6 | +author: msakande |
| 7 | +reviewer: frogglew |
| 8 | +ms.service: azure-ai-model-inference |
| 9 | +ms.topic: how-to |
| 10 | +ms.date: 04/30/2025 |
| 11 | +ms.author: mopeakande |
| 12 | +ms.reviewer: frogglew |
| 13 | +ms.custom: references_regions, tool_generated |
| 14 | +--- |
| 15 | + |
| 16 | +# How to use image models in the model catalog |
| 17 | + |
| 18 | +This article explains how to use _image_ models in the AI Foundry model catalog. Some models have unique parameters or data format requirements. |
| 19 | + |
| 20 | +## Prerequisites |
| 21 | + |
| 22 | +To use image models in your application, you need: |
| 23 | + |
| 24 | +[!INCLUDE [how-to-prerequisites](../includes/how-to-prerequisites.md)] |
| 25 | + |
| 26 | +* An image model deployment on Azure AI Foundry. This article uses a __Mistral OCR__ model deployment. |
| 27 | + |
| 28 | +## Use image model |
| 29 | + |
| 30 | +1. Authenticate using an API key. First, deploy the model to generate the endpoint URL and an API key to authenticate against the service. In this example, the endpoint and key are strings holding the endpoint URL and the API key. The API endpoint URL and API key can be found on the **Deployments + Endpoint** page once the model is deployed. |
| 31 | + |
| 32 | + If you're using Bash: |
| 33 | + |
| 34 | + ```bash |
| 35 | + export AZURE_API_KEY = "<your-api-key>" |
| 36 | + ``` |
| 37 | + |
| 38 | + If you're in PowerShell: |
| 39 | + |
| 40 | + ```powershell |
| 41 | + $Env:AZURE_API_KEY = "<your-api-key>" |
| 42 | + ``` |
| 43 | + |
| 44 | + If you're using Windows command prompt: |
| 45 | + |
| 46 | + ``` |
| 47 | + export AZURE_API_KEY = "<your-api-key>" |
| 48 | + ``` |
| 49 | + |
| 50 | +1. Run a basic code sample. Different image models accept different data formats. In this example, _Mistral OCR 25.03_ supports only base64 encoded data; document url or image url isn't supported. Paste the following code into a shell. |
| 51 | + |
| 52 | + ```http |
| 53 | + curl --request POST \ |
| 54 | + --url https://<your_serverless_endpoint>/v1/ocr \ |
| 55 | + --header 'Authorization: <api_key>' \ |
| 56 | + --header 'Content-Type: Application/json' \ |
| 57 | + --data '{ |
| 58 | + "model": "mistral-ocr-2503", |
| 59 | + "document": { |
| 60 | + "type": "document_url", |
| 61 | + "document_name": "test", |
| 62 | + "document_url": "data:application/pdf;base64,JVBER..." |
| 63 | + } |
| 64 | + }' |
| 65 | + ``` |
| 66 | +
|
| 67 | +**More code samples for Mistral OCR 25.03** |
| 68 | +
|
| 69 | +```bash |
| 70 | +input_file_path="assets/2201.04234v3.pdf" |
| 71 | +base64_value=$(base64 "$input_file_path") |
| 72 | +input_base64_value="data:application/pdf;base64,${base64_value}" |
| 73 | +# echo $input_base64_value |
| 74 | + |
| 75 | +# Prepare JSON data |
| 76 | +payload_body=$(cat <<EOF |
| 77 | +{ |
| 78 | + "model": "mistral-ocr-2503", |
| 79 | + "document": { |
| 80 | + "type": "document_url", |
| 81 | + "document_url": "$input_base64_value" |
| 82 | + }, |
| 83 | + "include_image_base64": true |
| 84 | +} |
| 85 | +EOF |
| 86 | +) |
| 87 | +
|
| 88 | +echo "$payload_body" | curl ${AZURE_AI_CHAT_ENDPOINT}/v1/ocr \ |
| 89 | + -H "Content-Type: application/json" \ |
| 90 | + -H "Authorization: Bearer ${AZURE_AI_CHAT_KEY}" \ |
| 91 | + -d @- -o ocr_pdf_output.json |
| 92 | +``` |
| 93 | +
|
| 94 | +
|
| 95 | +## Model-specific parameters |
| 96 | +
|
| 97 | +Some image models only support specific data formats. Mistral OCR 25.03, for example, requires `base64 encoded image data` for their `document_url` parameter. The following table lists the supported and unsupported data formats for image models in the model catalog. |
| 98 | +
|
| 99 | +| Model | Supported | Not supported | |
| 100 | +| :---- | ----- | ----- | |
| 101 | +| Mistral OCR 25.03 | base64 encoded image data | document url, image url | |
| 102 | +| dall-e-3 | document url, image url, b64_json | base64 encoded image data | |
| 103 | +| gpt-image-1 | base64 encoded image data, image url | document url | |
| 104 | +
|
| 105 | +
|
| 106 | +## Related content |
| 107 | +
|
| 108 | +* [How to use Image Generation Models on Azure OpenAI](.././how-to/dall-e?tabs=gpt-image-1.md) |
| 109 | +
|
| 110 | +::: zone-end |
| 111 | +
|
0 commit comments