|
| 1 | +--- |
| 2 | +title: Use summarization Docker containers on-premises |
| 3 | +titleSuffix: Azure AI services |
| 4 | +description: Use Docker containers for the summarization API to summarize text, on-premises. |
| 5 | +services: cognitive-services |
| 6 | +author: jboback |
| 7 | +manager: nitinme |
| 8 | +ms.service: cognitive-services |
| 9 | +ms.subservice: language-service |
| 10 | +ms.topic: how-to |
| 11 | +ms.date: 08/15/2023 |
| 12 | +ms.author: jboback |
| 13 | +keywords: on-premises, Docker, container |
| 14 | +--- |
| 15 | + |
| 16 | +# Use summarization Docker containers on-premises |
| 17 | + |
| 18 | +Containers enable you to host the Summarization API on your own infrastructure. If you have security or data governance requirements that can't be fulfilled by calling Summarization remotely, then containers might be a good option. |
| 19 | + |
| 20 | +## Prerequisites |
| 21 | + |
| 22 | +* If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/cognitive-services/). |
| 23 | +* [Docker](https://docs.docker.com/) installed on a host computer. Docker must be configured to allow the containers to connect with and send billing data to Azure. |
| 24 | + * On Windows, Docker must also be configured to support Linux containers. |
| 25 | + * You should have a basic understanding of [Docker concepts](https://docs.docker.com/get-started/overview/). |
| 26 | +* A <a href="https://portal.azure.com/#create/Microsoft.CognitiveServicesTextAnalytics" title="Create a Language resource" target="_blank">Language resource </a> with the free (F0) or standard (S) [pricing tier](https://azure.microsoft.com/pricing/details/cognitive-services/text-analytics/). For disconnected containers, the DC0 tier is required. |
| 27 | +* For disconnected containers, |
| 28 | + |
| 29 | +[!INCLUDE [Gathering required parameters](../../../containers/includes/container-gathering-required-parameters.md)] |
| 30 | + |
| 31 | +## Host computer requirements and recommendations |
| 32 | + |
| 33 | +[!INCLUDE [Host Computer requirements](../../../../../includes/cognitive-services-containers-host-computer.md)] |
| 34 | + |
| 35 | +The following table describes the minimum and recommended specifications for the summarization container skills. Listed CPU/memory combinations are for a 4000 token input (conversation consumption is for all the aspects in the same request). |
| 36 | + |
| 37 | +| Container Type | Recommended number of CPU cores | Recommended memory | Notes | |
| 38 | +|----------------------------|----------------------------------|--------------------|-------| |
| 39 | +| Summarization CPU container| 16 | 48 GB | | |
| 40 | +| Summarization GPU container| 2 | 24 GB | Requires an Nvidia GPU that supports Cuda 11.8 with 16GB VRAM.| |
| 41 | + |
| 42 | +CPU core and memory correspond to the `--cpus` and `--memory` settings, which are used as part of the `docker run` command. |
| 43 | + |
| 44 | + |
| 45 | +## Get the container image with `docker pull` |
| 46 | + |
| 47 | +The Summarization container image can be found on the `mcr.microsoft.com` container registry syndicate. It resides within the `azure-cognitive-services/textanalytics/` repository and is named `summarization`. The fully qualified container image name is, `mcr.microsoft.com/azure-cognitive-services/textanalytics/summarization` |
| 48 | + |
| 49 | +To use the latest version of the container, you can use the `latest` tag. You can also find a full list of [tags on the MCR](https://mcr.microsoft.com/azure-cognitive-services/textanalytics/summarization). |
| 50 | + |
| 51 | +Use the [`docker pull`](https://docs.docker.com/engine/reference/commandline/pull/) command to download a container image from the Microsoft Container Registry. |
| 52 | + |
| 53 | +``` |
| 54 | +docker pull mcr.microsoft.com/azure-cognitive-services/textanalytics/summarization:cpu |
| 55 | +``` |
| 56 | +for CPU containers, |
| 57 | +``` |
| 58 | +docker pull mcr.microsoft.com/azure-cognitive-services/textanalytics/summarization:gpu |
| 59 | +``` |
| 60 | +for GPU containers. |
| 61 | + |
| 62 | +[!INCLUDE [Tip for using docker list](../../../../../includes/cognitive-services-containers-docker-list-tip.md)] |
| 63 | + |
| 64 | +## Download the summarization models |
| 65 | + |
| 66 | +A pre-requisite for running the summarization container is to download the models first. This can be done by running one of the following commands: |
| 67 | + |
| 68 | +```bash |
| 69 | +docker run -v {HOST_MODELS_PATH}:/models mcr.microsoft.com/azure-cognitive-services/textanalytics/summarization downloadModels=ExtractiveSummarization billing={ENDPOINT_URI} apikey={API_KEY} |
| 70 | +docker run -v {HOST_MODELS_PATH}:/models mcr.microsoft.com/azure-cognitive-services/textanalytics/summarization downloadModels=AbstractiveSummarization billing={ENDPOINT_URI} apikey={API_KEY} |
| 71 | +docker run -v {HOST_MODELS_PATH}:/models mcr.microsoft.com/azure-cognitive-services/textanalytics/summarization downloadModels=ConversationSummarization billing={ENDPOINT_URI} apikey={API_KEY} |
| 72 | +``` |
| 73 | +It's not recommended to download models for all skills inside the same `HOST_MODELS_PATH`, as the container loads all models inside the `HOST_MODELS_PATH`. Doing so would use a large amount of memory. It's recommended to only download the model for the skill you need in a particular `HOST_MODELS_PATH`. |
| 74 | + |
| 75 | +In order to ensure compatibility between models and the container, re-download the utilized models whenever you create a container using a new image version. When using a disconnected container, the license should be downloaded again after downloading the models. |
| 76 | + |
| 77 | +## Run the container with `docker run` |
| 78 | + |
| 79 | +Once the container is on the host computer, use the following command to run the containers. The container will continue to run until you stop it. (note the `rai_terms=accept` part) |
| 80 | + |
| 81 | +```bash |
| 82 | +docker run -p 5000:5000 -v {HOST_MODELS_PATH}:/models mcr.microsoft.com/azure-cognitive-services/textanalytics/summarization eula=accept rai_terms=accept billing={ENDPOINT_URI} apikey={API_KEY} |
| 83 | +``` |
| 84 | + |
| 85 | +Or if you are running a GPU container, use the this command instead. |
| 86 | +```bash |
| 87 | +docker run -p 5000:5000 --gpus all -v {HOST_MODELS_PATH}:/models mcr.microsoft.com/azure-cognitive-services/textanalytics/summarization eula=accept rai_terms=accept billing={ENDPOINT_URI} apikey={API_KEY} |
| 88 | +``` |
| 89 | +If there is more than one GPU on the machine, replace `--gpus all` with `--gpus device={DEVICE_ID}`. |
| 90 | + |
| 91 | + |
| 92 | +> [!IMPORTANT] |
| 93 | +> * The docker commands in the following sections use the back slash, `\`, as a line continuation character. Replace or remove this based on your host operating system's requirements. |
| 94 | +> * The `Eula`, `Billing`, `rai_terms` and `ApiKey` options must be specified to run the container; otherwise, the container won't start. For more information, see [Billing](#billing). |
| 95 | +
|
| 96 | +To run the *Summarization* container, execute the following `docker run` command. Replace the placeholders below with your own values: |
| 97 | + |
| 98 | +| Placeholder | Value | Format or example | |
| 99 | +|-------------|-------|---| |
| 100 | +| **{API_KEY}** | The key for your Language resource. You can find it on your resource's **Key and endpoint** page, on the Azure portal. |`xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`| |
| 101 | +| **{ENDPOINT_URI}** | The endpoint for accessing the summarization API. You can find it on your resource's **Key and endpoint** page, on the Azure portal. | `https://<your-custom-subdomain>.cognitiveservices.azure.com` | |
| 102 | + |
| 103 | + |
| 104 | +```bash |
| 105 | +docker run --rm -it -p 5000:5000 --memory 4g --cpus 1 \ |
| 106 | +mcr.microsoft.com/azure-cognitive-services/textanalytics/summarization |
| 107 | +Eula=accept \ |
| 108 | +Billing={ENDPOINT_URI} \ |
| 109 | +ApiKey={API_KEY} |
| 110 | +``` |
| 111 | + |
| 112 | +This command: |
| 113 | + |
| 114 | +* Runs a *Summarization* container from the container image |
| 115 | +* Allocates one CPU core and 4 gigabytes (GB) of memory |
| 116 | +* Exposes TCP port 5000 and allocates a pseudo-TTY for the container |
| 117 | +* Automatically removes the container after it exits. The container image is still available on the host computer. |
| 118 | + |
| 119 | +[!INCLUDE [Running multiple containers on the same host](../../../../../includes/cognitive-services-containers-run-multiple-same-host.md)] |
| 120 | + |
| 121 | +## Query the container's prediction endpoint |
| 122 | + |
| 123 | +The container provides REST-based query prediction endpoint APIs. |
| 124 | + |
| 125 | +Use the host, `http://localhost:5000`, for container APIs. |
| 126 | + |
| 127 | +<!-- ## Validate container is running --> |
| 128 | + |
| 129 | +[!INCLUDE [Container's API documentation](../../../../../includes/cognitive-services-containers-api-documentation.md)] |
| 130 | + |
| 131 | +## Run the container disconnected from the internet |
| 132 | + |
| 133 | +[!INCLUDE [configure-disconnected-container](../../../containers/includes/configure-disconnected-container.md)] |
| 134 | + |
| 135 | +## Stop the container |
| 136 | + |
| 137 | +[!INCLUDE [How to stop the container](../../../../../includes/cognitive-services-containers-stop.md)] |
| 138 | + |
| 139 | +## Troubleshooting |
| 140 | + |
| 141 | +If you run the container with an output [mount](../../concepts/configure-containers.md#mount-settings) and logging enabled, the container generates log files that are helpful to troubleshoot issues that happen while starting or running the container. |
| 142 | + |
| 143 | +[!INCLUDE [Azure AI services FAQ note](../../../containers/includes/cognitive-services-faq-note.md)] |
| 144 | + |
| 145 | +## Billing |
| 146 | + |
| 147 | +The summarization containers send billing information to Azure, using a _Language_ resource on your Azure account. |
| 148 | + |
| 149 | +[!INCLUDE [Container's Billing Settings](../../../../../includes/cognitive-services-containers-how-to-billing-info.md)] |
| 150 | + |
| 151 | +For more information about these options, see [Configure containers](../../concepts/configure-containers.md). |
| 152 | + |
| 153 | +## Summary |
| 154 | + |
| 155 | +In this article, you learned concepts and workflow for downloading, installing, and running summarization containers. In summary: |
| 156 | + |
| 157 | +* Summarization provides Linux containers for Docker |
| 158 | +* Container images are downloaded from the Microsoft Container Registry (MCR). |
| 159 | +* Container images run in Docker. |
| 160 | +* You must specify billing information when instantiating a container. |
| 161 | + |
| 162 | +> [!IMPORTANT] |
| 163 | +> This container is not licensed to run without being connected to Azure for metering. Customers need to enable the containers to communicate billing information with the metering service at all times. Azure AI containers do not send customer data (e.g. text that is being analyzed) to Microsoft. |
| 164 | +
|
| 165 | +## Next steps |
| 166 | + |
| 167 | +* See [Configure containers](../../concepts/configure-containers.md) for configuration settings. |
0 commit comments