You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Custom Language service features enable you to deploy your project to more than one region, making it much easier to access your project globally while managing only one instance of your project in one place.
25
+
Custom language service features enable you to deploy your project to more than one region. This capability makes it much easier to access your project globally while you manage only one instance of your project in one place.
25
26
26
-
Before you deploy a project, you can assign **deployment resources** in other regions. Each deployment resource is a different Language resource from the one you use to author your project. You deploy to those resources and then target your prediction requests to that resource in their respective regions and your queries are served directly from that region.
27
+
Before you deploy a project, you can assign *deployment resources* in other regions. Each deployment resource is a different Language resource from the one that you use to author your project. You deploy to those resources and then target your prediction requests to that resource in their respective regions and your queries are served directly from that region.
27
28
28
-
When creating a deployment, you can select which of your assigned deployment resources and their corresponding regions you would like to deploy to. The model you deploy is then replicated to each region and accessible with its own endpoint dependent on the deployment resource's custom subdomain.
29
+
When you create a deployment, you can select which of your assigned deployment resources and their corresponding regions you want to deploy to. The model you deploy is then replicated to each region and accessible with its own endpoint dependent on the deployment resource's custom subdomain.
29
30
30
31
## Example
31
32
32
-
Suppose you want to make sure your project, which is used as part of a customer support chatbot, is accessible by customers across the US and India. You would author a project with the name **ContosoSupport** using a _West US 2_ Language resource named **MyWestUS2**. Before deployment, you would assign two deployment resources to your project - **MyEastUS** and **MyCentralIndia** in _East US_ and _Central India_, respectively.
33
+
Suppose you want to make sure your project, which is used as part of a customer support chatbot, is accessible by customers across the United States and India. You author a project with the name `ContosoSupport` by using a West US 2 Language resource named `MyWestUS2`. Before deployment, you assign two deployment resources to your project: `MyEastUS` and `MyCentralIndia` in East US and Central India, respectively.
34
+
35
+
When you deploy your project, you select all three regions for deployment: the original West US 2 region and the assigned ones through East US and Central India.
33
36
34
-
When deploying your project, You would select all three regions for deployment: the original _West US 2_ region and the assigned ones through _East US_ and _Central India_.
37
+
You now have three different endpoint URLs to access your project in all three regions:
35
38
36
-
You would now have three different endpoint URLs to access your project in all three regions:
37
-
* West US 2: `https://mywestus2.cognitiveservices.azure.com/language/:analyze-conversations`
38
-
* East US: `https://myeastus.cognitiveservices.azure.com/language/:analyze-conversations`
39
-
* Central India: `https://mycentralindia.cognitiveservices.azure.com/language/:analyze-conversations`
39
+
***West US 2**: `https://mywestus2.cognitiveservices.azure.com/language/:analyze-conversations`
The same request body to each of those different URLs serves the exact same response directly from that region.
43
+
The same request body to each of those different URLs serves the exact same response directly from that region.
42
44
43
45
## Validations and requirements
44
46
45
-
Assigning deployment resources requires Microsoft Entra authentication. Microsoft Entra ID is used to confirm you have access to the resources you are interested in assigning to your project for multi-region deployment. In the Language Studio, you can automatically [enable Microsoft Entra authentication](https://aka.ms/rbac-language) by assigning yourself the _Cognitive Services Language Owner_ role to your original resource. To programmatically use Microsoft Entra authentication, learn more from the [Azure AI services documentation](../../../authentication.md?source=docs&tabs=powershell&tryIt=true#authenticate-with-azure-active-directory).
47
+
Assigning deployment resources requires Microsoft Entra authentication. Microsoft Entra ID is used to confirm that you have access to the resources that you want to assign to your project for multiregion deployment. In Language Studio, you can automatically [enable Microsoft Entra authentication](https://aka.ms/rbac-language) by assigning yourself the Azure Cognitive Services Language Owner role to your original resource. To programmatically use Microsoft Entra authentication, learn more from the [Azure AI services documentation](../../../authentication.md?source=docs&tabs=powershell&tryIt=true#authenticate-with-azure-active-directory).
46
48
47
-
Your project name and resource are used as its main identifiers. Therefore, a Language resource can only have a specific project name in each resource. Any other projects with the same name will not be deployable to that resource.
49
+
Your project name and resource are used as its main identifiers. A Language resource can only have a specific project name in each resource. Any other projects with the same name can't be deployed to that resource.
48
50
49
-
For example, if a project **ContosoSupport** was created by resource **MyWestUS2** in _West US 2_ and deployed to resource **MyEastUS** in _East US_, the resource **MyEastUS** cannot create a different project called **ContosoSupport** and deploy a project to that region. Similarly, your collaborators cannot then create a project **ContosoSupport** with resource **MyCentralIndia** in _Central India_ and deploy it to either **MyWestUS2** or **MyEastUS**.
51
+
For example, if a project `ContosoSupport` was created by the resource `MyWestUS2` in West US 2 and deployed to the resource `MyEastUS` in East US, the resource `MyEastUS` can't create a different project called `ContosoSupport` and deploy a project to that region. Similarly, your collaborators can't then create a project `ContosoSupport` with the resource `MyCentralIndia` in Central India and deploy it to either `MyWestUS2` or `MyEastUS`.
50
52
51
-
You can only swap deployments that are available in the exact same regions, otherwise swapping will fail.
53
+
You can only swap deployments that are available in the exact same regions. Otherwise, swapping fails.
52
54
53
-
If you remove an assigned resource from your project, all of the project deployments to that resource will then be deleted.
55
+
If you remove an assigned resource from your project, all of the project deployments to that resource are deleted.
54
56
55
57
> [!NOTE]
56
58
> Orchestration workflow only:
57
59
>
58
-
> You **cannot** assign deployment resources to orchestration workflow projects with custom question answering or LUIS connections. You subsequently cannot add custom question answering or LUIS connections to projects that have assigned resources.
60
+
> You *can't* assign deployment resources to orchestration workflow projects with custom question answering or LUIS connections. Subsequently, you can't add custom question answering or LUIS connections to projects that have assigned resources.
59
61
>
60
-
> For multi-region deployment to work as expected, the connected CLU projects **must also be deployed** to the same regional resources you've deployed the orchestration workflow project to. Otherwise the orchestration workflow project will attempt to route a request to a deployment in its region that doesn't exist.
62
+
> For multiregion deployment to work as expected, the connected CLU projects *must also be deployed* to the same regional resources to which you deployed the orchestration workflow project. Otherwise, the orchestration workflow project attempts to route a request to a deployment in its region that doesn't exist.
61
63
62
64
Some regions are only available for deployment and not for authoring projects.
63
65
64
-
## Next steps
66
+
## Related content
65
67
66
68
Learn how to deploy models for:
69
+
67
70
*[Conversational language understanding](../../conversational-language-understanding/how-to/deploy-model.md)
68
71
*[Custom text classification](../../custom-text-classification/how-to/deploy-model.md)
# When to use conversational language understanding or orchestration workflow apps
16
16
17
-
When you create large applications, you should consider whether your use-case would be best served by a single conversational app (flat architecture), or multiple apps that are orchestrated.
17
+
When you create large applications, you should consider whether your usecase is best served by a single conversational app (flat architecture) or by multiple apps that are orchestrated.
18
18
19
+
## Orchestration overview
19
20
20
-
## Orchestration overview
21
+
Orchestration workflow is a feature that allows you to connect different projects from [LUIS](../../../LUIS/what-is-luis.md), [conversational language understanding](../overview.md), and [custom question answering](../../question-answering/overview.md) in one project. You can then use this project for predictions by using one endpoint. The orchestration project makes a prediction on which child project should be called, automatically routes the request, and returns with its response.
21
22
22
-
Orchestration workflow is a feature that allows you to connect different projects from [LUIS](../../../LUIS/what-is-luis.md)[conversational language understanding](../overview.md), and [custom question answering](../../question-answering/overview.md) in one project. You can then use this project for predictions using one endpoint. The orchestration project makes a prediction on which child project should be called, automatically routes the request, and returns with its response.
23
+
Orchestration involves two steps:
23
24
24
-
The key point is that orchestration involves two steps:
25
+
1. Predicting which child project to call. <!--The model that performs this classification can be trained either with a standard or an advanced recipe. (Please see footnotes on instructions for training with advanced recipe).-->
26
+
1. Routing the utterance to the destination child app and returning the child app's response.
25
27
26
-
1. Predicting which child project to call. <!--The model that performs this classification can be trained either with a standard or an advanced recipe. (Please see footnotes on instructions for training with advanced recipe).-->
27
-
2. Routing the utterance to the destination child app, and returning the child app's response.
28
-
29
-
### Advantages
28
+
### Orchestration advantages
30
29
31
30
* Clear decomposition and faster development:
32
-
* If your overall schema has a substantial number of domains, the orchestration approach can help decompose your application into several child apps (each serving a specific domain). For example, an automotive conversational app might have a *navigation domain*, a *media domain*, and so on.
33
-
* Developing each domain app in parallel is easier. People and teams with specific domain expertise can work on individual apps collaboratively and in parallel.
34
-
* Since each domain app is smaller, the development cycle becomes faster. Smaller sized domain apps take much less time to train than a single large app.
31
+
32
+
* If your overall schema has a substantial number of domains, the orchestration approach can help decompose your application into several child apps (each serving a specific domain). For example, an automotive conversational app might have a *navigation domain* or a *media domain*.
33
+
* Developing each domain app in parallel is easier. People and teams with specific domain expertise can work on individual apps collaboratively and in parallel.
34
+
* Because each domain app is smaller, the development cycle becomes faster. Smaller-sized domain apps take much less time to train than a single large app.
35
35
* More flexible [confidence score thresholds](/legal/cognitive-services/clu/clu-characteristics-and-limitations?context=/azure/ai-services/language-service/context/context#understand-confidence-scores):
36
-
* Since there are separate child apps serving each domain, it's easy to set separate thresholds for different child apps.
37
-
* AI quality improvements where appropriate:
38
-
* Some applications require that certain entities are domain restricted. Orchestration makes this easy to achieve. Once the orchestration project has predicted which child app should be called, the other child apps won't be called.
39
36
40
-
For example, if your app contains a `Person.Name` prebuilt entity, consider the utterance *"How do I use a jack?"*, in the context of a vehicle question. In this context, *jack* is an automotive tool, and shouldn’t be recognized as a person's name. Using orchestration, this utterance can be redirected to a child app created to answer such questions, which doesn’t have a `Person.Name` entity.
37
+
* Because separate child apps serve each domain, it's easy to set separate thresholds for different child apps.
38
+
* AI-quality improvements where appropriate:
39
+
40
+
* Some applications require that certain entities must be domain restricted. Orchestration makes this task easy to achieve. After the orchestration project predicts which child app should be called, the other child apps aren't called.
41
41
42
-
### Disadvantages
42
+
For example, if your app contains a `Person.Name` prebuilt entity, consider the utterance "How do I use a jack?" in the context of a vehicle question. In this context, *jack* is an automotive tool and shouldn't be recognized as a person's name. When you use orchestration, this utterance can be redirected to a child app created to answer such a question, which doesn't have a `Person.Name` entity.
43
+
44
+
### Orchestration disadvantages
43
45
44
46
* Redundant entities in child apps:
45
-
* If you need a particular prebuilt entity being returned in all utterances irrespective of the domain, for example `Quantity.Number` or `Geography.Location`, there is no way of adding an entity to the Orchestration app (it is an intent-only model). You would need to add it to all individual child apps.
47
+
48
+
* If you need a particular prebuilt entity being returned in all utterances irrespective of the domain, for example `Quantity.Number` or `Geography.Location`, there's no way of adding an entity to the orchestration app (it's an intent-only model). You would need to add it to all individual child apps.
46
49
* Efficiency:
47
-
* Orchestration apps take two model inferences. One for predicting which child app to call, another for the prediction in the child app. Inference times will typically be slower than single apps with a flat architecture.
50
+
51
+
* Orchestration apps take two model inferences. One for predicting which child app to call, and another for the prediction in the child app. Inference times are typically slower than single apps with a flat architecture.
48
52
* Train/test split for orchestrator:
49
-
* Training an orchestration app does not allow you to granularly split data between the testing and training sets. For example, you cannot train a 90-10 split for child app A, and then an 80-20 split for child app B. This may be a minor point, but worth keeping in mind.
53
+
54
+
* Training an orchestration app doesn't allow you to granularly split data between the testing and training sets. For example, you can't train a 90-10 split for child app A, and then train an 80-20 split for child app B. This limitation might be minor, but it's worth keeping in mind.
50
55
51
56
## Flat architecture overview
52
57
53
-
Flat architecture is the other method of developing conversational apps. Instead of using an orchestration app to send utterances to one of multiple child apps, you develop a singular (or flat) app to handle utterances.
58
+
Flat architecture is the other method of developing conversational apps. Instead of using an orchestration app to send utterances to one of multiple child apps, you develop a singular (or flat) app to handle utterances.
54
59
55
-
### Advantages
60
+
### Flat architecture advantages
56
61
57
62
* Simplicity:
58
-
* For small sized apps or domains, the orchestrator approach can be overly complex.
59
-
* Since all intents and entities are at the same app level, it might be easier to make changes to all of them together.
63
+
64
+
* For small-sized apps or domains, the orchestrator approach can be overly complex.
65
+
* Because all intents and entities are at the same app level, it might be easier to make changes to all of them together.
60
66
* It's easier to add entities that should always be returned:
61
-
* If you want certain prebuilt or list entities to be returned for all utterances, you only need to add it alongside other entities in a single app. If you use orchestration, as mentioned above, you would need to add it to every child app.
62
67
63
-
### Disadvantages
68
+
* If you want certain prebuilt or list entities to be returned for all utterances, you only need to add them alongside other entities in a single app. If you use orchestration, as mentioned, you need to add it to every child app.
69
+
70
+
### Flat architecture disadvantages
64
71
65
72
* Unwieldy for large apps:
66
-
* For large apps (say > 50 intents or entities) it can become difficult to keep track of evolving schemas and datasets. This is particularly evident in cases where the app has to serve several domains. For example an automotive conversational app might have a *navigation domain*, a *media domain*, and so on.
73
+
74
+
* For large apps (say, more than 50 intents or entities), it can become difficult to keep track of evolving schemas and datasets. This difficulty is evident in cases where the app has to serve several domains. For example, an automotive conversational app might have a *navigation domain* or a *media domain*.
67
75
* Limited control over entity matches:
68
-
* In a flat architecture, there is no way to restrict entities to be returned only in certain cases. You can accomplish this using orchestration by assigning those specific entities to particular child apps.
69
76
70
-
## Next steps
77
+
* In a flat architecture, there's no way to restrict entities to be returned only in certain cases. When you use orchestration, you can assign those specific entities to particular child apps.
0 commit comments