Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 56 additions & 1 deletion docs/Roles and Access Control/3.API Keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ To create and configure access control for API keys:

## Per-Request Keys

Per-request keys are used to manage access to user files for applications, enable open telemetry for tracing and realize cost control in a lifespan of a particular request.
Per-request keys are used to manage access to user files for applications, enable open telemetry for tracing and realize cost control in a lifespan of a particular request. They also play a key role for external applications accessing language models and applications deployed in AI DIAL.

Per-request keys are generated by AI DIAL Core, when it is making a request to the application and is valid only during the lifetime of this particular request from the Core to the application.

Expand Down Expand Up @@ -92,3 +92,58 @@ To provide the path of the folder for output files, we add `appdata` field to th

For tracing open telemetry, `traceparent` should be included in the request header. The open telemetry tracing does not interfere with the limits, statistics or file sharing.

### Access and Cost Control for External Applications

Applications in AI DIAL can use `routes` for communication through [registered in AI DIAL Core](https://github.com/epam/ai-dial-core?tab=readme-ov-file#dynamic-settings) endpoints, which may not necessarily adhere to the AI DIAL API. Routes, therefore, act as a bridging mechanism between the AI DIAL Core and external applications, facilitating seamless interactions.

Once a route with a designated endpoint is set up in AI DIAL Core, it allows client applications, such as AI DIAL Chat for example, to interact with this endpoint. Essentially, AI DIAL Core functions as an intermediary, handling authentication and authorization between the client and the external application linked to the route.

External applications, do not have direct access to the resources within AI DIAL. Still, they might need to retrieve user data or interact with other conversational agents and language models available in AI DIAL to perform their functions.

**Per-request keys are issued for routes to enable:**

* Access to language models and applications
* A dedicated workspace within a BLOB store for routes, allowing them to read and write files under `/Keys/<route_name>/`.
* The ability to fetch user information via the `/v1/user/info` endpoint.

To manage access and control costs for external applications behind routes, it is possible to assign specific roles for routes.

In the following example, a route `myApp` has a user role `app_user` assigned to it. This means, that a user with `app_user` role can access `myApp` route within the defined limits `requestsPerMin`.

```json
{
"routes": {
"myApp": {
"userRoles": ["app_user"] // user must have app_user role in order to access the route
}
},

"roles": {
"app_user": {
"limits": {
"myApp": {
"requestsPerMin": "1000", // user with the app_user role can call up to 1000 requests per min for the route myApp
}
}
}
}
}
```

> Refer to [AI DIAL Core config](https://github.com/epam/ai-dial-core/blob/development/sample/aidial.config.json) to see the full example.

##### Example

For instance, a user of AI DIAL Chat may request an external RAG (Retrieval-Augmented Generation) application to generate a response to a prompt based on an attached file. In this scenario, AI DIAL Chat uses a designated route to interact with the external RAG application. A per-request key is specifically generated for this interaction to ensure secure and authorized communication.

> **Note**: Access to models deployed in AI DIAL and request limits are determined by the roles set up in AI DIAL Core, which are assigned to both the route and the user.

The external RAG application, upon receiving the request, uses the per-request key to call a language model within AI DIAL. This model processes the attached file and generates the necessary response. Subsequently, the response is relayed back to the user in AI DIAL Chat, completing the interaction loop.