Enhancement: Custom Endpoint Permissions

[tip-dteller]

What features would you like to see added?

Allow usage of endpoints depending on User Group Claim.
Think of the following Scenario:

There are 2 endpoints in Custom section:
Endpoint A - OpenAI
Endpoint B - Google

# If you want to restrict access by groups
OPENID_REQUIRED_ROLE_TOKEN_KIND=id
OPENID_REQUIRED_ROLE_PARAMETER_PATH="roles"
OPENID_REQUIRED_ROLE="Your Group Name" <--- Should be List, comma separated

In order to route via teams, additional configuration params are required:
permissionsRole <- Should match the name of the group in AD\Entra
Example:
SecurityGroup - Team-A
permissionsRole: "${OPENAI_GROUP_CLAIM_NAME}"
The Config.yaml would look like:

    endpoints:
      custom:
      - name: "ChatGPT"
        apiKey: "${LITELLM_OPENAI_API_KEY}"
        baseURL: "http://litellm.default:4000"
        models:
          default: ["gpt-4-turbo"]
          fetch: true
        titleConvo: true
        titleModel: "gpt-4-turbo"
        summarize: false
        summaryModel: "gpt-4-turbo"
        forcePrompt: false
        modelDisplayLabel: "ChatGPT"
        permissionsRole: "${OPENAI_GROUP_CLAIM_NAME}" <-- New parameter
      - name: "Google"
        apiKey: "${LITELLM_GOOGLE_API_KEY}"
        baseURL: "http://litellm.default:4000"
        models:
          default: ["gemini"]
          fetch: true
        titleConvo: true
        titleModel: "Gemini"
        summarize: false
        summaryModel: "Gemini"
        forcePrompt: false
        modelDisplayLabel: "Gemini"
        permissionsRole: "${GOOGLE_GROUP_CLAIM_NAME}"

In UI, when Person Z from Team-A logs in with SSO with GroupClaim - team-a
The JSON from Azure Entra would contain:

{
  ...
  "_claim_names": {
   "groups": "team-a"
    },
    {
  "_claim_sources": {
    "src1": {
        "endpoint":"[Url to get this user's group membership from]"
        }
       }
     }
  ...
}

The user is then allowed to access the UI.
User clicks on "ChatGPT" Icon in the UI to start a chat with this endpoint.
Endpoint compares groupClaim with configuration and allows usage of Endpoint.

Person Z from Team-A tries to click on Google Endpoint and is thrown an error.

Assuming this works, we can then parse groups: team-a
to the proxy, LiteLLM in this case by adding this to the endpoint's configuration in the yaml:

...
        headers:
          team_id: "${OPENAI_GROUP_CLAIM_NAME}"
          Content-Type: "application/json"
...

On LiteLLM side, its possible to configure a team based routing and associate it with this group so it would work.

More details

None

Which components are impacted by your request?

Endpoints

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

There will be role based access controls but not quite like this

[danny-avila]

Mainly through a method that is accessible to all login methods. This is one of my next big projects slated for development and PRs are welcome after that is developed for other specific login methods

[tip-dteller]

sounds great, im just thinking out loud, with the scenario i've described above.
since LibreChat cannot control user expenses (at this point in time), its necessary to integrate with LiteLLM.
and in order to fine tune that control, a certain commonality is required.