Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 46 additions & 58 deletions docs/my-website/docs/proxy/docker_quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,15 +69,35 @@ Setup your config.yaml with your azure model.

Note: When using the proxy with a database, you can also **just add models via UI** (UI is available on `/ui` route).

### 1.1 Set up a Database

**Requirements**
- Need a postgres database (e.g. [Supabase](https://supabase.com/), [Neon](https://neon.tech/), etc)

```yaml
model_list:
- model_name: gpt-4o
- model_name: gpt-5-mini
litellm_params:
model: azure/my_azure_deployment
model: azure/gpt-5-mini
api_base: os.environ/AZURE_API_BASE
api_key: "os.environ/AZURE_API_KEY"
api_version: "2025-01-01-preview" # [OPTIONAL] litellm uses the latest azure api_version by default

- model_name: gpt-4o
litellm_params:
model: openai/gpt-4o
api_key: os.environ/OPENAI_API_KEY


general_settings:
master_key: sk-1234
database_url: "postgresql://<user>:<password>@<host>:<port>/<dbname>" # 👈 KEY CHANGE
```

Save config.yaml as `litellm_config.yaml` (used in 3.2).



---

### Model List Specification
Expand Down Expand Up @@ -244,65 +264,10 @@ curl -X POST 'http://0.0.0.0:4000/chat/completions' \

Track Spend, and control model access via virtual keys for the proxy

### 3.1 Set up a Database

**Requirements**
- Need a postgres database (e.g. [Supabase](https://supabase.com/), [Neon](https://neon.tech/), etc)


```yaml
model_list:
- model_name: gpt-4o
litellm_params:
model: azure/my_azure_deployment
api_base: os.environ/AZURE_API_BASE
api_key: "os.environ/AZURE_API_KEY"
api_version: "2025-01-01-preview" # [OPTIONAL] litellm uses the latest azure api_version by default

general_settings:
master_key: sk-1234
database_url: "postgresql://<user>:<password>@<host>:<port>/<dbname>" # 👈 KEY CHANGE
```

Save config.yaml as `litellm_config.yaml` (used in 3.2).
![virtual_keys_demo](../../img/virtualkey.gif)

---

**What is `general_settings`?**

These are settings for the LiteLLM Proxy Server.

See All General Settings [here](http://localhost:3000/docs/proxy/configs#all-settings).

1. **`master_key`** (`str`)
- **Description**:
- Set a `master key`, this is your Proxy Admin key - you can use this to create other keys (🚨 must start with `sk-`).
- **Usage**:
- **Set on config.yaml** set your master key under `general_settings:master_key`, example -
`master_key: sk-1234`
- **Set env variable** set `LITELLM_MASTER_KEY`

2. **`database_url`** (str)
- **Description**:
- Set a `database_url`, this is the connection to your Postgres DB, which is used by litellm for generating keys, users, teams.
- **Usage**:
- **Set on config.yaml** set your `database_url` under `general_settings:database_url`, example -
`database_url: "postgresql://..."`
- Set `DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname>` in your env

### 3.2 Start Proxy

```bash
docker run \
-v $(pwd)/litellm_config.yaml:/app/config.yaml \
-e AZURE_API_KEY=d6*********** \
-e AZURE_API_BASE=https://openai-***********/ \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-latest \
--config /app/config.yaml --detailed_debug
```


### 3.3 Create Key w/ RPM Limit

Create a key with `rpm_limit: 1`. This will only allow 1 request per minute for calls to proxy with this key.
Expand Down Expand Up @@ -499,6 +464,29 @@ LiteLLM Proxy uses the [LiteLLM Python SDK](https://docs.litellm.ai/docs/routing

`litellm_settings` are module-level params for the LiteLLM Python SDK (equivalent to doing `litellm.<some_param>` on the SDK). You can see all params [here](https://github.com/BerriAI/litellm/blob/208fe6cb90937f73e0def5c97ccb2359bf8a467b/litellm/__init__.py#L114)

**What is `general_settings`?**

These are settings for the LiteLLM Proxy Server.

See All General Settings [here](http://localhost:3000/docs/proxy/configs#all-settings).

1. **`master_key`** (`str`)
- **Description**:
- Set a `master key`, this is your Proxy Admin key - you can use this to create other keys (🚨 must start with `sk-`).
- **Usage**:
- **Set on config.yaml** set your master key under `general_settings:master_key`, example -
`master_key: sk-1234`
- **Set env variable** set `LITELLM_MASTER_KEY`

2. **`database_url`** (str)
- **Description**:
- Set a `database_url`, this is the connection to your Postgres DB, which is used by litellm for generating keys, users, teams.
- **Usage**:
- **Set on config.yaml** set your `database_url` under `general_settings:database_url`, example -
`database_url: "postgresql://..."`
- Set `DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname>` in your env


## Support & Talk with founders

- [Schedule Demo 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version)
Expand Down
56 changes: 34 additions & 22 deletions docs/my-website/docs/proxy/virtual_keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,27 +13,6 @@ Track Spend, and control model access via virtual keys for the proxy

:::

## Setup

Requirements:

- Need a postgres database (e.g. [Supabase](https://supabase.com/), [Neon](https://neon.tech/), etc)
- Set `DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname>` in your env
- Set a `master key`, this is your Proxy Admin key - you can use this to create other keys (🚨 must start with `sk-`).
- ** Set on config.yaml** set your master key under `general_settings:master_key`, example below
- ** Set env variable** set `LITELLM_MASTER_KEY`

(the proxy Dockerfile checks if the `DATABASE_URL` is set and then initializes the DB connection)

```shell
export DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname>
```


You can then generate keys by hitting the `/key/generate` endpoint.

[**See code**](https://github.com/BerriAI/litellm/blob/7a669a36d2689c7f7890bc9c93e04ff3c2641299/litellm/proxy/proxy_server.py#L672)

## **Quick Start - Generate a Key**
**Step 1: Save postgres db url**

Expand Down Expand Up @@ -66,6 +45,39 @@ curl 'http://0.0.0.0:4000/key/generate' \
--data-raw '{"models": ["gpt-3.5-turbo", "gpt-4"], "metadata": {"user": "[email protected]"}}'
```

**Expected Response:**
```json
{
"key": "sk-1234567890abcdef",
"expires": "2024-01-15T10:30:00Z",
"user_id": "[email protected]",
"team_id": null,
"models": ["gpt-3.5-turbo", "gpt-4"],
"spend": 0.0,
"max_budget": null
}
```

🎉 **Success!** Your virtual key `sk-1234567890abcdef` is ready to use!

**Step 4: Test your key**

```bash
curl 'http://0.0.0.0:4000/chat/completions' \
-H 'Authorization: Bearer sk-1234567890abcdef' \
-H 'Content-Type: application/json' \
-d '{
"model": "gpt-3.5-turbo",
"messages": [{"role": "user", "content": "Hello! This is a test."}]
}'
```

#### Generate keys from UI

![virtual_keys](../../img/virtualkey.gif)



## 🔁 Scheduled Key Rotations (NEW in v1.77.5)

LiteLLM can now rotate **virtual keys automatically** on a schedule you define.
Expand Down Expand Up @@ -116,7 +128,7 @@ Get spend per:
- key - via `/key/info` [Swagger](https://litellm-api.up.railway.app/#/key%20management/info_key_fn_key_info_get)
- user - via `/user/info` [Swagger](https://litellm-api.up.railway.app/#/user%20management/user_info_user_info_get)
- team - via `/team/info` [Swagger](https://litellm-api.up.railway.app/#/team%20management/team_info_team_info_get)
- end-users - via `/end_user/info` - [Comment on this issue for end-user cost tracking](https://github.com/BerriAI/litellm/issues/2633)
- end-users - via `/customer/info` [Swagger](https://litellm-api.up.railway.app/#/Customer%20Management)

**How is it calculated?**

Expand Down
Binary file added docs/my-website/img/getting_started_logging.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/my-website/img/virtualkey.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading