feat: add initial benchmarking suite, Channels benchmark

[westbrook-ai]

No description provided.

[silentoplayz]

Open WebUI Benchmark Suite - Test Report

System Context:

• OS: Ubuntu 24.04.3 LTS
• Hardware: Intel i7-12700K (20 threads), 64GB RAM
• Python Version: 3.13 (via Pyenv)
• Installation Method: Editable (pip install -e .) inside a fresh virtualenv.

1. Installation & Compatibility

✅ Confirmed: The suite installs and runs successfully on Python 3.13. All dependencies (including aiohttp, locust, pandas) resolved correctly without conflicts.

2. Issues Found

A. Missing Benchmark (channels-ws)
The README lists channels-ws as an available benchmark, but the CLI does not recognize it.

$ owb run channels-ws -m 5
Unknown benchmark: channels-ws

Note: I also tried channel-ws just in case, but that failed as well.

B. Initial 403 Forbidden
On the very first run after starting the Docker container via ./run.sh, I received a 403 Forbidden error on /api/v1/channels/create. To resolve this, I manually toggled on the Channels feature in the admin settings from the admin account.

3. Performance Data (Stress Test)

I ran a stress test scaling up to 100 concurrent users on the default profile.

• Max Sustainable Users: 60
• Breaking Point: At 60 users, latency jumped from ~1.1s to ~2.0s. At 100 users, P95 latency hit ~9.3s.

Summary Output:

Testing 10 to 100 concurrent users (step: 10)

━━━ Level 5/10: 50 users ━━━
  Requests: 599 | Avg: 1163.6ms | P95: 1649.2ms | Errors: 0.0%

━━━ Level 6/10: 60 users ━━━
  Requests: 529 | Avg: 2018.4ms | P95: 2856.4ms | Errors: 0.0%

━━━ Level 10/10: 100 users ━━━
  Requests: 454 | Avg: 5795.6ms | P95: 9345.8ms | Errors: 0.0%

                           Final Results Summary                            
┏━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━┳━━━━━━━━┓
┃ Benchmark   ┃ Max Users ┃ Requests ┃ Avg Time ┃ P95 Time ┃  RPS ┃ Errors ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━╇━━━━━━━━┩
│ Channel     │        60 │     4793 │   2018ms │   2856ms │ 15.8 │   0.0% │
│ Concurrency │           │          │          │          │      │        │
└─────────────┴───────────┴──────────┴──────────┴──────────┴──────┴────────┘

[westbrook-ai]

Thanks @silentoplayz, the issues you mentioned above should now be addressed!