Skip to content

Commit b71b22f

Browse files
committed
🖉 Update AI benchmarks
1 parent d4e43ca commit b71b22f

File tree

1 file changed

+19
-11
lines changed

1 file changed

+19
-11
lines changed

readme.md

Lines changed: 19 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -24,21 +24,29 @@ Results:
2424
<!-- include src/AI.Benchmarks/BenchmarkDotNet.Artifacts/results/AI.Benchmarks.ModelPerformance-report-github.md -->
2525
```
2626
27-
BenchmarkDotNet v0.14.0, Windows 11 (10.0.22631.4751/23H2/2023Update/SunValley3)
28-
Intel Core i9-10900T CPU 1.90GHz, 1 CPU, 20 logical and 10 physical cores
29-
.NET SDK 9.0.200-preview.0.25057.12
27+
BenchmarkDotNet v0.14.0, Ubuntu 24.04.1 LTS (Noble Numbat)
28+
AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
29+
.NET SDK 8.0.112
3030
[Host] : .NET 8.0.12 (8.0.1224.60305), X64 RyuJIT AVX2
3131
DefaultJob : .NET 8.0.12 (8.0.1224.60305), X64 RyuJIT AVX2
3232
3333
3434
```
35-
| Method | Client | Provider | Model | Mean | Error | StdDev | Median |
36-
|------- |------------------ |--------- |-------------- |--------:|---------:|---------:|--------:|
37-
| **Chat** | **aai-gpt-4o** | **Azure AI** | **gpt-4o** | **1.536 s** | **0.1220 s** | **0.3298 s** | **1.445 s** |
38-
| **Chat** | **aai-gpt-4o-mini** | **Azure AI** | **gpt-4o-mini** | **1.691 s** | **0.1988 s** | **0.5608 s** | **1.467 s** |
39-
| **Chat** | **oai-gpt-4o** | **OpenAI** | **gpt-4o** | **2.299 s** | **0.1650 s** | **0.4544 s** | **2.287 s** |
40-
| **Chat** | **oai-gpt-4o-mini** | **OpenAI** | **gpt-4o-mini** | **2.738 s** | **0.2487 s** | **0.7135 s** | **2.653 s** |
41-
| **Chat** | **xai-grok-2-latest** | **xAI** | **grok-2-latest** | **1.614 s** | **0.1312 s** | **0.3849 s** | **1.565 s** |
42-
| **Chat** | **xai-grok-beta** | **xAI** | **grok-beta** | **1.656 s** | **0.1114 s** | **0.3231 s** | **1.676 s** |
35+
| Method | Client | Provider | Model | Mean | Error |
36+
|------- |------------------ |--------- |-------------- |-----:|------:|
37+
| **Chat** | **aai-gpt-4o** | **Azure AI** | **gpt-4o** | **NA** | **NA** |
38+
| **Chat** | **aai-gpt-4o-mini** | **Azure AI** | **gpt-4o-mini** | **NA** | **NA** |
39+
| **Chat** | **oai-gpt-4o** | **OpenAI** | **gpt-4o** | **NA** | **NA** |
40+
| **Chat** | **oai-gpt-4o-mini** | **OpenAI** | **gpt-4o-mini** | **NA** | **NA** |
41+
| **Chat** | **xai-grok-2-latest** | **xAI** | **grok-2-latest** | **NA** | **NA** |
42+
| **Chat** | **xai-grok-beta** | **xAI** | **grok-beta** | **NA** | **NA** |
43+
44+
Benchmarks with issues:
45+
ModelPerformance.Chat: DefaultJob [Client=aai-gpt-4o]
46+
ModelPerformance.Chat: DefaultJob [Client=aai-gpt-4o-mini]
47+
ModelPerformance.Chat: DefaultJob [Client=oai-gpt-4o]
48+
ModelPerformance.Chat: DefaultJob [Client=oai-gpt-4o-mini]
49+
ModelPerformance.Chat: DefaultJob [Client=xai-grok-2-latest]
50+
ModelPerformance.Chat: DefaultJob [Client=xai-grok-beta]
4351

4452
<!-- src/AI.Benchmarks/BenchmarkDotNet.Artifacts/results/AI.Benchmarks.ModelPerformance-report-github.md -->

0 commit comments

Comments
 (0)