@@ -24,29 +24,21 @@ Results:
24
24
<!-- include src/AI.Benchmarks/BenchmarkDotNet.Artifacts/results/AI.Benchmarks.ModelPerformance-report-github.md -->
25
25
```
26
26
27
- BenchmarkDotNet v0.14.0, Ubuntu 24.04.1 LTS (Noble Numbat )
28
- AMD EPYC 7763 , 1 CPU, 4 logical and 2 physical cores
29
- .NET SDK 8 .0.112
27
+ BenchmarkDotNet v0.14.0, Windows 11 (10.0.22631.4751/23H2/2023Update/SunValley3 )
28
+ Intel Core i9-10900T CPU 1.90GHz , 1 CPU, 20 logical and 10 physical cores
29
+ .NET SDK 9 .0.200-preview.0.25057.12
30
30
[Host] : .NET 8.0.12 (8.0.1224.60305), X64 RyuJIT AVX2
31
31
DefaultJob : .NET 8.0.12 (8.0.1224.60305), X64 RyuJIT AVX2
32
32
33
33
34
34
```
35
- | Method | Client | Provider | Model | Mean | Error |
36
- | ------- | ------------------ | --------- | -------------- | -----:| ------:|
37
- | ** Chat** | ** aai-gpt-4o** | ** Azure AI** | ** gpt-4o** | ** NA** | ** NA** |
38
- | ** Chat** | ** aai-gpt-4o-mini** | ** Azure AI** | ** gpt-4o-mini** | ** NA** | ** NA** |
39
- | ** Chat** | ** oai-gpt-4o** | ** OpenAI** | ** gpt-4o** | ** NA** | ** NA** |
40
- | ** Chat** | ** oai-gpt-4o-mini** | ** OpenAI** | ** gpt-4o-mini** | ** NA** | ** NA** |
41
- | ** Chat** | ** xai-grok-2-latest** | ** xAI** | ** grok-2-latest** | ** NA** | ** NA** |
42
- | ** Chat** | ** xai-grok-beta** | ** xAI** | ** grok-beta** | ** NA** | ** NA** |
43
-
44
- Benchmarks with issues:
45
- ModelPerformance.Chat: DefaultJob [ Client=aai-gpt-4o]
46
- ModelPerformance.Chat: DefaultJob [ Client=aai-gpt-4o-mini]
47
- ModelPerformance.Chat: DefaultJob [ Client=oai-gpt-4o]
48
- ModelPerformance.Chat: DefaultJob [ Client=oai-gpt-4o-mini]
49
- ModelPerformance.Chat: DefaultJob [ Client=xai-grok-2-latest]
50
- ModelPerformance.Chat: DefaultJob [ Client=xai-grok-beta]
35
+ | Method | Client | Provider | Model | Mean | Error | StdDev | Median |
36
+ | ------- | ------------------ | --------- | -------------- | --------:| ---------:| ---------:| --------:|
37
+ | ** Chat** | ** aai-gpt-4o** | ** Azure AI** | ** gpt-4o** | ** 2.414 s** | ** 0.5845 s** | ** 1.6197 s** | ** 1.670 s** |
38
+ | ** Chat** | ** aai-gpt-4o-mini** | ** Azure AI** | ** gpt-4o-mini** | ** 2.748 s** | ** 0.5885 s** | ** 1.6599 s** | ** 1.954 s** |
39
+ | ** Chat** | ** oai-gpt-4o** | ** OpenAI** | ** gpt-4o** | ** 2.646 s** | ** 0.2275 s** | ** 0.6563 s** | ** 2.493 s** |
40
+ | ** Chat** | ** oai-gpt-4o-mini** | ** OpenAI** | ** gpt-4o-mini** | ** 2.845 s** | ** 0.2176 s** | ** 0.6173 s** | ** 2.738 s** |
41
+ | ** Chat** | ** xai-grok-2-latest** | ** xAI** | ** grok-2-latest** | ** 1.629 s** | ** 0.1238 s** | ** 0.3651 s** | ** 1.618 s** |
42
+ | ** Chat** | ** xai-grok-beta** | ** xAI** | ** grok-beta** | ** 1.731 s** | ** 0.1186 s** | ** 0.3402 s** | ** 1.697 s** |
51
43
52
44
<!-- src/AI.Benchmarks/BenchmarkDotNet.Artifacts/results/AI.Benchmarks.ModelPerformance-report-github.md -->
0 commit comments