Skip to content

Commit e1366be

Browse files
authored
Update llama-sagemaker-benchmark.md (#1534)
1 parent e5f0003 commit e1366be

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

llama-sagemaker-benchmark.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -94,9 +94,9 @@ The Best Throughput configuration maximizes the number of tokens that are genera
9494

9595
| Model | Quantization | Instance | concurrent requests | Latency (ms/token) median | Throughput (tokens/second) | On-demand cost ($/h) in us-west-2 | Time to generate 1 M tokens (minutes) | cost to generate 1M tokens ($) |
9696
| ----------- | ------------ | --------------- | ------------------- | ------------------------- | -------------------------- | --------------------------------- | ------------------------------------- | ------------------------------ |
97-
| Llama 2 7B | None | ml.g5.12xlarge | 20 | 43.99524 | 449.9423027 | $7.09 | 37.04 | $4.38 |
98-
| Llama 2 13B | None | ml.g5.12xlarge | 20 | 67.4027465 | 295.6378071 | $7.09 | 18.72 | $2.21 |
99-
| Llama 2 70B | None | ml.p4d.24xlarge | 20 | 59.798591 | 321.5369158 | $37.69 | 16.61 | $10.43 |
97+
| Llama 2 7B | None | ml.g5.12xlarge | 20 | 43.99524 | 449.9423027 | $7.09 | 33.59 | $3.97 |
98+
| Llama 2 13B | None | ml.p4d.12xlarge | 20 | 67.4027465 | 668.0204881 | $37.69 | 24.95 | $15.67 |
99+
| Llama 2 70B | None | ml.p4d.24xlarge | 20 | 59.798591 | 321.5369158 | $37.69 | 51.83 | $32.56 |
100100

101101
### Best Latency Deployment
102102

@@ -117,4 +117,4 @@ We hope the benchmark will help companies deploy Llama 2 optimally based on thei
117117

118118
---
119119

120-
Thanks for reading! If you have any questions, feel free to contact me on [Twitter](https://twitter.com/_philschmid) or [LinkedIn](https://www.linkedin.com/in/philipp-schmid-a6a2bb196/).
120+
Thanks for reading! If you have any questions, feel free to contact me on [Twitter](https://twitter.com/_philschmid) or [LinkedIn](https://www.linkedin.com/in/philipp-schmid-a6a2bb196/).

0 commit comments

Comments
 (0)