We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent f29155e commit df205a9Copy full SHA for df205a9
README.md
@@ -143,6 +143,7 @@ They are all prefixed with `tgi_`. Descriptions will be added to the table below
143
| `tgi_batch_inference_batch_size` | `histogram` | method = "prefill" or "next_token" | |
144
| `tgi_batch_inference_duration` | `histogram` | method = "prefill" or "next_token", makeup | |
145
| `tgi_batch_inference_forward_duration` | `histogram` | method = "prefill" or "next_token", makeup | |
146
+| `tgi_batch_inference_tokproc_duration` | `histogram` | method = "prefill" or "next_token", makeup | |
147
| `tgi_batch_next_tokens` | `histogram` | | Prefill only |
148
| `tgi_batch_current_size` | `gauge` | | |
149
| `tgi_batch_input_tokens` | `gauge` | | |
0 commit comments