Commit 21f053a
authored
increase timeout to receive first token (#263)
Based on discussion from Slack:
https://allenai.slack.com/archives/C07530Y79Q9/p1743707972179919?thread_ts=1743706445.356419&cid=C07530Y79Q9
The timeout we set for the Tulu3 405B release was a little too short for
all models. This was causing Molmo to get "overloaded" errors when it
shouldn't have. @codeviking said that Molmo's TTFT p99 was 10s, so we're
going a little above that to play it safe.1 parent 4eb0dd1 commit 21f053a
1 file changed
+6
-7
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
126 | 126 | | |
127 | 127 | | |
128 | 128 | | |
129 | | - | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
130 | 132 | | |
131 | 133 | | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
| 134 | + | |
136 | 135 | | |
137 | 136 | | |
138 | 137 | | |
| |||
287 | 286 | | |
288 | 287 | | |
289 | 288 | | |
290 | | - | |
| 289 | + | |
291 | 290 | | |
292 | 291 | | |
293 | 292 | | |
| |||
386 | 385 | | |
387 | 386 | | |
388 | 387 | | |
389 | | - | |
| 388 | + | |
390 | 389 | | |
391 | 390 | | |
392 | 391 | | |
| |||
0 commit comments