Claude Sonnet 3.7 exceeds 200k context window

[jswans33]

Which version of the app are you using?

3.7.2

Which API Provider are you using?

Anthropic

Which Model are you using?

Claude 3.7 Sonnet)

What happened?

constantly exceeding rate limit without dropping context when using 3.7:

400 {"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 141273 + 64000 > 204698, decrease input length or max_tokens and try again"}}

Tokens:
Context Window:
140.7k out of total shown 200.0k
Cache:
+117.9k→2.5m

Steps to reproduce

1. get your context window close to 141k tokens and then it is overwith

Relevant API REQUEST output

Additional context

No response

[mrubens]

Thanks for the details - I will try to fix

[mrubens]

@jswans33 are you using Anthropic directly?

[chuckjewell]

I see this a lot and am a direct anthropic user. Its usually when it tries to get cute and load a too large file but also frequently happens if it runs tests which have animation elements for progress or too long output.

Its not a huge deal but the issue is that once it happens, the session is unrecoverable. It would be nice to have some kind of max size filtering or a way to get back to a useful session after hitting this limit.

[mrubens]

I found some issues in the logic around this - will try my best to get a fix out today.

[chuckjewell]

Great news! I just now lost a good session to this bug and know a few other devs that say this is their biggest issue.

Thank you!

[mrubens]

Working on #1194

[tseven]

Thank you for your work on this fantastic tool!

Using 3.7.4 which should include the context fix, but it may not be including the "thinking" tokens generated with claude 3.7. I just experienced this

400 {"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 143789 + 64000 > 204698, decrease input length or max_tokens and try again"}}

This was using the default thinking tokens of ~16k.

[mrubens]

Thank you for your work on this fantastic tool!

Using 3.7.4 which should include the context fix, but it may not be including the "thinking" tokens generated with claude 3.7. I just experienced this

400 {"type":"error","error":{"type":"invalid_request_error","message":"input length and max_tokens exceed context limit: 143789 + 64000 > 204698, decrease input length or max_tokens and try again"}}

This was using the default thinking tokens of ~16k.

3.7.4 doesn’t have the fix I worked on today. Hope to get it out shortly in 3.7.5. Sorry for the hassle!

[tseven]

Oh gotcha, my bad, thanks for the update!

[jswans33]

@mrubens yes, sorry just saw this - I am using anthropic only.

[mrubens]

Can you all try 3.7.5 and see if it's any better? Thank you! @jswans33 @chuckjewell @tseven

[TypQxQ]

I tried resuming the task after updating to 3.7.5 and get following error when running trough Anthropic provider with Claude 3.7:

429 {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 20,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}

[mrubens]

I tried resuming the task after updating to 3.7.5 and get following error when running trough Anthropic provider with Claude 3.7:

429 {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 20,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}

Might be progress? The rate limits are really annoying - it's what got me to just use OpenRouter. But if you buy $100 of Anthropic credits and/or contact their sales team from a work email it might help.