Avoid un-skippable errors by "bailing out"

[lucacri]

The main way I use rule code is to let it run on the background on a second monitor and I keep an eye on it every once in a while, which is great. Unfortunately, there are times where there are some errors that are unavoidable anymore (example: 400 {"type":"error","error":{"type":"invalid_request_error","message":"prompt is too long: 210564 tokens > 200000 maximum"}}) and the system keeps on retrying forever.

It would be great if we introduced a new role that will have the behavior of "bailing out" the task from either problems that are only fixable by using a different model (Example, needing one with has a bigger context size), as well, maybe the option to... Step in if the task has been failing for several times instead of saying that the model might be too weak to continue.

If we do it right, this would be fantastic when using weaker models locally because most Most of the times they can do a lot, but then they get stuck on some steps that a bigger model would be able to fix it quickly.

This could lead to being able to let it run overnight with a local model and still having and good result in the morning.