Error code 503 during API Request

[Fluxus00]

Hello Guys, I am using InvenTree and its API frequently. When I am performing an API request it happens irregularity that i get an error "503 - Service not Available" but when I check the server it still runs normal. The whole error message also contains parts of an http code containing the "Service under maintenance" website of InvenTree.
The error seems to appear randomly, smaller scripts or single API requests working as expected for most of the time. But for my larger scripts, it is almost certain, that they get interrupted by this error. The error always appears at different parts of the script and I was not able to identify something consistent.
Since this error is very annoying I would like to ask if this error is known to someone, how i might be able to fix it and what information i could provide to make finding a solution easier.

[eeintech]

@Fluxus00 Don't have an idea about what's happening, how about implementing re-tries in your script as a workaround, does it help?

[wolflu05]

Have you tried slowing down consecutive requests by adding some delay inbetween?

[Fluxus00]

Thank you both for your answers. I decided to use a combination of both of your suggestions. Like this:
try:
code
except:
time.sleep(0.05)
same code again
I know it is not the best way to handle it but in my case it works. Furthermore i will post the whole error message here later if i encounter it again so you get a better grip of what i mean.

[wolflu05]

May you can build a wrapper for this, or we implement retry directly in the python sdk? @matmair ?

[matmair]

Retry could lead to double entry. My suggestion is to look into why the server is not performant enough to handle the requests or slowing them down a bit. If you do a stock-remove call for example we have no way to tell if it was already done.

[wolflu05]

That's a good aspect. Maybe testing out with only delays to see if your script is doing it's requests too fast in a row.

[matmair]

In my experience with large-scale Django applications, 503 tends to turn up when the database connection is congested (Most of the time by locking write calls). Maybe look into the metrics of your database and see if something is hitting a consistent ceiling. Postgres for example can be sped up a lot by optimising its settings - I have seen databases that handled 100k rows per second in/out without any problems.

[Fluxus00]

So after testing with different scripts, amount and type of requests I can say the error did not interrupt my scripts anymore. I will close the discussion later