`USER_AGENT` open-mastr is blocked

[FlorianK13]

Description of the issue

Our user-agent is set to open-mastr/{version('open-mastr')} python-requests/{version('requests')} when requesting the data from BNetzA. I faced the following issue:

1. Normal download with open-mastr User Agent is blocked with this error code:
   requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

2. Manually Setting user-agent=None

3. Downloading MaStR without a problem.

This is a rather serious issue. Do you have similar problems @nesnoj @pt-kkraemer ?

Possible solutions

I see three possible explanations:

• Maybe it is also a combination of IP and user agent that is blocked?
• user-agent = open-mastr is blocked at times of high traffic
• it was a coincidence and they do not specifically block user-agent = open-mastr

[FlorianK13]

Few minutes later, I was again able to download with the default open-mastr USER-AGENT. Explanations:

• It was coincidence: There was high load at BNetzA Server and that is why my request was declined. When I tried again with USER-AGENT=None, the load at BNetzA was lower
• At high load times they first block open-mastr users

[pt-kkraemer]

Our run this morning went smoothly and without any problems. A quick test run just right now was also working as intended.

We are currently part of a TUM Data Clinic course, maybe this could have caused a larger than usual amount of eduroam IPs. What do you think @Ortimis ?

[Ortimis]

Highly unlikely @pt-kkraemer

[FlorianK13]

I researched shortly and it seems like eduroam has no own IP range, so you get the one assigned from your local access point.
I'd leave that issue open, if the problem occurs again we can note it here.

[nesnoj]

I encountered the same last week in the GH actions tests, retriggering did the trick.
If the problem persists and it is caused by the user agent we could implement some client-side random user agent string selection.