PeerDAS: rate limit when syncing #8033
Description
Describe the bug
Sometimes we can not sync a peer up to head, the #1 error of "Batch download error" is rate limit:
...
2025-07-03 20:34:01.640 | Jul-03 13:34:01.639[sync] verbose: Batch download error id=Head, startEpoch=1240, status=Downloading, peer=16...2dKKZW, code=REQUEST_ERROR_INVALID_REQUEST, errorMessage=sNaPpYuV~2rate limited |
-- | -- | --
| | 2025-07-03 20:34:01.639 | Jul-03 13:34:01.639[sync] verbose: Batch download error id=Head, startEpoch=1242, status=Downloading, peer=16...2dKKZW, code=REQUEST_ERROR_INVALID_REQUEST, errorMessage=sNaPpYuV~2rate limited |
| | 2025-07-03 20:34:01.638 | Jul-03 13:34:01.637[sync] verbose: Batch download error id=Head, startEpoch=1241, status=Downloading, peer=16...2dKKZW, code=REQUEST_ERROR_INVALID_REQUEST, errorMessage=sNaPpYuV~2rate limited |
| | 2025-07-03 20:34:01.627 | Jul-03 13:34:01.627[sync] verbose: Batch download error id=Head, startEpoch=1236, status=Downloading, peer=16...2dKKZW, code=REQUEST_ERROR_INVALID_REQUEST, errorMessage=sNaPpYuV~2rate limited |
| | 2025-07-03 20:34:01.551 | Jul-03 13:34:01.551[sync] verbose: Batch download error id=Head, startEpoch=1239, status=Downloading, peer=16...2dKKZW, code=REQUEST_ERROR_INVALID_REQUEST, errorMessage=sNaPpYuV~2rate limited |
| | 2025-07-03 20:34:01.551 | Jul-03 13:34:01.551[sync] verbose: Batch download error id=Head, startEpoch=1237, status=Downloading, peer=16...2dKKZW, code=REQUEST_ERROR_INVALID_REQUEST, errorMessage=sNaPpYuV~2rate limited |
| | 2025-07-03 20:34:01.550 | Jul-03 13:34:01.550[sync] verbose: Batch download error id=Head, startEpoch=1235, status=Downloading, peer=16...2dKKZW, code=REQUEST_ERROR_INVALID_REQUEST, errorMessage=sNaPpYuV~2rate limited |
| | 2025-07-03 20:34:01.549 | Jul-03 13:34:01.549[sync] verbose: Batch download error id=Head, startEpoch=1238, status=Downloading, peer=16...2dKKZW, code=REQUEST_ERROR_INVALID_REQUEST, errorMessage=sNaPpYuV~2rate limited |
| | 2025-07-03 20:34:01.549 | Jul-03 13:34:01.548[sync] verbose: Batch download error id=Head, startEpoch=1233, status=Downloading, peer=16...2dKKZW, code=REQUEST_ERROR_INVALID_REQUEST, errorMessage=sNaPpYuV~2rate limited |
| | 2025-07-03 20:34:01.548 | Jul-03 13:34:01.548[sync] verbose: Batch download error id=Head, startEpoch=1232, status=Downloading, peer=16...2dKKZW, code=REQUEST_ERROR_INVALID_REQUEST, errorMessage=sNaPpYuV~2rate limited
...
there are are logs like:
2025-07-03 20:34:22.740
Jul-03 13:34:22.740[sync] verbose: Batch download error id=Head, startEpoch=1199, status=Downloading, peer=16...bA5Tw4, code=REQUEST_ERROR_UNKNOWN_ERROR_STATUS, errorMessage=HsNaPpYLX:Rate limited. There are already 2 active requests with the same protocol, status=139
Expected behavior
we should manage how many active requests in a peer
also it seems we download batches too aggressive, we should only download n epochs ahead
Steps to reproduce
No response
Additional context
No response
Operating system
Linux
Lodestar version or commit hash
peerDAS