[Statistics] 429 Too many requests resulting in "Failed to fetch the info from the server"

[rafa-br34]

For some reason, Statistics absolutely bombards Spotify's servers with requests (or they increased the rate limit), thus resulting in a 429 Too Many Requests response. To make matters worse, it appears apiFetch does not retry the request after temporary errors like these.

Spotify_24-12-25_23-11-08.485.mp4

[Zinibow]

I am having this same issue, any fixes?

[Imfreaknasty]

any updates

[lowcomplex-mint]

no updates?

[rafa-br34]

@Mossbraker seems to be working on a PR for this exact issue: 0f09692

[Mossbraker]

@Mossbraker seems to be working on a PR for this exact issue: 0f09692

I was looking into it. I tried various workarounds, but nothing worked. If you read issues in the Spicetify CLI repo such as spicetify/cli#3670, the cause seems to be that Spotify has implemented aggressive rate limiting on certain API endpoints. This is not related to Spicetify, but rather a result of Spotify's rate limiting and there's no simple workaround. Below is a quote from the Spicetify maintainer:

There are other ways to fetch stuff (spclient endpoints that work with usertoken) but we don't have a list of those so you have to find them yourself. Spotify is breaking stuff constantly and everyone developing for spicetify is well aware of this. We don't have to tell people that they have to find workarounds for stuff that broke or can be broken. We are just a wrapper around existing stuff. There is information on our docs about CosmosAsync that you should only use it for internal endpoints (which are far gone rn); we have made this method work again because Spotify broke it for external URLs and bunch of custom apps/extensions used it. It's up to you how you call spotify api.

Future Possibilities

1. When Spotify Reopens App Registration

Below are two possible ways to rebuild Spicetify Statistics to use OAuth authentication similar to other projects like Your Spotify and Statify, and many websites such as statsforspotify.com.

However, Spotify closed registration for new apps in December, so it is not currently possible to create new Client IDs, which OAuth requires.

A. Implement OAuth with user provided Client ID. Users would need to:

1. Create a Spotify Developer App at https://developer.spotify.com/dashboard
2. Add redirect URI: http://localhost:8888/callback
3. Enter Client ID in Stats app settings
4. Click "Authenticate with OAuth"

This method would currently only work for users who have previously created an app and have a Client ID. Anyone who doesn't have a Client ID would have to wait for registration to reopen. I implemented this in a branch in my fork, but I don't have a Client ID so I haven't tested it.

B. Developer provides their Client ID for OAuth Implementation

• harbassan creates a Spotify Developer App at https://developer.spotify.com/dashboard
• harbassan would have to host/maintain infrastructure.

This could be implemented now if harbassan already has a Client ID, or later once registration reopens, but would require them to host/maintain infrastructure.

2. Alternative Endpoints

The Spicetify maintainer mentioned that alternative spclient endpoints exist but are undocumented. These internal endpoints may not be subject to the same rate limits but would require reverse engineering. Any new endpoint could be rate limited or break entirely at some point in the future without notice.

Best Option

I think option 1A would be best, but it requires waiting for Spotify to reopen app registration and, as far as I know, they haven't provided any timeframe as to when this might happen. However, there may be alternative methods which I haven't thought of. Anyway, this is just my understanding of the situation, and I'm no expert.

[rafa-br34]

The Rate Limiting Issue

According to spicetify/cli#3670, Spotify has implemented aggressive rate limiting on certain API endpoints. Key findings:

1. Rate limits are account-level, not session-level
2. Cooldown periods can be up to 24 hours
3. Even 5-10 requests can trigger rate limits
4. The limit applies regardless of the fetch method used (CosmosAsync, direct fetch, etc.)

Ouch. In that case, we will probably either have to wait for apps or collect statistics over a long period of time and cache them.

[harbassan]

@Mossbraker thanks for the investigation into this, I would also lean towards 1A or further investigation into internal endpoints, i'm sure we can patch something in to capture all outgoing calls to see what the app itself does for fetching track info.

[harbassan]

If anyone is coming here in the meantime, check out #242, it should fix it for the most part (if not, don't make another issue please, I can't keep track otherwise!).