prowlarr needs an additional header for certain indexers that use flaresolverr

[durzo]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

I have been seeing failures for multiple indexers where by even when using flaresolverr/byparr the indexer returns a 403 error which is cloudflare blocking the request.

I spent a good few hours debugging this, and have found a solution thats working across the board for me.

The environment:

• prowlarr and flaresolverr in their respective docker containers, not using a vpn, on my home internet connection (no VPS).
• Add an indexer that is cloudflare protected, for example kickasstorrents.to any BaseUrl

The Problem:

1. Prowlarr sends the initial GET request to /17/all and successfully detects Cloudflare
2. Prowlarr passes the request to Flaresolverr, which solves CF and passes back the cf_clearance cookie
3. Prowlarr throws away the Flaresolverr response body and sends the request again, using the new cf_clearance cookie - BUT - it adds its own header of Accept-Encoding: gzip
4. Cloudflare/KAT seems to be able to detect this is not a valid request if the Accept-Encoding header is present and responds with the 403 from nginx - perhaps they know only Prowlarr adds this header?

The investigation:
Using curl, I was able to send the same request without the Accept-Encoding header and received the expected valid response from KAT. I then noticed that curl also adds its own header Accept: */*

I can't remove the Accept-Encoding: gzip header as its baked into prowlarr source, but we can add the Accept: */* header into the definition.

The solution:
I copied the kickasstorrents-to.yml definition into Definitions/Custom/, renamed it, and added two parts:

1. add custom user-agent setting, this may not be needed as i think prowlarr takes the UA from flaresolverr?
2. add the new Accept: */* header, this seems to be real fix.

The diff looks like:

--- ../kickasstorrents-to.yml	2026-01-24 17:05:38.000000000 +0000
+++ kickasstorrents-to-UA.yml	2026-01-25 19:32:38.590689381 +0000
@@ -1,6 +1,6 @@
 ---
-id: kickasstorrents-to
-name: kickasstorrents.to
+id: kickasstorrents-to-ua
+name: kickasstorrents.to-ua
 description: "kickasstorrents.to is a Public KickAssTorrent clone for MOVIES / TV / GENERAL"
 language: en-US
 type: public
@@ -125,6 +125,11 @@
 settings:
   - name: info_flaresolverr
     type: info_flaresolverr
+  - name: useragent
+    type: text
+    label: User-Agent
+  - name: info_useragent
+    type: info_useragent
 
 search:
   paths:
@@ -132,6 +137,10 @@
     - path: "{{ if .Keywords }}search/?q={{ .Keywords }}{{ else }}17/All/{{ end }}"
     - path: "{{ if .Keywords }}search/?page=2&q={{ .Keywords }}{{ else }}17/All/?page=2{{ end }}"
 
+  headers:
+    User-Agent: ["{{ .Config.useragent }}"]
+    Accept: ["*/*"]
+
   rows:
     selector: table.data > tbody > tr:has(a[href^="magnet:?xt="])
 

Using the custom definition, the flow is the same as before - Prowlarr detects cloudflare and sends to flaresolverr, then uses the cookie to send another request directly to KAT but this time adds the Accept: */* header, and we have success!

Expected Behavior

cloudflare protected indexers work

Steps To Reproduce

steps listed above

Environment

- OS: Ubuntu 24.04 as docker host
- Prowlarr: I tested on stable and develop container builds
- Docker Install: Yes
- Using Reverse Proxy: No
- Browser: N/A

What branch are you running?

Develop

Trace Logs? Not Optional

files attached:

• prowlarr-before uses stock kickasstorrents-to.yml
• prowlarr-after uses patched (diff submitted above) kickasstorrents-to.yml as a custom definition

prowlarr-before.txt

prowlarr-after.txt

Trace Logs have been provided as applicable. Reports will be closed if the required logs are not provided.

• I have read and followed the steps in the wiki link above and provided the required trace logs - the logs contain trace - that are relevant and show this issue.

[durzo]

[AngusNB]

I tried your modifications with extratorrent.st. I didn't add the Agent references just the Accept: ["/"] and changed the naming.
With the added lines I can add the indexer and it works with byparr again.

[durzo]

Thank you for confirming!

I don't know why this isn't getting any traction from the developers.. did I open it wrong?

[AngusNB]

I don't know why this isn't getting any traction from the developers.. did I open it wrong?

Not sure. It probably doesn't impact many indexers. Extratorrent was the only flaresolverr reliant indexer I had a problem with. I tested btstate and use TorrentLeech with no issues.