[FANBOX][cf-ch] All FANBOX options fatal with demjson3.JSONDecodeError (already using temp cookie, old and updated, tested with and without)

[localappdata]

Prerequisites

[x] Did you read FAQ section in readme.md?
[x] Did you test with the latest releases or commit?
[x] Did you search for existing issues in Issues?

Description

All FANBOX options currently fail during the post content fetching logic from post.info
(login OK, basic fetches OK, initial post status fetch OK, post.info for content FATAL).

Steps to Reproduce

1. Run any version using fix(cloudflare): proof of concept to prevent TLS fingerprinting from setting cloudflare off #1456 (latest commit/release recommended)
2. Use any f* option (f3 recommended, f2-f6 for deeper troubleshooting or f1 if applicable)
3. Wait for the CLI to fatal (will happen instantly if user goes with f3)

Expected behavior:

Ability to fetch content after calling post.info.

Actual behavior:

CLI fatals on HTML and JSON entrapment (see second title prefix).

Version

Latest commit 8608b8c was used. Also tested with a bunch of older and much older versions.
Will happen with any commit or release from the past 8 months or so (assuming a temp cookie is passed).

Logs

Easiest example to reproduce and log (f3 option)
DEBUG option (always on for highest verbosity and easier troubleshooting).
post.info immediately triggers access denial on cf challenge failure and breaks the logic.

(Hundreds of KB from the HTML response were not attached.)

PixivUtil20251112 - INFO - Starting with argument: [pixivutil2.py]
PixivUtil20251112 - INFO - Setting log level to: DEBUG
PixivUtil20251112 - INFO - No default cookie jar available, creating... 
PixivUtil20251112 - INFO - Using default DB Path: *.sqlite
PixivUtil20251112 - INFO - Using Blacklist Tags: * items
PixivUtil20251112 - INFO - Using Blacklist Members: * members
PixivUtil20251112 - INFO - Trying to log in with saved cookie
PixivUtil20251112 - INFO - Logging in, return url: https://www.pixiv.net
PixivUtil20251112 - INFO - Login successful
PixivUtil20251112 - INFO - Logged in using cookie
PixivUtil20251112 - INFO - My User Id: *
PixivUtil20251112 - INFO - Premium User: *
PixivUtil20251112 - INFO - Locale = 
PixivUtil20251112 - INFO - Using Blacklist Tags: * items
PixivUtil20251112 - INFO - Using Blacklist Members: * members
PixivUtil20251112 - INFO - Download FANBOX by post id mode (f3)
PixivUtil20251112 - INFO - Trying to log in FANBOX with saved cookie
PixivUtil20251112 - INFO - Logging in with cookie to Fanbox, return url: https://www.fanbox.cc
PixivUtil20251112 - INFO - FANBOX Login successful
PixivUtil20251112 - DEBUG - Getting post detail from https://api.fanbox.cc/post.info?postId=*
PixivUtil20251112 - DEBUG - (HTML response for cf challenge failure and access denial)
PixivUtil20251112 - ERROR - Unknown Error: Can not decode value starting with character '<'
Traceback (most recent call last):
  File "pixivutil2.py", line 1768, in main
    np_is_valid, op_is_valid, selection = main_loop(ewd, op_is_valid, selection, np_is_valid, args, options)
                                          ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pixivutil2.py", line 1500, in main_loop
    menu_fanbox_download_by_post_id(op_is_valid, args, options)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pixivutil2.py", line 998, in menu_fanbox_download_by_post_id
    post = __br__.fanboxGetPostById(post_id)
  File "\common\PixivBrowserFactory.py", line 1099, in fanboxGetPostById
    js = self.fanboxGetPostJsonById(post_id)
  File "\common\PixivBrowserFactory.py", line 1131, in fanboxGetPostJsonById
    js = demjson3.decode(p_response)
  File "<PYTHON_PATH>\site-packages\demjson3.py", line 6070, in decode
    result = j.decode(
        txt,
    ...<2 lines>...
        return_stats=(return_stats or write_stats),
    )
  File "<PYTHON_PATH>\site-packages\demjson3.py", line 5257, in decode
    raise errors[0]
demjson3.JSONDecodeError: Can not decode value starting with character '<'
PixivUtil20251112 - ERROR - Unknown Error, please check the log file: (<class 'demjson3.JSONDecodeError'>,
JSONDecodeError("Can not decode value starting with character '<'",
position=position_marker(offset=0,line=1,column=0), severity='error'), <traceback object at MEMORY_ADDR>)
PixivUtil20251112 - INFO - EXIT: -1

[Flandre5carlet]

Same here, both on the new build and on the 20250426 build. Must be recent because Fanbox downloads worked fine as recently as last Saturday (F1, at least)

[Nandaka]

demjson3.JSONDecodeError: Can not decode value starting with character '<'

I can only think that the requested json wasn't returned correctly. Instead of json, it return html page.

[NHOrus]

Getting artist information from https://api.fanbox.cc/creator.get?userId=268 
Unknown Error, please check the log file: (<class 'mechanize._response.get_seek_wrapper_class.<locals>.httperror_seek_wrapper'>, <httperror_seek_wrapper (urllib.error.HTTPError instance) at 0x7f6a1e8df920 whose wrapped object = <HTTPError 400: 'Bad Request'>>, <traceback object at 0x7f6a1e707340>) 

[samuelgallagher21]

Usually if you're getting back a html page rather than json, that's because your address is being blocked by cloudflare or the cookiestring is invalid (maybe you've changed addresses or are using the English version of pixiv fanbox?)

[localappdata]

Thanks for the suggestion and apologies for the late reply samuelgallagher but haven't changed addresses or locale and the problem happened all of a sudden.

Tried a few VPN nodes as suggested by Nomake Wan in #1489 just in case. No dice, cf-ch still fails and breaks the JSON.

[nomakewan]

Since after "fixing" my issue with Fanbox login (by switching to Chrome) I ran into this problem, I decided to check the logs to see what was going on. Sure enough this issue is that when a request is placed, eventually the JSON response will be replaced with raw HTML. I converted the HTML found in the pixivutil.log file into an actual HTML file and opened it, which resulted in:

So yeah, it would appear Pixiv does not like all these robot requests.

EDIT: As a note I must stress that this only happens to PixivUtil2. Even immediately after getting this error I can still browse Fanbox from my web browsers without any issues whatsoever.

[kannasama]

It's also important to remember to update your browser's user-agent in config.ini. If Chrome or Firefox or whatever browser you used has been updated since the last time you ran the script, even if you update the rest of the cookie values, a mismatched user-agent will also result in this error.

[nomakewan]

My user-agent matches my browser perfectly and always has. I always update it alongside all the cookie-related parameters.

[sk19vn]

I'm running into the same issue, has this been resolved yet?