On some dates I get the following error:JSONDecodeError: Expecting value

[apavlo89]

I get the error below on some dates. Why that is and how can I fix it? Thank you :/

  File "C:\Users\apavl\reddit\download_data_from_reddit.py", line 95, in <module>
    data = get_pushshift_data(key_word, after, before, sub_reddit)

  File "C:\Users\apavl\reddit\download_data_from_reddit.py", line 15, in get_pushshift_data
    data = json.loads(r.text)

  File "C:\Users\apavl\anaconda3\envs\reddit\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)

  File "C:\Users\apavl\anaconda3\envs\reddit\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())

  File "C:\Users\apavl\anaconda3\envs\reddit\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None

JSONDecodeError: Expecting value

[sb-barbarossa]

Hi!
Not sure if this is still relevant for you right now, but I "fixed" this issue by catching the JSONDecodeError where get_pushshift_data() is called in the main - in both lines 93 and 106 - and just repeated the method call.
So something like this:

try:
data = get_pushshift_data(key_word, after, before, sub_reddit)
except json.decoder.JSONDecodeError:
print('JSON decode error, retrying API request...')
data = get_pushshift_data(key_word, after, before, sub_reddit)

This is obviously not a fix, but the error occurred very inconsistently for me and wouldn't occur two times in a row on the same API request. At the very least, it has made the process more stable for me.