The JAX docs state this:

In the context of JAX transformations, Python exceptions should be considered side-effects: this means that intentionally raising an error within a pure_callback breaks the API contract, and the behavior of the resulting program is undefined.

Our callbacks can raise, so switching has_side_effect to True seems like the safer option. I don't know if this has any runtime impact right now (didn't notice any while testing).