Currently Beam has a dependency on a very old version of dill, because upgrading dill proved challenging, which causes significant inconvenience to existing users, see:
#22893 , #32842, #32617, #24458.
We plan to resolve this inconvenience by making cloudpickle the default pickler of choice and remove the mandatory dependency on dill.
Migration plan: s.apache.org/beam-cloudpickle-next-steps
Prior work (for historical reasons/additional context):
#15472, https://s.apache.org/beam-picklers, BEAM-13386.