Skip to content

Commit eb1f35c

Browse files
iindyktfx-copybara
authored andcommitted
Workaround that hopefully fixes pickling segfaults with Python 3.10.
Beam depends on dill version with seemingly outdated logic for CodeType pickling which causes occasional segfaults in Python 3.10. This change patches the pickling logic to a newer version. This is a temporary workaround until Beam depends on a newer version of dill (or this workaround is propagated to Beam) and we update the beam dependency. PiperOrigin-RevId: 546023626
1 parent 2b437f1 commit eb1f35c

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

tensorflow_transform/beam/impl.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@
8080
from tensorflow_transform.tf_metadata import dataset_metadata
8181
from tensorflow_transform.tf_metadata import metadata_io
8282
from tensorflow_transform.tf_metadata import schema_utils
83+
from tfx_bsl import beam as tfx_bsl_beam
8384
from tfx_bsl.coders import example_coder
8485
from tfx_bsl.telemetry import collection as telemetry
8586
from tfx_bsl.telemetry import util as telemetry_util
@@ -94,6 +95,8 @@
9495

9596
from tensorflow_metadata.proto.v0 import schema_pb2
9697

98+
tfx_bsl_beam.fix_code_type_pickling()
99+
97100
_TransformFnPathType = str
98101

99102
Context = context.Context

0 commit comments

Comments
 (0)