Skip to content

Commit a14023c

Browse files
tf-transform-teamtfx-copybara
authored andcommitted
Make sure that WriteTransformFn.publish_outputs is idempotent.
PiperOrigin-RevId: 700817208
1 parent 9ab3ebc commit a14023c

File tree

1 file changed

+6
-0
lines changed

1 file changed

+6
-0
lines changed

tensorflow_transform/beam/tft_beam_io/transform_fn_io.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,9 +102,15 @@ def publish_outputs(unused_element, metadata_source_path,
102102
if not tf.io.gfile.exists(self._path):
103103
tf.io.gfile.makedirs(self._path)
104104

105+
if tf.io.gfile.exists(metadata_path):
106+
tf.io.gfile.rmtree(metadata_path)
105107
tf.io.gfile.rename(metadata_source_path, metadata_path, overwrite=True)
108+
109+
if tf.io.gfile.exists(transform_fn_path):
110+
tf.io.gfile.rmtree(transform_fn_path)
106111
tf.io.gfile.rename(
107112
transform_fn_source_path, transform_fn_path, overwrite=True)
113+
108114
# TODO(b/211615643): Remove the exists check once importing TFIO in S3
109115
# addresses NotFoundError.
110116
if tf.io.gfile.exists(base_temp_dir):

0 commit comments

Comments
 (0)