Skip to content

Commit 8a28387

Browse files
committed
Automated rollback of commit efdaeb7
PiperOrigin-RevId: 379358557
1 parent efdaeb7 commit 8a28387

File tree

3 files changed

+0
-140
lines changed

3 files changed

+0
-140
lines changed

RELEASE.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,6 @@
66

77
## Bug Fixes and Other Changes
88

9-
* Optimized certain stats generators that needs to materialize the input
10-
RecordBatches.
119
* Depends on `protobuf>=3.13,<4`.
1210

1311
## Known Issues

tensorflow_data_validation/types.py

Lines changed: 0 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -136,40 +136,3 @@ def __len__(self) -> int:
136136

137137
def __bool__(self) -> bool:
138138
return bool(self._steps)
139-
140-
141-
# Do not use multiple threads to encode record batches, as parallelism
142-
# should be managed by beam.
143-
_ARROW_CODER_IPC_OPTIONS = pa.ipc.IpcWriteOptions(use_threads=False)
144-
145-
146-
# TODO(b/190756453): Make this into the upstream
147-
# (preference: Arrow, Beam, tfx_bsl).
148-
class _ArrowRecordBatchCoder(beam.coders.Coder):
149-
"""Custom coder for Arrow record batches."""
150-
151-
def encode(self, value: pa.RecordBatch) -> bytes:
152-
sink = pa.BufferOutputStream()
153-
writer = pa.ipc.new_stream(
154-
sink, value.schema, options=_ARROW_CODER_IPC_OPTIONS)
155-
writer.write_batch(value)
156-
writer.close()
157-
return sink.getvalue().to_pybytes()
158-
159-
def decode(self, encoded: bytes) -> pa.RecordBatch:
160-
reader = pa.ipc.open_stream(encoded)
161-
result = reader.read_next_batch()
162-
try:
163-
reader.read_next_batch()
164-
except StopIteration:
165-
pass
166-
else:
167-
raise ValueError("Expected only one RecordBatch in the stream.")
168-
return result
169-
170-
def to_type_hint(self):
171-
return pa.RecordBatch
172-
173-
174-
beam.coders.typecoders.registry.register_coder(pa.RecordBatch,
175-
_ArrowRecordBatchCoder)

tensorflow_data_validation/types_test.py

Lines changed: 0 additions & 101 deletions
This file was deleted.

0 commit comments

Comments
 (0)