Add input Reset in ExperimentalDecodeProgram at "end" of decoding infeed. This seems release some memory since currently after switching away from decode programs, some memories is still hold for their input pipeline in trainer host, which is not necessary for resettable case, but can cause OOM.

PiperOrigin-RevId: 489023208

Author: lingvo-bot

lingvo/core/program.py

@@ -1798,6 +1798,12 @@ def Run(self, sess=None, threadpool=None):
     self.WaitThread([infeed_future, decode_future] + post_futures)
     elapsed_secs = time.time() - start_time
 
+    # Reset again after decoding to release input generator memories for other
+    # programs.
+    if self._task.input.params.resettable:
+      tf.logging.info('Resetting input_generator after decoding.')
+      self._task.input.Reset(sess)
+
     return_future = None
     if threadpool:
       return_future = self._ThreadCall(