Rec automatic optimization, special behavior of layers

A list of layers with special behavior when inside a recurrent loop vs outside recurrent loop (optimized out of the loop).

• RecLayer / RnnCellLayer. Inside loop, they have hidden state, and do one step. Outside loop, they operate on the time sequence. Determined whether the input has a time dim.

• TwoDLSTMLayer

• SelfAttentionLayer (deprecated)

• KenLmStateLayer

• EditDistanceTableLayer

• MaskedComputationLayer

• UnmaskLayer

• WindowLayer. Inside loop, keeps the previous N (window_size - 1) frames as hidden state such that you have [B,window_size,...]. Assuming window_right=0 and window_left=window_size - 1. Outside loop, just adds the window axis (with an efficient implementation).

• CumsumLayer. For input x: inside loop, does output = prev:output + x. Outside loop, wraps tf.cumsum.