Skip to content

Rec automatic optimization, special behavior of layers

Albert Zeyer edited this page Aug 9, 2021 · 16 revisions

A list of layers with special behavior when inside a recurrent loop vs outside recurrent loop (optimized out of the loop).

  • RecLayer / RnnCellLayer. Inside loop, they have hidden state, and do one step. Outside loop, they operate on the time sequence. Determined whether the input has a time dim.

  • TwoDLSTMLayer

  • SelfAttentionLayer (deprecated)

  • KenLmStateLayer

  • EditDistanceTableLayer

  • MaskedComputationLayer

  • UnmaskLayer

Clone this wiki locally