Layers with special behavior on dynamic spatial axes

This is a list of layers with special behavior on dynamic spatial axes, i.e. axes with dynamic sequence lengths because considering the padding or sequence lengths is important for correct behavior.

• SoftmaxOverSpatialLayer, will make sure that the padded frames are masked away.
• ReduceLayer. sum, max etc will ignore the padded frames.
• BatchNormLayer (and batch norm in general on any layer)
• (NormLayer actually should have special behavior and ignore padded frames, but incorrectly it does not currently)

(This list is currently incomplete.)

Somewhat related is the option recurrent on each layer class. recurrent=False implies that sequence lengths do not matter as well as the ordering of frames. But this is not exactly the same. E.g. ConvLayer has recurrent=True but ConvLayer does not make use of the sequence lengths.

The obvious example of a layer where the dynamic spatial axes do not matter is LinearLayer.