Activation and norm sequence #2
Description
A thoughtful piece of work! I’d like to ask why the code applies batch normalization after the use of a restrictive activation function. Wouldn’t this cause a shift in the distribution of information before normalization?