When you execute on CUDA of LayerNorm you get his message:
Generalized LayerNorm is currently not implemented on GPU
See: https://github.com/OpenNMT/CTranslate2/blob/master/src/ops/layer_norm_gpu.cu#L36
This task is to complete the implementation to allow to move the computation of these layers in Whisper to GPU:
https://github.com/OpenNMT/CTranslate2/blob/master/src/models/whisper.cc#L392