Checking MXNet using cuDNN

[patrick-nicolas]

Adding cuDNN to train a transformer does not improve the performance (NVIDIA mentioned a 3x to 6x performance jump) - gpu(0). cuDNN 8.6.0 was generating a warning. I downgraded to 8.1.0: warning is gone but still no improvement.
Is there any utils class/method to verify that libcudnn in cuda/lib64 is actually loaded and used my MXNet?

---

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_May__3_18:49:52_PDT_2022
Cuda compilation tools, release 11.7, V11.7.64
Build cuda_11.7.r11.7/compiler.31294372

cuDNN 8.1.0
Scala 2.13.5
MXNet 1.9.1
Lib :mxnet-native-cu102mkl:1.9.1
DJL 0.18.0

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.73.08 Driver Version: 510.73.08 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:1E.0 Off | 0 |
| N/A 38C P0 85W / 70W | 7039MiB / 15360MiB | 87% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 21841 C /home/hadoop/jdk-11/bin/java 7037MiB |
+-----------------------------------------------------------------------------+