(PreconditionNotMet) The meta data must be valid when call the mutable data function.

[piperino11]

Hi, when try to training paddleOCR i receved this error:

• 系统环境/System Environment：Ubuntu 22.04.3 LTS
• 版本号/Version：Paddle： PaddleOCR： paddlepaddle-gpu 2.5.1.post120
• 运行指令/Command Code： python3 tools/train.py -c en_PP-OCRv3_rec_TARGA_prova.yml
• 完整报错/Complete Error Message：

W1124 16:46:14.279534 1907812 gpu_resources.cc:106] The GPU compute capability in your current machine is 90, which is not supported by Paddle, it is recommended to install the corresponding wheel package according to the installation information on the official Paddle website.
W1124 16:46:14.279567 1907812 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 9.0, Driver API Version: 12.2, Runtime API Version: 12.0
W1124 16:46:14.280475 1907812 gpu_resources.cc:149] device: 0, cuDNN Version: 8.9.
[2023/11/24 16:46:14] ppocr INFO: train dataloader has 6 iters
[2023/11/24 16:46:14] ppocr INFO: valid dataloader has 6 iters
[2023/11/24 16:46:14] ppocr INFO: load pretrain successful from /home/a.scaiella/PaddleOCR/ESPERIMENTI/TEXT_RECOGNITION_english/en_PP-OCRv3_rec_train/best_accuracy
[2023/11/24 16:46:14] ppocr INFO: During the training process, after the 0th iteration, an evaluation is run every 23 iterations
Could not load symbol cublasGetSmCountTarget from libcublas.so.11. Error: /home/a.scaiella/anaconda3/envs/paddle/lib/./libcublas.so.11: undefined symbol: cublasGetSmCountTarget
Traceback (most recent call last):
File "tools/train.py", line 227, in
main(config, device, logger, vdl_writer)
File "tools/train.py", line 198, in main
program.train(config, train_dataloader, valid_dataloader, device, model,
File "/home/a.scaiella/PaddleOCR/tools/program.py", line 308, in train
loss = loss_class(preds, batch)
File "/home/a.scaiella/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/nn/layer/layers.py", line 1254, in call
return self.forward(*inputs, **kwargs)
File "/home/a.scaiella/PaddleOCR/ppocr/losses/rec_multi_loss.py", line 47, in forward
loss = loss_func(predicts['ctc'],
File "/home/a.scaiella/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/nn/layer/layers.py", line 1254, in call
return self.forward(*inputs, **kwargs)
File "/home/a.scaiella/PaddleOCR/ppocr/losses/rec_ctc_loss.py", line 38, in forward
loss = self.loss_func(predicts, labels, preds_lengths, label_lengths)
File "/home/a.scaiella/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/nn/layer/layers.py", line 1254, in call
return self.forward(*inputs, **kwargs)
File "/home/a.scaiella/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/nn/layer/loss.py", line 1217, in forward
return paddle.nn.functional.ctc_loss(
File "/home/a.scaiella/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/nn/functional/loss.py", line 1900, in ctc_loss
loss_out = warpctc(
File "/home/a.scaiella/anaconda3/envs/paddle/lib/python3.8/site-packages/paddle/nn/functional/loss.py", line 1861, in warpctc
loss_out = _C_ops.warpctc(
RuntimeError: (PreconditionNotMet) The meta data must be valid when call the mutable data function.
[Hint: Expected valid() == true, but received valid():0 != true:1.] (at ../paddle/phi/core/dense_tensor.cc:118)

ANy solution?