This repository was archived by the owner on Nov 17, 2023. It is now read-only.
mxnet 1.9.1 cuda support for Jetson AGX Orin (Jetpack 5.0 with cuda 11.4) #21166
Description
Description
mxnet (1.9.1) is raising a cuda error with cuda 11.4 on Jetson AGX Orin series with Jetpack 5.0
Error Message
Check failed: err == cudaSuccess (209 vs. 0) : mxnet_generic_kernel ErrStr:no kernel image is available for execution on the device
To Reproduce
I ran the installation steps recommended for the jetson and built things manually from source with cuda path and other variables.
Everything went fine with the make and importing mxnet and version check. When I run the gpu check the error pops up.
What have you tried to solve it?
- Verified cuda installation, no errors raise during make
- Other libraries like pytorch GPU are working fine
Environment
Ubuntu 20.04, Nvidia jetson AGX Orin developer kit
Environment Information
----------Python Info----------
Version : 3.8.10
Compiler : GCC 9.4.0
Build : ('default', 'Nov 14 2022 12:59:47')
Arch : ('64bit', 'ELF')
------------Pip Info-----------
Version : 22.3.1
Directory : /home/flashsys002/py38/lib/python3.8/site-packages/pip
----------MXNet Info-----------
Version : 1.9.1
Directory : /home/flashsys002/mxnet/python/mxnet
Commit hash file "/home/flashsys002/mxnet/python/mxnet/COMMIT_HASH" not found. Not installed from pre-built package or built from source.
Library : ['/home/flashsys002/mxnet/lib/libmxnet.so', '/home/flashsys002/mxnet/python/mxnet/../../lib/libmxnet.so']
Build features:
✔ CUDA
✔ CUDNN
✖ NCCL
✖ CUDA_RTC
✖ TENSORRT
✖ CPU_SSE
✖ CPU_SSE2
✖ CPU_SSE3
✖ CPU_SSE4_1
✖ CPU_SSE4_2
✖ CPU_SSE4A
✖ CPU_AVX
✖ CPU_AVX2
✔ OPENMP
✖ SSE
✖ F16C
✖ JEMALLOC
✔ BLAS_OPEN
✖ BLAS_ATLAS
✖ BLAS_MKL
✖ BLAS_APPLE
✖ LAPACK
✖ MKLDNN
✖ OPENCV
✖ CAFFE
✖ PROFILER
✖ DIST_KVSTORE
✖ CXX14
✖ INT64_TENSOR_SIZE
✔ SIGNAL_HANDLER
✖ DEBUG
✖ TVM_OP
----------System Info----------
Platform : Linux-5.10.65-tegra-aarch64-with-glibc2.29
system : Linux
node : flashsys002-desktop
release : 5.10.65-tegra
version : #1 SMP PREEMPT Mon May 16 20:58:07 PDT 2022
----------Hardware Info----------
machine : aarch64
processor : aarch64
Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 12
On-line CPU(s) list: 0-11
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 3
Vendor ID: ARM
Model: 1
Model name: ARMv8 Processor rev 1 (v8l)
Stepping: r0p1
CPU max MHz: 2201.6001
CPU min MHz: 115.2000
BogoMIPS: 62.50
L1d cache: 768 KiB
L1i cache: 768 KiB
L2 cache: 3 MiB
L3 cache: 6 MiB
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp uscat ilrcpc flagm
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/mxnet, DNS: 0.0029 sec, LOAD: 0.4289 sec.
Error open Gluon Tutorial(en): http://gluon.mxnet.io, HTTP Error 404: Not Found, DNS finished in 0.07585453987121582 sec.
Error open Gluon Tutorial(cn): https://zh.gluon.ai, <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)>, DNS finished in 0.035755157470703125 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0161 sec, LOAD: 0.4418 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0113 sec, LOAD: 0.2129 sec.
Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.021700382232666016 sec.
----------Environment----------
MXNET_HOME="/home/flashsys002/mxnet/"
KMP_DUPLICATE_LIB_OK="True"
KMP_INIT_AT_FORK="FALSE"