NV TR-rtx execution provider

[rajeshkAymar]

i am trying to use NV TR-rtx execution provider and successfully integrated it with my code. Buts it look performance is not better than TensorRT execution provider.

I created the NV TR RTX engine , i tried to reuse the engine and performance is not better than the NV TR.

025-07-18 13:55:44.826156442 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 49022, index: 1, mask: {2, 26, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:44.826156217 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 49021, index: 0, mask: {1, 25, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:44.826202330 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 49023, index: 2, mask: {3, 27, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:44.826227178 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 49025, index: 3, mask: {4, 28, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:44.826289790 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 49026, index: 4, mask: {5, 29, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:44.826339731 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 49027, index: 5, mask: {6, 30, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:44.826364490 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 49028, index: 6, mask: {7, 31, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:44.826397704 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 49029, index: 7, mask: {8, 32, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:44.826461447 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 49030, index: 8, mask: {9, 33, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:44.826478281 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 49031, index: 9, mask: {10, 34, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:44.826559286 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49032, index: 10, mask: {11, 35, }
2025-07-18 13:55:44.826585419 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49033, index: 11, mask: {12, 36, }
2025-07-18 13:55:44.826615581 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49034, index: 12, mask: {13, 37, }
2025-07-18 13:55:44.826658931 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49035, index: 13, mask: {14, 38, }
2025-07-18 13:55:44.826844270 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49041, index: 19, mask: {20, 44, }
2025-07-18 13:55:44.826926550 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49039, index: 17, mask: {18, 42, }
2025-07-18 13:55:44.827336999 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49037, index: 15, mask: {16, 40, }
2025-07-18 13:55:44.828459584 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49040, index: 18, mask: {19, 43, }
2025-07-18 13:55:44.828690907 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49044, index: 22, mask: {23, 47, }
2025-07-18 13:55:44.839320010 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49036, index: 14, mask: {15, 39, }
2025-07-18 13:55:44.839326527 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49043, index: 21, mask: {22, 46, }
2025-07-18 13:55:44.839330724 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49038, index: 16, mask: {17, 41, }
2025-07-18 13:55:44.846032574 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 49042, index: 20, mask: {21, 45, }
2025-07-18 13:55:45.041629216 [V:onnxruntime:ORT_LOG_FATAL, nv_execution_provider.cc:1214 NvExecutionProvider] [NvTensorRTRTX EP] TensorRT version is 10000
2025-07-18 13:55:45.041658550 [V:onnxruntime:ORT_LOG_FATAL, nv_execution_provider.cc:1215 NvExecutionProvider] [NvTensorRTRTX EP] CUDA version is 12020
2025-07-18 13:55:45.041670797 [V:onnxruntime:ORT_LOG_FATAL, nv_execution_provider.cc:1217 NvExecutionProvider] [NvTensorRTRTX EP] Nv provider options: device_id: 0, nv_max_partition_iterations: 1000, nv_min_subgraph_size: 1, nv_max_workspace_size: 3221225472, nv_dump_subgraphs: 0, nv_weight_stripped_engine_enable: 1, nv_onnx_model_folder_path: , nv_engine_decryption_enable: 0, nv_engine_decryption_lib_path: , nv_force_sequential_engine_build: 0, nv_context_memory_sharing_enable: 0, nv_sparsity_enable: 0, nv_auxiliary_streams: -1, nv_cuda_graph_enable: 0, nv_dump_ep_context_model: 1, nv_ep_context_file_path: , nv_ep_context_embed_mode: 0, nv_cache_prefix: , nv_onnx_model_bytestream_size_: 0, nv_op_types_to_exclude:
2025-07-18 13:55:45.041689495 [V:onnxruntime:ORT_LOG_FATAL, nv_execution_provider_custom_ops.cc:72 CreateTensorRTCustomOpDomainList] [NvTensorRTRTX EP] Getting all registered TRT plugins from TRT plugin registry ...
2025-07-18 13:55:45.142822074 [I:onnxruntime:ORT_LOG_FATAL, nv_execution_provider_custom_ops.cc:83 CreateTensorRTCustomOpDomainList] [NvTensorRTRTX EP] Default plugins successfully loaded.
2025-07-18 13:55:45.142919389 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:28 BFCArena] Creating BFCArena for Cuda with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 initial_growth_chunk_size_bytes: 2097152 max_power_of_two_extend_bytes: 1073741824 memory limit: 18446744073709551615 arena_extend_strategy: 0
2025-07-18 13:55:45.142928740 [V:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:65 BFCArena] Creating 21 bins of max chunk size 256 to 268435456
2025-07-18 13:55:45.142939541 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:28 BFCArena] Creating BFCArena for CudaPinned with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 initial_growth_chunk_size_bytes: 2097152 max_power_of_two_extend_bytes: 1073741824 memory limit: 18446744073709551615 arena_extend_strategy: 0
2025-07-18 13:55:45.142946526 [V:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:65 BFCArena] Creating 21 bins of max chunk size 256 to 268435456
2025-07-18 13:55:45.142956354 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:28 BFCArena] Creating BFCArena for Cpu with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 initial_growth_chunk_size_bytes: 2097152 max_power_of_two_extend_bytes: 1073741824 memory limit: 18446744073709551615 arena_extend_strategy: 0
2025-07-18 13:55:45.142964454 [V:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:65 BFCArena] Creating 21 bins of max chunk size 256 to 268435456
2025-07-18 13:55:45.146118934 [I:onnxruntime:ORT_LOG_FATAL, nv_execution_provider_utils.h:465 TRTGenerateId] [NvTensorRTRTX EP] Model name is onnx_VRLow_250_ctx.onnx
2025-07-18 13:55:45.146158212 [I:onnxruntime:ORT_LOG_FATAL, nv_execution_provider.cc:1529 GetSubGraph] [NvTensorRTRTX EP] TensorRT subgraph MetaDef name TRTKernel_graph_TRTKernel_graph_tf2onnx_15342500236894012246_0_11187674933298151044_0
2025-07-18 13:55:45.146218844 [W:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:403 ValidateEPCtxNode] [NvTensorRTRTX EP] Engine was compiled for a different compatibility level and might not work or perform suboptimal
2025-07-18 13:55:45.146228530 [W:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:404 ValidateEPCtxNode] [NvTensorRTRTX EP] The compute capability of the engine: 86+
2025-07-18 13:55:45.146236030 [W:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:405 ValidateEPCtxNode] [NvTensorRTRTX EP] The compute capability of the GPU: 86
2025-07-18 13:55:45.146265077 [V:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:328 GetEpContextFromGraph] [NvTensorRTRTX EP] GetEpContextFromGraph engine_cache_path: /usr/g/sdc_image_pool/rajesh/iungo_steps-9.57-sandbox/config/cj/onnx/ctx_models/NvTensorRTRTXExecutionProvider_TRTKernel_graph_tf2onnx_15342500236894012246_0_0_strong_typed.stripped.engine
2025-07-18 13:55:45.146284874 [V:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:339 GetEpContextFromGraph] [NvTensorRTRTX EP] NvTensorRTRTXExecutionProvider_TRTKernel_graph_tf2onnx_15342500236894012246_0_0_strong_typed.engine exists.
2025-07-18 13:55:45.180026924 [V:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:362 GetEpContextFromGraph] [NvTensorRTRTX EP] DeSerialized NvTensorRTRTXExecutionProvider_TRTKernel_graph_tf2onnx_15342500236894012246_0_0_strong_typed.engine
2025-07-18 13:55:46.358587086 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:346 AllocateRawInternal] Extending BFCArena for Cuda. bin_num:12 (requested) num_bytes: 1681920 (actual) rounded_bytes:1681920
2025-07-18 13:55:46.358844652 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:205 Extend] Extended allocation by 2097152 bytes.
2025-07-18 13:55:46.358851793 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:208 Extend] Total allocated bytes: 2097152
2025-07-18 13:55:46.358856814 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:211 Extend] Allocated memory at 0x7f29e9800000 to 0x7f29e9a00000
2025-07-18 13:55:46.359155461 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:346 AllocateRawInternal] Extending BFCArena for Cuda. bin_num:12 (requested) num_bytes: 1681920 (actual) rounded_bytes:1681920
2025-07-18 13:55:46.359325494 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:205 Extend] Extended allocation by 2097152 bytes.
2025-07-18 13:55:46.359333974 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:208 Extend] Total allocated bytes: 4194304
2025-07-18 13:55:46.359341663 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:211 Extend] Allocated memory at 0x7f29e9a00000 to 0x7f29e9c00000
2025-07-18 13:55:46.416625604 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 365, index: 0, mask: {1, 25, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:46.416664951 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 366, index: 1, mask: {2, 26, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:46.416689712 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 367, index: 2, mask: {3, 27, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:46.416729670 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 368, index: 3, mask: {4, 28, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:46.416747747 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 369, index: 4, mask: {5, 29, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:46.416798438 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 370, index: 5, mask: {6, 30, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:46.416814403 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 371, index: 6, mask: {7, 31, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:46.416864550 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 372, index: 7, mask: {8, 32, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:46.416910767 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 373, index: 8, mask: {9, 33, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:46.416935133 [E:onnxruntime:ORT_LOG_FATAL, env.cc:227 ThreadMain] pthread_setaffinity_np failed for thread: 374, index: 9, mask: {10, 34, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.
2025-07-18 13:55:46.416975607 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 375, index: 10, mask: {11, 35, }
2025-07-18 13:55:46.417002286 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 376, index: 11, mask: {12, 36, }
2025-07-18 13:55:46.417059138 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 377, index: 12, mask: {13, 37, }
2025-07-18 13:55:46.417085478 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 378, index: 13, mask: {14, 38, }
2025-07-18 13:55:46.417154862 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 379, index: 14, mask: {15, 39, }
2025-07-18 13:55:46.417163613 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 380, index: 15, mask: {16, 40, }
2025-07-18 13:55:46.417239362 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 382, index: 17, mask: {18, 42, }
2025-07-18 13:55:46.417264493 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 383, index: 18, mask: {19, 43, }
2025-07-18 13:55:46.417321434 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 384, index: 19, mask: {20, 44, }
2025-07-18 13:55:46.417352779 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 385, index: 20, mask: {21, 45, }
2025-07-18 13:55:46.417386443 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 386, index: 21, mask: {22, 46, }
2025-07-18 13:55:46.418785986 [V:onnxruntime:ORT_LOG_FATAL, nv_execution_provider.cc:1214 NvExecutionProvider] [NvTensorRTRTX EP] TensorRT version is 10000
2025-07-18 13:55:46.418800737 [V:onnxruntime:ORT_LOG_FATAL, nv_execution_provider.cc:1215 NvExecutionProvider] [NvTensorRTRTX EP] CUDA version is 12020
2025-07-18 13:55:46.418808636 [V:onnxruntime:ORT_LOG_FATAL, nv_execution_provider.cc:1217 NvExecutionProvider] [NvTensorRTRTX EP] Nv provider options: device_id: 0, nv_max_partition_iterations: 1000, nv_min_subgraph_size: 1, nv_max_workspace_size: 3221225472, nv_dump_subgraphs: 0, nv_weight_stripped_engine_enable: 1, nv_onnx_model_folder_path: , nv_engine_decryption_enable: 0, nv_engine_decryption_lib_path: , nv_force_sequential_engine_build: 0, nv_context_memory_sharing_enable: 0, nv_sparsity_enable: 0, nv_auxiliary_streams: -1, nv_cuda_graph_enable: 0, nv_dump_ep_context_model: 1, nv_ep_context_file_path: , nv_ep_context_embed_mode: 0, nv_cache_prefix: , nv_onnx_model_bytestream_size_: 0, nv_op_types_to_exclude:
2025-07-18 13:55:46.418823642 [V:onnxruntime:ORT_LOG_FATAL, nv_execution_provider_custom_ops.cc:72 CreateTensorRTCustomOpDomainList] [NvTensorRTRTX EP] Getting all registered TRT plugins from TRT plugin registry ...
2025-07-18 13:55:46.418969377 [I:onnxruntime:ORT_LOG_FATAL, nv_execution_provider_custom_ops.cc:83 CreateTensorRTCustomOpDomainList] [NvTensorRTRTX EP] Default plugins successfully loaded.
2025-07-18 13:55:46.419004743 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:28 BFCArena] Creating BFCArena for Cuda with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 initial_growth_chunk_size_bytes: 2097152 max_power_of_two_extend_bytes: 1073741824 memory limit: 18446744073709551615 arena_extend_strategy: 0
2025-07-18 13:55:46.419012895 [V:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:65 BFCArena] Creating 21 bins of max chunk size 256 to 268435456
2025-07-18 13:55:46.419020064 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:28 BFCArena] Creating BFCArena for CudaPinned with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 initial_growth_chunk_size_bytes: 2097152 max_power_of_two_extend_bytes: 1073741824 memory limit: 18446744073709551615 arena_extend_strategy: 0
2025-07-18 13:55:46.419025850 [V:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:65 BFCArena] Creating 21 bins of max chunk size 256 to 268435456
2025-07-18 13:55:46.419035277 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:28 BFCArena] Creating BFCArena for Cpu with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 initial_growth_chunk_size_bytes: 2097152 max_power_of_two_extend_bytes: 1073741824 memory limit: 18446744073709551615 arena_extend_strategy: 0
2025-07-18 13:55:46.419041299 [V:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:65 BFCArena] Creating 21 bins of max chunk size 256 to 268435456
2025-07-18 13:55:46.419081935 [I:onnxruntime:ORT_LOG_FATAL, nv_execution_provider_utils.h:465 TRTGenerateId] [NvTensorRTRTX EP] Model name is onnx_VRHigh_250_ctx.onnx
2025-07-18 13:55:46.419105827 [I:onnxruntime:ORT_LOG_FATAL, nv_execution_provider.cc:1529 GetSubGraph] [NvTensorRTRTX EP] TensorRT subgraph MetaDef name TRTKernel_graph_TRTKernel_graph_tf2onnx_11481305936560953865_0_17797625797846353329_0
2025-07-18 13:55:46.419150123 [W:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:403 ValidateEPCtxNode] [NvTensorRTRTX EP] Engine was compiled for a different compatibility level and might not work or perform suboptimal
2025-07-18 13:55:46.419156650 [W:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:404 ValidateEPCtxNode] [NvTensorRTRTX EP] The compute capability of the engine: 86+
2025-07-18 13:55:46.419162850 [W:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:405 ValidateEPCtxNode] [NvTensorRTRTX EP] The compute capability of the GPU: 86
2025-07-18 13:55:46.419180163 [V:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:328 GetEpContextFromGraph] [NvTensorRTRTX EP] GetEpContextFromGraph engine_cache_path: /usr/g/sdc_image_pool/rajesh/iungo_steps-9.57-sandbox/config/cj/onnx/ctx_models/NvTensorRTRTXExecutionProvider_TRTKernel_graph_tf2onnx_11481305936560953865_0_0_strong_typed.stripped.engine
2025-07-18 13:55:46.419192809 [V:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:339 GetEpContextFromGraph] [NvTensorRTRTX EP] NvTensorRTRTXExecutionProvider_TRTKernel_graph_tf2onnx_11481305936560953865_0_0_strong_typed.engine exists.
2025-07-18 13:55:46.423384609 [V:onnxruntime:ORT_LOG_FATAL, onnx_ctx_model_helper.cc:362 GetEpContextFromGraph] [NvTensorRTRTX EP] DeSerialized NvTensorRTRTXExecutionProvider_TRTKernel_graph_tf2onnx_11481305936560953865_0_0_strong_typed.engine
2025-07-18 13:55:46.431328497 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 381, index: 16, mask: {17, 41, }
2025-07-18 13:55:46.455313263 [V:onnxruntime:ORT_LOG_FATAL, env.cc:220 ThreadMain] pthread_setaffinity_np succeed for thread: 387, index: 22, mask: {23, 47, }
2025-07-18 13:55:47.513445758 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:346 AllocateRawInternal] Extending BFCArena for Cuda. bin_num:12 (requested) num_bytes: 1681920 (actual) rounded_bytes:1681920
2025-07-18 13:55:47.513676298 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:205 Extend] Extended allocation by 2097152 bytes.
2025-07-18 13:55:47.513683184 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:208 Extend] Total allocated bytes: 2097152
2025-07-18 13:55:47.513688654 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:211 Extend] Allocated memory at 0x7f2959200000 to 0x7f2959400000
2025-07-18 13:55:47.513906666 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:346 AllocateRawInternal] Extending BFCArena for Cuda. bin_num:12 (requested) num_bytes: 1681920 (actual) rounded_bytes:1681920
2025-07-18 13:55:47.514048391 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:205 Extend] Extended allocation by 2097152 bytes.
2025-07-18 13:55:47.514053988 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:208 Extend] Total allocated bytes: 4194304
2025-07-18 13:55:47.514058695 [I:onnxruntime:ORT_LOG_FATAL, bfc_arena.cc:211 Extend] Allocated memory at 0x7f2959400000 to 0x7f2959600000