Free previous model vram when switch to another model?

[jmjoy]

When I switch model sam2:large to sam2:latest, I got error message:

2025-11-18 11:31:40.660864176 [E:onnxruntime:, inference_session.cc:2544 operator()] Exception during initialization: /onnxruntime_src/onnxruntime/core/framework/bfc_arena.cc:359 void* onnxruntime::BFCArena::AllocateRawInternal(size_t, bool, onnxruntime::Stream*) Failed to allocate memory for requested buffer of size 3211264                                                        
                                                                                                                                                                                               
2025-11-18 11:31:40.660 | ERROR    | osam.types._model:__init__:44 - Failed to create inference session with providers ['CUDAExecutionProvider', 'CPUExecutionProvider']. Falling back to ['CPUExecutionProvider']: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: /onnxruntime_src/onnxruntime/core/framework/bfc_arena.cc:359 void* onnxruntime::BFCArena::AllocateRawInternal(size_t, bool, onnxruntime::Stream*) Failed to allocate memory for requested buffer of size 3211264                                                                           
                                                                                                                                                                                               
2025-11-18 11:31:40.746850217 [W:onnxruntime:, graph.cc:120 MergeShapeInfo] Error merging shape info for output. '/conv_s0/Conv_output_0' source:{1,32,256,256} target:{}. Falling back to lenient merge.                                                                                                                                                                                     
2025-11-18 11:31:40.747383860 [W:onnxruntime:, graph.cc:120 MergeShapeInfo] Error merging shape info for output. '/conv_s1/Conv_output_0' source:{1,64,128,128} target:{}. Falling back to lenient merge.                                                                                                                                                                                     
2025-11-18 11:31:41.116 | INFO     | osam.types._model:__init__:57 - Initialized inference sessions with providers ['CPUExecutionProvider']

There are two model in used, cause vram OOM and fallback to CPUExecutionProvider, which is very slow.

So, is better to free previous model vram when switch to another model?

[wkentaro]

i'll check if we are releasing the memory when switching the model, which I intended

[wkentaro]

@jmjoy thanks for the report, this issue should be resolved by #1715, please let me know if you still observe an issue 🙏