Skip to content

Commit c63087a

Browse files
author
Ashish Garg
committed
htp_shared_memory: fix inference failures in genai
When using the htp_shared_memory feature, we see that the buffer freed using rpcmem_free is not passed the right address. This leads to memory exhaustion and leads to inference failure.
1 parent e46c0d8 commit c63087a

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

onnxruntime/core/providers/qnn/qnn_allocator.cc

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -181,7 +181,9 @@ void HtpSharedMemoryAllocator::Free(void* allocation_address) {
181181
// Avoid throwing exceptions as this may be running from a destructor.
182182
try {
183183
// take ownership of shared memory and free at end of scope
184-
auto shared_memory = WrapSharedMemoryWithUniquePtr(allocation_address, rpcmem_lib_->Api());
184+
const size_t allocation_offset = AllocationOffsetFromStartOfHeader();
185+
void* raw_allocation_address = (void*)((std::byte*)allocation_address - allocation_offset);
186+
auto shared_memory = WrapSharedMemoryWithUniquePtr(raw_allocation_address, rpcmem_lib_->Api());
185187

186188
// destroy header
187189
allocation_header.~AllocationHeader();

0 commit comments

Comments
 (0)