Skip to content

Commit 73cb971

Browse files
committed
Update on "[flat_tensor] Persist FreeableBuffers of external constants in method"
## Problem Currently, the FlatTensorDataMap persists tensors, and returns a FreeableBuffer with an empty free function. The NamedDataMap should not persist data, as most cases (eg. delegate) will want it to be freed. Ownership should be on the caller; `get_data` returns a FreeableBuffer that 'owns' the data. The FreeableBuffer in turn is owned by the caller. NOTE: this doesn't support the case where we want to share plain tensors between methods/pte files at runtime. A custom NDM could support that use-case. ## This diff: 1. Introduces a 'NamedData' struct to method.h. This holds a key and a FreeeableBuffer. 2. Iterate over all the flatbuffer tensors to count the constants tagged with EXTERNAL. NOTE: this will increase load time for all users. Potentially allocate chunks of 16 and use a linked list to store external constants, or store this number in PTE file. 3. Allocate space for num_external_constants using the method allocator. 4. Iterate over all flatbuffer tensors and use the named_data_map to resolve EXTERNAL tensors into the array of NamedData. 5. Pass the resolved external constants to tensor_parser, along with NDM (used for mutable external tensors). 6. Resolved external tensors are stored inside method. They are freed when the method is destructed. Some notes: https://docs.google.com/document/d/1_PBi4JgODuClUPD4PCUWrKNjyUH54zOUHGUJ3QHDNes/edit?tab=t.0#heading=h.blsvwraxss7g Differential Revision: [D69477027](https://our.internmc.facebook.com/intern/diff/D69477027/) [ghstack-poisoned]
2 parents 04ab3b5 + ecc6556 commit 73cb971

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

runtime/executor/method.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -392,7 +392,7 @@ Error Method::parse_external_constants(const NamedDataMap* named_data_map) {
392392
new (&external_constants_[n_external_constants_].buffer)
393393
FreeableBuffer(std::move(buffer.get()));
394394

395-
n_external_constants_ = i + 1;
395+
n_external_constants_ += 1;
396396
}
397397
return Error::Ok;
398398
}

runtime/executor/tensor_parser.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ ET_NODISCARD Result<executorch::aten::Tensor> parseTensor(
3535
MemoryManager* memory_manager,
3636
const executorch_flatbuffer::Tensor* s_tensor,
3737
const NamedDataMap* named_data_map = nullptr,
38-
Span<NamedData> entries = {});
38+
Span<NamedData> external_constants = {});
3939

4040
ET_NODISCARD Result<BoxedEvalueList<executorch::aten::Tensor>> parseTensorList(
4141
const flatbuffers::Vector<int32_t>* tensor_indices,

0 commit comments

Comments
 (0)