Skip to content

Commit 53df398

Browse files
Regan Stehlefacebook-github-bot
authored andcommitted
Add num_instructions program function
Summary: Add num_instructions program function to executorch program methods, because FDLA finds this information useful. FDLA can use this information to gauge how big the method allocator needs to be. While FDLA doesn't have an exact formula yet, most models are fine with a small allocation of 16Kb, whereas some require much more (LLM CRIA requires .3MB, Surface Typing requires .5MB). Checking the number of instructions in the model is a fairly good indicator of this need, as most FDLA models have under 100 instructions, and the latter models have ~1000 instructions. This order of magnitude difference means that, while not an exact science, FDLA can read the number of instructions and surmise that models with above 100 instructions deserve a large size method allocator block, and regular models can take the default size method allocator block. FDLA's method allocator is not dynamically sized, FDLA grants all models with the same size method allocator right now, but it would be an egregious waste of memory to grant all models with a .5 MB allocator (the new top line with Surface Typing model) when most models only need 16 KB. FDLA's method allocator is not dynamic because the core who manages the memory (FDLAMCU) is different from the core who executes the inference (FDLADSP). This thread discusses the same https://fb.workplace.com/groups/pytorch.edge.users/permalink/1523803391823282/ Differential Revision: D66504180
1 parent a9565aa commit 53df398

File tree

3 files changed

+30
-0
lines changed

3 files changed

+30
-0
lines changed

runtime/executor/method_meta.cpp

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -210,5 +210,25 @@ Result<int64_t> MethodMeta::memory_planned_buffer_size(size_t index) const {
210210
return s_plan_->non_const_buffer_sizes()->Get(index + 1);
211211
}
212212

213+
size_t MethodMeta::num_instructions() const {
214+
const auto chains = s_plan_->chains();
215+
if (chains == nullptr) {
216+
return 0;
217+
}
218+
const auto num_chains = chains->size();
219+
auto num_instructions = 0;
220+
for (size_t i = 0; i < num_chains; ++i) {
221+
auto s_chain = chains->Get(i);
222+
if (s_chain == nullptr) {
223+
continue;
224+
}
225+
auto s_instructions = s_chain->instructions();
226+
if (s_instructions != nullptr) {
227+
num_instructions += s_instructions->size();
228+
}
229+
}
230+
return num_instructions;
231+
}
232+
213233
} // namespace runtime
214234
} // namespace executorch

runtime/executor/method_meta.h

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -184,6 +184,13 @@ class MethodMeta final {
184184
* @returns The size in bytes on success, or an error on failure.
185185
*/
186186
Result<int64_t> memory_planned_buffer_size(size_t index) const;
187+
188+
/**
189+
* Get the number of instructions in this method.
190+
*
191+
* @returns The number of instructions.
192+
*/
193+
ET_EXPERIMENTAL size_t num_instructions() const;
187194

188195
/**
189196
* DEPRECATED: Use num_memory_planned_buffers() instead.

runtime/executor/test/method_meta_test.cpp

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,9 @@ TEST_F(MethodMetaTest, MethodMetaApi) {
9292
method_meta->non_const_buffer_size(1).error(),
9393
Error::InvalidArgument); // Deprecated API
9494

95+
// Number instructions in method is nonzero
96+
EXPECT_NE(method_meta->num_instructions(), 0);
97+
9598
// Missing method fails
9699
EXPECT_EQ(
97100
program_->method_meta("not_a_method").error(), Error::InvalidArgument);

0 commit comments

Comments
 (0)