Avoid inserting Convert operations for irregular ov::Result case #33402
Conversation
github-actions
bot
added
category: NPU
dmatveev
left a comment
dmatveev
left a comment
There was a problem hiding this comment.
A very promising start!
src/plugins/intel_npu/src/plugin/npuw/partitioning/online/snapshot.hpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_npu/src/plugin/npuw/partitioning/online/snapshot.cpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_npu/src/plugin/npuw/partitioning/online/snapshot.cpp
Outdated
Show resolved
Hide resolved
src/plugins/intel_npu/src/plugin/npuw/partitioning/online/snapshot.cpp
Outdated
Show resolved
Hide resolved
nikita-kud
force-pushed
the
fix_F16IC_handling
branch
from
f472bd5 to
543da0d
Compare
| } | ||
|
|
||
| auto firstGroup = *(gset.begin()); | ||
| for(auto output_layer: firstGroup->getOutputs()) { |
There was a problem hiding this comment.
This is why I added Group::getOutputs method
|
|
||
| auto matches = m_layer_matches.at(reptag->id()); | ||
|
|
||
| if(gset.size() <= 1) { |
There was a problem hiding this comment.
Is it even possible to get empty or just one element set for the repeating tag?
There was a problem hiding this comment.
In theory, yes, a single-element set is possible. E.g. the algorithm has found 32 repeating blocks A and 31 repeating blocks B, A+B merge would result in 31 AB blocks + 1 A leftover.
Those blocks are clean up later in the repeating blocks pipeline but may retain their reptags in the process (so it really depends on when this method is called).
| return lrs.count(this_layer_name) > 0; | ||
| }); | ||
|
|
||
| NPUW_ASSERT(layer_bank_iter != matches.end()); |
There was a problem hiding this comment.
So I assume we will have at least one match if we have more than one group.
dmatveev
left a comment
dmatveev
left a comment
There was a problem hiding this comment.
Well done, LGTM! Hope it still works!
|
|
||
| auto matches = m_layer_matches.at(reptag->id()); | ||
|
|
||
| if(gset.size() <= 1) { |
There was a problem hiding this comment.
In theory, yes, a single-element set is possible. E.g. the algorithm has found 32 repeating blocks A and 31 repeating blocks B, A+B merge would result in 31 AB blocks + 1 A leftover.
Those blocks are clean up later in the repeating blocks pipeline but may retain their reptags in the process (so it really depends on when this method is called).
nikita-kud
force-pushed
the
fix_F16IC_handling
branch
from
543da0d to
cc1ac68
Compare
nikita-kud
changed the title
nikita-kud
force-pushed
the
fix_F16IC_handling
branch
2 times, most recently
from
51c805e to
05f47b7
Compare
|
|
||
| pugi::xml_node node = doc.append_child("ensemble"); | ||
| node.append_attribute("gflops") = std::to_string(ens.gflops).data(); | ||
| node.append_attribute("irregular_results") = std::to_string(ens.irregular_results).data(); |
There was a problem hiding this comment.
It looks like there are no tests for printing/parsing this structure, please let me know if I missed something
|
|
||
| #include "model_generator.hpp" | ||
|
|
||
| #include "openvino/op/ops.hpp" |
There was a problem hiding this comment.
changes by clang format
nikita-kud
force-pushed
the
fix_F16IC_handling
branch
from
05f47b7 to
12c16e2
Compare
3 participants
Details:
Essentially the partitioning ignores
::intel_npu::NPUW_F16ICoption for the irregularov::Resultconsumer.Tickets: