关于Expert数量限制的一些问题 #2749
Unanswered
sitabulaixizawaluduo
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
`float max_logit = 0.f;
// Find top-k
PRAGMA_UNROLL
for (int k = 0; k < top_k; ++k) {
int max_bit = 0;
float max_val = -std::numeric_limits::infinity();
int bit = 1;
for (int e = 0; e < experts; ++e) {
const auto val = logits[ti * experts + e];
// const auto val = logits[e * tokens + ti];
if ((mask & bit) && val > max_val) {
max_bit = bit;
max_val = val;
}
bit *= 2;
}
mask -= max_bit;
if (k == 0) {
max_logit = max_val;
}
}`
从这段代码看,是不是意味着能够支持最大的expert数量只有32?感觉解答
Beta Was this translation helpful? Give feedback.
All reactions