Skip to content

Commit 363bcbc

Browse files
committed
XTC: Sorting before resizing
1 parent a4a8d60 commit 363bcbc

File tree

1 file changed

+5
-2
lines changed

1 file changed

+5
-2
lines changed

base/llama-addon.cpp

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,8 +56,11 @@ void llama_sample_xtc_addon(struct llama_context * ctx, llama_token_data_array *
5656
}
5757
}
5858
}
59-
60-
candidates->sorted = false;
59+
// sorting with new logits
60+
std::sort(candidates->data, candidates->data + candidates->size, [](const llama_token_data & a, const llama_token_data & b) {
61+
return a.logit > b.logit;
62+
});
63+
//resizing now that penalized tokens are at the back
6164
candidates->size = candidates->size - removed;
6265

6366
llama_set_time(ctx, t_start_sample_us);

0 commit comments

Comments
 (0)