Skip to content

Commit e128165

Browse files
committed
Revert "server : support multi-modal context checkpoints and prompt caching (ikawrakow#1398)"
This reverts commit 433531d.
1 parent be97779 commit e128165

File tree

10 files changed

+552
-700
lines changed

10 files changed

+552
-700
lines changed

common/common.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -422,7 +422,7 @@ struct gpt_params {
422422
float slot_prompt_similarity = 0.1f;
423423

424424
bool do_checkpoint = false; // do checkpoint for recurrent models only
425-
int32_t ctx_checkpoints_n = 32; // max number of context checkpoints per slot
425+
int32_t ctx_checkpoints_n = 8; // max number of context checkpoints per slot
426426
int32_t ctx_checkpoints_interval = 512; // minimum number of tokens between each context checkpoints
427427
int32_t ctx_checkpoints_tolerance = 5; // the number of tokens before the full prompt to create the checkpoint
428428
int32_t cache_ram_mib = 8192; // -1 = no limit, 0 - disable, 1 = 1 MiB, etc.

common/regex-partial.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,7 @@ std::string regex_to_reversed_partial_regex(const std::string & pattern) {
101101
sequence->back() += *it;
102102
auto is_star = *it == '*';
103103
++it;
104-
if (it != end && is_star) {
104+
if (is_star) {
105105
if (*it == '?') {
106106
++it;
107107
}

0 commit comments

Comments
 (0)