RWKV

[alienatorZ]

I cannot get my rwkv model to load. I tried the example but I keep getting model load failure.
Here is an example of what I am getting.

┌───────────────────────────────────────────────────┐
│ Fiber v2.46.0 │
│ http://127.0.0.1:8080 │
│ (bound on host 0.0.0.0 and port 8080) │
│ │
│ Handlers ............ 23 Processes ........... 1 │
│ Prefork ....... Disabled PID .............. 1386 │
└───────────────────────────────────────────────────┘

llama.cpp: loading model from /models/rwkv
error loading model: unknown (magic, version) combination: 67676d66, 00000065; is this really a GGML file?
llama_init_from_file: failed to load model
llama.cpp: loading model from /models/rwkv
error loading model: unknown (magic, version) combination: 67676d66, 00000065; is this really a GGML file?
gptjllama_init_from_file: failed to load model
LLAMA ERROR: failed to load model from /models/rwkv
mpt_model_load: loading model from '/models/rwkv' - please wait ...
mpt_model_load: invalid model file '/models/rwkv' (bad magic)
gptj_model_load: loading model from '/models/rwkv' - please wait ...
GPT-J ERROR: failed to load model from /models/rwkvgptj_model_load: invalid model file '/models/rwkv' (bad magic)
GPT-J ERROR: failed to load model from /models/rwkvgpt_neox_model_load: invalid model file '/models/rwkv' (bad magic)
gpt_neox_bootstrap: failed to load model from '/models/rwkv'
bert_load_from_file: invalid model file '/models/rwkv' (bad magic)
bert_bootstrap: failed to load model from '/models/rwkv'
gptj_model_load: invalid model file '/models/rwkv' (bad magic)
gptj_bootstrap: failed to load model from '/models/rwkv'
gpt2_model_load: invalid model file '/models/rwkv' (bad magic)
gpt2_bootstrap: failed to load model from '/models/rwkv'
dollyv2_model_load: invalid model file '/models/rwkv' (bad magic)
dolly_bootstrap: failed to load model from '/models/rwkv'
mpt_model_load: invalid model file '/models/rwkv' (bad magic)
mpt_bootstrap: failed to load model from '/models/rwkv'
replit_model_load: invalid model file '/models/rwkv' (bad magic)
replit_bootstrap: failed to load model from '/models/rwkv'
starcoder_model_load: invalid model file '/models/rwkv' (bad magic)
starcoder_bootstrap: failed to load model from '/models/rwkv'

It doesnt seem to be even trying rwkv. Am i missing something??

[mudler]

you need to set the rwkv backend for the model.

See the rwkv example here: https://github.com/go-skynet/LocalAI/blob/2c918378652522648bc45dd6b5410c5e1713d9ae/examples/rwkv/models/gpt-3.5-turbo.yaml#L9

[alienatorZ]

ahh ok thank you!
Is there a way to reset the model state so that it wont remember previous inputs?

[BradKML]

Any developments on new architectures like Mamba and Jamba?