Ollama embedding error context length exceeded for jina-embeddings-v2-zh

[chenyuz3]

I noticed that for no matter what models used for vualt QA embedding, the default context length/chunk size is 2048 tokens, which may lead to a reduction of retrieval performance as any note longer than that will be simply abandoned. Many embedding models have larger context window by default so may be we can add an option in settings for that?

[logancyang]

@chenyuz3 Ollama has model-specific settings which you can set using /set parameter .... Embedding context length is model-specific, so it shouldn't be set on the client side imo.

[chenyuz3]

Thx @logancyang , I have created a new model based on the original model with a context length larger, however I can still see some errors stating the context window is still not enough for some notes, I wonder how do copilot coupe with such situation? Does it simply cut off the note or something else?

[logancyang]

@chenyuz3 copilot does nothing when it exceeds the context length, the error you see should fail the chunk entirely so it won't appear in the index.

It is up to the backend to decide whether to truncate. In the past Ollama truncated the excessive length silently without error, now it seems it no longer does that, which is better.

What embedding model are you using and how long exactly is your context length? Copilot's chunk is 4000 chars, i.e. roughly 1000 tokens, should be short enough for most embedding models.

[chenyuz3]

@chenyuz3 copilot does nothing when it exceeds the context length, the error you see should fail the chunk entirely so it won't appear in the index.

It is up to the backend to decide whether to truncate. In the past Ollama truncated the excessive length silently without error, now it seems it no longer does that, which is better.

What embedding model are you using and how long exactly is your context length? Copilot's chunk is 4000 chars, i.e. roughly 1000 tokens, should be short enough for most embedding models.

@logancyang I use jina-embeddings-v2-zh as the embedding model with 8192 context length. I used to use bge-m3 models which also had a 8192 context length though at that time there were no embedding error found.

[logancyang]

@chenyuz3 I see. Could be language related since it's Chinese? I will test with jina-embeddings-v2-zh and see what's up.

[chenyuz3]

@chenyuz3 I see. Could be language related since it's Chinese? I will test with jina-embeddings-v2-zh and see what's up.

@logancyang The bge-m3 model I used to use is also multilingual (including Chinese). I should say there were one or two errors
(code 400) when using it, though the copilot plugin did not say it was a context length issue.