Any fix or workaround for "does not support matryoshka representation, changing output dimensions will lead to poor results"?

[jeremychoi]

Could I get any hints about how to fix the issue like "does not support matryoshka representation, changing output dimensions will lead to poor results" error?

I'm trying to use the sentence-transformers/all-MiniLM-L6-v2(served through vllm remotely) embedding model, but when I make a request to the vector-io API like the following, I get the 400 error. Here's an example.

request:

curl -X 'POST' \
  'http://localhost:8321/v1/vector-io/query' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "vector_db_id": "vector-db-id",
  "query": "hello",
  "params": {
    "additionalProp1": {}
  }
}'

response with 400 error:
Changing "sentence-transformers/all-MiniLM-L6-v2" does not support matryoshka representation, changing output dimensions will lead to poor results._

Below is my config file and code to register a vector db and 384(proper embedding dimension) for the all-MiniLM-L6-v2 :

LlamaStack config file:

providers:
  inference:
  - provider_id: vllm-inference
    provider_type: remote::vllm
    config:
      url: ${env.VLLM_URL}
      max_tokens: ${env.VLLM_MAX_TOKENS:=4096}
      api_token: ${env.VLLM_API_TOKEN:=fake}
      tls_verify: ${env.VLLM_TLS_VERIFY:=false}
      
  - provider_id: vllm-embedding
    provider_type: remote::vllm
    config:
      url: ${env.VLLM_EMBEDDING_URL}
      max_tokens: ${env.VLLM_MAX_TOKENS:=4096}
      api_token: ${env.VLLM_EMBEDDING_API_TOKEN:=fake}
      tls_verify: ${env.VLLM_TLS_VERIFY:=false}
...
models:
- metadata: {}
  model_id: ${env.INFERENCE_MODEL}
  provider_id: vllm-inference
  model_type: llm
- metadata: 
    embedding_dimension: ${env.EMBEDDING_DIMENSION:=384}
  model_id: ${env.EMBEDDING_MODEL:=sentence-transformers/all-MiniLM-L6-v2}
  provider_id: vllm-embedding
  model_type: embedding

code to register

            vector_db = self.client.vector_dbs.register(
                vector_db_id=self.vector_db_id,
                embedding_model=embedding_model,
                embedding_dimension=int(os.getenv("EMBEDDING_DIMENSION", "384")),
                provider_id=vector_provider.provider_id,
            )

Is there a way to fix this, or workaround or is my configuration incorrect? I didn't change any output dimensions. Any tips will be much appreciated.

Side notes:

• when I tested with the inline::sentence-transformers instead of the remote one, it worked without an issue.
• when I tested the same remote one with LlamaIndex for retrieval, it worked without an issue.