Create new haystack elasticsearch instance in docker / update embedding dims in existing elastic search instance

[heyt0pe]

Hello, I just got started with Haystack not too long ago and I've been to run a simple question and answer python script and also been able to set up the UI and rest API using docker-compose. For elastic search in docker, I already set the image to "elasticsearch:7.9.1" and not the instance with countries and capitals, so that it will start an empty elasticsearch instance and I can add my documents myself.

But now, whenever I want to set up anything using Elastic search as my document store, I get a series of errors. If I leave the default value for embedding_dims as 768, I get an error saying "Embedding dim. of model (128) doesn't match embedding dim. in DocumentStore (768).Specify the arg embedding_dim when initializing ElasticsearchDocumentStore()" and whenever I specify the embedding_dim to 128, I get another error saying there is a conflict with the existing mapping "'Mapper for [embedding] conflicts with existing mapping: [mapper [embedding] has different dims]'". I want to believe I get this error because the first time I ran Elastic search, it's embedding dims where set to 768 which is also visible when I visit http://localhost:9200/document/_mapping . Now I want to know if there's any way to get around this error, the only way so far is using an entirely different document store like FAISS, but is there a way I can fix this issue while still using Elastic Search. Thanks for your time

[julian-risch]

Hi @heyt0pe I just responded to your question in the over discussion you started here: #3329 🙂