Add / enable new OpenAI embedding models to class OpenAIEmbeddings

[vanetreg]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Last week OpenAI released 2 new embedding models, one is cheaper, the other is better than ada-002, so pls.
consider to change default ada-002 to
text-embedding-3-small
and add as option:
text-embedding-3-large
Today I tried the first when instantiated the class but it was not accepted...

Motivation

Want to use better or cheaper model for embeddings.

Proposal (If applicable)

No response

[baskaryan]

new models should already work https://python.langchain.com/docs/integrations/text_embedding/openai. @vanetreg how were you instantiating?

[vanetreg]

hey @baskaryan
exactly the way it is in the docs linked, using small:
embeddings = OpenAIEmbeddings(model="text-embedding-3-large")
In file: langgraph_crag.ipynb, here:
vectorstore = Chroma.from_documents(
documents=doc_splits,
collection_name="rag-chroma",
embedding=OpenAIEmbeddings(model="text-embedding-3-small"),
)

Versions:
langchain: 0.1.5
langgraph: 0.0.23

Note, latest:
libs/partners/openai/langchain_openai/embeddings/base.py
contains hardcoded ada-002

[baskaryan]

and you've got langchain-openai installed and are importing from

from langchain_openai import OpenAIEmbeddings

not from langchain_community, right? if so, what's the error you're getting?

[vanetreg]

yes, I import that way:
from langchain_openai import OpenAIEmbeddings
I got warning:
Warning: model not found. Using cl100k_base encoding.

Additional version info:
langchain-openai: 0.0.5

[vanetreg]

hey @baskaryan
any update? :)

[baskaryan]

are you getting an error or just a warning? warning fixed in #16964 and will be released in langchain-openai 0.0.6, but that shouldn't be blocking anything

[gykung]

yes, I import that way: from langchain_openai import OpenAIEmbeddings I got warning: Warning: model not found. Using cl100k_base encoding.

Additional version info: langchain-openai: 0.0.5

Any idea why the documentation at langchain includes the warning "Warning: model not found. Using cl100k_base encoding."?
https://python.langchain.com/docs/integrations/text_embedding/openai

I get this warning too when I run the following:
vectorstore = Chroma.from_documents(texts,OpenAIEmbeddings(model="text-embedding-3-small"),persist_directory="db/")

[dbonates]

fixed on langchain Version: 0.1.8

maybe worth update langchain-openai to 0.0.6 too, if needed.

[shawnesquivel]

bump the upgrade to langchain-openai from 0.0.4 to 0.0.6 fixed it for me