langchain-openai couldn't work while the openai package works fine at the same parameters

[mengmdpc]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

'''python
from openai import OpenAI

client = OpenAI(
    api_key="EMPTY",
    base_url="http://222.19.236.173:8003/v1"
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
    model="glm4",
)

print(chat_completion)
'''
The code above works fine
'''python
from langchain_openai import OpenAI
import os

llm = OpenAI(base_url="http://location-of-my-server:8003/v1", 
             api_key="EMPTY",
             model="glm4")

result = llm.invoke("nihao")
print(result)
'''
The second code block shows the code with problems.

Description

I use the same parameters to create a client/llm with openai package and langchain-openai package to use the glm model through the same openai api base_url.
I expected to got reply from the llm.
The openai client works fine, shows the parameters are correct, but the langchain-openai llm report a error
NotFoundError: Error code: 404 - {'detail': 'Not Found'}

System Info

Package Version

---

aiohappyeyeballs 2.4.0
aiohttp 3.10.5
aiosignal 1.3.1
annotated-types 0.7.0
anyio 3.7.1
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
asttokens 2.0.5
async-lru 2.0.4
async-timeout 4.0.3
attrs 24.2.0
Babel 2.11.0
beautifulsoup4 4.12.3
bleach 4.1.0
Brotli 1.1.0
certifi 2024.8.30
cffi 1.17.0
charset-normalizer 3.3.2
click 8.1.7
colorama 0.4.6
comm 0.2.1
dataclasses-json 0.6.7
debugpy 1.6.7
decorator 5.1.1
defusedxml 0.7.1
distro 1.9.0
exceptiongroup 1.2.2
executing 0.8.3
fastapi 0.112.2
fastjsonschema 2.16.2
frozenlist 1.4.1
greenlet 3.0.3
h11 0.14.0
h2 4.1.0
hpack 4.0.0
httpcore 1.0.5
httpx 0.27.2
hyperframe 6.0.1
idna 3.8
importlib_resources 6.4.0
ipykernel 6.28.0
ipython 8.27.0
jedi 0.19.1
Jinja2 3.1.4
jiter 0.5.0
json5 0.9.6
jsonpatch 1.33
jsonpointer 3.0.0
jsonschema 4.23.0
jsonschema-specifications 2023.12.1
jupyter_client 8.6.0
jupyter_core 5.7.2
jupyter-events 0.10.0
jupyter-lsp 2.2.0
jupyter_server 2.14.1
jupyter_server_terminals 0.4.4
jupyterlab 4.2.5
jupyterlab-pygments 0.1.2
jupyterlab_server 2.27.3
langchain 0.3.2
langchain-community 0.2.15
langchain-core 0.3.9
langchain-openai 0.2.2
langchain-text-splitters 0.3.0
langgraph 0.2.16
langgraph-checkpoint 1.0.8
langserve 0.2.3
langsmith 0.1.131
MarkupSafe 2.1.3
marshmallow 3.22.0
matplotlib-inline 0.1.6
mistune 2.0.4
multidict 6.0.5
mypy-extensions 1.0.0
nbclient 0.8.0
nbconvert 7.16.4
nbformat 5.10.4
nest-asyncio 1.6.0
notebook 7.2.2
notebook_shim 0.2.3
numpy 1.26.4
openai 1.51.0
orjson 3.10.7
overrides 7.4.0
packaging 24.1
pandocfilters 1.5.0
parso 0.8.3
pip 24.2
pkgutil_resolve_name 1.3.10
platformdirs 3.10.0
prometheus-client 0.14.1
prompt-toolkit 3.0.43
psutil 5.9.0
pure-eval 0.2.2
pycparser 2.22
pydantic 2.8.2
pydantic_core 2.20.1
Pygments 2.15.1
pyproject-toml 0.0.10
PySocks 1.7.1
python-dateutil 2.9.0.post0
python-json-logger 2.0.7
pytz 2024.1
pywin32 305.1
pywinpty 2.0.10
PyYAML 6.0.2
pyzmq 25.1.2
referencing 0.30.2
regex 2024.7.24
requests 2.32.3
requests-toolbelt 1.0.0
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rpds-py 0.10.6
Send2Trash 1.8.2
setuptools 72.1.0
six 1.16.0
sniffio 1.3.1
soupsieve 2.5
SQLAlchemy 2.0.32
sse-starlette 1.8.2
stack-data 0.2.0
starlette 0.38.4
tenacity 8.5.0
terminado 0.17.1
tiktoken 0.7.0
tinycss2 1.2.1
toml 0.10.2
tornado 6.4.1
tqdm 4.66.5
traitlets 5.14.3
typing_extensions 4.12.2
typing-inspect 0.9.0
urllib3 2.2.2
uvicorn 0.30.6
wcwidth 0.2.5
webencodings 0.5.1
websocket-client 1.8.0
wheel 0.43.0
win-inet-pton 1.1.0
yarl 1.9.7
zstandard 0.22.0

---

Windows 11 23H2 (I have the same problem on my macbook)
python = 3.10

[dosubot[bot]]

Hello @mengmdpc! I'm here to help you with any bugs, questions, or contributions you may have.

I found some similar issues that might help you resolve the NotFoundError: Error code: 404 - {'detail': 'Not Found'} you're encountering with the langchain_openai package:

1. ChatOpenAI and OpenAI give different behaviors when using local openai_api_base: This issue suggests that the OpenAI class might be trying to access a URL that doesn't exist on your local server. The ChatOpenAI class might be working fine because it interacts with different endpoints. You might need to check the URL construction in the OpenAI class to ensure it matches the available endpoints on your server ^[1].

2. NotFoundError for OpenAI with gpt-3.5-turbo model: This issue highlights that chat models like gpt-3.5 and gpt-4 require using ChatOpenAI from langchain_openai. Ensure that the model you're using is compatible with the class you're using to interact with it ^[2].

These issues suggest checking the endpoint URLs and ensuring compatibility between the model and the class used. Adjusting these might resolve the error you're facing.

To continue talking to Dosu, mention @dosu.

[mkhludnev]

try to set_debug(True) and check logs.

[mengmdpc]

[llm/error] [llm:OpenAI] [151ms] LLM run errored with error:
"NotFoundError(\"Error code: 404 - {'detail': 'Not Found'}\")Traceback (most recent call last):\n\n\n  File \"C:\\Users\\mengm\\anaconda3\\envs\\langchain\\Lib\\site-packages\\langchain_core\\language_models\\llms.py\", line 774, in _generate_helper\n    try:\n         \n\n\n  File \"C:\\Users\\mengm\\anaconda3\\envs\\langchain\\Lib\\site-packages\\langchain_openai\\llms\\base.py\", line 360, in _generate\n    )\n      \n\n\n  File \"C:\\Users\\mengm\\anaconda3\\envs\\langchain\\Lib\\site-packages\\openai\\_utils\\_utils.py\", line 274, in wrapper\n    return func(*args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^\n\n\n  File \"C:\\Users\\mengm\\anaconda3\\envs\\langchain\\Lib\\site-packages\\openai\\resources\\completions.py\", line 528, in create\n    suffix: Optional[str] | NotGiven = NOT_GIVEN,\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n  File \"C:\\Users\\mengm\\anaconda3\\envs\\langchain\\Lib\\site-packages\\openai\\_base_client.py\", line 1260, in post\n    cast_to: Type[ResponseT],\n                           ^^^\n\n\n  File \"C:\\Users\\mengm\\anaconda3\\envs\\langchain\\Lib\\site-packages\\openai\\_base_client.py\", line 937, in request\n    remaining_retries: Optional[int] = None,\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n  File \"C:\\Users\\mengm\\anaconda3\\envs\\langchain\\Lib\\site-packages\\openai\\_base_client.py\", line 1041, in _request\n    stream=stream,\n\n\nopenai.NotFoundError: Error code: 404 - {'detail': 'Not Found'}"

@dosu @mkhludnev here is the log with set_bug(True) enabled. It seems the langchain-openai is still using the openai package. I have no idea why the langchian-openai doesn't work. Thank U for your reply.

[dosubot[bot]]

The NotFoundError you're encountering suggests that the langchain-openai package is trying to access an endpoint that doesn't exist on your server. This could be due to a mismatch between the model and the class used to interact with it. The langchain-openai package uses a helper function to select the appropriate model class based on the provider, and it seems like it might not be correctly configured for the glm4 model.

Here are a few things you can check or try:

1. Model Compatibility: Ensure that the glm4 model is supported by the langchain-openai package. If it's not explicitly supported, you might need to use a different model or check if there's a specific class or configuration required for glm4.

2. Endpoint Configuration: Double-check the base URL and ensure that the endpoint you're trying to access is correctly set up on your server. The 404 error indicates that the specific path or resource is not found.

3. Provider Specification: Make sure that the model provider is correctly specified or inferred. The langchain-openai package uses a function to infer the model provider, and if it's not correctly set, it might default to an incorrect provider.

If these steps don't resolve the issue, you might need to look into the specific implementation details of the langchain-openai package to ensure compatibility with your setup.

To continue talking to Dosu, mention @dosu.