Links to sources when using Web Search + GPT-5

[michnovka]

What happened?

I use gpt-5 with my agent, and I have noticed often when it mentions information from Web Search (the one from Librechat, not OpenAI's) it gives an artifact like ︎turn0search2 instead of the link.

I was able to reproduce this always with GPT-5 agent, but never with GPT-4.1 or other models. Those link the sources properly.

Also, this only happened with GPT-5 + Responses API. But GPT4.1+responses or GPT5+conversations or any other combo you can think of - I could not reproduce.

Ive found info about this on wiki but I am not sure if this is GPT-5 LLM bug, or its about how LC interprets the response from the LLM.

Version Information

ghcr.io/danny-avila/librechat-dev latest 2041804cc3a4 4 hours ago 1.56GB

Steps to Reproduce

1. create and Agent using OpenAI's GPT-5
2. enable Web Search tool with the agent (the one using Librechat - firecrawl + searxng + jina, not OpenAI's)
3. ask agent something which requires web search
4. notice no hyperlinks to sources, only turnXsearchY:

---

if you test with other models you will get it linking properly, see gpt-4.1:

What browsers are you seeing the problem on?

No response

Relevant log output

N/A

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

It's up to the LLM to use this format and not the codebase.

[michnovka]

I see, upon inspection of the reply from from the "broken" reponses they are like

turn0search2

while when it works there is some unicode prefix:

\ue202turn0search2

So the LLM is messing up the generated text? there is no way LC strips this somehow during some pre or postprocessing of the stream?

[danny-avila]

Yes, LC does not strip this. Copy+paste the response where they are not generated vs. when they are

[michnovka]

Its like this when not generated:

Big-city picture (how to interpret)
- London has the largest concentration of non-English main languages, with the lowest share of English-as-main-language among regions (78.4%). This implies London contains the largest share of Spanish main-language speakers in absolute terms. ︎turn0search1
- Interactive local-authority maps show Spanish among the mapped top-10 non-English languages; you can drill into each city’s local authorities for counts/percentages (e.g., London boroughs; Birmingham; Manchester; Leeds; Liverpool; Sheffield; Bristol; Leicester; Nottingham; Newcastle; Cardiff). ︎turn0search3

and like this when generated:

### Distribution Across Big Cities

- **London** has by far the largest Spanish-speaking community, especially in the South and West regions, such as Kensington and Chelsea\ue202turn0search0.
- Other major English cities such as **Manchester and Birmingham** also host vibrant Spanish-speaking communities, primarily due to Spanish expatriates, students, and immigrants from Latin America and Spain\ue202turn0search1.
- Significant populations are also found in **Scotland (over 12,000 Spanish-born residents) and Wales (over 3,000)** as of the 2021/22 census\ue202turn0search0.

I've really never had this issue other than with agent using gpt5

[michnovka]

Today I got this:

here is the text:

Short answer: The self-hosted Firecrawl backend does not include the hosted “Fire-engine” that powers advanced web search; basic search endpoints exist, but “web search” in self-hosting is limited and may require bringing your own search/SERP provider and proxies. 󰡂turn0search0󰡂turn0search5󰡂turn0search1

## What’s available self-hosted
- Core APIs (scrape, crawl, extract) are supported in self-hosting. 󰡂turn0search2󰡂turn0search4
- A Search API is documented (search the web and optionally scrape results), but the fully managed search experience relies on Firecrawl’s hosted infrastructure. Self-hosting lacks “Fire-engine” (IP rotation, anti-bot/robot detection handling, etc.). 󰡂turn0search1󰡂turn0search0󰡂turn0search5

## Implications for you
- To replicate “search” on-prem, you typically need:
  - A SERP/search backend (e.g., your own index, a third‑party SERP API) feeding URLs to crawl. 󰡂turn0search1
  - Your own proxies/headless browser setup to handle bot detection at scale, since Fire-engine isn’t included in self-host. 󰡂turn0search0󰡂turn0search5
- Some tooling (MCP server) exposes a flag to enable web search, but when pointed at a self-hosted instance you’re still responsible for the underlying search provider and infra. 󰡂turn0search7

so is this again LLM bug? or did the special chars get somewhere parsed by LC?

[danny-avila]

It's an issue with the LLM again. Some do better with the default web search instructions, some on or off. I might provide more info about the underlying prompt so it can be guided better via custom instructions.