Crashing in Firefox when dealing with large conversations and long generations

[ThomasBaruzier]

What happened?

Firefox tab is crashing around 50% of the time when dealing with long generations over long context queries

Steps to Reproduce

Use LibreChat
Paste 30-50k token codebase, ask questions, wait. I measured 45k tokens in my current conversation, generations included.
Works better with DeepSeek API, since it's slow and the crash is random, the likelihood of crashes is higher. Odds skyrockets when generating in the background and actively using another tab. Finally, there is a 20-30% crash when the generation ends.

What browsers are you seeing the problem on?

Firefox

Relevant log output

No logs, Firefox does not seem to log tab crashes

Nothing meaningful in /app/api/logs/error-2024-10-15.log either:

{"level":"error","message":"[handleAbortError] AI response error; aborting request: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-002:streamGenerateContent?alt=sse: [429 Too Many Requests] Resource has been exhausted (e.g. check quota).","stack":"Error: [429 Too Many Requests] Resource has been exhausted (e.g. check quota).\n    at makeRequest (/app/api/node_modules/@google/generative-ai/dist/index.js:263:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async generateContentStream (/app/api/node_modules/@google/generative-ai/dist/index.js:649:22)\n    at async GoogleClient.getCompletion (/app/api/app/clients/GoogleClient.js:699:22)\n    at async GoogleClient.sendCompletion (/app/api/app/clients/GoogleClient.js:881:13)\n    at async GoogleClient.sendMessage (/app/api/app/clients/BaseClient.js:574:24)\n    at async AskController (/app/api/server/controllers/AskController.js:130:20)\n    at async /app/api/server/routes/ask/google.js:23:5"}
{"level":"error","message":"[handleAbortError] AI response error; aborting request: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-exp-0827:streamGenerateContent?alt=sse: [500 Internal Server Error] An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting","stack":"Error: [500 Internal Server Error] An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting\n    at makeRequest (/app/api/node_modules/@google/generative-ai/dist/index.js:263:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async generateContentStream (/app/api/node_modules/@google/generative-ai/dist/index.js:649:22)\n    at async GoogleClient.getCompletion (/app/api/app/clients/GoogleClient.js:699:22)\n    at async GoogleClient.sendCompletion (/app/api/app/clients/GoogleClient.js:881:13)\n    at async GoogleClient.sendMessage (/app/api/app/clients/BaseClient.js:574:24)\n    at async AskController (/app/api/server/controllers/AskController.js:130:20)\n    at async /app/api/server/routes/ask/google.js:23:5"}

I am sorry this bug report does not offer proper logging, I'd be glad to help with some guidance.

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

I’m not able to reproduce it on Firefox, and Chrome doesn’t seem to have this issue.

[danny-avila]

I’m not sure why it would be worse on Firefox vs. chrome, but I have no RAM issues on either and I’ve never experienced a crash when generating a lot of text.

[ThomasBaruzier]

Today the amount of crashes is low enough for me, but I was getting a really high amount of them (75-90% chance when using DeepSeek v2.5 to go up to 8k output). Today I do the same, but with a 2-5% chance of crashing.
The difference between now and before is that I don't work anymore with a program that filled up my ram up to 10 of my 16gbs on my Linux laptop.
So either the performance has been improved or the available ram is a key factor.

[RAPHCVR]

I have a pretty powerfull pc but still have this issue. It is indeed much less present on chromium. Also, as I said before, there is some styling that are present on chromium but not on firefox (the "thought" widget for example, does not appear on firefox it seems). And finally, the issue is also present with friends of mine using deepseek R1 too on firefox.

[thomas-333]

I'm seeing issues as well with Firefox. Hard to pin down exactly, but during long generations at times I can hear my CPU fans spin up and the other day while using Deepseek R1 all my 32GB of memory was fully utilised. I know this doesn't help debugging, but if I can isolate it I'll post again.

[danny-avila]

Adding to the conversation my notes.

After doing some profiling of render speeds, it seems react-markdown will just slow down on re-render with large texts: https://github.com/orgs/remarkjs/discussions/1027

This is not new to me but I under-estimated the amount of text where performance is felt.

On re-render, the outlier is the Markdown component in terms of performance (increases render time by amount of text)

Normally, AI do not go on streaming a lot of content in one Markdown block unless specifically prompted or has innate propensity to do so (like deepseek-reasoner).

This issue presents itself less when using Agents as they use a new message paradigm of content blocks.

We can do the same for legacy endpoints to break up into blocks, separated by length by the rough amount of text it takes to cover a regular-sized screen.

The only challenge that remains is making sure the new "reasoning" UI is still accounted for when broken up into blocks.