Error: RangeError: Maximum call stack size exceeded, when upserting a large csv file

[UgoWadoux]

Hi everyone !

I am building chatbot to retrieve information from a CSV file. I have no problem doing it with a “small” file (about 5000 lines and 1.9 Mo). But when I am doing that with a larger file 86 Mo, it doesn't work.

I have tried different things :
First I was getting a blank screen. I saw that files couldn't be more than 50 mb, so I increase that.
I tried modifying it in the code and also running it in a docker and setting the environment variable FLOWISE_FILE_SIZE_LIMIT=150mb.
The two options worked when saving my chat flow, but now I have another error :

Created a chunk of size 5974, +
2024-04-10 15:11:39 flowise-1 | which is longer than the specified 1000
2024-04-10 15:11:40 flowise-1 | Created a chunk of size 1027, +
2024-04-10 15:11:40 flowise-1 | which is longer than the specified 1000
2024-04-10 15:11:40 flowise-1 | Created a chunk of size 1479, +
2024-04-10 15:11:40 flowise-1 | which is longer than the specified 1000
2024-04-10 15:11:40 flowise-1 | Created a chunk of size 1002, +
2024-04-10 15:11:40 flowise-1 | which is longer than the specified 1000
2024-04-10 15:11:40 flowise-1 | 2024-04-10 13:11:40 [ERROR]: Maximum call stack size exceeded
2024-04-10 15:11:40 flowise-1 | RangeError: Maximum call stack size exceeded
2024-04-10 15:11:40 flowise-1 | at Csv_DocumentLoaders.init (/usr/local/lib/node_modules/flowise/node_modules/flowise-components/dist/nodes/documentloaders/Csv/Csv.js:65:25)
2024-04-10 15:11:40 flowise-1 | at async buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:361:36)
2024-04-10 15:11:40 flowise-1 | at async upsertVector (/usr/local/lib/node_modules/flowise/dist/utils/upsertVector.js:117:32)
2024-04-10 15:11:40 flowise-1 | at async Object.upsertVectorMiddleware (/usr/local/lib/node_modules/flowise/dist/services/vectors/index.js:6:9)
2024-04-10 15:11:40 flowise-1 | at async createInternalUpsert (/usr/local/lib/node_modules/flowise/dist/controllers/vectors/index.js:26:16)
2024-04-10 15:11:40 flowise-1 | 2024-04-10 13:11:40 [ERROR]: [server]: Error: RangeError: Maximum call stack size exceeded
2024-04-10 15:11:40 flowise-1 | Error: RangeError: Maximum call stack size exceeded
2024-04-10 15:11:40 flowise-1 | at buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:405:19)
2024-04-10 15:11:40 flowise-1 | at async upsertVector (/usr/local/lib/node_modules/flowise/dist/utils/upsertVector.js:117:32)
2024-04-10 15:11:40 flowise-1 | at async Object.upsertVectorMiddleware (/usr/local/lib/node_modules/flowise/dist/services/vectors/index.js:6:9)
2024-04-10 15:11:40 flowise-1 | at async createInternalUpsert (/usr/local/lib/node_modules/flowise/dist/controllers/vectors/index.js:26:16)

I don't know if this is a bug, or maybe I have not configured something to support larger files.

My chat flow:

Thanks in advance for any help or suggestions!