[Bug] Use online big model to generate the problem, status details show that the number of problems is 100%, but it is stuck in queue

[fit2cloud-lijiaxin]

Contact Information

No response

MaxKB Version

v1.10.2-lts

Problem Description

使用在线大模型生成问题，状态详情显示问题数目100%,但是卡在排队中

Steps to Reproduce

1.勾选70个文档，全部使用阿里百炼deep-seek生成问题 ，剩最后10个文档 一直在排队中,问题数目100%
2.点击取消生成问题，又卡在取消中。看gunicorn 日志也没报错，
3. 重启maxkb,起来以后 状态就是失败
4.再次生成问题，仅执行未成功的分段，这次都正常了，问题数目没变
如图每个文档分段基本只有一两个，每个分段内容上千字符

The expected correct result

问题数目100%时，状态希望是生成中，或者成功

Related log output

Additional Information

No response

[fit2cloud-lijiaxin]

gunicorn.log

[fit2cloud-lijiaxin]

[Shenguobin0102]

你好，感谢反馈。我们在测试环境使用了84个文档进行了向量化和问题生成，操作步骤按照你描述中的实施，没有发现描述中的问题。针对你遇到的问题做以下解释：
问题一：剩最后10个文档 一直在排队中
解释：MaxKB 在向量化和问题生成时，最多只有10个线程同时执行，任务一直在排队中，可能是团队其他人也在做向量化和问题生成，他们的任务占住了线程，所以会有这个问题。我们测试过程中，只有我自己的任务在跑，所以没有出现一直在排队中的情况。出现排队时间长的任务时，耐心等待即可。
问题二：取消后，状态一直在取消中
解释：取消任务时，只有执行到该任务时，取消才会生效，根据问题一中的状态，这些排队中的任务还没有轮到它们执行，所以对这些任务发取消指令后，会等到这些任务执行时才会取消，所以长时间处于取消中也是正常的。
问题三：重启后，状态为失败
解释：重启后状态为失败正常，因为异常打断问题生成任务，导致至少一个分段的问题生成是失败的，就会更新任务状态为失败。
问题四：再次生成问题，仅执行未成功的分段，这次都正常了，问题数目没变
解释：我们测试过程中，对失败的、部分成功未完成全部分段问题生成的文档重新生成问题（仅执行未成功的分段），最终结果全部成功。这里有个现象需要解释一下，就是取消问题生成后，如果没有生成问题失败的分段，文档的问题生成状态会为成功，即使有部分分段没有完成问题生成。

此外，确实存在一个问题，对分段生成问题失败的文档做取消生成操作，取消操作不生效，已经提交研发修复了。
附上测试信息：
共86个文档，最大分段数：80，最小分段数：9，文件数据见截图。
向量化耗时1.5小时，问题生成耗时：55分钟，结果：全部成功

[shaohuzhang1]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Hello, thanks for the feedback. We used 84 documents to vectorize and generate problems in the test environment. The operation steps were implemented as described, and no problems were found in the description. The following explanation is given to the problems you are facing:
Problem 1: The last 10 documents left are always in line
Explanation: When MaxKB is vectorized and problem generation, there are only 10 threads executed at the same time. The tasks are always queued. Maybe other people in the team are also doing vectorized and problem generation. Their tasks occupy threads, so there will be this problem. During our testing, only my own tasks were running, so there was no situation where I was always in line. When there is a long queue, just wait patiently.
Question 2: After cancellation, the status is being cancelled
Explanation: When canceling a task, cancellation will only take effect when the task is executed. According to the status in Question 1, these queued tasks are not yet their turn to execute. Therefore, after issuing a cancellation command to these tasks, it will wait until these tasks are executed. Therefore, it is normal to be cancelled for a long time.
Problem 3: After restarting, the status is failed
Explanation: The state fails after restarting, because the exception interrupts the problem generation task, resulting in the problem generation of at least one segment failing, and the task status will be updated to fail.
Problem 4: Generate the problem again, only the unsuccessful segmentation was executed. This time it was normal, the number of problems did not change.
Explanation: During our testing, we regenerate the documents generated by the failed and partially successful and unfinished all segment problems (only unsuccessful segments were executed), and the final results were all successful. There is a phenomenon that needs to be explained here. After canceling the problem generation, if there is no segment that fails to generate the problem, the problem generation status of the document will be successful, even if some segments do not complete the problem generation.

In addition, there is indeed a problem. The cancellation operation is performed on the document that failed to generate segments. The cancellation operation does not take effect and has been submitted for R&D and repair.
Attached test information:
There are 86 documents in total, the maximum number of segments: 80, the minimum number of segments: 9, and the file data is shown in the screenshot.
Vectorization takes 1.5 hours, problem generation takes 55 minutes, result: all successful