Summary Index: data does not be cleared when upload an exists document

[Yanchongyun]

Self Checks

• I have read the Contributing Guide and Language Policy.
• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
• Please do not modify this template :) and fill in all the required fields.

Dify version

1.14.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Step:

1. Upload a file and turn on [Enable Summary]. Chunk summary is successful created.
2. Upload same file to same dataset. (From now, Summary setting is disappered)
3. Summary will not be regenrated at all. But summary data is still in database. (Vector data seems like same)
4. Regerate summary from button at the top of the page. New summary generated but old data still in the database.

Postion:
When upload same file, knowledge_config.duplicate is True and only go into DuplicateDocumentIndexingTask.

dify/api/services/dataset_service.py

Lines 2307 to 2312 in eae44cf

	if document_ids:
	DocumentIndexingTaskProxy(dataset.tenant_id, dataset.id, document_ids).delay()
	if duplicate_document_ids:
	DuplicateDocumentIndexingTaskProxy(
	dataset.tenant_id, dataset.id, duplicate_document_ids
	).delay()

✔️ Expected Behavior

• Generate new summary when upload same file.
• Clear old summary pg data.
• Clear old summary vector data.

❌ Actual Behavior

No response

[dosubot]

This is a confirmed bug. The DuplicateDocumentIndexingTask properly cleans up old document segments (from both PostgreSQL and the vector store) before re-indexing, but it completely skips summary data handling [1].

Specifically, the duplicate indexing flow:

• ✅ Deletes old DocumentSegment records and their vectors [2]
• ✅ Re-runs IndexingRunner to reprocess the document [3]
• ❌ Does not clean up old summary data (PostgreSQL or vector)
• ❌ Does not call generate_summary_index_task after re-indexing completes

By contrast, the normal document_indexing_task does trigger summary generation after successful indexing [4].

The fix would need two additions to the duplicate document indexing task:

1. Before re-indexing: Call SummaryIndexService.delete_summaries_for_segments() [5] to clean up old summary records and their vectors for the segments being deleted.

2. After re-indexing succeeds: Add the same conditional logic from document_indexing_task to queue generate_summary_index_task.delay() — checking that the dataset uses high-quality indexing, has summary_index_setting enabled, and the document has need_summary=True.

It's worth noting that a very similar issue was reported for the document_indexing_update_task (updating documents via service API) in #35950 — both stem from the same root cause: the summary index feature was added to the initial indexing flow but the duplicate/update flows were never updated accordingly.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[shifang0511]

I've submitted a PR #36964 to fix this issue. Looking forward to review!