BUG: properly clean up orphan objects when removing pages

[luis-cd]

This fixes an issue where removing pages after writing left orphan objects in the PDF writer.

The fix forces a rebuild of the internal structure to clean orphaned objects, improving stability and avoiding potential PDF corruption.

Closes #3418

[codecov]

Codecov Report

❌ Patch coverage is 90.90909% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.96%. Comparing base (71a1bd1) to head (25a9684).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
pypdf/_writer.py	90.90%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3432      +/-   ##
==========================================
- Coverage   96.97%   96.96%   -0.02%     
==========================================
  Files          54       54              
  Lines        9337     9358      +21     
  Branches     1711     1712       +1     
==========================================
+ Hits         9055     9074      +19     
- Misses        168      170       +2     
  Partials      114      114              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[stefan6419846]

Thanks for the PR. Besides the exception handling being untested, I fear that this will have a quite large hit on performance and break with edge cases like incremental mode or similar. The docs are rather clear that clean=True will just clear references.

I am not sure if we are able to solve this issue without too much overhead or should prefer to document that to remove orphans, the corresponding method should be called at the end. If we still miss some unused objects with this, I prefer to focus on fixing the behavior there instead.

[luis-cd]

Hi — thanks again for the review and for pointing out the performance / semantic concerns.

Short summary / proposal

Rather than forcing a heavy serialize→read→clone rebuild inside remove_page(clean=True) (which changes semantics and can be expensive), I propose:

• Add a lightweight PdfWriter.remove_orphan_objects() that performs an in-memory mark-and-sweep (graph reachability) to remove unreachable indirect objects from the writer’s internal pool.
• Keep a heavy PdfWriter.rebuild() as an explicit fallback that does serialize→read→clone (bulletproof but expensive).
• Optionally expose writer.write(..., rebuild=False) as a convenience flag (default False), but do not change remove_page(clean=True) semantics.

Why this approach

• Preserves current documented behavior of clean=True.
• Mark-and-sweep is O(number_of_objects + references) and avoids reserializing large streams — much faster than full rebuild for large PDFs.
• Users opt in to the expensive operation; no surprising perf hit by default.

Edge cases / safety

• Incremental mode: do not run sweep or rebuild automatically in incremental=True. Suggested behavior: remove_orphan_objects() no-ops and logs a warning (or we can raise if maintainers prefer).
• Encryption / object streams / names trees: include these roots in traversal and add tests for encrypted files.
• Fallback: rebuild() remains when sweep is insufficient.

Request for guidance

1. Are you comfortable with adding remove_orphan_objects() + rebuild() as explicit APIs, instead of changing remove_page(clean=True)?
2. Preferred incremental-mode behavior: no-op + logger_warning (recommended) or raise an error?
3. Any additional roots or special catalog entries I should include in traversal?

Thanks again for the feedback — happy to adjust based on your guidance.

[stefan6419846]

1. We should not need remove_orphan_objects, we already have compress_identical_objects(remove_orphans=True), possibly with adding remove_identicals=False. I do not see the need for adding new parameters or the rebuild() method.
2. The user should know when he uses incremental mode. I do not see an explicit need for adding a new warning here.
3. According to PdfWriter remove_page/compress_identical_objects does not clean up properly #3418 (comment), fonts and images are not deleted correctly at the moment. They should be included. I do not see any need for handling encrypted documents in a special manner.

[luis-cd]

Given the feedback from the maintainers, I understand that the approach in this PR (modifying remove_page and adding _rebuild_via_io) is not the preferred solution due to potential performance impact and maintenance concerns.

I will close this PR and instead prepare a new one focusing on improving compress_identical_objects(remove_orphans=True) so that it properly removes fonts, images, and other unreferenced objects, as suggested in the discussion.

Thanks for the detailed review and guidance!