Page image not being rendered during rewrite operations

[djoltes]

I'm continuing development of a redaction solution, and recently noticed that in some cases pages are rendered with the redactions in place, but without the actual text backing the page. There seems to be no pattern: in some cases even a 1-page document is affected, while in others I'll see a mix of correctly rendered pages along with others where all that's visible are the redaction blocks.

No idea why this started happening recently...any ideas appreciated. I've been doing page-by-page intermediate saves of documents, which has never caused a problem. I may look at that as a possible cause.

[JorjMcKie]

Hard to tell without seeing the code.
Storing stuff to an existing page may go wrong when there are non-localized geometry changes as explained in "Misplaced Item Insertions on PDF Pages".
Try page.clean_contents() immediately after loading a page.

[djoltes]

Sure, here's a one-pager with some fake PII in it -- original and redacted versions. The original file was created with Word and exported to PDF. The weird thing is that there seems to be no rhyme or reason for which pages exhibit this.

Oh, and at this point the code does not strip the initial text from the full page rect.
Redaction_test.pdf
Redaction_test_redacted.pdf

[JorjMcKie]

Here is the explanation:
You must clean the page apearance commands before inserting the redactions!
The PDF creator (Word) might have left behind a mess of "geometry changes" of all sorts, causing your redact rectangle(s) to be potentially anything - except what you are expecting. The following does the job and leaves behind an empty page (except for two underlines of the links - which is inevitable:

doc=fitz.open("Redaction_test.pdf")
page=doc[0]
page.is_wrapped
False
page.clean_contents()
page.is_wrapped
True
page.add_redact_annot(page.rect)
'Redact' annotation on page 0 of Redaction_test.pdf
page.apply_redactions()
True
doc.ez_save("original-redacted.pdf")

[djoltes]

Hmm, that's odd because I've tried using clean_contents() (I've also tried wrap_contents()) and I still get the same output. This block is the initial page handling code I'm now using, and it gives the same results.

    for page in doc:
        # wrap_contents is needed for fixing alignment issues with rect boxes 
        page.clean_contents()
        print("Page #: {a}".format(a = pageNo))
        logging.info("Page #: {a}".format(a = pageNo))
        fileTotalPages += 1 # increment page count
        myText = str(page.get_text().encode("utf8"))
        myText = re.sub("\n", " ", myText)

[djoltes]

Ahhh, I may have found the problem. One of the tasks involves using Textract to handle signature annotations, and it turns out something in that code was triggering an 'illicit' redaction of all the other text. I think I have it locked, but I'll run more tests.

Thanks as always for the back-and-forth. If you're ever near Boston, remind me I owe you a few pints!

[JorjMcKie]

😂 accepted!