images= PDF_REDACT_IMAGE_PIXELS arg of apply_redactions()

[tigrankh]

Hi,

I'm sorry in advance, since my question is a little long

The question is about the value PDF_REDACT_IMAGE_PIXELS of images param.
when i use apply_redactions() with default value i get a result like this:

I read in the documentation about a bug related to transparency, when images=PDF_REDACT_IMAGE_PIXELS, so i went to debug the images inside.
Using page.get_images() returned a list of 4 items:

[
    (59, 0, 1, 1, 8, 'DeviceGray', '', 'FXX1', ''), 
    (60, 0, 1200, 1518, 8, 'DeviceRGB', '', 'FXX2', 'JPXDecode'), 
    (61, 264, 1, 1, 8, 'DeviceGray', '', 'FXX3', ''), 
    (62, 264, 800, 1012, 8, 'DeviceRGB', '', 'FXX4', 'JPXDecode')
]

So basically i have 2 images that have heights: # 60 and # 62
as can be seen # 62 also indicates presence of a stencil mask which is object # 264

For the sake of experiment I started extracting and saving objects # 60 and # 62 individually.
This is the code i used:

data = page.get_text("dict")
new_doc.insert_page(0)
img1 = doc.extract_image(60)["image"]
img2 = doc.extract_image(62)["image"]
new_doc[0].insert_image(Rect(*data["blocks"][1]["bbox"]), stream=img2)
#new_doc[0].insert_image(Rect(*data["blocks"][0]["bbox"]), stream=img1)

block[0] is the object # 60

writing # 60 resulted in a slightly less colorful version of original doc:

This is example of original:

I apologize for not being able to share the whole thing.

Then i saved block[1] which is object # 62 and got the same bad/blurry version that i shared at the top.

looks like even without apply_redactions() i got similarly bad result.

but hey, # 62 had some smask object.
So i went ahead to extract smask object # 264 and tried to create a combined pixmap with this:

pix = fitz.Pixmap(doc.extract_images(62)["image"])
mask = fitz.Pixmap(doc.extract_images(264)["image"])

combo = fitz.Pixmap(pix, mask)

just like its shown here: https://pymupdf.readthedocs.io/en/latest/faq.html

and I got an error, that color and mask should have the same size.

I checked, indeed the mask size is much bigger than the pix.

I think i'm doing it right, but since i'm a relatively new user to pymupdf, maybe not.

Could you please help me understand if i'm doing something wrong ?
How can the size of the smask image be bigger than the image its applied to ?
Also, could you please explain the nature of the bug related to this value: PDF_REDACT_IMAGE_PIXELS

As an experiment I used images=0 param to resolve the output pdf issue after redaction, but since images can still be fully recovered, that is not a good option for me.

Thank you!

[JorjMcKie]

This cannot remain as an issue without reproducible data.
So I am converting to a discussion.

[JorjMcKie]

There are 3 options for applying redactions: whiten pixels overlapping a redaction rect, leave the image as is, or remove the image completely.
For an image with transparency (presence of a mask), only the last two options work correctly.

But:
You can extract the image as a pixmap with transparency (see here for how this works).
Then clear the relevant overlapping pixmap area yourself using pix.set_rect(). Compute the correct subrectangle using:

1. the bbox of the image on the page, intersected with the redact rect, let's call this "subrect".
2. compute the matrix which converts the bbox to the pixmap irect as mat = bbox.torect(irect).
3. the pixmap area to replace by white then is subrect * mat.

Then replace the original image with the modified pixmap - see here for a recipe.

[tigrankh]

thanks for getting back.
as i mentioned in my original post, when i try to create a new pixmap from the image's pixmap and mask's pixmap, i'm getting an error that they are not of the same size. in fact the mask is bigger (i've checked it by using extract_image)

If I understood correctly the approach you suggested is based on the fact that the combined pixmap is successfully created or i misunderstood?

[tigrankh]

strangely, even just saving one of the images into the file caused this blurry result.
in that scenario I haven't even used redactions

[tigrankh]

I could swear that i tried this before and it didn't work, but whatever. I finally was able to create an image with mask on it with insert_image() method.

img2 = doc.extract_image(62)["image"]
mimg = doc.extract_image(264)["image"]

new_doc[0].insert_image(Rect(*data["blocks"][1]["bbox"]), stream=img2, mask=mimg)

This experiment basically shows that i can create an image and apply a mask that has arbitrary size.
but i'll still fail to create a combined pixmap, because sizes of image and the mask are different

[JorjMcKie]

BTW "same size" in this context means "same width and height" - not the file sizes or whatever.

[tigrankh]

if you are referring to my comment about size errors, sure, that is understood, that size is about width and hight.
in my example, the size of an image is 800x1012, while the size of the mask is 2399x3035.
this is what i see in IRECT

[JorjMcKie]

Well, cases like this one are simply not supported.
Looks weird to me though, because the width-height ratio is only almost and not exactly the same between base image and the mask.
But without the file at hand ... I cannot tell what is really going on there.

[tigrankh]

I used an approach similar to what you suggested above, except since i'm unable to combine pixmaps, i manually whited out the redaction box from both the image and its mask. Then i inserted a new masked image and replaced the original one with it

        page.clean_contents()

        old_xref = 62

        data = page.get_text("dict")
        image_without_mask = doc.extract_image(60)["image"]

        # Step 1: extract image and its mask
        img_with_mask = doc.extract_image(62)["image"]
        mask = doc.extract_image(264)["image"]

        # Step 2: get pixmaps from streams
        pm_image_with_mask = fitz.Pixmap(img_with_mask)
        pm_mask = fitz.Pixmap(mask)

        # Step 3: get page params
        page_pixmap = page.get_pixmap()

        # Step 4: compute ratios of image/mask to page
        redaction_rect = (205.15096, 10.7828, 305.5, 61.5)

        image_width_ratio = pm_image_with_mask.width / page_pixmap.width
        image_height_ratio = pm_image_with_mask.height / page_pixmap.height

        mask_width_ratio = pm_mask.width / page_pixmap.width
        mask_height_ratio = pm_mask.height / page_pixmap.height

        # Step 5: Compute redaction rectangles on the image and the mask
        image_red_rect = (
            redaction_rect[0]*image_width_ratio,
            redaction_rect[1]*image_height_ratio,
            redaction_rect[2]*image_width_ratio,
            redaction_rect[3]*image_height_ratio
        )
        mask_red_rect = (
            redaction_rect[0] * mask_width_ratio,
            redaction_rect[1] * mask_height_ratio,
            redaction_rect[2] * mask_width_ratio,
            redaction_rect[3] * mask_height_ratio
        )

        # Step 6: Set rectangles in pixmaps
        pm_image_with_mask.set_rect(Rect(*image_red_rect), color=(255, 255, 255))
        pm_mask.set_rect(Rect(*mask_red_rect), color=(255,))

        # Step 7: Get the new streams
        new_image = pm_image_with_mask.tobytes()
        new_mask = pm_mask.tobytes()

        # Step 8: Insert new image
        new_xref = page.insert_image(Rect(*data["blocks"][1]["bbox"]), stream=new_image, mask=new_mask)

        # Step 9: Copy definition of stream of the new image
        doc.xref_copy(new_xref, old_xref)

        # Step 10: there now is a second /Contents object, showing new image
        cont_xrefs = page.get_contents()

        # Step 11: make sure that new /Contents is forgotten
        page.set_contents(cont_xrefs[0])
        page.clean_contents()

Then when doing redaction i'm using apply_redaction(images=0), since I basically already did what the default images option was doing.

It seems a little hacky to me, but it solves the problem

[JorjMcKie]

Very well done!
Yes: a bit hacky - but just a bit, it could be worse 😁. On the other hand, you got your result.

[tigrankh]

Indeed, it could be worse :D
Thank you very much for all the help!