Getting confused with transformations

[foranuj]

Hi,

I have a PDF that contains images,
TestSample.pdf

The images here are transformed and only part of these images are displayed. I want to develop a viewer that when I double click on any of the sub images shows the underlying image, and marks the section that's displayed on the PDF page.

I have already managed to launch the correct image in a viewer class, that looks like this,

My ImageViewer2 looks like this,

`class ImageViewer2(Gtk.Window):
def init(self, pixbuf, img_transform):
Gtk.Window.init(self, title="Image Viewer")

    self.pixbuf = pixbuf
    self.img_transform = img_transform

    self.drawing_area = Gtk.DrawingArea()
    self.drawing_area.connect("draw", self.on_draw)
    self.add(self.drawing_area)

    # Set the window size based on the original image dimensions
    width, height = self.pixbuf.get_width(), self.pixbuf.get_height()
    self.set_default_size(width, height)
    self.show_all()

def on_draw(self, widget, cr):
    Gdk.cairo_set_source_pixbuf(cr, self.pixbuf, 0, 0)
    cr.paint()

    # Get the image rectangle and matrix from the img_transform
    img_rect = self.img_transform[0][0]
    img_matrix = self.img_transform[0][1]

    cr.save()

    # Convert the fitz.Matrix to cairo.Matrix
    a, b, c, d, e, f = img_matrix
    cairo_matrix = cairo.Matrix(a, b, c, d, e, f)
    cr.transform(cairo_matrix)

    cr.rectangle(img_rect.x0, img_rect.y0, img_rect.width, img_rect.height)
    cr.set_source_rgba(0, 0, 1, 0.5)
    cr.fill()

    cr.restore()`

However, I can't manage to display the rectangle that should mark the part of the image that's displayed in the original PDF.

My thought process is as follows,

1. I find the xref of the image on which we had a mouse click
2. Get the pixmap, pixmap = fitz.Pixmap(fitz_page.parent.extract_image(img_xref)["image"])
3. Get the transforms applied to the original image, img_transform = fitz_page.get_image_rects(img_xref, transform=True)
4. Construct a pixbuf
5. And call ImageViewer2(pixbuf, img_transform)
6. Given that I know the transform and the bbox for the original image, I should be able to transform the cairo surface and draw the bounding box rectangle on the image to mark the section that was displayed,

Hoping I'm missing something small here and someone can help.

Thanks in advance,
Anuj

[JorjMcKie]

I'm sure I am missing something here. Still the following suggestion:

• First all, method page.get_image_info(xrefs=True) will give you all info on each image on the page (even if it has no xref responsible for display! Just an aside).
• Take an image's bbox in the previous list.
• You make the pixmap of the image as before.
• Compute matrix = bbox.torect(pix.irect).
• To compute the corresponding coordinates of something inside bbox (a point, a sub rectangle, ...) to coordinates wrt to the Pixmap IRect, just do point * matrix, or subrect * matrix.

Does that help?

BTW, you can also do this with the image instead of making a pixmap for cairo:

imgdoc = fitz.open(doc.extract_image(xref)["image"])  # open image as MuPDF document (also possible!)
pdfbytes = imgdoc.convert_to_pdf()  # convert this to a PDF
imgpdf = fitz.open("pdf", pdfbytes)  # the PDF document version of the image
imgpage = imgpdf[0]  # document page corresponding to the image
matrix = bbox.torect(imgpage.rect)  # transformation matrix bbox -> imgage page
# take a sub rectangle of the image's bbox on original page,
# and draw its location on the image full page
imgpage.draw_rect(subrect * matrix, color=fitz.pdfcolor["red"])
imgpdf.save(...)
# alternatively make a pixmap from that page again and save as PNG / JPEG

[foranuj]

Not quite what I'm expecting, Attaching an image that displays the issue.

You can see that I double clicked in the PDF on the first image in the left, A new panel opens up, which shows the full image that's on the page.

However, the transparent blue box covers the entire image. I want it to show only the displayed portion. I guess the question probably becomes, how do I get access to the subrect on the original PDF page, since if I use the image rectangle, then it'll cover the full image.

[JorjMcKie]

Sure you need the image's bbox part on the page, that is actually visible. There is no way to find that sub-rectangle by looking at the image properties.
The best you can have is bbox & page.rect: I noticed that some of these images are not fully contained in page.rect - their bboxes have negative coordinate values.
But whether other stuff on the page is partly covering cannot be determined that way. Also note that this coverage need not at all leave behind a visible sub-rectangle of bbox in the general case: it could just be a corner or a hole inside the bbox and what not. So just assuming the visible part of any image is a rectangle will lead to nowhere.
There is page.get_bboxlog() which lists all rectangles on the page covered by anything: text, images, vector graphics, etc. The sequence in that list is the sequence in which the page appearance is created: later items cover ealier items if the bboxes overlap. Maybe that helps you here.

[foranuj]

Thanks for your reply, I work on this stuff weekend to weekend, so apologies for the late response.