Filter out unwanted/empty drawings from Page.get_drawings()

[megh-khaire]

Hello, I have a PDF file that looks something like this: lech102.pdf
I want to extract all images from this PDF file.
Sample of the expected output:

I tried using a simple image extraction script to extract these images from the pdf file, however, that did not work and I realized the images I want to extract are actually vector graphics. So I implemented the following script (based on your drawing extraction script) to extract the vector images and save them as pictures:
To save time jump directly to extract_drawings

import os
import fitz


matrix = fitz.Matrix(3, 3)
border = (-1, -1, 1, 1)


def unify_rectangles(rect1, rect2):
    # Extract coordinates of top-left and bottom-right corners for both rectangles
    x1_tl, y1_tl, x1_br, y1_br = rect1
    x2_tl, y2_tl, x2_br, y2_br = rect2

    # Find the minimum and maximum x and y values
    x_min = min(x1_tl, x2_tl)
    y_min = min(y1_tl, y2_tl)
    x_max = max(x1_br, x2_br)
    y_max = max(y1_br, y2_br)

    # Create and return the union rectangle
    union_rect = (x_min, y_min, x_max, y_max)
    return union_rect


def unify_overlapping_rectangles(rect1, rect2):
    # Extract coordinates of top-left and bottom-right corners
    x1_tl, y1_tl, x1_br, y1_br = rect1
    x2_tl, y2_tl, x2_br, y2_br = rect2

    # Check if the rectangles overlap on both the x and y axis
    if x1_tl <= x2_br and x1_br >= x2_tl and y1_tl <= y2_br and y1_br >= y2_tl:
        return unify_rectangles(rect1, rect2)

    # If the rectangles do not overlap on either axis, they do not overlap
    return []


def extract_drawings(doc, drawings_dir):
    drawings = []
    for page in doc:
        page_drawings = []
        page.add_redact_annot(page.rect)
        page.apply_redactions(images=fitz.PDF_REDACT_IMAGE_REMOVE)
        page.clean_contents()

        for drawing in page.get_drawings():
            rect = drawing["rect"]
            if rect.height <= 10 or rect.width <= 10: continue
            page_drawings.append(rect)

        num_page_drawings = len(page_drawings)
        for i in range(num_page_drawings):
            if page_drawings[i]:
                for j in range(num_page_drawings):
                # if the current rectangle overlaps with any other rectangle enlarge it
                    if i!= j and page_drawings[j]:
                        bounding_box = unify_overlapping_rectangles(page_drawings[i], page_drawings[j])
                        if bounding_box:
                            page_drawings[i] = bounding_box
                            page_drawings[j] = None

        drawings = list(set(page_drawings))  # remove any duplicates
        for i, rect in enumerate(drawings):
            if rect:
                pix = page.get_pixmap(matrix=matrix, clip=rect+border)
                name = "drawing-%03i-%02i.png" % (page.number, i)
                pix.save(os.path.join(drawings_dir, name))

This script works however it creates a lot of unwanted pictures that have whitespaces and/or a few lines/rectangles:

Could someone please help me out in filtering out these unwanted pictures?
PS: PyMuPDF is amazing ❤️
Thank you!

System: Windows 10 (64 bit)
Python Version: 3.11.2
PyMuPDF Version: 1.21.1

[JorjMcKie]

Thank you very much for the feedback! Also chapeau for your advanced use of PyMuPDF features👏!

You could use Pixmap color counting to exclude insignificant vector graphics:

1. After cleaning / redacting the page, make a pixmap of it.
2. After consolidating the vector graphics the way you already do, inspect the remaining for minimum relevance:
   • use pix.color_count(colors=True/False, clip=rect) to check if the drawing rect is just one color or only a few, optionally returning which colors these are
   • use pix.color_topusage(clip=rect) a variant letting you check if rect is "almost" unicolor: returns (ratio, pixel) with 0 <= ratio <= 1 and "pixel" being that most used color tuple
   • or invest in the effort of making separate little pixmaps for the drawing rectangles and then check check them for .is_unicolor.
3. Exclude "line-like" drawings: rect.width << rect.height etc. They often are generated instead of lines by converters DOCX -> PDF (MS Word, LibreOffice). Well, you already exclude those having tiny width or height. But maybe 10 * rect.width < rect.height might also make sense.

[megh-khaire]

Thank you for the quick reply 🙇
I've made changes to the script according to your suggestions:

def valid_image(page, rect):
    pix = page.get_pixmap(matrix=matrix, clip=rect)
    if pix.is_unicolor: return False
    max_ratio, _ = pix.color_topusage(clip=rect)
    if max_ratio > 0.9 : return False
    return True
    
# ==============================================================
# Inside extract_drawings:
        for drawing in page.get_drawings():
            rect = drawing["rect"]
            if rect.height <= MIN_DIMENTSION or rect.width <= MIN_DIMENTSION: continue
            if 5 * rect.width < rect.height: continue
            if not valid_image(page, rect): continue
            page_drawings.append(rect)

However, function color_topusage is returning a ZeroDivisionError.
Here is the stack trace:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "__main__.py", line 79, in <module>
    extract_drawings(cleaned_doc, drawings_dir)
  File "drawing.py", line 67, in extract_drawings
    if not valid_image(page, rect): continue
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "drawing.py", line 49, in valid_image
    max_ratio = pix.color_topusage(clip=rect)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\.venv\Lib\site-packages\fitz\fitz.py", line 7096, in color_topusage
    return (cnt / allpixels, maxpixel)
            ~~~~^~~~~~~~~~~
ZeroDivisionError: division by zero

Am I doing something wrong here?

[JorjMcKie]

if the pixmap itself already is restricted to the rectangle, then omit clip from .color_topusage(). Also, the rectangle must not be empty: allpixels (number of pixels in the pixmap) obviously is zero.

[megh-khaire]

Also, the rectangle must not be empty: allpixels (number of pixels in the pixmap) obviously is zero.

Right, I was just wondering how the rectangle is empty because I had a height and width check before calling color_topusage

if the pixmap itself already is restricted to the rectangle, then omit clip from .color_topusage()

This fixed the issue.

The script now removes almost 80% of the unwanted/insignificant drawings.
Thank you for your help 🙏