Augmented Image.getbbox method?

[p3rAsperaAdAstra]

I have made a matplotlib script for plotting PXRD measurements and I thought it would be neat if I could add an image of a molecule to the upper right of the plot. So what I did is to use RDKit to generate a raster image (PIL image) from a SMILES code of a molecule. That gets passed to the OffsetImage class and then to the AnnotationBox class so that it can be displayed in the upper right.

There was a particular problem though that irked me while implementing this: The images that come from RDKit always have a white background and aren't cropped to the content of the image. I found out that PIL provides the Image.getbbox() method, but that only works for, as I understand, perfect black pixels an completely transparent pixels, if the right keyword is used.

It didn't work for my white bg image, which I apparently can only obtain as a white bg image (a limitation by RDKit). That is why I wrote my own hacky code to find out where the edges of my image are. Those can then be cropped using the Image.crop() method.

So I started thinking: Can the concept of "cropping to content" not be extended to an arbitrary background color and paired with a tolerance factor in case the provided image is a little grainy?

The general idea is the following: Create a stand-alone function, or method, which takes both a user provided color and a tolerance factor and then crops the image according to the following logic:

1. Iterate over rows and columns.
2. Find the color distance for every pixel to the user provided color according to a given color space like sRGB.
3. Normalise the color distances to values between 0 and 1. (Would require finding out what the max color dist for each color space is)
4. Compare all normalised color distances to the user-provided tolerance factor.
5. If the distance is below or equal to the tolerance, the bbox gets updated.

So it's basically a generalisation of the existing method regarding the background color as well as a tweak to how precise the match between bg and match color needs to be.

This is what I have implemented so far, but it is far from polished:

from PIL import Image
from math import sqrt

def crop_to_content(image,bg=(255,255,255),tolerance=0.04):

	'''Crop to content function with arbitrary bg colour and percentual colour distance tolerance.'''

	def get_color_distance(pixel,bg,color_space='sRGB'):

		'''Obtains the color distance between two pixels according to a specified color space.
		Returned values are normalised to be between 0 and 1.'''

		if color_space.lower() == 'srgb':
			norm = sqrt(3* 255**2)
			cdist = sqrt((pixel[0] - bg[0])**2 + (pixel[1] - bg[1])**2 + (pixel[2] - bg[2])**2)

			return cdist/norm

		elif color_space.lower() == 'redman':
			pass # need to implement
		elif color_space.lower() == 'cie94':
			pass # need to implement
		elif color_space.lower() == 'ciede2000':
			pass # need to implement


	w,h = image.size # obtain the number of rows and columns of the pixel array
	margins = {'l':0,'r':w,'t':0,'b':h} # stores the bounds (l:left, r:right, t:top, b:bottom) of the white margins.
	
	# four almost identical iterations over either rows or columns to determine at which index a row
	# or column doesn't consist of only pure white pixels anymore.
	for i in range(w): # left to right column scan.
		col = [image.getpixel((i,j))[:3] for j in range(h)] # all pixels of the current column
		dists = [get_color_distance(pixel,bg) for pixel in col] # all colour distances between the column's pixels and the bg.
		all_remove = all([dis <= tolerance for dis in dists]) # boolean check to see if column can be cropped.
		if all_remove == False: # all distances must be smaller than the tolerance factor.
			margins['l'] = i-1 # always use one col/row before the first non-white pixel is found.
			break

	for i in range(w)[::-1]: # right to left column scan.
		col = [image.getpixel((i,j))[:3] for j in range(h)] # all pixels of the current column
		dists = [get_color_distance(pixel,bg) for pixel in col] # all colour distances between the column's pixels and the bg.
		all_remove = all([dis <= tolerance for dis in dists]) # boolean check to see if column can be cropped.
		
		if all_remove == False: # all distances must be smaller than the tolerance factor.
			margins['r'] = i+1 # always use one col/row before the first non-white pixel is found.
			break

	for i in range(h): # top to bottom row scan.
		row = [image.getpixel((j,i))[:3] for j in range(w)] # all pixels of the current column
		dists = [get_color_distance(pixel,bg) for pixel in row] # all colour distances between the column's pixels and the bg.
		all_remove = all([dis <= tolerance for dis in dists]) # boolean check to see if column can be cropped.
		
		if all_remove == False: # all distances must be smaller than the tolerance factor.
			margins['t'] = i-1 # always use one col/row before the first non-white pixel is found.
			break

	for i in range(h)[::-1]: # bottom to top row scan.
		row = [image.getpixel((j,i))[:3] for j in range(w)] # all pixels of the current column
		dists = [get_color_distance(pixel,bg) for pixel in row] # all colour distances between the column's pixels and the bg.
		all_remove = all([dis <= tolerance for dis in dists]) # boolean check to see if column can be cropped.
		
		if all_remove == False: # all distances must be smaller than the tolerance factor.
			margins['b'] = i+1 # always use one col/row before the first non-white pixel is found.
			break

	bbox = (margins['l'],margins['t'],margins['r'],margins['b']) # the tuple in order of left, top, right, bottom passed to the crop method.
	image = image.crop(bbox) # crop the image exactly at the edge of the content.

	return image 


image = Image.open('to_crop.png') # open example image.
cropped = crop_to_content(image) # preform crop.
cropped.save('cropped.png') # save image.

This is the reference image I'm using:

This is an additional function for generating a PIL image from a SMILES code, if you are interested/have RDKit installed:

from rdkit import Chem
from rdkit.Chem import Draw


def get_rdkit_image(SMILES='CN(C)CC1CCCCC1(C2=CC(=CC=C2)OC)O'):

	'''Obtains a PIL image object from a SMILES using RDKit.'''

	mol = Chem.MolFromSmiles(SMILES)
	img = Draw.MolToImage(mol, size=(300,300))

	return img

Is this a good idea?

[radarhere]

As an aside, for the reference image you mentioned, a simpler solution would be to just invert the white to become black, and then get the bounding bbox.

from PIL import Image, ImageOps
im = Image.open("input.png")
bbox = ImageOps.invert(im.convert("RGB")).getbbox()
im.crop(bbox).save("output.png")

gives

[radarhere]

Alternatively, for an image without a white background, you could use ImageDraw.floodfill() to fill the edges with black. ImageDraw.floodfill() has a thresh argument that could provide you with the tolerance that you are looking for as well.

from PIL import Image, ImageDraw
im = Image.open("input.png")
im_filled = im.copy()
bbox_color = (255, 255, 255, 255)
for corner in ((0, 0), (im.width-1, im.height-1)):
    if im.getpixel(corner) == bbox_color:
        ImageDraw.floodfill(im_filled, corner, (0, 0, 0, 0), thresh=0)
bbox = im_filled.getbbox()

im.crop(bbox).save("output.png")

[radarhere]

If you were just talking about a different color, I'd suggest making changes to ImagingGetBBox. But that uses a mask, and you're also talking about using a threshold.

Maybe a variation on my floodfill() approach is the simplest option? See what you think.

[radarhere]

Closing, as a lack of response here indicates that there may not be a strong need for this feature.