Try CIEDE colour distance #2
Description
Instead of Euclidean distance in RGB, try other metrics, esp CIEDE2000 as used in Poco, Mayhua & Heer (2018).
They also use pixel patches around target pixels as a way to smooth the colours a bit. Could find targets based on clustering to do this.