[Feature] New utility for unassigned faces

[Enzo1603]

I have searched the existing feature requests to make sure this is not a duplicate request.

• Yes

The feature

Dear immich Team,

My feature request refers to the following closed issue: #10277.

It would be highly beneficial if there were a way to filter photos containing unassigned faces, allowing users to review and assign these faces one by one if desired.

One possible solution is to add a new filter option specifically for photos with unassigned faces. Alternatively, considering the recent introduction of the Utilities section, this functionality could be integrated there, alongside the duplicate detection feature.

Thank you for considering this suggestion.
(I hope I did not miss that you already work on it or that somebody already requested this :) )

Platform

• Server
• Web
• Mobile

[Thecrafterja]

I would also really like to see this feature.

[Eyeball753]

This confuses me so much cause I swear that was a feature that's now gone. Months ago, when I imported all my pictures, the people screen was full of recognized faces that I'd assign names to or hide. Now I added a picture and no new faces popped up in the people screen. I had to manually find that one picture, click on the detected face, create a new person and then name them.

That's what led me to this post. How it works now makes no sense.

[jforseth210]

This may not be exactly the same (but #16863 was closed as a duplicate, so close enough), but I think an, "Is this also _____" option within the "/people/" page would be cool. The edit faces menu shows similar faces, but there's no easy way to say, "Yes, this is also ______".

I wrote a quick script to do it:

import { getFaces, getAllPeople, init } from "@immich/sdk";
import  open  from 'open';
import readline from 'readline';
const API_KEY = "";
const BASE_URL = "";
const PERSON_ID = "";
init({
  baseUrl: BASE_URL,
  apiKey: API_KEY,
});
function input(promptText = '') {
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });
  return new Promise(resolve => {
    rl.question(promptText, answer => {
      rl.close();
      resolve(answer);
    });
  });
}
async function main() {
  const people = await getAllPeople({
    withHidden: false,
    closestPersonId: PERSON_ID,
  });
  for (const person of people.people) {
    // Open the person's page in the default browser
    await open(`${BASE_URL}/people/${person.id}`);
    await input(`Opened ${person.id}. Press Enter to continue...`);
  } 
}
main();

But a UI with a quick check/x would be better

[desmond206x]

Searched exactly for this.
I want to assign all faces if possible in all photos, but how to find the fotos with unassigned faces?

But one additional thing.

Why is there no indicator that this image has unassigned faces?

[ireun]

Okay, maybe this will help you a little. Remember it's just PoC, no warranties, read the code and understand what it does. DANGER AHEAD!

I've got around 50k images in my library, a few thousands of these from weddings, probably another few thousands of screenshots (with faces in fb-messenger bubbles), tiktok-videos, and what not. Ergo - lots of faces (28 613 to be exact).

Because of that I've set my ML face detection settings pretty rigorously: buffalo_l, 0.65, 0.4, 3.

This method requires few hours of work, but it provides probably best fine-tuned clusters that you can get.

Really low max_distance leads to multiple people that you will have to manually merge together.
Since WebUI will let you only merge 5 people at a time - use immich power tools to merge these, or my script, it will marge n_closest_persons starting from person_id

1. Go into the person, copy the uuid from url
2. Sort via similarity and count how many people are the same person as the person that you have selected. I was merging in the increments of 10, so if 33 closest 'people' are the same person i'd merge 30 using my scripts, and rest using WebUI
3. update the person_uuid, n_closest_to_merge, immich_access_token in the scripts
4. run

import requests.utils

cookies = {
    "immich_access_token": "<token>",  # provide your own
    "immich_auth_type": "password",
    "immich_is_authenticated": "true"}

# edit these
person_id = "xxx-xxx-xxx-xxx-xxx"
n_closest_to_merge = 10
origin = "https://photos.immich.app"

nearby_people = requests.get(
    f"{origin}/api/people?closestPersonId={person_uuid}&withHidden=false",
    cookies=cookies).json()['people']

nearby_people_uuids = [people['id'] for people in nearby_people]

response = requests.post(f"{origin}/api/people/{person_uuid}/merge",
                         json={"ids": nearby_people_uuids[1:n_closest_to_merge + 1]},
                         cookies=requests.utils.cookiejar_from_dict(cookies))

After merging everything you could, set a name for people that are important to you.
You will still have many faces unassigned to people (since max_distance was set really low)

Now query the DB for UUIDs of assets that contain unassigned faces, example below, i have ordered by resolution, use offset and limit your results. Copy result into assets in the script.

select distinct *
from (select "assetId"
--        ("boundingBoxX2" - "boundingBoxX1") * ("boundingBoxY2" - "boundingBoxY1")
      from asset_face
      where "personId" is null
      order by ("boundingBoxX2" - "boundingBoxX1") * ("boundingBoxY2" - "boundingBoxY1") asc) as afaI
offset 0 limit 500;

The script below goes though assets, finds missing faces, queries for nearby (named) ones and in case it finds a named person in the near-n (size parameter) people - it asks if this is the same person. The name of possible person will be in the title, as well as in the console. Enter to confirm, Escape to skip, you may also use buttons. The face is marked on the thumbnail using red rectangle.

My advise: go through 1-2k of UUIDs (many of them will not have a nearby named person, so don't worry) then run the 'face recognition` job, it will probably be able to assign some faces for you, then query the DB again with an offset to skip the ones that you have gone through.

from io import BytesIO
from tkinter import *

import PIL
import requests.utils
from PIL import Image, ImageDraw
from PIL import ImageTk, Image

cookies = {
    "immich_access_token": "<token>",  # provide your own
    "immich_auth_type": "password",
    "immich_is_authenticated": "true"}

s = requests.Session()
requests.utils.add_dict_to_cookiejar(s.cookies, cookies)

# UUIDs of assets taken from DB
assets = ['xxx-e649-426e-a11e-xxx', 'xxx-db86-463e-be4a-xxx',
          'xxx-4af9-4837-836a-xxx', 'xxx-7e51-441e-989b-xxx',
          'xxx-d24c-4014-a9f7-xxx', 'xxx-bc94-49d5-96e9-xxx', ...]

# edit these
origin = "https://photos.immich.app"


def getFaces(asset_uuid):
    response = s.get(f"{origin}/api/faces?id={asset_uuid}")
    if 'error' in response.json():
        return []

    return response.json()


def getClosestPeople(face_uuid):
    response = s.get(f"{origin}/api/people?closestAssetId={face_uuid}&size=3")
    return response.json()["people"]


root = Tk()
root.geometry("550x500+300+150")
root.resizable(width=True, height=True)

asset_offset = 0
possible_match = ""
curr_face = ""


def next_img(key=None):
    global asset_offset, possible_match, curr_face
    for asset_id in assets[asset_offset:]:
        asset_offset += 1
        print(asset_offset)
        for face in getFaces(asset_id):
            if face['person']: # skip faces that are assigned
                continue
            people = getClosestPeople(face["id"])
            named_people_names = [person['name'] for person in people if person['name'] != '']
            if named_people_names:
                curr_face = face["id"]
                # person_thumb = s.get(f"{origin}/api/people/{people[0]['id']}/thumbnail")
                # person_thumb_img = Image.open(BytesIO(person_thumb.content))
                picture_thumb = s.get(f"{origin}/api/assets/{asset_id}/thumbnail")
                picture_thumb_img = Image.open(BytesIO(picture_thumb.content)).convert('RGB')

                draw = ImageDraw.Draw(picture_thumb_img)
                thumb_shrink_ratio = (picture_thumb_img.size[0] / face['imageWidth'])

                draw.line((int(face['boundingBoxX1'] * thumb_shrink_ratio),
                           int(face['boundingBoxY2'] * thumb_shrink_ratio),
                           int(face['boundingBoxX1'] * thumb_shrink_ratio),
                           int(face['boundingBoxY1'] * thumb_shrink_ratio)), fill=128)
                draw.line((int(face['boundingBoxX2'] * thumb_shrink_ratio),
                           int(face['boundingBoxY2'] * thumb_shrink_ratio),
                           int(face['boundingBoxX2'] * thumb_shrink_ratio),
                           int(face['boundingBoxY1'] * thumb_shrink_ratio)), fill=128)
                draw.line((int(face['boundingBoxX1'] * thumb_shrink_ratio),
                           int(face['boundingBoxY1'] * thumb_shrink_ratio),
                           int(face['boundingBoxX2'] * thumb_shrink_ratio),
                           int(face['boundingBoxY1'] * thumb_shrink_ratio)), fill=128)
                draw.line((int(face['boundingBoxX1'] * thumb_shrink_ratio),
                           int(face['boundingBoxY2'] * thumb_shrink_ratio),
                           int(face['boundingBoxX2'] * thumb_shrink_ratio),
                           int(face['boundingBoxY2'] * thumb_shrink_ratio)), fill=128)

                print(f"possibly {named_people_names} in asset {origin}/photos/{asset_id}")
                root.title(f"{named_people_names[0]}?")
                possible_match = [person for person in people if person['name'] != ''][0]['id']

                picture_thumb_img.thumbnail((1024, 1024), PIL.Image.Resampling.LANCZOS)
                img = ImageTk.PhotoImage(picture_thumb_img)
                panel.configure(image=img)
                panel.image = img
                return


def assign_face(key=None):
    print(f"PUT {origin}/api/faces/{possible_match} with payload id: {curr_face}")

    if True:
        s.put(f"{origin}/api/faces/{possible_match}", json={"id": curr_face})

    next_img()


panel = Label(root)
buttons = Frame(root)
buttons.pack(pady=5)
btn = Button(buttons, text='Skip', command=next_img).pack(side="left")
btn2 = Button(buttons, text='Correct', command=assign_face).pack(side="right")
panel.pack()
root.bind("<Escape>", next_img)
root.bind("<Return>", assign_face)

root.mainloop()