images in a shared folder are not processed upon addition to user's timeline path

[val-kulkov]

Which version of recognize are you using?

9.0.7

Enabled Modes

Object recognition, Face recognition, Video recognition, Music recognition

TensorFlow mode

GPU mode

Downstream App

Files App

Which Nextcloud version do you have installed?

31.0.9

Which Operating system do you have installed?

Fedora Linux 42 (LXC container in Proxmox 9)

Which database are you running Nextcloud on?

PostgreSQL 17.4

Which Docker container are you using to run Nextcloud? (if applicable)

N/A

How much RAM does your server have?

16GiB

What processor Architecture does your CPU have?

x86_64

Describe the Bug

My use case is as follows:

• Nextcloud users have their images processed by Recognize, including face recognition
• Nextcloud users share their folders with other Nextcloud users
• those other users add shared folders to their "Timeline Path" in Memories -> Settings
• those other users have "Face Recognition" enabled in their personal settings

The other users who add shared folders to their "Timeline Path" never see the images in the shared folders processed by Recognize for face recognition.

However, if the owner of a shared folder adds an image to the shared folder AFTER the other user has placed the shared folder into their "Timeline Path", Recognize processes the image for both the owner and the other user.

Here is another way to illustrate what is happening. User3 is the owner of a shared folder and user1 and user2 have added the shared folder to their "Timeline Path". The Nextcloud admin runs "recognize:recrawl" and "recognize:classify" to reclassify everything from scratch. This results in the following:

nextcloud=# SELECT user_id, count(id) FROM oc_recognize_face_detections GROUP BY user_id;
 user_id | count 
---------+-------
 user1   |   271
 user2   |   271
 user3   |   271
(3 rows)

Now if the owner shares the same folder with user4 and user4 adds the shared folder to their "Timeline Path", nothing happens. The above SELECT query produces the same output, which means that user4 will never see detected faces in their "People (Recognize)" tab.

User4 will only see the detected faces if "recognize:classify" is run again, which should not be necessary at all because all images are already classified, and re-classification will produce the same results for user1, user2 and user3.

I should note that the data in "oc_recognize_face_detections" table for a given image are exactly the same for user1, user2 and user3. An analysis of the output of "occ recognize:classify" command reveals that each image in a shared folder is processed by Recognize only once, but each image is recorded in a separate table entry for each user who has access to this shared image. This appears wasteful because one table entry should be enough, and this one entry should be available to all users who are given access to the shared folder.

As a workaround, one can copy entries from user1 to user4, with necessary replacement of the value of "user_id" field in "oc_recognize_face_detections". But having only one table entry for an image would be obviously a better solution.

Expected Behavior

A Nextcloud user should have face detection results available to them upon addition of a shared folder to their "Timeline Path" in Memories -> Settings.

To Reproduce

1. Put images in a folder and have them processed by Recognize for face detection.
2. Share the folder with some Nextcloud users.
3. Have these Nextcloud users add the shared folder to their "Timeline Path" in Memories -> Settings.
4. Observe that these Nextcloud users will never see face detections for images in the shared folder.

Debug log

No response

[val-kulkov]

I should add that "facerecognition" creates only one table entry for a detected face in a given image (see "oc_facerecog_persons" table), and this table entry becomes available to all users who have access to the image through image/folder sharing. As a result, all detected faces/persons become instantly available to Nextcloud users who add the shared folder to their "Timeline Path".

Having said that, "facerecognition" app messes up processing of images in Team Folders by processing an image in a Team Folder anew for each user who has access to the Team Folder (what a waste of computing power!), and by creating a respective table entry as if that user were an owner of that image. This creates tons of duplicates, but this matter is going to be discussed in a separate bug report in the "facerecognition" section.

[marcelklehr]

Hi @val-kulkov
Thank you for taking the time to give feedback! I will investigate and fix this, if I can reproduce it.

[marcelklehr]

Would you be up for a bit of testing? I have prepared the following PR which should/might fix this: #1384 I will also test it myself, but only earliest next week.

[val-kulkov]

Would you be up for a bit of testing? I have prepared the following PR which should/might fix this: #1384 I will also test it myself, but only earliest next week.

After applying #1384, processing of images in shared folders works like a charm. The user who received a share did not even have to log in. The background process took care of everything.

Thank you @marcelklehr! Your response has been super fast and super efficient.

P.S. I still think that having one table record per person / face detection would be more efficient than a separate table record for all users with access to a shared image. In organizations where images are shared with hundreds or thousands of users, "oc_recognize_face_detection" may become unnecessarily large and slow to handle.

[val-kulkov]

Hold on... I am sorry I have rushed my previous comment. With #1384 applied, "oc_recognize_face_detections" is populated correctly for a user who received a share. However, face clustering is not performed, and no new records are added to "oc_recognize_face_clusters". Running "occ recognize:cluster-faces" on the terminal fixes this problem, but apparently a manual intervention should not be required.

[marcelklehr]

face clustering is not performed

Recognize checks whether a clustering run is needed every 12h, so you might need to wait a bit, but we can also create the cluster job in the face detection copy routine to make it feel more instant. I'll do that next week.

[val-kulkov]

Recognize checks whether a clustering run is needed every 12h, so you might need to wait a bit, but we can also create the cluster job in the face detection copy routine to make it feel more instant. I'll do that next week.

Thank you for clarifying this to me. 12h period for a clustering run seems reasonable. I simply haven't immersed myself in the code enough to learn this. And there is not a lot of documentation on what's going on behind the scenes - please do not take this comment as criticism, your work on Recognize means a lot to me and to all the folks who use Recognize. THANK YOU.