Allowed memory exhausted

[zycker]

Which version of recognize are you using?

8.2.0

Enabled Modes

Face recognition

TensorFlow mode

WASM mode

Downstream App

Memories App

Which Nextcloud version do you have installed?

30.0.4

Which Operating system do you have installed?

Proxmox 8.3.2 LXC container

Which database are you running Nextcloud on?

unknown

Which Docker container are you using to run Nextcloud? (if applicable)

27.4.1

How much RAM does your server have?

16GB

What processor Architecture does your CPU have?

x86_64

Describe the Bug

No face detected for just my user (180Go of pictures), works fine for the other users.
Manually run, here is the ouput :

`docker exec --user www-data -it nextcloud-aio-nextcloud php -d memory_limit=15000M occ recognize:cluster-faces --batch-size 200

Clustering face detections for user User2

ClusterDebug: Retrieving face detections for user User2

ClusterDebug: Not enough face detections found

Clustering face detections for user User3

ClusterDebug: Retrieving face detections for user User3

ClusterDebug: Not enough face detections found

Clustering face detections for user User1

ClusterDebug: Retrieving face detections for user User1

ClusterDebug: Found 500 fresh detections. Adding 50 old detections and 3727 sampled detections from already existing clusters. Calculating clusters on 4277 detections.

ClusterDebug: Clustering complete. Total num of clustered detections: 183

Clustering face detections for user User4

ClusterDebug: Retrieving face detections for user User4

ClusterDebug: Not enough face detections found

Clustering face detections for user User5

ClusterDebug: Retrieving face detections for user User5

ClusterDebug: Not enough face detections found

Clustering face detections for user admin

ClusterDebug: Retrieving face detections for user admin

ClusterDebug: Found 500 fresh detections. Adding 50 old detections and 3120 sampled detections from already existing clusters. Calculating clusters on 3670 detections.`

No face detection for User1.

I'm running Nextcloud AIO, I've seen similar issues here that where solved by increasing Nextcloud Memory limit, here is mine :
NEXTCLOUD_MEMORY_LIMIT=16000M

Expected Behavior

Faces being detected

To Reproduce

Install Recognize and Memories, wait for cron job to run.

Debug log

Allowed memory size of 1073741824 bytes exhausted (tried to allocate 655360 bytes) at /var/www/html/custom_apps/recognize/lib/Clustering/MrdBallTree.php#450

[github-actions]

Hello 👋

Thank you for taking the time to open this issue with recognize. I know it's frustrating when software
causes problems. You have made the right choice to come here and open an issue to make sure your problem gets looked at
and if possible solved.
I try to answer all issues and if possible fix all bugs here, but it sometimes takes a while until I get to it.
Until then, please be patient.
Note also that GitHub is a place where people meet to make software better together. Nobody here is under any obligation
to help you, solve your problems or deliver on any expectations or demands you may have, but if enough people come together we can
collaborate to make this software better. For everyone.
Thus, if you can, you could also look at other issues to see whether you can help other people with your knowledge
and experience. If you have coding experience it would also be awesome if you could step up to dive into the code and
try to fix the odd bug yourself. Everyone will be thankful for extra helping hands!
One last word: If you feel, at any point, like you need to vent, this is not the place for it; you can go to the forum,
to twitter or somewhere else. But this is a technical issue tracker, so please make sure to
focus on the tech and keep your opinions to yourself. (Also see our Code of Conduct. Really.)

I look forward to working with you on this issue
Cheers 💙

[marcelklehr]

Hey @zycker

Allowed memory size of 1073741824 bytes

This seems to indicate the limit is at 1GiB when the cron job is run, so somehow AIO is not setting the right memory limit it seems.

[szaimen]

@zycker can you post the output of sudo docker inspect nextcloud-aio-nextcloud | grep -i memory here?

[zycker]

root@docker:~# docker inspect nextcloud-aio-nextcloud | grep -i memory "Memory": 0, "MemoryReservation": 0, "MemorySwap": 0, "MemorySwappiness": null, "PHP_MEMORY_LIMIT=1024M",

It seems the memory limit isn't applied.
As you've write in another issue szaimen, I've stopped the mastercontainer, removed it and recreated it but it doesn't change the result.

I'm using Portainer with this stack :
`version: "3.8"

volumes:
nextcloud_aio_mastercontainer:
name: nextcloud_aio_mastercontainer # This line is not allowed to be changed

services:
nextcloud:
image: nextcloud/all-in-one:latest
restart: always
container_name: nextcloud-aio-mastercontainer # This line is not allowed to be changed
volumes:
- nextcloud_aio_mastercontainer:/mnt/docker-aio-config # This line is not allowed to be changed
- /var/run/docker.sock:/var/run/docker.sock:ro # May be changed on macOS, Windows or docker rootless. See the applicable documentation. If adjusting, don't forget to also set 'DOCKER_SOCKET_PATH'!
ports:
- 80:80 # Can be removed when running behind a web server or reverse proxy (like Apache, Nginx and else). See https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md
- 8080:8080
- 8443:8443 # Can be removed when running behind a web server or reverse proxy (like Apache, Nginx and else). See https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md
environment: # Is needed when using any of the options below
- APACHE_PORT=11000 # Is needed when running behind a web server or reverse proxy (like Apache, Nginx and else). See https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md
- SKIP_DOMAIN_VALIDATION=true
- NEXTCLOUD_ENABLE_DRI_DEVICE=true
# - APACHE_IP_BINDING=127.0.0.1 # Should be set when running behind a web server or reverse proxy (like Apache, Nginx and else) that is running on the same host. See https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md
# - COLLABORA_SECCOMP_DISABLED=false # Setting this to true allows to disable Collabora's Seccomp feature. See https://github.com/nextcloud/all-in-one#how-to-disable-collaboras-seccomp-feature
# - DOCKER_SOCKET_PATH=/var/run/docker.sock # Needs to be specified if the docker socket on the host is not located in the default '/var/run/docker.sock'. Otherwise mastercontainer updates will fail. For macos it needs to be '/var/run/docker.sock'
# - DISABLE_BACKUP_SECTION=false # Setting this to true allows to hide the backup section in the AIO interface.
# - NEXTCLOUD_DATADIR=/mnt/ncdata # Allows to set the host directory for Nextcloud's datadir. See https://github.com/nextcloud/all-in-one#how-to-change-the-default-location-of-nextclouds-datadir
- NEXTCLOUD_MOUNT=/mnt/ # Allows the Nextcloud container to access the chosen directory on the host. See https://github.com/nextcloud/all-in-one#how-to-allow-the-nextcloud-container-to-access-directories-on-the-host
- NEXTCLOUD_UPLOAD_LIMIT=10G # Can be adjusted if you need more. See https://github.com/nextcloud/all-in-one#how-to-adjust-the-upload-limit-for-nextcloud
# - NEXTCLOUD_MAX_TIME=3600 # Can be adjusted if you need more. See https://github.com/nextcloud/all-in-one#how-to-adjust-the-max-execution-time-for-nextcloud
- NEXTCLOUD_MEMORY_LIMIT=14000M`

[szaimen]

It seems the memory limit isn't applied.
As you've write in another issue szaimen, I've stopped the mastercontainer, removed it and recreated it but it doesn't change the result.

You need to restart the containers via the aio interface after adjusting the value. Otherwise is the change not applied to the sibling containers as docker containers are immutable.

[zycker]

Indeed, after doing that the result changed :
docker inspect nextcloud-aio-nextcloud | grep -i memory "Memory": 0, "MemoryReservation": 0, "MemorySwap": 0, "MemorySwappiness": null, "PHP_MEMORY_LIMIT=14000M"

I will see tomorrow if the cron job runs without errors.

Thank you !

[marcelklehr]

Thanks for chiming in, @szaimen 💙

[zycker]

5 days later, it is now working properly, thank for your help to you both !