comparing small (10-20kb) jpgs in upython

[kjm1102]

I've got an outdoor esp32cam woken from deepsleep to take a photo by a passive infra red sensor. The sensor claims to have some fancy human body model but gives ~10 to 20 false wakeups/day. By false wakeup I mean not a warm blooded creature or any sign of movement anywhere in the frame.

I've tried comparing photo file sizes between current & previous photos to reduce the image count but it's a blunt tool for a problem that needs more finesse. Limiting wakeup rate has also helped (the false wake ups are sometimes bursty) but I'm still left with ~5 to 10 false wakeups/day.

All the photo comparison hits online post 2018 involve openCV, pillow or other libs not available for upython. Posts from before then tend to have a lot of math but not much in the way of python code I could use as a starting point. Tiny ML doesn't seem to fit the bill either & looks to have a really steep learning curve

I need a better way to find differences between 2 jpgs than simply comparing files size. Anyone else come up against this type of problem & found a upython solution?

[2dof]

I'm not sure what type of "human body model" (I didn't delve into it) it use, but if You have false detection from small animals and "ghosts" (i.e no movements/ sun shade changes and etc.) that means preprocessing fails (motion detection and object extraction) , infrared sensor works here as motion detector ( i assume it is sensor - not a ccd infrared camera).
If you don't have the option to tune ( since it is ML model then propably not) I recommend just use morphological Operations (like erode, dilation etc) in grayscale image/binary image some theory)
these are basic operations that are easy to implement even in micropython ( 2d matrix operations).

the simplest way ( based on your infra red sensor as motion detector working as "trigger"):
The first step : motion detection ( based in camera photo trigered by sensor)

• Calculate the absolute difference of the two images: base photo (grayscale) ( photo of your outdoor without objects - it can be couple photos in several dayligh conditions) , if you par car or you leave other objects on area then you can compare photo made in some intervals)
• perform image erosion ( ant other operations if nedded) of output image.
• calculate area (pixels) of object/ objects and compare with some threshold value) to detect motion. threshold value is some narrow scope of pixels area , to if you set right value you get instantly "human" detection.

second step:
it just comaring areas (no of pixes) to classify them as human size objects. also You can do other processing: for example if camera catch
area outside of your porch/driveway etc you can just define "cut-off" area from photo ( then camera has to be static).

if resolution of image is big, then resizing can significantly speed-up process. For fast development I I recommend use first python (even with open-cv) to just build your prototype and perform testing and chose parametrs , and when all will be as you planned -> migrate to upython.

[rkompass]

Tensorflow lite is available for micropython: This site has firmware for the ESP32 both for with or without SPIRAM.
It states:

The purpose of this project is to make a custom micropython firmware that installs tensorflow lite for micro controllers and allows for experimentation.

There is an example for image classification (after reducing to grayscale 96x96 pixels).

[kjm1102]

I think what I need is just some simple motion detection code that identifies movement by comparing each new photo to the previous one. The C programmers seem to have an EloquentSurveillance.h lib for this, although I've yet to see how it works. All the online references to it (eg https://github.com/eloquentarduino/EloquentSurveillance) show how to access it but I can never find the actual C++ jpg comparison code.

I briefly thought https://github.com/remibert/pycameresp/blob/main/modules/lib/motion/motioncore.py might have a upython jpg comparer in it but after half an hour of raking through the the .py files I can't seem to find it. I guess it must be in the 'specially modified ESP32CAM firmware'?

[2dof]

If You analyse motioncore.py You notice that motion detection is based on differences_detection threshold number of calculated
differences = self.compare(display) which are some image properties (like difference in histogram, etc).

For EloquentSurveillance in Motion.h detection is based on calculation of number of pixel difference (Red part of image _image.m_pMCUBufR ) of decoded jpg.

If You do not know what of them (motion detectors) you want to use, or if You want to implement Your own then ( Like I used to do):
answer for question:

• does detection area have selected "Region of interest" ( indoor area/ driveway etc)
• does it have trees in background (during windy weather a lot of movement , clouds etc)
• type of Night lighting and if it is activated by day clock ( mostly for shades appearance)
• does motion detector will activate video recording or just do series of photo
• if camera will be fixed in place or moving.
• if there are other significant parameters affecting detection

Save some photo from your esp32cam (series) in tests conditions, play with them with python open-CV ( for example Visualize differences between two images ) ant other tutorials to learn the simplest methods (mostly calculation and counting some sort of difference in pixels) ( play with "converting to greyscale, or just take R-part of image etc, use different morphology operation) - keep it simple for simple porting code to upython.