calling gesture recognizer model and getting results

[vsingh1986]

Hi Thanks for the mediapipe lib, i had successfuly build the library and integrated it in my project, i am also able to run your example. In my project i am able to send the frames for landmark detection and create landmarks on the frame.

I do not have software background, and i am doing this for personal use out of pure curiosity.

I want to recognize the hand gesture, in the sample python code of mediapipe they provide a method of sending the frames to a model "gesture_recognizer.task" here is the python code snippets for it

#python code for initializing the gesture recognizer
num_hands = 2
model_path = "C:\Users\vivek\Downloads\gesture_recognizer.task"
GestureRecognizer = mp.tasks.vision.GestureRecognizer
GestureRecognizerOptions = mp.tasks.vision.GestureRecognizerOptions
VisionRunningMode = mp.tasks.vision.RunningMode
self.lock = threading.Lock()
self.current_gestures = []
options = GestureRecognizerOptions(
base_options=python.BaseOptions(model_asset_path=model_path),
running_mode=VisionRunningMode.LIVE_STREAM,
num_hands = num_hands,
result_callback=self.__result_callback)
recognizer = GestureRecognizer.create_from_options(options)

#python code for getting the gestures

 if results.multi_hand_landmarks:
            for hand_landmarks in results.multi_hand_landmarks:
                mp_drawing.draw_landmarks(frame, hand_landmarks, mp_hands.HAND_CONNECTIONS)
                mp_image = mp.Image(image_format=mp.ImageFormat.SRGB, data=np_array)
                recognizer.recognize_async(mp_image, timestamp)  #this line calls the recognizer
                timestamp = timestamp + 1 # should be monotonically increasing, because in LIVE_STREAM mode
             self.put_gestures(frame)

Can you please tell me how to implement this part in your library. I have been trying but i am stuck as i do not technically understand your code and whats happening in it.

Thanks for the help.