Realtime streaming of the Sadtalker

[MalikTayyabTanveer]

Hello everyone,

I'm working on real-time streaming avatars and have tested "pirender" model on my Mac. It's impressively fast. However, I was wondering if it's possible to generate video in real time using "pirender."

I noticed that during each inference, the following steps are processed:

• 3DMM Extraction for the source image
• mel
• audio2exp
• FaceRender

If we use a single image for every inference, would running only FaceRender be enough to achieve real-time performance? Am I correct in thinking this? or how we make the SadTalker real-time.

Does anyone have suggestions or ideas on how to make this fully real-time? If you're interested in discussing real-time avatars, feel free to join the conversation!