unstable faceswap / swims /

[bolekcg]

Hello community .
First of all thanks for all to creators .

I did a small test with reactor but i've got few question tho

reactor_result.mp4

1: Why my face is sliding over ? How to overcome that issue ?
2. Why my face is blurred on some frames and other times it's pretty sharp.

Thank you in advance for all suggestions.

[Buumcode]

1: Why my face is sliding over?

Reactor uses "CROP AND PASTE (BLEND) FACE TECHNOLOGY" in face swapping. You can look how it's work here: https://www.photopea.com/tuts/swap-faces-online/
or here:
https://www.youtube.com/watch?v=lz285umjn-g

This technology working good ONLY on faces which look straight forward (on you). If face looking to the side (90° turn) - it's not work at all.

How to overcome that issue?

Try to use Inswapper-128 + GPEN-BFR-512 (with face_restore_visibility 0.70-0.80) - it's gives maximum quality with turned to the side faces, as in your example file (45° turn face).

2. Why my face is blurred on some frames and other times it's pretty sharp.

It's lack of technology. Face Swapping (crop and paste face) is unstable.

a) Face size is matter (Inswapper-128 maked for 128x128 pixels face size) - see picture below.
b) Face turn is matter.
c) And you will always have visible difference between frames (twitching) in video face swapping - because every frame in video is IMAGE and have swapped personally by program, one-by-one.

Here is 128 pixels square:

[bolekcg]

Thanks @Buumcode for stopping by!

Ad1. Yeah ... I'm aware how thats's working in basics ( crop/paste )... but I thought there's some additional going on under the REActor nodes

Try to use Inswapper-128 + GPEN-BFR-512 (with face_restore_visibility 0.70-0.80) - it's gives maximum quality with turned to the side faces, as in your example file (45° turn face).

I will check that ... don't remember well my settings for this one as I did many of them

It's lack of technology. Face Swapping (crop and paste face) is unstable.

As mentioned above i thought maybe it's using something more than that ....

That way or that way thanks for all answers ... just a last question ...
What do you think would be good idea to now polish that faceswap ( because it's very fast and i do l) in terms of consistency what would be your suggestion to get best stable result and speed of REActor ??

I already tried MOCHA examples... but results differs as mentioned here MOCHA

Thanks again.

[Buumcode]

I'm aware how thats's working in basics ( crop/paste )... but I thought there's some additional going on under the REActor nodes.

Yes, model (face swapping model) do WARP face - turn it to side on needed degree, not just straight paste as in Photoshop. Thats all.

What do you think would be good idea

In often, I can't get good result even on image2image face swap. No word about video. Video is fantastic goal.
Thanks to link on Mocha, never heard about them. Did take a look later.