[feature request] Face restoration algorithms #187
Description
I use a model called codeformer (based on esrgan I think) that's trained specifically on human faces. Combined with a bit of scripting the included tool scans for faces, "upscale" them (basically generating a new face with the same structure) and pastes it back onto the original image. This winds up pretty wobbly so a stabilization type filter (like vs_Temporalfix) tends to clean this up a bit.
This makes a pretty drastic improvement in quality for real life video upscaling when the original video is SD or lower res.
It's also pretty funny when it hallucinates and does weird shit.
Topaz AI and a few other tools have been using similar methods for years in their products, but tend to hide them behind super pricey subscription models.
There's a few of the face restoration models out now, most of them have pretty permissible licenses too. I think this could do wonders for this tool.
Image below is pretty accurate with my results as well.
CodeFormer - https://github.com/sczhou/CodeFormer S-Lab Licence. (not very permissive tbh)
GFPGAN - https://github.com/TencentARC/GFPGAN - Apache Licence, (more open, also 95% the same as codeformer)
vs_temporalfix - https://github.com/pifroggi/vs_temporalfix - MIT Licence (getting this to run with your program would probably be more effort than it's worth imho)