Thoughts/Ideas/Future-Plans

[Trentonom0r3]

Starting this thread both for myself, and others to discuss thoughts and ideas regarding ezsynth.

My goal with this project is to provide a simpler way to use ebsynth video stylization, while enhancing the original methods with newer, updated processes.

[Trentonom0r3]

#15 (comment)

Dense Correspondence Map Improvement

• Initial Improvement would first, come from "inpainting" of black pixels in warped maps.
• This would need to be done in a seamless and intelligent way, in order to maintain temporal coherence.

Approach

• Naive approach is to, with each iteration of map creation, create a mask showing the black pixel areas, and inpaint them.
• A potential better approach would be to warp using both forward AND backward flow, possibly reducing black area artifacts.
• Then using a combination of either feature matching or optical flow, interpolate the missing pixel values.
• More to come

[baaoh]

Hi! Is it feasible to turn this into a Comfy UI node? I'm not sure how it would work exactly, but the nodes can receive inputs and send outputs, making it a very easy way to use code