Why facial landmarks are not recognized correctly

[rezalahmi]

I think something is wrong!!!! the model can detect 68 landmarks but for more landmarks and mesh, generate wrong landmarks

[sophiajwagner]

Hi, could you solve this issue? I have the same problem.

[rezalahmi]

Hi, could you solve this issue? I have the same problem.

Hi, Unfortunately no

[t-martyniuk]

Hi!
@rezalahmi , is it the demo.py script you're running, but with different type_of_output values? are you sure that the input to both inference runs is the same?

I think the problem is that the model expects a loose head crop (see Section 4 "Method" of the paper), and here the image you're using has much more space around the head than it's required. The demo script uses different branches of the network for the "68 points" type of output and all the other subsets of points (2D branch and reprojections of 3D branch, respectively). While the 2D branch is quite robust to different inputs, - the 3D branch is quite sensitive to the head crop provided.
Hope this helps to understand better the discrepancy in the results.
For better results, please use either off-the-shelf or custom head detector, and use offset of ~10% (I refer you to

DAD-3DHeads/model_training/data/flame_dataset.py

Line 94 in 3acc5c2

def _parse_anno(self, item_anno: Dict[str, Any]) -> Dict[str, Any]:

for the context, L96-99).

[kllaodong]

I cropped the other test images by referring to Example Figure 1.jpeg, and the detection of face landmarks was relatively more accurate. However, the detection accuracy of side faces is relatively low.