How to use Segment Mesh in MeshLib Python

[ParvGuptaSpCT]

Hello Devs,
I want to segment the mesh using the 2 Point method shown in this video Teeth Segmentation with Segment Mesh Tool can you guide me though the process?

Thanks

[Grantim]

Hello!

You might find this #1627 helpful.
In general the idea there is to build "Surrounding Contour" around picked pointes and then separate its "left" part with respect to possible tunnels.

Also it might be worth mentioning that we use this metric for contouring:

MeshLib/source/MRMesh/MREdgeMetric.h

Lines 32 to 37 in f3c4381

	/// returns edge's metric that depends both on edge's length and on the angle between its left and right faces
	/// \param angleSinFactor multiplier before dihedral angle sine in edge metric calculation (positive to prefer concave angles, negative - convex)
	/// \param angleSinForBoundary consider this dihedral angle sine for boundary edges;
	/// this metric is symmetric: m(e) == m(e.sym())
	[[nodiscard]] MRMESH_API EdgeMetric edgeCurvMetric( const Mesh & mesh, float angleSinFactor = 2, float angleSinForBoundary = 0 );
	[[nodiscard]] MRMESH_API EdgeMetric edgeCurvMetric( const MeshTopology& topology, const VertCoords& points, float angleSinFactor = 2, float angleSinForBoundary = 0 );

[ParvGuptaSpCT]

Hello @Grantim,
thanks for the help, it seems to be working for me

Parv

[ParvGuptaSpCT]

Hello @Grantim
While trying the suggested method I am facing following issues

The surrounding contour is covering a bigger area than expected while using default settings

1. While using only 2 points

the input points for this result are shown in blue

points = [{'_x':` 14.166236408203876, '_y': -8.929631832164553, '_z':-1.4242551689431409 }, {'_x': 8.53928811041473, '_y': -8.55186543358381, '_z':-7.265178806037206 }]

Although while trying on MeshInspector UI I was able to get the expected output with the same points by changing the direction, but in finding correct direction in code is where i'm stuck.

2. Using 4 or more points with z+ direction

points = [
    {"_x": 8.404852014189412,"_z": -7.065228515801127,"_y": -8.234302203598272}, {"_x": 12.351485832771028,"_z": -2.4020179645101445,"_y": -12.974353002901431}, {"_x": 12.599123574750896,"_z": -0.5432661122871352,"_y": -11.400677276978216}, {"_x": 14.356418637909899,"_z": -1.4822625020066695,"_y": -9.218963694141813}, {"_x": 10.24002311068657,"_z": 1.3388566768771923,"_y": -7.883055938853044}, {"_x": 8.794909154245008,"_z": -0.26603828381105643,"_y": -6.867481766630053}, {"_x": 8.246684539477961,"_z": -3.753572629940942,"_y": -6.858298782153508}
]

The goal is to segment the middle tooth
Is there any way to tackle this issue with out the change in direction or are there any other settings i can change?

you can find the mesh here

here's the code snippet for reference:

# add key vertices
keyVerts = mm.std_vector_Id_VertTag()
        
for point in points:
    pt = mesh.getClosestVertex(mm.findProjection(
            mm.Vector3f(point["_x"], point["_z"], point["_y"]),
            mesh
        ).mtp)
    keyVerts.append(pt)

# setting edge curve metric
metric = mm.edgeCurvMetric(mesh)

# setting orthogonal direction
direction = mm.Vector3f(0, 0, 1 if jawType == "max" else -1)

# creating cutting contour
contour = mm.surroundingContour(
    mesh,
    keyVerts,
    metric,
    direction
)

# cut mesh
segmentFaces = mm.fillContourLeftByGraphCut(
    mesh.topology,
    contour,
    metric
)

[Grantim]

Hello @ParvGuptaSpCT !

Looks like changing amount of points will not really help here, knowing the correct direction is important for this algorithm.
Usually direction is taken from previous workflow stages, or input by user (or obtained by AI if there is one).

You can surely try to guess the direction but it will not work in all cases:

# setting orthogonal direction
meshDir = mm.Vector3f(mesh.dirArea())
pointsDir = mesh.points[keyVerts[0]] - mesh.points[keyVerts[1]]
guessDir = mm.cross(mm.cross(pointsDir,meshDir),pointsDir)
if mm.dot(guessDir,meshDir) < 0:
    guessDir = -guessDir

direction = guessDir.normalized()

(it seems to work in this case, but it is not a robust solution)

---

Also, it looks like _y and _z should be swapped in the code sample

[ParvGuptaSpCT]

Hello @Grantim!
Thank for the quick reply!
I swapped the _y and _z was swapped in the input but i guess i missed to change that in the code.

[ParvGuptaSpCT]

Hello @Grantim!
Thanks for the solution, it seems to be working on cases i have tried.

Although i saw some case where even on MeshInstector UI I am unable to get the correct segment (small chunk of the desired mesh is missing).

You can find the meshes here

mesh_1 points:

[{'_x': 45.78063438730129, '_y': -1.5167375525175615, '_z': -16.721158105003376}, {'_x': 43.01257131635327, '_y': -1.9243602634587136, '_z': -27.03330569302635}]

mesh_2 points:

[{'_x': 42.655812567397604, '_y': -1.4706659441365844, '_z': -19.293636027480318}, {'_x': 39.587005358274496, '_y': -2.8878166177394893, '_z': -28.706359335489566}]

[{'_x': 32.88501852589948, '_y': -3.303556440717479, '_z': -14.175732769431654}, {'_x': 28.528823620579416, '_y': -4.025835641475606, '_z': -24.508905703968132}]

PS: _y and _z are correct unlike last time

[Grantim]

Hello!
This is known issue, sometimes it is hard to segment molars with just two points, that is why in MeshInspector UI we allow to add 3rd point to segmentation.
Tuning direction and two points might work sometimes, but it is not robust

[ParvGuptaSpCT]

Thanks @Grantim!!

[Ruturaj-Arolkar]

Hi Grantim, what does "Tuning direction " in this case mean ?
Flipping the vertical axis of projection or the direction from clockwise to anticlockwise traversal between the 2 points over the mesh ?

[Grantim]

Hello @Ruturaj-Arolkar

By "Tuning" I have meant small changes in coordinates.

[Ruturaj-Arolkar]

Hello @Grantim,
Is there a way to capture the vertex data of a specific color (Blue in the attached image) from the visualize curvature tool ?
The goal is to use that array to predict the path for the half segmented molars shown in the above images.

[Grantim]

Hello!

You sure can use mrmeshnumpy.getNumpyMeanCurvature to get curvature for all vertices, and then accumulate VertBitSet with desired vertices, but in general edgeCurvMetric already takes into account surface curvature, so I don't think that will make much difference.

You can try playing with edgeCurvMetric parameters to fine tune segmentation metric