Question about orienting mesh

[bttalisman]

I’ve got two entities:

AnchorEntityA
	- ModelEntityA

AnchorEntityB
	-ModelEntityB

These AnchorEntities each have an arbitrary position and orientation, and the ModelEntity children have an independent scale value. Each ModelEntity has a simple cube model. My goal is to subtract mesh B from mesh A. I want the Meshes oriented and scaled relative to each other just as their parent entities are in my scene.

Here's how I create the Mesh objects:

var modelEntityA = entityA.findChild(name: "scaledCubeCollisionZone", type: ModelEntity.self)!
var modelEntityB = entityB.findChild(name: "scaledCubeCollisionZone", type: ModelEntity.self)!

var meshA = Mesh.init(modelEntityA)
var meshB = Mesh.init(modelEntityB)

let newMesh = meshA.subtracting(meshB)

It looks like each Mesh object gets a Transform from its ModelEntity, but this Transform only transforms the child relative to its parent. I want the Mesh to include the parent entity's Transform in addition to the ModelEntity Transform. I've tried composing these entity transforms:

modelEntityA.transform = Transform(matrix: entityA.transform.matrix * modelEntityA.transform.matrix)
modelEntityB.transform = Transform(matrix: entityB.transform.matrix * modelEntityB.transform.matrix)

But it just fails this assertion:

extension Plane {
    init(unchecked normal: Vector, w: Double) {
        assert(normal.isNormalized)              

How should I be doing this?

[nicklockwood]

If you display the model entities with composed transforms in RealityKit (without Euclid) do the positions look correct? I think you might need to do the transform multiplication the other way around

[bttalisman]

I've got a parent Entity, which implements HasAnchoring. This is where I set the position and orientation.
Then that parent has a chilld Entity, which is loaded from a usdz file, and has a ModelEntity buried inside. This child has a little position and scale info also. Everything looks exactly as I want it.

Are you suggesting I need to reorganize my entity structure itself?

[nicklockwood]

No, I just mean that I'm not sure if multiplying the transforms as you are doing has the same effect as nesting transformed entities, so I thought that would be worth checking by removing the transform from the parent entity and applying the combined transform to the children (just temporarily, to see if it looks the same)

[bttalisman]

ahh, gotcha. Thanks, I'll try that

[bttalisman]

As I'm experimenting with this, I'm getting a lot of assertion failures regarding Vector.isNormalized. I've inspected the vector length, and it's really close to 1.0, but not close enough. Is there any way of loosening up the tolerance?

[nicklockwood]

Maybe just delete the assert for now, it should be fine

[bttalisman]

I figured out what I need to do. Looking back it seems pretty straightforward, but it definitely wasn't obvious to me. Thanks for the snappy responses.

guard let modelEntityA = entityA.findChild(name: "collisionZone", type: ModelEntity.self) else { return }
guard let modelEntityB = entityB.findChild(name: "collisionZone", type: ModelEntity.self) else { return }


// save these so we can replace them on the modelEntities when we're done
let originallTransformA = modelEntityA.transform
let originalTransformB = modelEntityB.transform

// we will need to undo the transform done to the ModelEntity
var inverseTransform = Euclid.Transform(modelEntityA.transformMatrix(relativeTo: nil).inverse)

// temporarily apply the world-coord transform to the modelEntities
// Euclid applies that transform to the mesh
modelEntityA.transform = Transform(matrix: modelEntityA.transformMatrix(relativeTo: nil))
modelEntityB.transform = Transform(matrix: modelEntityB.transformMatrix(relativeTo: nil))

var meshA = Mesh(modelEntityA)
var meshB = Mesh(modelEntityB)
meshB = meshB.withMaterial(Mesh.Material(UIColor.red))
 
var newMesh = meshA.subtracting(meshB)
newMesh.transform(by: inverseTransform)     // undo the transform applied by euclid

do {
    modelEntityA.removeFromParent()
    
    // build a new ModelEntity with the new mesh
    let newModelEntity = try ModelEntity(newMesh)
    newModelEntity.transform = originalTransformA
    modelEntityB.transform = originalTransformB
    
    entityA.addChild(newModelEntity)
                            

} catch {
}