fuzzy/torn selection region

[nmz787-intel]

when using the example GLTF, disabling all layers, then selecting just nwell, and then clicking a shape and zooming in, the region (presumably part of a particular cell, given all cells are still show according to the GUI) is torn and fuzzy, and this changes with 3D rotation:

[algofoogle]

This is a 3D rendering artefact called “Z fighting” when polygons overlap exactly (or close enough). There is probably not much that can be done about this, because of how the ability to do cell selection competes with how the GDS itself is structured. Maybe it’s possible to do something like:

• Applying a tiny offset on the “visual depth” of the selected cell at render time, so it is always calculated to be slightly closer (at least in the Z buffer) than the geometry it is fighting with… if that’s even possible (?); OR
• Changing the rendering order so that the selected cell always clears the Z buffer and renders each of its layers after the same respective layer of all other geometry at the same level, effectively “painting over” the non-selected stuff (and specifically on a layer-by-layer basis to prevent it from “leaking through” layers at a different level); OR
• Maybe instead rendering the whole selected cell (all layers at once) after everything else is rendered, and doing it with some transparency so as to “tint” the regions of geometry that would be selected on top of all higher layers too; OR
• Doing some other effect like an overlaid wireframe render of all the geometry to show selection, instead of solid green polygons…
  … but to be honest these are all a lot of mucking around and probably not worth the trouble :)

[algofoogle]

Actually maybe my first idea could trivially be implemented, and it's called "depth biasing" or a polygonOffset -- this could be applied to all geometry in the current selection (and I think it would need to be negative values, i.e. closer to the camera, so it "wins" over non-selected geometry). I will try to look into this more.

[algofoogle]

The commented-out lines shown below sort of do this:

function highlightObject(graph_node) {
  highlighted_objects.push(graph_node);

  const cell = GDS.cells[graph_node.cell_name];

  for (let i = 0; i < cell.meshes_names.length; i++) {
    const color = new THREE.Color();
    const instancedMesh = GDS.meshes[cell.meshes_names[i]].threejs_instanced_mesh;
    // const original_mat = instancedMesh.material;
    // const biased_mat = original_mat.clone();
    // biased_mat.polygonOffset = true;
    // biased_mat.polygonOffsetFactor = -1;
    // biased_mat.polygonOffsetUnits = -1;
    // instancedMesh.material = biased_mat;
    // instancedMesh.userData.original_material = original_mat;

    instancedMesh.getColorAt(graph_node.instanced_mesh_idx, color);
    highlighted_prev_colors.push(color.clone());

    instancedMesh.setColorAt(graph_node.instanced_mesh_idx, highlight_color);
    instancedMesh.instanceColor.needsUpdate = true;
  }
}

function turnOffHighlight() {
  for (let i = 0; i < highlighted_objects.length; i++) {
    const graph_node = highlighted_objects[i];

    const cell = GDS.cells[graph_node.cell_name];
    for (let i = 0; i < cell.meshes_names.length; i++) {
      const instancedMesh = GDS.meshes[cell.meshes_names[i]].threejs_instanced_mesh;
      instancedMesh.setColorAt(graph_node.instanced_mesh_idx, highlighted_prev_colors[i].clone());
      instancedMesh.instanceColor.needsUpdate = true;
      // instancedMesh.material = instancedMesh.userData.original_material;
    }
  }
  highlighted_objects = [];
  highlighted_prev_colors = [];
}

This is me fumbling with Three.js (which I haven't used before) and I suspect these problems:

1. It shouldn't have to generate a new material every time. There's probably just 1 original material that needs to be cloned and modified, then shared around.
2. Likewise, doing it the way I showed keeps making new clones and doesn't clean them up.
3. Because all the meshes are instanced, every instance of the same cell (as the selection) ALSO gets this effect applied, which means that adjacent cells of the same type have the same Z-fighting issue (though others do not, which is why I say this "sort of" works).

I suspect the solution to point 3 is to replace the selected cell with a copy of the mesh (instead of an instance) and apply the biased material to that copy... while hiding (or deleting) the selected instance... and then switching things back again in turnOffHighlight().