Currently the single selecting uses Physics.RaycastAll and then picks the first valid hit. This doesn't work reliably because the order is random.
From Unity docs: "RaycastHit[] An array of RaycastHit objects. Note that the order of the results is undefined."
https://docs.unity3d.com/ScriptReference/Physics.RaycastAll.html