Providing validation function with ValidateNotSelfIntersecting test successfully prevents creating invalid geometries when finishing drawing, but it has no effect on editing in polygon mode.
Reproduce:
Provide validation function for polygon mode in options
Change existing polygon into self-intersecting shape
Sandbox: https://jsbin.com/deqisir/edit?html,output