I found that flattenedPointsAreClockwise may return wrong value for self-intersecting polygons.
For example it returns true for:
(0,0) (9,0) (0,10) (10,10)
But returns false for:
0,0) (11,0) (0,10) (10,10)
To be more specific - the returned values says whether the polygon is "mostly clockwise" or not. It is not information that all neighbouring edges are clockwise. And for self-intersecting points orientation is undefined.
Suggestion: add
assert(pointsAreSelfIntersecting(points) == false)
More informations:
https://en.wikipedia.org/wiki/Curve_orientation