[css-borders-4] Constrain radii for concave opposite corners (#12272)

* [css-borders-4] Constrain radii for concave opposite corners

If the hulls of opposite corners overlap, find a scale factor that would prevent the overlap, and apply to all radii.

Resolution: #12098 (comment)
Closes #12098

* Fix hull computation for convex/concave

Author: noamr

css-borders-4/Overview.bs

@@ -394,7 +394,8 @@ Each shadow of [=/element=]'s 'box shadow' is shaped by the [=border contour pat
 To compute an [=/element=] |element|'s <dfn>border contour path</dfn> given an an [=edge=] |targetEdge| and an optional number |spread| (default 0):
 	1. Let |outerLeft|, |outerTop|, |outerRight|, |outerBottom| be |element|'s [=unshaped edge|unshaped=] [=border edge=].
 	1. Let |topLeftHorizontalRadius|, |topLeftVericalRadius|, |topRightHorizontalRadius|, |topRightVerticalRadius|, |bottomRightHorizontalRadius|,
-		|bottomRightVerticalRadius|, |bottomLeftHorizontalRadius|, and |bottomLeftVerticalRadius| be |element| [=border edge=]'s radii.
+		|bottomRightVerticalRadius|, |bottomLeftHorizontalRadius|, and |bottomLeftVerticalRadius| be |element| [=border edge=]'s radii,
+		scaled by |element|'s [=opposite corner scale factor=].
 	1. Let |topLeftShape|, |topRightShape|, |bottomRightShape|, and |bottomLeftShape| be |element|'s [=computed value|computed=] 'corner-*-shape' values.
 	1. Let |targetLeft|, |targetTop|, |targetRight|, |targetBottom| [=unshaped edge|unshaped=] |targetEdge|.
 	1. Let |path| be a new path [[SVG2]].
@@ -503,6 +504,47 @@ To compute the <dfn>corner path</dfn> given a rectangle |cornerRect|, a rectangl
 	1. Return |cornerPath|.
 </div>
 
+<h4 id=corner-shape-constrain-radii>
+Constraining opposite radii</h4>
+
+When concave 'corner-shape' values are present (the [=superellipse parameter=] is negative), diagonally opposite corners might overlap each other.
+
+<div class="example">
+	<p>The following example would create overlapping corners if not constrained.</p>
+	<pre class="lang-css">
+		div {
+			corner-shape: scoop;
+			border-top-left-radius: 80%;
+			border-bottom-right-radius: 80%;
+		}
+	</pre>
+</div>
+
+To prevent this, the four radii are constrained to prevent overlaps.
+This is done by computing a hull polygon for each of the opposite corners, and finding the highest downscale factor which, if applied to both corners, would make it so that the polygons would not intersect.
+
+<div algorithm="constrain-radii-for-concave-corner-shape">
+To compute the <dfn>opposite corner scale factor</dfn> given an [=/element=] |element|:
+	1. Let |rect| be |element|'s [=border box=].
+	1. Let |topRightHull| be a the [=normalized inner corner hull=] given |element|'s [=computed value|computed=] 'corner-top-right-shape',
+		mapped to the rectangle (|rect|'s [=width dimension=] - |element|'s [=computed value|computed=] horizontal 'border-top-right-radius', 0, |rect|'s [=computed value|computed=] 'border-top-right-radius').
+	1. Let |bottomRightHull| be a the [=normalized inner corner hull=] given |element|'s [=computed value|computed=] 'corner-bottom-right-shape',
+		rotated by 90deg with (0.5, 0.5) as an origin,
+		and mapped to the rectangle (|rect|'s [=width dimension=] - |element|'s [=computed value|computed=] horizontal 'border-bottom-right-radius', |rect|'s [=height dimension=] - |element|'s [=computed value|computed=] vertical 'border-bottom-right-radius',
+		|element|'s [=computed value|computed=] 'border-bottom-right-radius').
+	1. Let |bottomLeftHull| be a the [=normalized inner corner hull=] given |element|'s [=computed value|computed=] 'corner-bottom-right-shape',
+		rotated by 180deg with (0.5, 0.5) as an origin,
+		and mapped to the rectangle (0, |rect|'s [=height dimension=] - |element|'s [=computed value|computed=] vertical 'border-bottom-left-radius',
+		|element|'s [=computed value|computed=] 'border-bottom-left-radius').
+	1. Let |topLeftHull| be a the [=normalized inner corner hull=] given |element|'s [=computed value|computed=] 'corner-top-left-shape',
+		rotated by 270deg with (0.5, 0.5) as an origin,
+		mapped to (0, 0, |element|'s [=computed value|computed=] 'border-top-left-radius').
+	1. Let |scaleFactorA| be the highest number which, if both |topLeftHull| and |bottomRightHull| were scaled by, using their first point as the origin, those polygons would not intersect.
+	1. Let |scaleFactorB| be the highest number which, if both |topRightHull| and |bottomLeftHull| were scaled by, using their first point as the origin, those polygons would not intersect.
+	1. Return <code>min(1, |scaleFactorA|, |scaleFactorB|)</code>.
+</div>
+
+
 <h4 id=corner-shape-value>
 'corner-shape' values</h4>
 
@@ -660,6 +702,20 @@ To compute the <dfn>normalized superellipse half corner</dfn> given a [=superell
 		1. Return |convexHalfCorner|.
 </dl>
 
+<div algorithm="superellipse-param-to-hull">
+To compute the <dfn>normalized inner corner hull</dfn> given a [=superellipse parameter=] |curvature|:
+	1. If |curvature| is greater than or equal to zero, return a triangle betwen « (1, 1), (1, 0), (0, 1) ».
+	1. Let |axisLineA| be a line between <code>(1, 0)</code> and <code>(1, 1)</code>.
+	1. Let |axisLineB| be a line between <code>(0, 1)</code> and <code>(1, 1)</code>.
+	1. Let |normalizedHalfCorner| be the [=normalized superellipse half corner=] given |curvature|.
+	1. Let |halfCornerPoint| be <code>(|normalizedHalfCorner|, 1 - |normalizedHalfCorner|)</code>.
+	1. Let |lineFromCenterToHalfCorner| be a line between <code>(0, 0)</code> and |halfCornerPoint|.
+	1. Let |tangentLine| be the line perpendicular to |lineFromCenterToHalfCorner|, at |halfCornerPoint|.
+	1. Let |intersectionA| be the intersection between |axisLineA| and |tangentLine|.
+	1. Let |intersectionB| be the intersection between |axisLineB| and |tangentLine|.
+	1. Return a pentagon between the points « (1, 1), (1, 0), |intersectionA|, |intersectionB|, (0, 1), (1, 1) ».
+</div>
+
 To interpolate a [=superellipse parameter=] |s| to an interpolation value between 0 and 1, return the [=normalized superellipse half corner=] given |s|.
 
 To convert a <<number [0,1]>> |interpolationValue| back to a [=superellipse parameter=], switch on |interpolationValue|: