Skip to content

Commit 84d39b6

Browse files
authored
ExecutionTest::UnaryHalfOpTest#AcosHalf: Update tolerance (microsoft#6690)
Update tolerance for ExecutionTest::UnaryHalfOpTest#AcosHalf test. Enables implementations to calculate `acos` for fp16 type by converting to fp32, doing fp32 math, and then converting back to fp16 using round-to-nearest-even conversing (RTNE) per D3D11 spec. For more details please see issue microsoft#6179. As mentioned in the linked issue, for these floating point tests a fixed point tolerance does not really make sense. It should vary based on the magnitude of the expected value. But we are already using this approach in many similar test cases and the simplest fix now is to update the tolerance to accommodate the fp32-to-ft16 conversion. Fixes microsoft#6179
1 parent 4353db3 commit 84d39b6

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

tools/clang/unittests/HLSLExec/ShaderOpArithTable.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1562,7 +1562,7 @@
15621562
</Row>
15631563
<Row Name="AcosHalf">
15641564
<Parameter Name="Validation.Type">Epsilon</Parameter>
1565-
<Parameter Name="Validation.Tolerance">0.0008</Parameter>
1565+
<Parameter Name="Validation.Tolerance">0.0010</Parameter>
15661566
<Parameter Name="ShaderOp.Text"> struct SUnaryFPOp {
15671567
float16_t input;
15681568
float16_t output;

0 commit comments

Comments
 (0)