|
54 | 54 |
|
55 | 55 | Focusing on the decimal result, consider this example: |
56 | 56 |
|
57 | | -bin_prev: 0x10000000000003 * 2^23 |
58 | | -bin: 0x10000000000004 * 2^23 |
59 | | -bin_next: 0x10000000000005 * 2^23 |
| 57 | +bin_prev: 0x1000000000001D * 2^23 |
| 58 | +bin: 0x1000000000001E * 2^23 |
| 59 | +bin_next: 0x1000000000001F * 2^23 |
60 | 60 |
|
61 | 61 | ulp = 8.388608 * 10^6 |
62 | | -dec_prev: 37778931862957186.875392 * 10^6 |
63 | | -dec: 37778931862957195.264000 * 10^6 |
64 | | -dec_next: 37778931862957203.652608 * 10^6 |
| 62 | +dec_prev: 37778931862957404.979200 * 10^6 |
| 63 | +dec: 37778931862957413.367808 * 10^6 |
| 64 | +dec_next: 37778931862957421.756416 * 10^6 |
65 | 65 |
|
66 | | - d0 d1 c u1 u0 next |
67 | | - ----|----|----|----|----|-*--|----|----|----|----|----|----|----|----|----|--*-|--- |
68 | | - 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 |
| 66 | + d0 d1 c u1 u0 next |
| 67 | + ----|----|----|----|----|-*--|----|----|----|----|----|----|----|--*-|---- |
| 68 | + 9 0 1 2 3 4 5 6 7 8 9 0 1 2 |
69 | 69 | |___________________|___________________| |
70 | 70 | 1ulp |
71 | 71 |
|
72 | | -The 1ulp falls in [1.0, 10.0), the acceptable range for dec is +/-0.5ulp. |
| 72 | +The 1ulp falls in [1.0, 10.0), the acceptable range for dec is +/-0.5ulp: |
| 73 | +dec - 0.5ulp: 37778931862957409.173504 * 10^6 |
| 74 | +dec + 0.5ulp: 37778931862957417.562112 * 10^6 |
73 | 75 |
|
74 | 76 | Rounding dec to an integer involves these cases: |
75 | 77 | 1. Fractional part < x.5: round down by 1 (as shown) |
|
0 commit comments