|
| 1 | +# CWE-230: Improper Handling of Missing Values |
| 2 | + |
| 3 | +In python, some datasets use NaN (not-a-number) to represent the missing data. This can be problematic as the NaN values are unordered. The NaN value should be stripped before as they can cause surprising or undefined behaviours in the statistics functions that sort or count occurrences [[2024 doc.python.org]](https://docs.python.org/3/library/statistics.html) Any ordered comparison of a number to a not-a-number value are False. A counter-intuitive implication is that not-a-number values are not equal to themselves. |
| 4 | + |
| 5 | +This behavior is compliant with IEEE 754[[2024 Wikipedia]](https://en.wikipedia.org/wiki/IEEE_754) a hardware induced compromise. |
| 6 | +The [example01.py](example01.py) code demonstrates various comparisons of float('NaN') all resulting in False |
| 7 | +```python |
| 8 | +""" Code Example """ |
| 9 | + |
| 10 | +foo = float('NaN') |
| 11 | +print(f"foo={foo} type = {type(foo)}") |
| 12 | + |
| 13 | + |
| 14 | +print(foo == float("NaN") or |
| 15 | + foo is float("NaN") or |
| 16 | + foo < 3 or |
| 17 | + foo == foo or |
| 18 | + foo is None |
| 19 | + ) |
| 20 | + |
| 21 | +``` |
| 22 | +## Non-Compliant Code Example |
| 23 | + |
| 24 | +This noncompliant code example [[2024 docs.python.org]](https://docs.python.org/3/reference/expressions.html#value-comparisons) attempts a direct comparison with NaN in |
| 25 | + |
| 26 | +_value == float("NaN"). |
| 27 | + |
| 28 | +*[noncompliant01.py](noncompliant01.py):* |
| 29 | + |
| 30 | +```python |
| 31 | +""" Non-compliant Code Example """ |
| 32 | + |
| 33 | + |
| 34 | +def balance_is_positive(value: str) -> bool: |
| 35 | + """Returns True if there is still enough value for a transaction""" |
| 36 | + _value = float(value) |
| 37 | + if _value == float("NaN") or _value is float("NaN") or _value is None: |
| 38 | + raise ValueError("Expected a float") |
| 39 | + if _value <= 0: |
| 40 | + return False |
| 41 | + else: |
| 42 | + return True |
| 43 | + |
| 44 | + |
| 45 | +##################### |
| 46 | +# attempting to exploit above code example |
| 47 | +##################### |
| 48 | +print(balance_is_positive("0.01")) |
| 49 | +print(balance_is_positive("0.001")) |
| 50 | +print(balance_is_positive("NaN")) |
| 51 | + |
| 52 | +``` |
| 53 | + |
| 54 | +The balance_is_positive method returns True for all 3 cases instead of throwing an ValureError exception for balance_is_positive("NaN") |
| 55 | + |
| 56 | +## Compliant Solution |
| 57 | + |
| 58 | +The `compliant01.py` the method Decimal.quantize is used to gain control over known rounding errors in floating point values. |
| 59 | + |
| 60 | +The decision by the balance_is_positive method is to ROUND_DOWN instead of the default ROUND_HALF_EVEN. |
| 61 | + |
| 62 | +*[compliant01.py](compliant01.py):* |
| 63 | + |
| 64 | +```python |
| 65 | +# SPDX-FileCopyrightText: OpenSSF project contributors |
| 66 | +# SPDX-License-Identifier: MIT |
| 67 | +""" Compliant Code Example """ |
| 68 | +from decimal import ROUND_DOWN, Decimal |
| 69 | + |
| 70 | + |
| 71 | +def balance_is_positive(value: str) -> bool: |
| 72 | + """Returns True if there is still enough value for a transaction""" |
| 73 | + # TODO: additional input sanitation for expected type |
| 74 | + _value = Decimal(value) |
| 75 | + # TODO: exception handling |
| 76 | + return _value.quantize(Decimal(".01"), rounding=ROUND_DOWN) > Decimal("0.00") |
| 77 | + |
| 78 | + |
| 79 | +##################### |
| 80 | +# attempting to exploit above code example |
| 81 | +##################### |
| 82 | +print(balance_is_positive("0.01")) |
| 83 | +print(balance_is_positive("0.001")) |
| 84 | +print(balance_is_positive("NaN")) |
| 85 | + |
| 86 | +``` |
| 87 | + |
| 88 | +Decimal throws a decimal.InvalidOperation for NaN values, the controlled rounding causes only "0.01" to return True. |
| 89 | + |
| 90 | +In `compliant02.py` we use the math.isnan to very if the value passed is a valid float value. |
| 91 | + |
| 92 | +*[compliant02.py](compliant02.py):* |
| 93 | + |
| 94 | +```python |
| 95 | +# SPDX-FileCopyrightText: OpenSSF project contributors |
| 96 | +# SPDX-License-Identifier: MIT |
| 97 | +""" Compliant Code Example """ |
| 98 | +import math |
| 99 | + |
| 100 | + |
| 101 | +def balance_is_positive(value: str) -> bool: |
| 102 | + """Returns True if there is still enough value for a transaction""" |
| 103 | + _value = float(value) |
| 104 | + if math.isnan(_value) or _value is None: |
| 105 | + raise ValueError("Expected a float") |
| 106 | + if _value <= 0: |
| 107 | + return False |
| 108 | + else: |
| 109 | + return True |
| 110 | + |
| 111 | + |
| 112 | +##################### |
| 113 | +# attempting to exploit above code example |
| 114 | +##################### |
| 115 | +print(balance_is_positive("0.01")) |
| 116 | +print(balance_is_positive("0.001")) |
| 117 | +print(balance_is_positive("NaN")) |
| 118 | + |
| 119 | +``` |
| 120 | + |
| 121 | +The balance_is_poitive method will raise an ValueError for NaN values. |
| 122 | + |
| 123 | +## Automated Detection |
| 124 | + |
| 125 | +|Tool|Version|Checker|Description| |
| 126 | +|:----|:----|:----|:----| |
| 127 | +|Bandit|1.7.4 on Python 3.10.4|Not Available|| |
| 128 | +|flake8|flake8-4.0.1 on python 3.10.4||FS002 '.format' used| |
| 129 | + |
| 130 | +## Related Guidelines |
| 131 | + |
| 132 | +||| |
| 133 | +|:---|:---| |
| 134 | +|[SEI CERT Coding Standard for Java](https://wiki.sei.cmu.edu/confluence/display/java/SEI+CERT+Oracle+Coding+Standard+for+Java)|[IDS06-J. Exclude unsanitized user input from format strings](https://wiki.sei.cmu.edu/confluence/display/java/IDS06-J.+Exclude+unsanitized+user+input+from+format+strings)| |
| 135 | +|[ISO/IEC TR 24772:2013](https://wiki.sei.cmu.edu/confluence/display/java/Rule+AA.+References#RuleAA.References-ISO/IECTR24772-2013)|Injection RST| |
| 136 | +|[MITRE CWE Pillar](http://cwe.mitre.org/)|[CWE-703: Improper Check or Handling of Exceptional Conditions (mitre.org)](https://cwe.mitre.org/data/definitions/703.html)| |
| 137 | +|[MITRE CWE Pillar](http://cwe.mitre.org/)|[CWE-230: Improper Handling of Missing Values](https://cwe.mitre.org/data/definitions/230.html)| |
| 138 | + |
| 139 | +## Bibliography |
| 140 | + |
| 141 | +||| |
| 142 | +|:---|:---| |
| 143 | +|[[Python 3.10.4 docs]](https://docs.python.org/3/library/string.html#formatstrings)|Format String Syntax. Available from: <https://docs.python.org/3/library/string.html#formatstrings> \[Accessed 22 July 2025]| |
| 144 | +|[Python docs](https://docs.python.org/3/)|<https://docs.python.org/3/library/math.html#math.nan> \[Accessed 22 July 2025]| |
| 145 | +|[Python docs](https://docs.python.org/3/)|Python Value comparisons<https://docs.python.org/3/reference/expressions.html#value-comparisons> \[Accessed 22 July 2025]| |
| 146 | +|[[Wikipedia 2024]](https://realpython.com/python-string-formatting/)|IEEE 754: <https://en.wikipedia.org/wiki/IEEE_754> \[Accessed 22 July 2025]| |
0 commit comments