Skip to content

Commit a8040b2

Browse files
authored
Merge branch 'ossf:main' into jp-translation-regex1
2 parents 0127b0b + 586eae3 commit a8040b2

12 files changed

+429
-45
lines changed

docs/Concise-Guide-for-Evaluating-Open-Source-Software.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,8 @@ Licensing frameworks, while not directly security-related, significantly impact
5757
|------|-------------|:--------:|
5858
| **License Clarity** | Verify that every component has a license, that it's a widely-used [OSI license](https://opensource.org/licenses) if it's OSS, and that it's consistent with your intended use. Projects that won't provide clear license information are less likely to follow other good practices that lead to secure software. | |
5959
| **Name Verification** | Check if a similar name is more popular - that could indicate a typosquatting attack. | |
60-
| **Usage Metrics** | Assess if it has significant use. Software with many users or large users may be inappropriate for your use. However, widely-used software is more likely to offer useful information on how to use it securely, and more people will care about its security. | |
60+
| **Adoption** | Assess if the software has significant use. Widely-used software is more likely to offer useful information on how to use it securely and more people will care about its security. | |
61+
| **Suitability** | Choose software that is a good solution for your problem. Avoid [Hype Driven Development](https://blog.daftcode.pl/hype-driven-development-3469fc2e9b22): Don't choose it merely because it's used by large companies or because it's the latest fad. | |
6162

6263
## Practical Testing
6364

docs/Correctly-Using-Regular-Expressions-Rationale.md

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -477,6 +477,45 @@ Many developers believe that regex notation is the same everywhere, even though
477477

478478
Such changes would take years to adopt. Even worse, these changes might not be accepted in some cases because some people may think that merely being possible to do something is adequate. We don’t agree; we think it’s important to make it _easy_ to do the secure action, not just possible, and it’s best to make avoidable mistakes les likely. These changes require implementations in many systems and modifications of many specifications; doing this has been historically challenging. Still, such changes would reduce the likelihood of these problems worldwide.
479479

480+
#### Status of adding \A and \z across ecosystems
481+
482+
As previously noted, one start is to have a _single_ regex syntax
483+
that _always_ means "match beginning of input and "match end of input"
484+
_even_ when a multi-line mode is enabled.
485+
This notation is especially important for security, because they make it
486+
practical to use regexes for input validation.
487+
488+
Many platforms already support \A and \z respectively for beginning-of-input
489+
and end-of-input.
490+
These platforms are
491+
Perl, .NET/C#, Java, PHP, PCRE, Golang, Rust crate regex, RE2, and Ruby.
492+
493+
If the following platforms made adjustments, the notation would
494+
be nearly universal:
495+
496+
* POSIX: On 2024-04-24 the Austin Group "accepted as marked"
497+
[bug 1919](https://www.austingroupbugs.net/view.php?id=1919) the
498+
proposed change to add \A and \z to extended regular expressions (EREs).
499+
They decided to not require the older BRE syntax to support it at this time,
500+
but do plan to add a NOTE saying "a future version of this standard is likely
501+
to require such characters to be supported in a regular expression.
502+
Implementors are encouraged to provide this as an extension using
503+
"\A" for the beginning and "\z" for the end of strings as they are
504+
already in widespread use for this purpose in other languages."
505+
* ECMAScript/JavaScript: In 2021 Ron Buckton (@rbuckton) created the proposal
506+
[Regular Expression Buffer Boundaries for ECMAScript](https://github.com/tc39/proposal-regexp-buffer-boundaries)
507+
to add \A and \z to ECMAScript/JavaScript, and it advanced to stage 2,
508+
but it seems to be stuck there. We intend to see if we can help it advance.
509+
* Python: Python supports \A, but it uses the unique \Z instead of the
510+
\z used everywhere else for end-of-string.
511+
We'll ask to see if \z could be supported in addition to \Z for end-of-string.
512+
We'll probably start with a minor git request (as this is a really
513+
small change), otherwise we'll create a PEP, depending on the desires
514+
of the Python community.
515+
In current versions of Python3 a \z in a regex raises an exception, so
516+
adding \z for end-of-string would be a backwards-compatible addition.
517+
See [CPython issue 133306](https://github.com/python/cpython/issues/133306).
518+
480519
## Authors and contributors
481520

482521
We would like to thank the following contributors:
Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
# CWE-754: Improper Check for Unusual or Exceptional Conditions - Float
2+
3+
Ensure to have handling for exceptional floating-point values.
4+
5+
The `float` class has the capability to interpret various input values as floating-point numbers. Some special cases can interpret input values as
6+
7+
* Positive Infinity
8+
* Negative Infinity
9+
* `NaN` (Not-a-Number)
10+
11+
These floating-point class values represent numbers that fall outside the typical range and exhibit unique behaviors. `NaN` (Not a Number) lacks a defined order and is not considered equal to any value, including itself. Hence, evaluating an expression such as `NaN == NaN` returns `False`.
12+
13+
## Non-Compliant Code Example
14+
15+
The `noncompliant01.py` intent is to ensure that adding objects will not exceed a total weight of `100` units. Validation fails as the code to test for exceptional conditions, such as `NaN` or `infinity`, is missing.
16+
17+
*[noncompliant01.py](noncompliant01.py):*
18+
19+
```py
20+
# SPDX-FileCopyrightText: OpenSSF project contributors
21+
# SPDX-License-Identifier: MIT
22+
"""Non-compliant Code Example"""
23+
24+
import sys
25+
26+
27+
class Package:
28+
"""Class representing a package object"""
29+
30+
def __init__(self):
31+
self.package_weight: float = 0.0
32+
self.max_package_weight: float = 100.0
33+
34+
def add_to_package(self, object_weight: str):
35+
"""Function for adding an object into the package"""
36+
value = float(object_weight)
37+
# This is dead code as value gets type cast to float,
38+
# hence will never be equal to string "NaN"
39+
if value == "NaN":
40+
raise ValueError("'NaN' not a number")
41+
# This is also dead code as value is type cast to float,
42+
# unusual inputs like -infinity will not get caught
43+
if isinstance(value, float) is False:
44+
raise ValueError("not a number")
45+
if self.package_weight + value > self.max_package_weight:
46+
raise ValueError("Addition would exceed maximum package weight.")
47+
48+
self.package_weight += value
49+
50+
def get_package_weight(self):
51+
"""Getter for outputting the package's current weight"""
52+
return self.package_weight
53+
54+
55+
#####################
56+
# exploiting above code example
57+
#####################
58+
package = Package()
59+
print(f"\nOriginal package's weight is {package.get_package_weight():.2f} units\n")
60+
for item in [100, "-infinity", sys.float_info.max, "NaN", -100]:
61+
print(f"package.add_to_package({item})")
62+
try:
63+
package.add_to_package(item)
64+
print(
65+
f"package.get_package_weight() = {package.get_package_weight():.2f}\n"
66+
)
67+
68+
except Exception as e:
69+
print(e)
70+
```
71+
72+
Some important considerations when dealing with floating-point values from `non-complaint01.py`.
73+
74+
* Setting a value above `sys.float_info.max` does not increase the held value. In some cases, incrementing `package_weight` with a high enough value may turn its value into `inf`.
75+
* Setting the added value to `-infinity` and `+infinity` causes the value of the `package_weight` to be infinite as well.
76+
* Adding `"NaN"`, which is not a valid value to `package_weight` will always return `"nan"`.
77+
78+
**Example `noncompliant01.py` output:**
79+
80+
```bash
81+
Original package's weight is 0.00 units
82+
83+
package.add_to_package(100)
84+
package.get_package_weight() = 100.00
85+
86+
package.add_to_package(-infinity)
87+
package.get_package_weight() = -inf
88+
89+
package.add_to_package(1.7976931348623157e+308)
90+
package.get_package_weight() = -inf
91+
92+
package.add_to_package(NaN)
93+
package.get_package_weight() = nan
94+
95+
package.add_to_package(-100)
96+
package.get_package_weight() = nan
97+
```
98+
99+
## Compliant Solution
100+
101+
Exceptional values and out-of-range values are handled in `compliant01.py`. Some negative values are also checked for due to the nature of the code example.
102+
The `isfinite` function from the `math` library is useful for checking for `NaN`, `infinity` and `-infinity` values. `math.isfinite` checks if a value is neither `infinite` nor a `NaN`.
103+
104+
Other functions from the `math` library that could be of use are `isnan`, which checks if an inputted value is `"NaN"`, and `isinf` (which checks if a value is positive or negative infinity).
105+
106+
*[compliant01.py](compliant01.py):*
107+
108+
```py
109+
# SPDX-FileCopyrightText: OpenSSF project contributors
110+
# SPDX-License-Identifier: MIT
111+
""" Compliant Code Example """
112+
113+
114+
import sys
115+
from math import isfinite, isnan
116+
from typing import Union
117+
118+
119+
class Package:
120+
"""Class representing a package object."""
121+
def __init__(self) -> None:
122+
self.package_weight: float = 0.0
123+
self.max_package_weight: float = 100.0
124+
125+
def add_to_package(self, object_weight: Union[str, int, float]) -> None:
126+
# TODO: input sanitation.
127+
# TODO: proper exception handling
128+
"""Add an object into the package after validating its weight."""
129+
try:
130+
value = float(object_weight)
131+
except (ValueError, TypeError) as e:
132+
raise ValueError("Input cannot be converted to a float.") from e
133+
if isnan(value):
134+
raise ValueError("Input is not a number")
135+
if not isfinite(value):
136+
raise ValueError("Input is not a finite number.")
137+
if value < 0:
138+
raise ValueError("Weight must be a non-negative number.")
139+
if self.package_weight + value > self.max_package_weight:
140+
raise ValueError("Addition would exceed maximum package weight.")
141+
142+
print(f"Adding an object that weighs {value} units to package")
143+
self.package_weight += value
144+
145+
def get_package_weight(self) -> float:
146+
"""Return the package's current weight."""
147+
return self.package_weight
148+
149+
150+
#####################
151+
# exploiting above code example
152+
#####################
153+
154+
155+
package = Package()
156+
print(f"\nOriginal package's weight is {package.get_package_weight():.2f} units\n")
157+
for item in [100, "-infinity", sys.float_info.max, "NaN", -100]:
158+
print(f"package.add_to_package({item})")
159+
try:
160+
package.add_to_package(item)
161+
print(
162+
f"package.get_package_weight() = {package.get_package_weight():.2f}\n"
163+
)
164+
165+
except Exception as e:
166+
print(e)
167+
```
168+
169+
This compliant code example will raise a `ValueError` for inputs that are `-infinity`, `infinity`, or `NaN`, with messages "Input is not a finite number" and "Input is not a number" respectively. It should also ensure weights are non-negative, returning "Weight must be a non-negative number" for negative inputs.
170+
171+
**Example `compliant01.py` output:**
172+
173+
```bash
174+
Original package's weight is 0.00 units
175+
176+
package.add_to_package(100)
177+
Adding an object that weighs 100.0 units to package
178+
package.get_package_weight() = 100.00
179+
180+
package.add_to_package(-infinity)
181+
Input is not a finite number.
182+
package.add_to_package(1.7976931348623157e+308)
183+
Addition would exceed maximum package weight.
184+
package.add_to_package(NaN)
185+
Input is not a number
186+
package.add_to_package(-100)
187+
Weight must be a non-negative number.
188+
```
189+
190+
## Automated Detection
191+
192+
|||||
193+
|:---|:---|:---|:---|
194+
|Tool|Version|Checker|Description|
195+
|bandit|1.7.4|no detection||
196+
197+
## Related Guidelines
198+
199+
|||
200+
|:---|:---|
201+
|[SEI CERT C Coding Standard](https://wiki.sei.cmu.edu/confluence/display/c/SEI+CERT+C+Coding+Standard)|[FLP04-C. Check floating-point inputs for exceptional values](https://wiki.sei.cmu.edu/confluence/display/c/FLP04-C.+Check+floating-point+inputs+for+exceptional+values)|
202+
|[SEI CERT Oracle Coding Standard for Java](https://wiki.sei.cmu.edu/confluence/display/java/SEI+CERT+Oracle+Coding+Standard+for+Java?src=sidebar)|[NUM08-J. Check floating-point inputs for exceptional values](https://wiki.sei.cmu.edu/confluence/display/java/NUM08-J.+Check+floating-point+inputs+for+exceptional+values)|
203+
|[CWE MITRE Pillar](http://cwe.mitre.org/)|[CWE-703: Improper Check or Handling of Exceptional Conditions](https://cwe.mitre.org/data/definitions/703.html)|
204+
|[MITRE CWE Base](https://cwe.mitre.org/)|[CWE-754: Improper Check for Unusual or Exceptional Conditions](https://cwe.mitre.org/data/definitions/754.html)|
Lines changed: 35 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,58 @@
11
# SPDX-FileCopyrightText: OpenSSF project contributors
22
# SPDX-License-Identifier: MIT
33
""" Compliant Code Example """
4+
5+
46
import sys
5-
from math import isinf, isnan
7+
from math import isfinite, isnan
8+
from typing import Union
69

710

811
class Package:
9-
def __init__(self):
10-
self.package_weight = float(1.0)
11-
12-
def put_in_the_package(self, user_input):
12+
"""Class representing a package object."""
13+
def __init__(self) -> None:
14+
self.package_weight: float = 0.0
15+
self.max_package_weight: float = 100.0
16+
17+
def add_to_package(self, object_weight: Union[str, int, float]) -> None:
18+
"""Add an object into the package after validating its weight."""
19+
# TODO: input sanitation.
20+
# TODO: proper exception handling
1321
try:
14-
value = float(user_input)
15-
except ValueError:
16-
raise ValueError(f"{user_input} - Input is not a number!")
17-
18-
print(f"value is {value}")
19-
20-
if isnan(value) or isinf(value):
21-
raise ValueError(f"{user_input} - Input is not a real number!")
22-
22+
value = float(object_weight)
23+
except (ValueError, TypeError) as e:
24+
raise ValueError("Input cannot be converted to a float.") from e
25+
if isnan(value):
26+
raise ValueError("Input is not a number")
27+
if not isfinite(value):
28+
raise ValueError("Input is not a finite number.")
2329
if value < 0:
24-
raise ValueError(
25-
f"{user_input} - Packed object weight cannot be negative!"
26-
)
30+
raise ValueError("Weight must be a non-negative number.")
31+
if self.package_weight + value > self.max_package_weight:
32+
raise ValueError("Addition would exceed maximum package weight.")
2733

28-
if value >= sys.float_info.max - self.package_weight:
29-
raise ValueError(f"{user_input} - Input exceeds acceptable range!")
34+
print(f"Adding an object that weighs {value} units to package")
3035
self.package_weight += value
3136

32-
def get_package_weight(self):
37+
def get_package_weight(self) -> float:
38+
"""Return the package's current weight."""
3339
return self.package_weight
3440

3541

3642
#####################
3743
# exploiting above code example
3844
#####################
45+
46+
3947
package = Package()
4048
print(f"\nOriginal package's weight is {package.get_package_weight():.2f} units\n")
41-
for item in [sys.float_info.max, "infinity", "-infinity", "nan"]:
49+
for item in [100, "-infinity", sys.float_info.max, "NaN", -100]:
50+
print(f"package.add_to_package({item})")
4251
try:
43-
package.put_in_the_package(item)
44-
print(f"Current package weight = {package.get_package_weight():.2f}\n")
52+
package.add_to_package(item)
53+
print(
54+
f"package.get_package_weight() = {package.get_package_weight():.2f}\n"
55+
)
56+
4557
except Exception as e:
4658
print(e)
Lines changed: 27 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,35 @@
11
# SPDX-FileCopyrightText: OpenSSF project contributors
22
# SPDX-License-Identifier: MIT
3-
""" Non-compliant Code Example """
3+
"""Non-compliant Code Example"""
4+
45
import sys
56

67

78
class Package:
9+
"""Class representing a package object"""
10+
811
def __init__(self):
9-
self.package_weight = float(1.0)
12+
self.package_weight: float = 0.0
13+
self.max_package_weight: float = 100.0
1014

11-
def put_in_the_package(self, object_weight):
15+
def add_to_package(self, object_weight: str):
16+
"""Function for adding an object into the package"""
1217
value = float(object_weight)
13-
print(f"Adding an object that weighs {value} units")
18+
# This is dead code as value gets type cast to float,
19+
# hence will never be equal to string "NaN"
20+
if value == "NaN":
21+
raise ValueError("'NaN' not a number")
22+
# This is also dead code as value is type cast to float,
23+
# unusual inputs like -infinity will not get caught
24+
if isinstance(value, float) is False:
25+
raise ValueError("not a number")
26+
if self.package_weight + value > self.max_package_weight:
27+
raise ValueError("Addition would exceed maximum package weight.")
28+
1429
self.package_weight += value
1530

1631
def get_package_weight(self):
32+
"""Getter for outputting the package's current weight"""
1733
return self.package_weight
1834

1935

@@ -22,9 +38,13 @@ def get_package_weight(self):
2238
#####################
2339
package = Package()
2440
print(f"\nOriginal package's weight is {package.get_package_weight():.2f} units\n")
25-
for item in [sys.float_info.max, "infinity", "-infinity", "nan"]:
41+
for item in [100, "-infinity", sys.float_info.max, "NaN", -100]:
42+
print(f"package.add_to_package({item})")
2643
try:
27-
package.put_in_the_package(item)
28-
print(f"Current package weight = {package.get_package_weight():.2f}\n")
44+
package.add_to_package(item)
45+
print(
46+
f"package.get_package_weight() = {package.get_package_weight():.2f}\n"
47+
)
48+
2949
except Exception as e:
3050
print(e)

0 commit comments

Comments
 (0)