Skip to content

Commit 955b378

Browse files
alzimmermsftalzimmermsft
authored
Fix numeric handling bug in JsonReader.readUntyped (Azure#44424)
Fix numeric handling bug in JsonReader.readUntyped
1 parent 1f9f7c0 commit 955b378

File tree

72 files changed

+1903
-3786
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

72 files changed

+1903
-3786
lines changed

‎eng/versioning/version_client.txt‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -138,8 +138,6 @@ com.azure:azure-identity-perf;1.0.0-beta.1;1.0.0-beta.1
138138
com.azure:azure-iot-deviceupdate;1.0.25;1.1.0-beta.1
139139
com.azure:azure-iot-modelsrepository;1.0.0-beta.1;1.0.0-beta.2
140140
com.azure:azure-json;1.4.0;1.5.0-beta.1
141-
com.azure:azure-json-gson;1.0.0-beta.3;1.0.0-beta.4
142-
com.azure:azure-json-reflect;1.0.0-beta.2;1.0.0-beta.3
143141
com.azure:azure-maps-traffic;1.0.0-beta.1;1.0.0-beta.2
144142
com.azure:azure-maps-weather;1.0.0-beta.3;1.0.0-beta.4
145143
com.azure:azure-maps-timezone;1.0.0-beta.2;1.0.0-beta.3
@@ -496,6 +494,7 @@ io.clientcore:annotation-processor-test;1.0.0-beta.1;1.0.0-beta.1
496494
# In the pom, the version update tag after the version should name the unreleased package and the dependency version:
497495
# <!-- {x-version-update;unreleased_com.azure:azure-core;dependency} -->
498496

497+
unreleased_com.azure:azure-json;1.5.0-beta.1
499498
unreleased_io.clientcore:core;1.0.0-beta.6
500499

501500
# Released Beta dependencies: Copy the entry from above, prepend "beta_", remove the current

‎sdk/clientcore/core/CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,17 @@
66

77
### Breaking Changes
88

9+
- `JsonNumber` previously would use `float` when the floating point number was small enough to fit in `float` but it
10+
now aligns behavior with `JsonReader.readUntyped()` where `double` will be the smallest floating point type used.
11+
This aligns with floating point number behavior in Java where `double` is the default if no type is specified.
12+
- Support for special numeric `INF`, `-INF`, and `+INF` values have been removed to align with behaviors of `Float`
13+
and `Double` in Java where only the `Infinity` variants are supported.
14+
915
### Bugs Fixed
1016

17+
- `JsonReader.readUntyped()` had incomplete support for untyped numerics. Numerics too large for `double` and `long` are
18+
now supported and a bug where exponents were not being parsed correctly is fixed.
19+
1120
### Other Changes
1221

1322
## 1.0.0-beta.5 (2025-02-14)

‎sdk/clientcore/core/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ could be called indefinitely returning the same integer without error until `nex
6767
forward.
6868

6969
`JsonReader` allows for type conversion between JSON types, such as trying to convert a JSON string to a number or vice
70-
versa, and for commonly used nonstandard JSON values, such as `NaN`, `INF`, `-INF`, `Infinity`, and `-Infinity`.
70+
versa, and for commonly used nonstandard JSON values, such as `NaN`, `Infinity`, `+Infinity`, and `-Infinity`.
7171

7272
`JsonReader` doesn't take ownership of the JSON input source and therefore doesn't close any resources if the JSON is
7373
provided using an `InputStream` or `Reader`.
@@ -84,7 +84,7 @@ and objects, APIs for writing JSON. `JsonWriter` is provided to allow for any un
8484
such as Jackson or GSON, as long as the implementation passes the tests provided by the package's test-jar
8585
(`JsonWriterContractTests`).
8686

87-
`JsonWriter` allows for commonly used nonstandard JSON values, such as `NaN`, `INF`, `-INF`, `Infinity`, and
87+
`JsonWriter` allows for commonly used nonstandard JSON values, such as `NaN`, `Infinity`, `+Infinity`, and
8888
`-Infinity`, to be written using `writeNumberField` or `writeRawValue`.
8989

9090
`JsonWriter` doesn't write null `byte[]`, `Boolean`, `Number`, or `String` values when written as a field,
@@ -115,7 +115,7 @@ by this package if an implementation isn't found on the classpath.
115115
#### JsonOptions
116116

117117
`JsonOptions` contains configurations that must be respected by all implementations of `JsonReader`s and `JsonWriter`s.
118-
At this time, there's only one configuration for determining whether non-numeric numbers, `NaN`, `INF`, `-INF`, `Infinity`,
118+
At this time, there's only one configuration for determining whether non-numeric numbers, `NaN`, `Infinity`, `+Infinity`,
119119
and `-Infinity` are supported in JSON reading and writing with a default setting of `true`, that non-numeric numbers
120120
are allowed.
121121

‎sdk/clientcore/core/checkstyle-suppressions.xml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,12 @@
2525
<suppress files="io.clientcore.core.http.pipeline.HttpInstrumentationPolicy.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
2626
<suppress files="io.clientcore.core.implementation.MethodHandleReflectiveInvoker.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
2727
<suppress files="io.clientcore.core.implementation.http.rest.LengthValidatingInputStream.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
28+
<suppress files="io.clientcore.core.implementation.instrumentation.fallback.FallbackInstrumentation.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
29+
<suppress files="io.clientcore.core.implementation.instrumentation.otel.OTelInstrumentation.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
2830
<suppress files="io.clientcore.core.instrumentation.logging.LoggingEvent.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
2931
<suppress files="io.clientcore.core.serialization.json.JsonReader.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
3032
<suppress files="io.clientcore.core.serialization.json.JsonWriteContext.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
3133
<suppress files="io.clientcore.core.serialization.json.JsonWriter.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
3234
<suppress files="io.clientcore.core.serialization.json.implementation.StringBuilderWriter.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
33-
<suppress files="io.clientcore.core.serialization.json.models.JsonNumber.java" checks="com.azure.tools.checkstyle.checks.UseCaughtExceptionCauseCheck" />
34-
<suppress files="io.clientcore.core.implementation.instrumentation.otel.OTelInstrumentation.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
35-
<suppress files="io.clientcore.core.implementation.instrumentation.fallback.FallbackInstrumentation.java" checks="com.azure.tools.checkstyle.checks.ThrowFromClientLoggerCheck" />
35+
<suppress files="io.clientcore.core.serialization.json.implementation.JsonUtils.java" checks="com.azure.tools.checkstyle.checks.UseCaughtExceptionCauseCheck" />
3636
</suppressions>

‎sdk/clientcore/core/src/main/java/io/clientcore/core/serialization/json/JsonReader.java‎

Lines changed: 6 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33

44
package io.clientcore.core.serialization.json;
55

6+
import io.clientcore.core.serialization.json.implementation.JsonUtils;
67
import io.clientcore.core.serialization.json.implementation.jackson.core.JsonFactory;
78
import io.clientcore.core.serialization.json.implementation.jackson.core.JsonParser;
89
import io.clientcore.core.serialization.json.implementation.jackson.core.io.JsonStringEncoder;
@@ -15,6 +16,8 @@
1516
import java.io.InputStreamReader;
1617
import java.io.Reader;
1718
import java.io.StringReader;
19+
import java.math.BigDecimal;
20+
import java.math.BigInteger;
1821
import java.nio.charset.StandardCharsets;
1922
import java.util.ArrayList;
2023
import java.util.Base64;
@@ -744,8 +747,8 @@ private <T> T readMapOrObject(IOExceptionCheckedFunction<JsonReader, T> valueRea
744747
* <ul>
745748
* <li>null if the starting token is null or {@link JsonToken#NULL}</li>
746749
* <li>true or false if the starting token is {@link JsonToken#BOOLEAN}</li>
747-
* <li>One of int, long, float, or double is the starting token is {@link JsonToken#NUMBER}, the smallest
748-
* containing value will be used if the number is an integer</li>
750+
* <li>One of int, long, {@link BigInteger}, double, or {@link BigDecimal} if the starting token is
751+
* {@link JsonToken#NUMBER}, the smallest containing value will be used</li>
749752
* <li>An array of untyped elements if the starting point is {@link JsonToken#START_ARRAY}</li>
750753
* <li>A map of String-untyped value if the starting point is {@link JsonToken#START_OBJECT}</li>
751754
* </ul>
@@ -781,26 +784,7 @@ private Object readUntypedHelper(int depth) throws IOException {
781784
} else if (token == JsonToken.BOOLEAN) {
782785
return getBoolean();
783786
} else if (token == JsonToken.NUMBER) {
784-
String numberText = getText();
785-
786-
if ("INF".equals(numberText)
787-
|| "Infinity".equals(numberText)
788-
|| "-INF".equals(numberText)
789-
|| "-Infinity".equals(numberText)
790-
|| "NaN".equals(numberText)) {
791-
// Return special Double values as text as not all implementations of JsonReader may be able to handle
792-
// them as Doubles when parsing generically.
793-
return numberText;
794-
} else if (numberText.contains(".")) {
795-
// Unlike integers always use Double to prevent floating point rounding issues.
796-
return Double.parseDouble(numberText);
797-
} else {
798-
try {
799-
return Integer.parseInt(numberText);
800-
} catch (NumberFormatException ex) {
801-
return Long.parseLong(numberText);
802-
}
803-
}
787+
return JsonUtils.parseNumber(getText());
804788
} else if (token == JsonToken.STRING) {
805789
return getString();
806790
} else if (token == JsonToken.START_ARRAY) {

‎sdk/clientcore/core/src/main/java/io/clientcore/core/serialization/json/implementation/JsonUtils.java‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
// Copyright (c) Microsoft Corporation. All rights reserved.
2+
// Licensed under the MIT License.
3+
package io.clientcore.core.serialization.json.implementation;
4+
5+
import java.math.BigDecimal;
6+
import java.math.BigInteger;
7+
8+
/**
9+
* Utility class containing helper methods for serialization.
10+
*/
11+
public final class JsonUtils {
12+
/**
13+
* Parses a number from a string.
14+
* <p>
15+
* Callers to this method should ensure that the number string isn't null or empty before calling this method.
16+
* <p>
17+
* For integer numbers, this method will return the smallest number type that can represent the number. Where
18+
* {@link Integer} is preferred over {@link Long} and {@link Long} is preferred over {@link BigInteger}.
19+
* <p>
20+
* For floating point numbers, {@link Double} will be preferred but {@link BigDecimal} will be used if the number
21+
* is too large to fit in a {@link Double}.
22+
* <p>
23+
* Unlike the JSON specification, this method will handle the special floating point representations
24+
* ({@code NaN}, {@code Infinity}, etc) and will return a {@link Double} for those values.
25+
*
26+
* @param numberString The string representation of the number.
27+
* @return The number represented by the string.
28+
* @throws NumberFormatException If the string is not a valid number.
29+
*/
30+
public static Number parseNumber(String numberString) {
31+
int length = numberString.length();
32+
33+
// Use the length of the number and checks for special values to determine if the number is a special
34+
// floating point representation.
35+
// The special floating point representations are: NaN, Infinity, +Infinity, and -Infinity.
36+
// These will be returned using Double.
37+
if (length == 3 && "NaN".equals(numberString)) {
38+
return Double.NaN;
39+
} else if (length == 8 && "Infinity".equals(numberString)) {
40+
return Double.POSITIVE_INFINITY;
41+
} else if (length == 9) {
42+
if ("+Infinity".equals(numberString)) {
43+
return Double.POSITIVE_INFINITY;
44+
} else if ("-Infinity".equals(numberString)) {
45+
return Double.NEGATIVE_INFINITY;
46+
}
47+
}
48+
49+
boolean floatingPoint = false;
50+
for (int i = 0; i < length; i++) {
51+
char c = numberString.charAt(i);
52+
if (c == '.' || c == 'e' || c == 'E') {
53+
floatingPoint = true;
54+
break;
55+
}
56+
}
57+
58+
return floatingPoint ? handleFloatingPoint(numberString) : handleInteger(numberString);
59+
}
60+
61+
private static Number handleFloatingPoint(String value) {
62+
// Floating point parsing will return Infinity if the String value is larger than what can be contained by
63+
// the numeric type. Check if the String contains the Infinity representation to know when to scale up the
64+
// numeric type.
65+
// Additionally, due to the handling of values that can't fit into the numeric type, the only time floating
66+
// point parsing will throw is when the string value is invalid.
67+
double d = Double.parseDouble(value);
68+
if (!Double.isInfinite(d)) {
69+
return d;
70+
}
71+
72+
return new BigDecimal(value);
73+
}
74+
75+
private static Number handleInteger(String value) {
76+
try {
77+
return Integer.parseInt(value);
78+
} catch (NumberFormatException failedInteger) {
79+
try {
80+
return Long.parseLong(value);
81+
} catch (NumberFormatException failedLong) {
82+
failedLong.addSuppressed(failedInteger);
83+
try {
84+
return new BigInteger(value);
85+
} catch (NumberFormatException failedBigDecimal) {
86+
failedBigDecimal.addSuppressed(failedLong);
87+
throw failedBigDecimal;
88+
}
89+
}
90+
}
91+
}
92+
}

‎sdk/clientcore/core/src/main/java/io/clientcore/core/serialization/json/implementation/jackson/core/JsonParser.java‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,8 +56,8 @@ public enum Feature {
5656
* (see section 3.2.4.1, Lexical Representation)
5757
* allows (tokens are quoted contents, not including quotes):
5858
*<ul>
59-
* <li>"INF" (for positive infinity), as well as alias of "Infinity"
60-
* <li>"-INF" (for negative infinity), alias "-Infinity"
59+
* <li>"Infinity" (for positive infinity)
60+
* <li>"-Infinity" (for negative infinity)
6161
* <li>"NaN" (for other not-a-numbers, like result of division by zero)
6262
*</ul>
6363
*<p>

‎sdk/clientcore/core/src/main/java/io/clientcore/core/serialization/json/implementation/jackson/core/base/ParserMinimalBase.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1113,7 +1113,7 @@ protected void _throwUnquotedSpace(int i, String ctxtDesc) throws JsonParseExcep
11131113
*/
11141114
protected String _validJsonValueList() {
11151115
if (Feature.ALLOW_NON_NUMERIC_NUMBERS.enabledIn(_features)) {
1116-
return "(JSON String, Number (or 'NaN'/'INF'/'+INF'), Array, Object or token 'null', 'true' or 'false')";
1116+
return "(JSON String, Number (or 'NaN'/'Infinity'/'+Infinity'/'-Infinity'), Array, Object or token 'null', 'true' or 'false')";
11171117
}
11181118
return "(JSON String, Number, Array, Object or token 'null', 'true' or 'false')";
11191119
}

‎sdk/clientcore/core/src/main/java/io/clientcore/core/serialization/json/implementation/jackson/core/json/ReaderBasedJsonParser.java‎

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -792,15 +792,7 @@ protected JsonToken _handleInvalidNumberStart(int ch, boolean negative) throws I
792792
}
793793
}
794794
ch = _inputBuffer[_inputPtr++];
795-
if (ch == 'N') {
796-
String match = negative ? "-INF" : "+INF";
797-
_matchToken(match, 3);
798-
if (Feature.ALLOW_NON_NUMERIC_NUMBERS.enabledIn(_features)) {
799-
return resetAsNaN(match, negative ? Double.NEGATIVE_INFINITY : Double.POSITIVE_INFINITY);
800-
}
801-
_reportError(
802-
"Non-standard token '" + match + "': enable JsonParser.Feature.ALLOW_NON_NUMERIC_NUMBERS to allow");
803-
} else if (ch == 'n') {
795+
if (ch == 'n') {
804796
String match = negative ? "-Infinity" : "+Infinity";
805797
_matchToken(match, 3);
806798
if (Feature.ALLOW_NON_NUMERIC_NUMBERS.enabledIn(_features)) {

0 commit comments

Comments
 (0)