issue #4514 tonumber function as part of roadmap #4287 #4605

asifabashar · 2025-10-20T19:23:31Z

Description

tonumber(, )
This function converts a string to a number.
Usage
You can use this function with the eval and where commands, in the WHERE clause of the from command, and as part of evaluation expressions with other commands.
The argument can be a string or the name of a field that contains a string. If the string contains a decimal point ( . ), then the tonumber function converts it to a double. Otherwise, the function converts the string to an integer.

Be aware that integers are supported differently in different product contexts:
The argument is optional. It defines the base of the number in the argument. It defaults to 10, which corresponds to the decimal system. You can set to a number between 2 and 36, inclusive.

If the tonumber function cannot parse a literal string to a number, the function returns an error.

Basic examples
The following example converts the string values from the store_sales field to numbers, and then stores the numbers in a field named n. This example uses the default base of 10.
... | eval n=tonumber(store_sales)

The following example takes the hexadecimal number and uses a base of 16 to return the number "164".

... | eval n=tonumber("0A4",16)
Resolves #4514

Related Issues

Resolves #4514

Check List

[ x] New functionality includes testing.
[x ] New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
New PPL command checklist all confirmed.
API changes companion pull request created.
Commits are signed per the DCO using --signoff or -s.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Asif Bashar <[email protected]>

… default duration is in seconds which will be used for duration format , added cast call for tostring with 1 argument Signed-off-by: Asif Bashar <[email protected]>

Signed-off-by: Asif Bashar <[email protected]>

…ensearch-project#4287 Signed-off-by: Asif Bashar <[email protected]>

Signed-off-by: Asif Bashar <[email protected]>

docs/user/ppl/functions/conversion.rst

ppl/src/main/antlr/OpenSearchPPLLexer.g4

penghuo · 2025-10-30T16:25:24Z

core/src/test/java/org/opensearch/sql/expression/function/udf/ToNumberFunctionTest.java

+  void testToNumberWithDecimalBase16() {
+    assertEquals(255.5, ToNumberFunction.toNumber("FF.8", 16));
+    assertEquals(16.25, ToNumberFunction.toNumber("10.4", 16));
+    assertEquals(171.6875, ToNumberFunction.toNumber("AB.B", 16));


Any standard we are following?
Seems SPL does not support tonumber("AB.B", 16)

If not, we should define the behaviour of tonumber as

if base=10, convert to int / double, depend on whether there is .

if base!=10, always convert to long, e.g. new BigInteger("FF", 16).

made the changes

@asifabashar what is change?
I still see test

@Test void testToNumberWithDecimalBase16() { assertEquals(255L, ToNumberFunction.toNumber("FF.8", 16)); assertEquals(16L, ToNumberFunction.toNumber("10.4", 16)); assertEquals(171L, ToNumberFunction.toNumber("AB.B", 16)); }

sorry, fixed it now

Signed-off-by: Asif Bashar <[email protected]>

asifabashar · 2025-11-03T17:44:27Z

@penghuo Please review.

penghuo · 2025-11-04T19:56:23Z

Please fix IT.

Signed-off-by: Asif Bashar <[email protected]>

asifabashar · 2025-11-06T06:58:02Z

Please help set PR label.

Signed-off-by: Asif Bashar <[email protected]>

core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java

Signed-off-by: Asif Bashar <[email protected]>

core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java

Signed-off-by: Asif Bashar <[email protected]>

penghuo · 2025-11-12T23:01:11Z

Looks good to me, Please merge conflict.

@dai-chen , please take another look.

Signed-off-by: Asif Bashar <[email protected]>

dai-chen · 2025-11-13T23:28:32Z

docs/user/ppl/functions/conversion.rst

+    os> source=people | eval int_value = tonumber('010101',2) | fields int_value | head 1
+    fetched rows / total rows = 1/1
+    +-----------+
+    | int_value |


Is int_value an integer or double?

dai-chen · 2025-11-13T23:31:53Z

docs/user/ppl/functions/conversion.rst

+
+The following usage options are available, depending on the parameter types and the number of parameters.
+
+Usage: tonumber(string, [base]) converts the value in first argument to provided base type string in second argument. If second argument is not provided, then it converts to base 10 number representation.


It's not converting 1st arg to provided base type in 2nd arg, right? The 2nd arg describe the base of 1st arg instead?

dai-chen · 2025-11-13T23:33:06Z

core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java

+    public Expression implement(
+        RexToLixTranslator translator, RexCall call, List<Expression> translatedOperands) {
+      Expression fieldValue = translatedOperands.get(0);
+      int base = 10;


dai-chen · 2025-11-13T23:37:32Z

core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java

+        result = bigInteger.longValue();
+      }
+    } catch (Exception e) {
+


Is this by design?

dai-chen · 2025-11-13T23:40:05Z

ppl/src/test/java/org/opensearch/sql/ppl/calcite/CalcitePPLToNumberFunctionTest.java

+import org.apache.calcite.test.CalciteAssert;
+import org.junit.Test;
+
+public class CalcitePPLToNumberFunctionTest extends CalcitePPLAbstractTest {


No IT? I think we need to also cover NULL and error case for invalid input.

asifabashar added 14 commits October 19, 2025 06:14

PPL tostring() implementation issue opensearch-project#4492

0ed23de

Signed-off-by: Asif Bashar <[email protected]>

removed sql changes

5a5b778

Signed-off-by: Asif Bashar <[email protected]>

doc changes

9d71a95

Signed-off-by: Asif Bashar <[email protected]>

docs changes

be2c2e2

Signed-off-by: Asif Bashar <[email protected]>

reverted string doc changes

fc763a4

Signed-off-by: Asif Bashar <[email protected]>

removed extra word

a04eb14

Signed-off-by: Asif Bashar <[email protected]>

added any type

590a8e6

Signed-off-by: Asif Bashar <[email protected]>

doc formatting fixes

6938e2c

Signed-off-by: Asif Bashar <[email protected]>

description for boolean example

314fccd

Signed-off-by: Asif Bashar <[email protected]>

added format_time call from calcite , added duration_millis as splunk…

0ee17b9

… default duration is in seconds which will be used for duration format , added cast call for tostring with 1 argument Signed-off-by: Asif Bashar <[email protected]>

added doc update to specifically set 2nd argument as optional

6e24aa3

Signed-off-by: Asif Bashar <[email protected]>

mentioned as value instead of number specifically

454cfc8

Signed-off-by: Asif Bashar <[email protected]>

fixed wrong bullet point

b221e8c

Signed-off-by: Asif Bashar <[email protected]>

issue opensearch-project#4514 tonumber function as part of roadmap op…

a0a89c4

…ensearch-project#4287 Signed-off-by: Asif Bashar <[email protected]>

asifabashar requested review from MaxKsyunz, Swiddis, YANG-DB, Yury-Fridlyand, acarbonetto, anirudha, dai-chen, derek-ho, forestmvey, joshuali925, kavithacm, mengweieric, penghuo, ps48, seankao-az and vamsimanohar as code owners October 20, 2025 19:23

asifabashar added 3 commits October 22, 2025 10:11

spotless apply

042d3e8

Signed-off-by: Asif Bashar <[email protected]>

merge conflict fix

38ce420

Signed-off-by: Asif Bashar <[email protected]>

merge conflict fix

58d8e1b

Signed-off-by: Asif Bashar <[email protected]>

penghuo requested changes Oct 30, 2025

View reviewed changes

added recommended changes

34f8635

Signed-off-by: Asif Bashar <[email protected]>

asifabashar requested a review from penghuo November 2, 2025 05:49

asifabashar added 4 commits November 4, 2025 18:13

fixed doctest for conversion.rst and applied spotless

8279bf3

Signed-off-by: Asif Bashar <[email protected]>

removed decimal point from hex

b5346a6

Signed-off-by: Asif Bashar <[email protected]>

added doctest fixes

7cf867b

Signed-off-by: Asif Bashar <[email protected]>

fixed merged conflicts

8d9db4e

Signed-off-by: Asif Bashar <[email protected]>

asifabashar added 2 commits November 5, 2025 23:11

removed unused variables

46f010b

Signed-off-by: Asif Bashar <[email protected]>

spotless

b7afa17

Signed-off-by: Asif Bashar <[email protected]>

penghuo added PPL Piped processing language enhancement New feature or request backport 2.19-dev labels Nov 6, 2025

penghuo requested changes Nov 6, 2025

View reviewed changes

core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java Show resolved Hide resolved

made recommended changes to simply parse to number, null when malformed

0f0125a

Signed-off-by: Asif Bashar <[email protected]>

asifabashar requested a review from penghuo November 6, 2025 18:25

penghuo reviewed Nov 7, 2025

View reviewed changes

core/src/main/java/org/opensearch/sql/expression/function/udf/ToNumberFunction.java Show resolved Hide resolved

asifabashar requested a review from penghuo November 9, 2025 00:56

hex max limit doc and unit tests

804db79

Signed-off-by: Asif Bashar <[email protected]>

asifabashar force-pushed the feature_tonumber branch from a745afb to 804db79 Compare November 9, 2025 00:58

fix to spotless

7588397

Signed-off-by: Asif Bashar <[email protected]>

Merge branch 'main' into feature_tonumber

04a5564

Signed-off-by: Asif Bashar <[email protected]>

dai-chen reviewed Nov 13, 2025

View reviewed changes


		The following usage options are available, depending on the parameter types and the number of parameters.

		Usage: tonumber(string, [base]) converts the value in first argument to provided base type string in second argument. If second argument is not provided, then it converts to base 10 number representation.

issue #4514 tonumber function as part of roadmap #4287 #4605

Are you sure you want to change the base?

issue #4514 tonumber function as part of roadmap #4287 #4605

Conversation

asifabashar commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

Uh oh!

Uh oh!

penghuo Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asifabashar commented Nov 3, 2025

Uh oh!

penghuo commented Nov 4, 2025

Uh oh!

asifabashar commented Nov 6, 2025

Uh oh!

Uh oh!

Uh oh!

penghuo commented Nov 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

asifabashar commented Oct 20, 2025 •

edited

Loading

penghuo Oct 30, 2025 •

edited

Loading