Skip to content

Commit 0937560

Browse files
acarbonettoacarbonetto
committed
Update JSON_APPEND, JSON_EXTEND, JSON_SET
Signed-off-by: Andrew Carbonetto <andrew.carbonetto@improving.com>
1 parent 8f1539f commit 0937560

File tree

4 files changed

+59
-41
lines changed

4 files changed

+59
-41
lines changed

‎docs/ppl-lang/functions/ppl-json.md‎

Lines changed: 26 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -206,23 +206,26 @@ Example:
206206

207207
**Description**
208208

209-
`json_set(json_string, [key, value...])` Inserts or updates one or more keys and their corresponding values from the specified JSON object.
209+
`json_set(json_string, array(<path>, <value>, ...))` Inserts or updates one or more values at the corresponding paths in the specified JSON object.
210210

211-
**Arguments type:** JSON_STRING, List<STRING, STRING>
211+
**Argument type:**
212+
- \<json_string\> must be a JSON_STRING.
213+
- \<path\> must be a STRING.
214+
- \<value\> can be a JSON_STRING.
212215

213216
**Return type:** JSON_STRING
214217

215218
An updated JSON object format.
216219

217220
Example:
218221

219-
os> source=people | eval updated = json_set('{"a":[{"b":1},{"b":2}]}', '$.a[*].b', 3) | head 1 | fields updated
222+
os> source=people | eval updated = json_set('{"a":[{"b":1},{"b":2}]}', array('$.a[*].b', '3', '$.a', '{"c":4}')) | head 1 | fields updated
220223
fetched rows / total rows = 1/1
221-
+-------------------------+
222-
| updated |
223-
+-------------------------+
224-
| {"a":[{"b":3},{"b":3}]} |
225-
+-------------------------+
224+
+---------------------------------+
225+
| updated |
226+
+---------------------------------+
227+
| {"a":[{"b":3},{"b":3},{"c":4}]} |
228+
+---------------------------------+
226229

227230

228231
### `JSON_DELETE`
@@ -267,9 +270,12 @@ Example:
267270

268271
**Description**
269272

270-
`json_append(json_string, [path_key, list of values to add])` appends values to end of an array within the json elements. Return the updated json object after appending.
273+
`json_append(json_string, [path_key, value,...])` appends values to end of an array at path_key within the json elements. Return the updated json object after appending.
271274

272-
**Argument type:** JSON_STRING, List<STRING>
275+
**Argument type:**
276+
- \<json_string\> must be a JSON_STRING.
277+
- \<path\> must be a STRING.
278+
- \<value\> can be a JSON_STRING.
273279

274280
**Return type:** JSON_STRING
275281

@@ -285,23 +291,23 @@ Append adds the value to the end of the existing array with the following cases:
285291

286292
Example:
287293

288-
os> source=people | eval append = json_append(`{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]}`,array('student', '{"name":"Tomy","rank":5}')) | head 1 | fields append
294+
os> source=people | eval append = json_append(`{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]}`, 'student', array('{"name":"Tomy","rank":5}')) | head 1 | fields append
289295
fetched rows / total rows = 1/1
290296
+-----------------------------------------------------------------------------------------------------------------------------------+
291297
| append |
292298
+-----------------------------------------------------------------------------------------------------------------------------------+
293299
|{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2},{"name":"Tomy","rank":5}]} |
294300
+-----------------------------------------------------------------------------------------------------------------------------------+
295301

296-
os> source=people | eval append = json_append(`{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]}`,array('teacher', 'Tom', 'Walt')) | head 1 | fields append
302+
os> source=people | eval append = json_append(`{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]}`, 'teacher', array('Tom', 'Walt')) | head 1 | fields append
297303
fetched rows / total rows = 1/1
298304
+-----------------------------------------------------------------------------------------------------------------------------------+
299305
| append |
300306
+-----------------------------------------------------------------------------------------------------------------------------------+
301307
|{"teacher":["Alice","Tom","Walt"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]} |
302308
+-----------------------------------------------------------------------------------------------------------------------------------+
303309

304-
os> source=people | eval append = json_append(`{"school":{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]}}`,array('school.teacher', array('Tom', 'Walt'))) | head 1 | fields append
310+
os> source=people | eval append = json_append(`{"school":{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]}}`, 'school.teacher', array(array('Tom', 'Walt'))) | head 1 | fields append
305311
fetched rows / total rows = 1/1
306312
+-------------------------------------------------------------------------------------------------------------------------+
307313
| append |
@@ -313,9 +319,12 @@ Example:
313319

314320
**Description**
315321

316-
`json_extend(json_string, [path_key, list of values to flatten])` extends the values to end of an array within the json elements. Return the updated json object after extending.
322+
`json_extend(json_string, [path_key, value,...])` extends values to end of an array at path_key within the json elements. Return the updated json object after extending.
317323

318-
**Argument type:** JSON_STRING, List<STRING>
324+
**Argument type:**
325+
- \<json_string\> must be a JSON_STRING.
326+
- \<path\> must be a STRING.
327+
- \<value\> can be a JSON_STRING.
319328

320329
**Return type:** JSON_STRING
321330

@@ -331,7 +340,7 @@ Extends adds the value to the end of the existing array with the following cases
331340

332341
Example:
333342

334-
os> source=people | eval extend = json_extend(`{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]}`, 'student', '{"name":"Tommy","rank":5}') | head 1 | fields extend
343+
os> source=people | eval extend = json_extend(`{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]}`, 'student', array('{"name":"Tommy","rank":5}')) | head 1 | fields extend
335344
fetched rows / total rows = 1/1
336345
+-----------------------------------------------------------------------------------------------------------------------------------+
337346
| extend |
@@ -347,7 +356,7 @@ Example:
347356
|{"teacher":["Alice","Tom","Walt"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]} |
348357
+-----------------------------------------------------------------------------------------------------------------------------------+
349358

350-
os> source=people | eval extend = json_extend(`{"school":{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]}}`, 'school.teacher', array('Tom', 'Walt')) | head 1 | fields extend
359+
os> source=people | eval extend = json_extend(`{"school":{"teacher":["Alice"],"student":[{"name":"Bob","rank":1},{"name":"Charlie","rank":2}]}}`, 'school.teacher', array(array('Tom', 'Walt'))) | head 1 | fields extend
351360
fetched rows / total rows = 1/1
352361
+-------------------------------------------------------------------------------------------------------------------------+
353362
| extend |

‎ppl-spark-integration/src/main/java/org/opensearch/sql/expression/function/JsonUtils.java‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,6 @@ static void updateNestedValue(Object currentObj, String[] pathParts, int depth,
4343
String currentKey = pathParts[depth];
4444

4545
if (depth == pathParts.length - 1) {
46-
// If it's the last key, append to the array
4746
currentMap.put(currentKey, valueToUpdate);
4847
} else {
4948
// Continue traversing

‎ppl-spark-integration/src/main/java/org/opensearch/sql/expression/function/SerializableUdf.java‎

Lines changed: 30 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -44,16 +44,16 @@
4444
public interface SerializableUdf {
4545

4646
abstract class SerializableAbstractFunction1<T1, R> extends AbstractFunction1<T1, R>
47-
implements Serializable {
48-
}
47+
implements Serializable {}
4948

5049
abstract class SerializableAbstractFunction2<T1, T2, R> extends AbstractFunction2<T1, T2, R>
51-
implements Serializable {
52-
}
50+
implements Serializable {}
5351

5452
abstract class SerializableAbstractFunction3<T1, T2, T3, R> extends AbstractFunction3<T1, T2, T3, R>
55-
implements Serializable {
56-
}
53+
implements Serializable {}
54+
55+
abstract class SerializeableUpdateNestedFunction<T1, T2, R> extends AbstractFunction2<T1, T2, R>
56+
implements Serializable {}
5757

5858
/**
5959
* Remove specified keys from a JSON string.
@@ -144,20 +144,25 @@ public String apply(String jsonStr, WrappedArray<String> elements) {
144144
}
145145
try {
146146
List<String> pathValues = JavaConverters.mutableSeqAsJavaList(elements);
147+
// don't update if the list is empty, or the list is not key-value pairs
147148
if (pathValues.isEmpty()) {
148149
return jsonStr;
149150
}
150151

151-
String path = pathValues.get(0);
152-
String[] pathParts = path.split("\\.");
153-
List<String> values = pathValues.subList(1, pathValues.size());
154-
155152
// Parse the JSON string into a Map
156153
Map<String, Object> jsonMap = objectMapper.readValue(jsonStr, Map.class);
157154

158-
// Append each value at the specified path
159-
for (String value : values) {
160-
Object parsedValue = parseValue(value); // Parse the value
155+
// Iterate through the key-value pairs and update the json
156+
var iter = pathValues.iterator();
157+
while (iter.hasNext()) {
158+
String path = iter.next();
159+
if (!iter.hasNext()) {
160+
// no value provided and cannot update anything
161+
break;
162+
}
163+
String[] pathParts = path.split("\\.");
164+
Object parsedValue = parseValue(iter.next());
165+
161166
appendNestedValue(jsonMap, pathParts, 0, parsedValue, false);
162167
}
163168

@@ -183,20 +188,25 @@ public String apply(String jsonStr, WrappedArray<String> elements) {
183188
}
184189
try {
185190
List<String> pathValues = JavaConverters.mutableSeqAsJavaList(elements);
191+
// don't update if the list is empty, or the list is not key-value pairs
186192
if (pathValues.isEmpty()) {
187193
return jsonStr;
188194
}
189195

190-
String path = pathValues.get(0);
191-
String[] pathParts = path.split("\\.");
192-
List<String> values = pathValues.subList(1, pathValues.size());
193-
194196
// Parse the JSON string into a Map
195197
Map<String, Object> jsonMap = objectMapper.readValue(jsonStr, Map.class);
196198

197-
// Append each value at the specified path
198-
for (String value : values) {
199-
Object parsedValue = parseValue(value); // Parse the value
199+
// Iterate through the key-value pairs and update the json
200+
var iter = pathValues.iterator();
201+
while (iter.hasNext()) {
202+
String path = iter.next();
203+
if (!iter.hasNext()) {
204+
// no value provided and cannot update anything
205+
break;
206+
}
207+
String[] pathParts = path.split("\\.");
208+
Object parsedValue = parseValue(iter.next());
209+
200210
appendNestedValue(jsonMap, pathParts, 0, parsedValue, true);
201211
}
202212

‎ppl-spark-integration/src/test/java/org/opensearch/sql/expression/function/SerializableJsonUdfTest.java‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -173,8 +173,8 @@ public void testJsonAppendFunctionAppendToExistingArray() {
173173
@Test
174174
public void testJsonAppendFunctionAppendArrayItemToExistingArray() {
175175
String jsonStr = "{\"arrayKey\":[\"value1\",\"value2\"]}";
176-
String expectedJson = "{\"arrayKey\":[\"value1\",\"value2\",[\"value3\",\"value4\"]]}";
177-
String result = jsonAppendFunction.apply(jsonStr, WrappedArray.make(new String[]{"arrayKey", "[\"value3\",\"value4\"]"}));
176+
String expectedJson = "{\"arrayKey\":[\"value1\",\"value2\",[\"value3\",\"value4\"],[\"value5\",\"value6\"]]}";
177+
String result = jsonAppendFunction.apply(jsonStr, WrappedArray.make(new String[]{"arrayKey", "[\"value3\",\"value4\"]", "arrayKey", "[\"value5\",\"value6\"]"}));
178178
assertEquals(expectedJson, result);
179179
}
180180

@@ -260,7 +260,7 @@ public void testJsonExtendFunctionAppendArrayItemToExistingArray() {
260260
// in that it flattens the given arrays
261261
String jsonStr = "{\"arrayKey\":[\"value1\",\"value2\"]}";
262262
String expectedJson = "{\"arrayKey\":[\"value1\",\"value2\",\"value3\",\"value4\",\"value5\",\"value6\"]}";
263-
String result = jsonExtendFunction.apply(jsonStr, WrappedArray.make(new String[]{"arrayKey", "[\"value3\",\"value4\"]", "[\"value5\",\"value6\"]"}));
263+
String result = jsonExtendFunction.apply(jsonStr, WrappedArray.make(new String[]{"arrayKey", "[\"value3\",\"value4\"]", "arrayKey", "[\"value5\",\"value6\"]"}));
264264
assertEquals(expectedJson, result);
265265
}
266266

0 commit comments

Comments
 (0)