[Initial Feedback] Recursive Tree Traversal

[springcomp]

A very useful feature of XSLT is to supports recursive tree traversal using XPath.
I believe such a feature would be a great addition to JMESPath.

I have drafted a very ugly working prototype for what could be a new descendant-expression

Specification

A new descendant-expression will be added as an alternative to the expression grammar rule:

descendant = ".." identifier
descendant-expression = ( expression ) 1*( descendant )

expression =/ descendant-expression

A descendant-expression is a projection that returns an array with all the elements of the JSON input with the specified name.

Examples

Given the following JSON input:

{
  "name": "first",
  "top": { "name": "second" },
  "nested": {
    "name": [
      1, "string", false,
      { "name": "third" }
    ]
  }
}

The following expressions are possible:

Expression	Result
..name	["first","second",[1,"string",false,{"name":"third"}],"third"]
..name.length(@)	[5, 6, 4, 5]
..name..name	["third"]

Compliance Tests

This JEP standardizes a syntax that was formely not valid. For this reason, one of the syntax.json compliance tests must be changed or removed altogether:

    {
      "expression": ".foo",
      "error": "syntax"
    },
    {
      "expression": "foo..bar",
-     "error": "syntax"
+     "result": null
    },

A new file descendant.json will be added to the compliance test suite.

[MrBones757]

I'm not exactly sure how you want feedback on this item - please let me know if you want this elsewhere / in another form.

I think adding this would be a great extension of capability overall, and the baseline functions outlined would go a long way.

One thing that comes to mind when looking at this, though, is how this might work with filter projections.
that is, for a nested payload (my example here may not be the best to accurately illustrate my thinking):

[
  {
    "name": "abc",
    "children": [
      {
        "name": "xyz"
      }
    ]
  }
]

could i create a query like: [?..name=="xyz"]
or would something like this be out of scope for the intended changes.

[springcomp]

@MrBones757 Your feedback made me ponder.

First you must be aware that a filter-expression using syntax such as [? <comparator> ] applies to an input array and evaluates its comparator expression for each item in the input array.

Your expression assumes an implicit JSON input array, and will apply the comparison to each item in the input.

That being said, we still have an issue with regards to supporting your proposal.

Currently, the proposal is such that the recursive descendant expression yields a projection. That is, the output is an array on which subsequent expressions are evaluated on each item. So, for instance, your ..name expression would yield either an empty array [] or some result, possibly multi-valued ["xyz"].

So we have to make a case wether we want to support such comparison expressions such as ["xyz"] == 'xyz' which are currently not defined.

Furthermore, from an implementation standpoint, an array and a projection are not handled in the same way internally. This makes it really difficult to handle thoses cases as the expression syntax would lend itself to mean.

The result you want could work with the current proposal when rewritten to: ..name|[?@ == 'xyz']

TL;DR; we need some more thought on how to properly handle arrays, projections for this proposal to go forward.

[MrBones757]

Thanks for your reply here!

I don't think this needs to be a requirement for the proposal to go forward - on its own this in a great feature. Additions / enhancements could indeed be tracked as a separate feature in the future if tacking this in more than one part makes sense. Overall, though, the ability to achieve what I described with a recursive conditional returning matching sub dictionaries as a list would be extremely useful in many situations! Thanks again.