Merge pull request #61 from jg-rp/projection

Add simple query projection

Author: jg-rp

CHANGELOG.md

@@ -1,9 +1,16 @@
 # Python JSONPath Change Log
 
-## Version 1.1.2 (unreleased)
+## Version 1.2.0 (unreleased)
+
+**Fixes**
 
 - Fixed handling of JSONPath literals in filter expressions. We now raise a `JSONPathSyntaxError` if a filter expression literal is not part of a comparison or function expression. See [jsonpath-compliance-test-suite#81](https://github.com/jsonpath-standard/jsonpath-compliance-test-suite/pull/81).
 
+**Features**
+
+- Added a `select` method to the JSONPath [query iterator interface](https://jg-rp.github.io/python-jsonpath/query/), generating a projection of each JSONPath match by selecting a subset of its values.
+- Added the `addne` and `addap` operations to [JSONPatch](https://jg-rp.github.io/python-jsonpath/api/#jsonpath.JSONPatch). `addne` (add if not exists) is like the standard `add` operation, but only adds object keys/values if the key does not exist. `addap` (add or append) is like the standard `add` operation, but assumes an index of `-` if the target index can not be resolved.
+
 ## Version 1.1.1
 
 **Fixes**

docs/advanced.md

@@ -39,15 +39,15 @@ user_names = jsonpath.findall(
 
 ## Function Extensions
 
-Add, remove or replace [filter functions](functions.md) by updating the [`function_extensions`](api.md#jsonpath.env.JSONPathEnvironment.function_extensions) attribute of a [`JSONPathEnvironment`](api.md#jsonpath.env.JSONPathEnvironment). It is a regular Python dictionary mapping filter function names to any [callable](https://docs.python.org/3/library/typing.html#typing.Callable), like a function or class with a `__call__` method.
+Add, remove or replace [filter functions](functions.md) by updating the [`function_extensions`](api.md#jsonpath.JSONPathEnvironment.function_extensions) attribute of a [`JSONPathEnvironment`](api.md#jsonpath.JSONPathEnvironment). It is a regular Python dictionary mapping filter function names to any [callable](https://docs.python.org/3/library/typing.html#typing.Callable), like a function or class with a `__call__` method.
 
 ### Type System for Function Expressions
 
 [Section 2.4.1](https://datatracker.ietf.org/doc/html/rfc9535#name-type-system-for-function-ex) of RFC 9535 defines a type system for function expressions and requires that we check that filter expressions are well-typed. With that in mind, you are encouraged to implement custom filter functions by extending [`jsonpath.function_extensions.FilterFunction`](api.md#jsonpath.function_extensions.FilterFunction), which forces you to be explicit about the [types](api.md#jsonpath.function_extensions.ExpressionType) of arguments the function extension accepts and the type of its return value.
 
 !!! info
 
-    [`FilterFunction`](api.md#jsonpath.function_extensions.FilterFunction) was new in Python JSONPath version 0.10.0. Prior to that we did not enforce function expression well-typedness. To use any arbitrary [callable](https://docs.python.org/3/library/typing.html#typing.Callable) as a function extension - or if you don't want built-in filter functions to raise a `JSONPathTypeError` for function expressions that are not well-typed - set [`well_typed`](api.md#jsonpath.env.JSONPathEnvironment.well_typed) to `False` when constructing a [`JSONPathEnvironment`](api.md#jsonpath.env.JSONPathEnvironment).
+    [`FilterFunction`](api.md#jsonpath.function_extensions.FilterFunction) was new in Python JSONPath version 0.10.0. Prior to that we did not enforce function expression well-typedness. To use any arbitrary [callable](https://docs.python.org/3/library/typing.html#typing.Callable) as a function extension - or if you don't want built-in filter functions to raise a `JSONPathTypeError` for function expressions that are not well-typed - set [`well_typed`](api.md#jsonpath.JSONPathEnvironment.well_typed) to `False` when constructing a [`JSONPathEnvironment`](api.md#jsonpath.JSONPathEnvironment).
 
 ### Example
 
@@ -137,7 +137,7 @@ env = MyEnv()
 
 ### Compile Time Validation
 
-Calls to [type-aware](#type-system-for-function-expressions) function extension are validated at JSONPath compile-time automatically. If [`well_typed`](api.md#jsonpath.env.JSONPathEnvironment.well_typed) is set to `False` or a custom function extension does not inherit from [`FilterFunction`](api.md#jsonpath.function_extensions.FilterFunction), its arguments can be validated by implementing the function as a class with a `__call__` method, and a `validate` method. `validate` will be called after parsing the function, giving you the opportunity to inspect its arguments and raise a `JSONPathTypeError` should any arguments be unacceptable. If defined, `validate` must take a reference to the current environment, an argument list and the token pointing to the start of the function call.
+Calls to [type-aware](#type-system-for-function-expressions) function extension are validated at JSONPath compile-time automatically. If [`well_typed`](api.md#jsonpath.JSONPathEnvironment.well_typed) is set to `False` or a custom function extension does not inherit from [`FilterFunction`](api.md#jsonpath.function_extensions.FilterFunction), its arguments can be validated by implementing the function as a class with a `__call__` method, and a `validate` method. `validate` will be called after parsing the function, giving you the opportunity to inspect its arguments and raise a `JSONPathTypeError` should any arguments be unacceptable. If defined, `validate` must take a reference to the current environment, an argument list and the token pointing to the start of the function call.
 
 ```python
 def validate(

docs/api.md

@@ -14,6 +14,9 @@
 ::: jsonpath.Query
     handler: python
 
+::: jsonpath.Projection
+    handler: python
+
 ::: jsonpath.function_extensions.FilterFunction
     handler: python
 

docs/async.md

@@ -2,7 +2,7 @@
 
 Largely motivated by its integration with [Python Liquid](https://jg-rp.github.io/liquid/jsonpath/introduction), Python JSONPath offers an asynchronous API that allows for items in a target data structure to be "fetched" lazily.
 
-[`findall_async()`](api.md#jsonpath.env.JSONPathEnvironment.findall_async) and [`finditer_async()`](api.md#jsonpath.env.JSONPathEnvironment.finditer) are [asyncio](https://docs.python.org/3/library/asyncio.html) equivalents to [`findall()`](api.md#jsonpath.env.JSONPathEnvironment.findall) and [`finditer()`](api.md#jsonpath.env.JSONPathEnvironment.finditer). By default, any class implementing the [mapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) or [sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence) interfaces, and a `__getitem_async__()` method, will have `__getitem_async__()` awaited instead of calling `__getitem__()` when resolving mapping keys or sequence indices.
+[`findall_async()`](api.md#jsonpath.JSONPathEnvironment.findall_async) and [`finditer_async()`](api.md#jsonpath.JSONPathEnvironment.finditer_async) are [asyncio](https://docs.python.org/3/library/asyncio.html) equivalents to [`findall()`](api.md#jsonpath.JSONPathEnvironment.findall) and [`finditer()`](api.md#jsonpath.JSONPathEnvironment.finditer). By default, any class implementing the [mapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) or [sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence) interfaces, and a `__getitem_async__()` method, will have `__getitem_async__()` awaited instead of calling `__getitem__()` when resolving mapping keys or sequence indices.
 
 ## Example
 

docs/exceptions.md

@@ -16,3 +16,36 @@ Each of the following exceptions has a `token` property, referencing the [`Token
 
 ::: jsonpath.JSONPathNameError
     handler: python
+
+::: jsonpath.JSONPointerError
+    handler: python
+
+::: jsonpath.JSONPointerEncodeError
+    handler: python
+
+::: jsonpath.JSONPointerResolutionError
+    handler: python
+
+::: jsonpath.JSONPointerIndexError
+    handler: python
+
+::: jsonpath.JSONPointerKeyError
+    handler: python
+
+::: jsonpath.JSONPointerTypeError
+    handler: python
+
+::: jsonpath.RelativeJSONPointerError
+    handler: python
+
+::: jsonpath.RelativeJSONPointerIndexError
+    handler: python
+
+::: jsonpath.RelativeJSONPointerSyntaxError
+    handler: python
+
+::: jsonpath.JSONPatchError
+    handler: python
+
+::: jsonpath.JSONPatchTestFailure
+    handler: python

docs/query.md

@@ -82,7 +82,7 @@ for value in it.values():
 
 ## Tee
 
-And finally there's `tee()`, which creates multiple independent queries from one query iterator. It is not safe to use the initial `Query` instance after calling `tee()`.
+[`tee()`](api.md#jsonpath.Query.tee) creates multiple independent queries from one query iterator. It is not safe to use the initial `Query` instance after calling `tee()`.
 
 ```python
 from jsonpath import query
@@ -92,3 +92,130 @@ it1, it2 = query("$.some[[email protected]]", data).tee()
 head = it1.head(10) # first 10 matches
 tail = it2.tail(10) # last 10 matches
 ```
+
+## Select
+
+[`select(*expressions, projection=Projection.RELATIVE)`](api.md/#jsonpath.Query.select) performs JSONPath match projection, selecting a subset of values according to one or more JSONPath query expressions relative to the match location. For example:
+
+```python
+from jsonpath import query
+
+data = {
+    "categories": [
+        {
+            "name": "footwear",
+            "products": [
+                {
+                    "title": "Trainers",
+                    "description": "Fashionable trainers.",
+                    "price": 89.99,
+                },
+                {
+                    "title": "Barefoot Trainers",
+                    "description": "Running trainers.",
+                    "price": 130.00,
+                    "social": {"likes": 12, "shares": 7},
+                },
+            ],
+        },
+        {
+            "name": "headwear",
+            "products": [
+                {
+                    "title": "Cap",
+                    "description": "Baseball cap",
+                    "price": 15.00,
+                },
+                {
+                    "title": "Beanie",
+                    "description": "Winter running hat.",
+                    "price": 9.00,
+                },
+            ],
+        },
+    ],
+    "price_cap": 10,
+}
+
+for product in query("$..products.*", data).select("title", "price"):
+    print(product)
+```
+
+Which selects just the `title` and `price` fields for each product.
+
+```text
+{'title': 'Trainers', 'price': 89.99}
+{'title': 'Barefoot Trainers', 'price': 130.0}
+{'title': 'Cap', 'price': 15.0}
+{'title': 'Beanie', 'price': 9.0}
+```
+
+Without the call to `select()`, we'd get all fields in each product object.
+
+```python
+# ...
+
+for product in query("$..products.*", data).values():
+    print(product)
+```
+
+```text
+{'title': 'Trainers', 'description': 'Fashionable trainers.', 'price': 89.99}
+{'title': 'Barefoot Trainers', 'description': 'Running trainers.', 'price': 130.0, 'social': {'likes': 12, 'shares': 7}}
+{'title': 'Cap', 'description': 'Baseball cap', 'price': 15.0}
+{'title': 'Beanie', 'description': 'Winter running hat.', 'price': 9.0}
+```
+
+We can select nested values too.
+
+```python
+# ...
+
+for product in query("$..products.*", data).select("title", "social.shares"):
+    print(product)
+```
+
+```text
+{'title': 'Trainers'}
+{'title': 'Barefoot Trainers', 'social': {'shares': 7}}
+{'title': 'Cap'}
+{'title': 'Beanie'}
+```
+
+And flatten the selection into a sequence of values.
+
+```python
+from jsonpath import Projection
+
+# ...
+
+for product in query("$..products.*", data).select(
+    "title", "social.shares", projection=Projection.FLAT
+):
+    print(product)
+```
+
+```text
+['Trainers']
+['Barefoot Trainers', 7]
+['Cap']
+['Beanie']
+```
+
+Or project the selection from the JSON value root.
+
+```python
+# ..
+
+for product in query("$..products[[email protected]]", data).select(
+    "title",
+    "social.shares",
+    projection=Projection.ROOT,
+):
+    print(product)
+
+```
+
+```text
+{'categories': [{'products': [{'title': 'Barefoot Trainers', 'social': {'shares': 7}}]}]}
+```

docs/quickstart.md

@@ -4,7 +4,7 @@ This page gets you started using JSONPath, JSON Pointer and JSON Patch wih Pytho
 
 ## `findall(path, data)`
 
-Find all objects matching a JSONPath with [`jsonpath.findall()`](api.md#jsonpath.env.JSONPathEnvironment.findall). It takes, as arguments, a JSONPath string and some _data_ object. It always returns a list of objects selected from _data_, never a scalar value.
+Find all objects matching a JSONPath with [`jsonpath.findall()`](api.md#jsonpath.JSONPathEnvironment.findall). It takes, as arguments, a JSONPath string and some _data_ object. It always returns a list of objects selected from _data_, never a scalar value.
 
 _data_ can be a file-like object or string containing JSON formatted data, or a Python [`Mapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) or [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence), like a dictionary or list. In this example we select user names from a dictionary containing a list of user dictionaries.
 
@@ -52,7 +52,7 @@ with open("users.json") as fd:
 
 ## `finditer(path, data)`
 
-Use [`jsonpath.finditer()`](api.md#jsonpath.env.JSONPathEnvironment.finditer) to iterate over instances of [`jsonpath.JSONPathMatch`](api.md#jsonpath.JSONPathMatch) for every object in _data_ that matches _path_. It accepts the same arguments as [`findall()`](#findall), a path string and data from which to select matches.
+Use [`jsonpath.finditer()`](api.md#jsonpath.JSONPathEnvironment.finditer) to iterate over instances of [`jsonpath.JSONPathMatch`](api.md#jsonpath.JSONPathMatch) for every object in _data_ that matches _path_. It accepts the same arguments as [`findall()`](#findallpath-data), a path string and data from which to select matches.
 
 ```python
 import jsonpath
@@ -96,7 +96,7 @@ The selected object is available from a [`JSONPathMatch`](api.md#jsonpath.JSONPa
 
 ## `compile(path)`
 
-When you have a JSONPath that needs to be matched against different data repeatedly, you can _compile_ the path ahead of time using [`jsonpath.compile()`](api.md#jsonpath.env.JSONPathEnvironment.compile). It takes a path as a string and returns a [`JSONPath`](api.md#jsonpath.JSONPath) instance. `JSONPath` has `findall()` and `finditer()` methods that behave similarly to package-level `findall()` and `finditer()`, just without the `path` argument.
+When you have a JSONPath that needs to be matched against different data repeatedly, you can _compile_ the path ahead of time using [`jsonpath.compile()`](api.md#jsonpath.JSONPathEnvironment.compile). It takes a path as a string and returns a [`JSONPath`](api.md#jsonpath.JSONPath) instance. `JSONPath` has `findall()` and `finditer()` methods that behave similarly to package-level `findall()` and `finditer()`, just without the `path` argument.
 
 ```python
 import jsonpath
@@ -137,7 +137,7 @@ other_users = path.findall(other_data)
 
 **_New in version 0.8.0_**
 
-Get a [`jsonpath.JSONPathMatch`](api.md#jsonpath.JSONPathMatch) instance for the first match found in _data_. If there are no matches, `None` is returned. `match()` accepts the same arguments as [`findall()`](#findall).
+Get a [`jsonpath.JSONPathMatch`](api.md#jsonpath.JSONPathMatch) instance for the first match found in _data_. If there are no matches, `None` is returned. `match()` accepts the same arguments as [`findall()`](#findallpath-data).
 
 ```python
 import jsonpath
@@ -228,7 +228,7 @@ sue_score = pointer.resolve("/users/99/score", data, default=0)
 print(sue_score)  # 0
 ```
 
-See also [`JSONPathMatch.pointer()`](api.md#jsonpath.match.JSONPathMatch.pointer), which builds a [`JSONPointer`](api.md#jsonpath.JSONPointer) from a `JSONPathMatch`.
+See also [`JSONPathMatch.pointer()`](api.md#jsonpath.JSONPathMatch.pointer), which builds a [`JSONPointer`](api.md#jsonpath.JSONPointer) from a `JSONPathMatch`.
 
 ## `patch.apply(patch, data)`
 
@@ -294,7 +294,7 @@ print(data)  # {'some': {'other': 'thing', 'foo': {'bar': [1], 'else': 'thing'}}
 
 ## What's Next?
 
-Read about the [Query Iterators](query.md) API or [user-defined filter functions](advanced.md#function-extensions). Also see how to make extra data available to filters with [Extra Filter Context](advanced.md#extra-filter-context).
+Read about the [Query Iterators](query.md) API or [user-defined filter functions](advanced.md#function-extensions). Also see how to make extra data available to filters with [Extra Filter Context](advanced.md#filter-variables).
 
 `findall()`, `finditer()` and `compile()` are shortcuts that use the default[`JSONPathEnvironment`](api.md#jsonpath.JSONPathEnvironment). `jsonpath.findall(path, data)` is equivalent to:
 

jsonpath/__about__.py

@@ -1,4 +1,4 @@
 # SPDX-FileCopyrightText: 2023-present James Prior <[email protected]>
 #
 # SPDX-License-Identifier: MIT
-__version__ = "1.1.2"
+__version__ = "1.2.0"

jsonpath/__init__.py

@@ -3,11 +3,14 @@
 # SPDX-License-Identifier: MIT
 
 from .env import JSONPathEnvironment
+from .exceptions import JSONPatchError
+from .exceptions import JSONPatchTestFailure
 from .exceptions import JSONPathError
 from .exceptions import JSONPathIndexError
 from .exceptions import JSONPathNameError
 from .exceptions import JSONPathSyntaxError
 from .exceptions import JSONPathTypeError
+from .exceptions import JSONPointerEncodeError
 from .exceptions import JSONPointerError
 from .exceptions import JSONPointerIndexError
 from .exceptions import JSONPointerKeyError
@@ -17,6 +20,7 @@
 from .exceptions import RelativeJSONPointerIndexError
 from .exceptions import RelativeJSONPointerSyntaxError
 from .filter import UNDEFINED
+from .fluent_api import Projection
 from .fluent_api import Query
 from .lex import Lexer
 from .match import JSONPathMatch
@@ -36,6 +40,8 @@
     "finditer_async",
     "finditer",
     "JSONPatch",
+    "JSONPatchError",
+    "JSONPatchTestFailure",
     "JSONPath",
     "JSONPathEnvironment",
     "JSONPathError",
@@ -45,6 +51,7 @@
     "JSONPathSyntaxError",
     "JSONPathTypeError",
     "JSONPointer",
+    "JSONPointerEncodeError",
     "JSONPointerError",
     "JSONPointerIndexError",
     "JSONPointerKeyError",
@@ -53,6 +60,7 @@
     "Lexer",
     "match",
     "Parser",
+    "Projection",
     "query",
     "Query",
     "RelativeJSONPointer",

jsonpath/env.py

@@ -353,7 +353,7 @@ def query(
             ...
         ```
         """
-        return Query(self.finditer(path, data, filter_context=filter_context))
+        return Query(self.finditer(path, data, filter_context=filter_context), self)
 
     async def findall_async(
         self,