Merge pull request #98 from jg-rp/v2

Python JSONPath Version 2

Author: jg-rp

.github/workflows/tests-no-regex.yaml

@@ -0,0 +1,19 @@
+name: test-no-regex
+on: [push, pull_request]
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          submodules: true
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.11"
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install --upgrade hatch
+      - run: hatch -e no-regex run test

CHANGELOG.md

@@ -1,5 +1,36 @@
 # Python JSONPath Change Log
 
+## Version 2.0.0 (unreleased)
+
+**JSONPath syntax changes**
+
+These breaking changes apply to Python JSONPath in its default configuration. We've also introduced a _strict mode_ where we follow the RFC 9535 specification exactly. See [optional dependencies](https://jg-rp.github.io/python-jsonpath/#optional-dependencies) and the [syntax guide](https://jg-rp.github.io/python-jsonpath/syntax/) for more information.
+
+- Using bracket notation, unquoted property names are no longer interpreted as quoted property names. These paths used to be equivalent, `$[foo]`, `$['foo']` and `$["foo"]`. Now, names without quotes start a _singular query selector_. With an implicit _root identifier_, `$.a[b]` is equivalent to `$.a[$.b]`. See [Singular query selector](https://jg-rp.github.io/python-jsonpath/syntax/#singular-query-selector) in the syntax guide.
+- In filter selector expressions, float literals now follow the specification. Previously `.1` and `1.` where allowed, now it must be `0.1` and `1.0`, with at least one digit either side of the decimal point.
+- Slice selector indexes and step now follow the specification. Previously leading zeros and negative zero were allowed, now they raise a `JSONPathSyntaxError`.
+- Whitespace is no longer allowed between a dot (`.` or `..`) and a name when using shorthand notation for the name selector. Whitespace before the dot oor double dot is OK.
+
+**JSONPath features**
+
+- Added the [Keys filter selector](https://jg-rp.github.io/python-jsonpath/syntax/#keys-filter-selector).
+- Added the [Singular query selector](https://jg-rp.github.io/python-jsonpath/syntax/#singular-query-selector).
+- We now use the [regex] package, if available, instead of `re` for match and search function extensions. See [optional dependencies](https://jg-rp.github.io/python-jsonpath/#optional-dependencies).
+- Added the `strict` argument to all [convenience functions](https://jg-rp.github.io/python-jsonpath/convenience/), the CLI and the `JSONPathEnvironment` constructor. When `strict=True`, all extensions to RFC 9535 and any lax parsing rules will be disabled.
+- Added class variable `JSONPathEnvironment.max_recursion_depth` to control the maximum recursion depth of descendant segments.
+- Added pretty exception messages.
+
+**Python API changes**
+
+- Renamed class variable `JSONPathEnvironment.fake_root_token` to `JSONPathEnvironment.pseudo_root_token`.
+
+**Low level API changes**
+
+These breaking changes will only affect you if you're customizing the JSONPath lexer or parser.
+
+- The tokens produced by the JSONPath lexer have changed. Previously we broadly skipped some punctuation and whitespace. Now the parser can make better choices about when to accept whitespace and do a better job of enforcing dots.
+- We've change the internal representation of compiled JSONPath queries. We now model segments and selectors explicitly and use terminology that matches RFC 9535.
+
 ## Version 1.3.2
 
 **Fixes**

docs/advanced.md

@@ -2,7 +2,7 @@
 
 ## Filter Variables
 
-Arbitrary variables can be made available to [filter expressions](syntax.md#filters-expression) using the _filter_context_ argument to [`findall()`](quickstart.md#findallpath-data) and [`finditer()`](quickstart.md#finditerpath-data). _filter_context_ should be a [mapping](https://docs.python.org/3/library/typing.html#typing.Mapping) of strings to JSON-like objects, like lists, dictionaries, strings and integers.
+Arbitrary variables can be made available to [filter selectors](syntax.md#filter-selector) using the `filter_context` argument to [`findall()`](quickstart.md#findallpath-data) and [`finditer()`](quickstart.md#finditerpath-data). `filter_context` should be a [mapping](https://docs.python.org/3/library/typing.html#typing.Mapping) of strings to JSON-like objects, like lists, dictionaries, strings and integers.
 
 Filter context variables are selected using a filter query starting with the _filter context identifier_, which defaults to `_` and has usage similar to `$` and `@`.
 
@@ -257,23 +257,3 @@ env = MyJSONPathEnvironment()
 query = env.compile("$.users[999]")
 # jsonpath.exceptions.JSONPathIndexError: index out of range, line 1, column 8
 ```
-
-### Subclassing Lexer
-
-TODO:
-
-### Subclassing Parser
-
-TODO:
-
-### Get Item
-
-TODO:
-
-### Truthiness and Existence
-
-TODO:
-
-### Filter Infix Expressions
-
-TODO:

docs/async.md

@@ -59,7 +59,3 @@ data = {
 best_a_team_players = jsonpath.findall_async("$.teams['A Team'][?rank >= 8]", data)
 
 ```
-
-## Custom Async Item Getting
-
-TODO:

docs/cli.md

@@ -62,6 +62,7 @@ optional arguments:
                         File to write resulting objects to, as a JSON array. Defaults to the standard
                         output stream.
   --no-type-checks      Disables filter expression well-typedness checks.
+  --strict              Compile and evaluate JSONPath expressions with strict compliance with RFC 9535.
 ```
 
 ## Global Options
@@ -191,6 +192,12 @@ _New in version 0.10.0_
 
 Disables JSONPath filter expression well-typedness checks. The well-typedness of a filter expression is defined by RFC 9535.
 
+#### `--strict`
+
+_New in version 2.0.0_
+
+Compile and evaluate JSONPath expressions with strict compliance with RFC 9535.
+
 ### `pointer`
 
 Resolve a JSON Pointer against a JSON document. One of `-p`/`--pointer` or `-r`/`--pointer-file` must be given. `-p` being a JSON Pointer given on the command line as a string, `-r` being the path to a file containing a JSON Pointer.

docs/convenience.md

@@ -0,0 +1,31 @@
+# Convenience Functions
+
+These package-level functions use the default [JSONPathEnvironment](api.md#jsonpath.JSONPathEnvironment), `jsonpath.DEFAULT_ENV` when `strict=False`, or the preconfigured strict environment, `jsonpath.STRICT_ENV` when `strict=True`.
+
+::: jsonpath.compile
+
+    handler: python
+
+::: jsonpath.findall
+
+    handler: python
+
+::: jsonpath.finditer
+
+    handler: python
+
+::: jsonpath.findall_async
+
+    handler: python
+
+::: jsonpath.finditer_async
+
+    handler: python
+
+::: jsonpath.match
+
+    handler: python
+
+::: jsonpath.query
+
+    handler: python

docs/functions.md

@@ -1,6 +1,6 @@
 # Filter Functions
 
-A filter function is a named function that can be called as part of a [filter selector](syntax.md#filters-expression) expression. Here we describe built-in filters. You can [define your own function extensions](advanced.md#function-extensions) too.
+A filter function is a named function that can be called as part of a [filter selector](syntax.md#filter-selector). Here we describe built in filters. You can [define your own function extensions](advanced.md#function-extensions) too.
 
 ## `count()`
 

docs/index.md

@@ -2,7 +2,7 @@
 
 JSONPath is a mini language for selecting values from data formatted in JavaScript Object Notation, or equivalent Python objects, like dictionaries and lists.
 
-Python JSONPath is a non-evaluating, read-only implementation of JSONPath, suitable for situations where JSONPath query authors are untrusted. We follow most of [RFC 9535](https://datatracker.ietf.org/doc/html/rfc9535). See [Notable differences](syntax.md#notable-differences) for a list of areas where we deviate from the standard.
+Python JSONPath is a non-evaluating, read-only implementation of JSONPath, suitable for situations where JSONPath query authors are untrusted. We follow [RFC 9535](https://datatracker.ietf.org/doc/html/rfc9535) and test against the [JSONPath Compliance Test Suite](https://github.com/jsonpath-standard/jsonpath-compliance-test-suite).
 
 We also include implementations of [JSON Pointer](pointers.md) ([RFC 6901](https://datatracker.ietf.org/doc/html/rfc6901)) and [JSON Patch](api.md#jsonpath.JSONPatch) ([RFC 6902](https://datatracker.ietf.org/doc/html/rfc6902)), plus methods for converting a [JSONPathMatch](api.md#jsonpath.JSONPathMatch) to a `JSONPointer`.
 
@@ -32,6 +32,14 @@ Or from [conda-forge](https://anaconda.org/conda-forge/python-jsonpath):
 conda install -c conda-forge python-jsonpath
 ```
 
+### Optional dependencies
+
+By default, and without any additional dependencies, the syntax supported by Python JSONPath is **very close** to RFC 9535. For strict compatibility with the specification, install [regex](https://pypi.org/project/regex/) and [iregexp-check](https://pypi.org/project/iregexp-check/) packages too.
+
+With these two packages installed, the [`match()`](functions.md#match) and [`search()`](functions.md#search) filter functions will use [regex](https://pypi.org/project/regex/) instead of `re` from the standard library, and will validate regular expression patterns against [RFC 9485](https://datatracker.ietf.org/doc/html/rfc9485).
+
+See the [syntax guide](syntax.md) for more information about strict compatibility with RFC 9535 and extensions to the specification.
+
 ## Example
 
 ```python

docs/pointers.md

@@ -10,7 +10,7 @@ JSON Pointers are a fundamental component of JSON Patch ([RFC 6902](https://data
 
     We have extended RFC 6901 to support:
 
-    - Interoperability with the JSONPath [keys selector](syntax.md#keys-or) (`~`)
+    - Interoperability with the JSONPath [keys selector](syntax.md#keys-selector) (`~`)
     - A special non-standard syntax for targeting **keys or indices themselves**, used in conjunction with [Relative JSON Pointer](#torel)
 
     **Keys Selector Compatibility**

docs/quickstart.md

@@ -4,18 +4,18 @@ This page gets you started using JSONPath, JSON Pointer and JSON Patch wih Pytho
 
 ## `findall(path, data)`
 
-Find all values matching a JSONPath expression using [`jsonpath.findall()`](api.md#jsonpath.JSONPathEnvironment.findall).
+Find all values matching a JSONPath query using [`jsonpath.findall()`](convenience.md#jsonpath.findall).
 
 This function takes two arguments:
 
-- `path`: a JSONPath expression as a string (e.g., `"$.users[*].name"`)
+- `path`: a JSONPath query as a string (e.g. `"$.users[*].name"`)
 - `data`: the JSON document to query
 
-It always returns a **list** of matched values, even if the path resolves to a single result or nothing at all.
+It **always** returns a list of matched values, even if the path resolves to a single result or nothing at all.
 
 The `data` argument can be:
 
-- A Python [`Mapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) (e.g., `dict`) or [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence) (e.g., `list`)
+- A Python [`Mapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) (e.g. `dict`) or [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence) (e.g. `list`)
 - A JSON-formatted string
 - A file-like object containing JSON
 
@@ -65,7 +65,7 @@ with open("users.json") as fd:
 
 ## `finditer(path, data)`
 
-Use [`jsonpath.finditer()`](api.md#jsonpath.JSONPathEnvironment.finditer) to iterate over instances of [`jsonpath.JSONPathMatch`](api.md#jsonpath.JSONPathMatch) for every object in _data_ that matches _path_. It accepts the same arguments as [`findall()`](#findallpath-data), a path string and data from which to select matches.
+Use [`jsonpath.finditer()`](convenience.md#jsonpath.finditer) to iterate over instances of [`jsonpath.JSONPathMatch`](api.md#jsonpath.JSONPathMatch) for every object in _data_ that matches _path_. It accepts the same arguments as [`findall()`](#findallpath-data), a query string and data from which to select matches.
 
 ```python
 import jsonpath
@@ -109,7 +109,7 @@ The selected object is available from a [`JSONPathMatch`](api.md#jsonpath.JSONPa
 
 ## `compile(path)`
 
-When you have a JSONPath that needs to be matched against different data repeatedly, you can _compile_ the path ahead of time using [`jsonpath.compile()`](api.md#jsonpath.JSONPathEnvironment.compile). It takes a path as a string and returns a [`JSONPath`](api.md#jsonpath.JSONPath) instance. `JSONPath` has `findall()` and `finditer()` methods that behave similarly to package-level `findall()` and `finditer()`, just without the `path` argument.
+When you have a JSONPath query that needs to be matched against different data repeatedly, you can compile the path ahead of time using [`jsonpath.compile()`](convenience.md#jsonpath.compile). It takes a query as a string and returns an instance of [`JSONPath`](api.md#jsonpath.JSONPath). `JSONPath` has `findall()` and `finditer()` methods that behave similarly to package-level `findall()` and `finditer()`, just without the `path` argument.
 
 ```python
 import jsonpath