Skip to content

Python JSONPath Version 2 #98

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
4bfcb7c
Version 2 WIP [skip ci]
jg-rp Aug 7, 2025
33fe76d
Rewrite parser WIP [skip ci]
jg-rp Aug 8, 2025
c7a10af
Fix canonical paths, compound paths and list literals
jg-rp Aug 9, 2025
e338a0c
Remove shorthand arguments to Property, Wild and Keys selectors
jg-rp Aug 10, 2025
b4cb9c2
Add "key" and "keys filter" JSONPath selectors
jg-rp Aug 11, 2025
7a55c02
Test "extra" JSONPath syntax
jg-rp Aug 12, 2025
ea84ed9
Singular query selector stub [skip ci]
jg-rp Aug 13, 2025
9a1886e
Implement the singular query selector
jg-rp Aug 13, 2025
616f438
Rename some selector classes and tidy.
jg-rp Aug 14, 2025
a5605a1
Introduce strict mode and use regex if available
jg-rp Aug 14, 2025
e98453d
Add strict lexer rules
jg-rp Aug 16, 2025
7e09eaf
Test for compliance in strict and lax mode
jg-rp Aug 16, 2025
6e1f3b7
Separate test cases for non-standard sytnax in to JSON files WIP
jg-rp Aug 18, 2025
225f686
Assert that non-standard syntax fails in strict mode
jg-rp Aug 18, 2025
e087f70
Remember to test async path too
jg-rp Aug 18, 2025
f6909ca
More tidy of test cases
jg-rp Aug 18, 2025
9a73434
Enforce recursion limit and more tidying
jg-rp Aug 19, 2025
71a43ba
More tests and refactor parser.parse_query
jg-rp Aug 20, 2025
f384b63
Pretty exception messages
jg-rp Aug 20, 2025
efb0f7d
Update docs WIP [skip ci]
jg-rp Aug 20, 2025
dd37e3d
Syntax docs WIP [skip ci]
jg-rp Aug 21, 2025
cfa891c
More docs [skip ci]
jg-rp Aug 21, 2025
6ce68d2
Add a `strict` argument to convenience functions
jg-rp Aug 22, 2025
bdbc677
Add a `strict` argument to the JSONPath CLI
jg-rp Aug 22, 2025
a8bace4
Add package level functions to docs and update CLI docs [skip ci]
jg-rp Aug 22, 2025
833a6d6
More docs updates [skip ci]
jg-rp Aug 23, 2025
f7bdc18
Check regex patterns against I-Regexp if available
jg-rp Aug 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/workflows/tests-no-regex.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: test-no-regex
on: [push, pull_request]

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
submodules: true
- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: "3.11"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade hatch
- run: hatch -e no-regex run test
22 changes: 1 addition & 21 deletions docs/advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Filter Variables

Arbitrary variables can be made available to [filter expressions](syntax.md#filters-expression) using the _filter_context_ argument to [`findall()`](quickstart.md#findallpath-data) and [`finditer()`](quickstart.md#finditerpath-data). _filter_context_ should be a [mapping](https://docs.python.org/3/library/typing.html#typing.Mapping) of strings to JSON-like objects, like lists, dictionaries, strings and integers.
Arbitrary variables can be made available to [filter selectors](syntax.md#filter-selector) using the `filter_context` argument to [`findall()`](quickstart.md#findallpath-data) and [`finditer()`](quickstart.md#finditerpath-data). `filter_context` should be a [mapping](https://docs.python.org/3/library/typing.html#typing.Mapping) of strings to JSON-like objects, like lists, dictionaries, strings and integers.

Filter context variables are selected using a filter query starting with the _filter context identifier_, which defaults to `_` and has usage similar to `$` and `@`.

Expand Down Expand Up @@ -257,23 +257,3 @@ env = MyJSONPathEnvironment()
query = env.compile("$.users[999]")
# jsonpath.exceptions.JSONPathIndexError: index out of range, line 1, column 8
```

### Subclassing Lexer

TODO:

### Subclassing Parser

TODO:

### Get Item

TODO:

### Truthiness and Existence

TODO:

### Filter Infix Expressions

TODO:
4 changes: 0 additions & 4 deletions docs/async.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,3 @@ data = {
best_a_team_players = jsonpath.findall_async("$.teams['A Team'][?rank >= 8]", data)

```

## Custom Async Item Getting

TODO:
7 changes: 7 additions & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ optional arguments:
File to write resulting objects to, as a JSON array. Defaults to the standard
output stream.
--no-type-checks Disables filter expression well-typedness checks.
--strict Compile and evaluate JSONPath expressions with strict compliance with RFC 9535.
```

## Global Options
Expand Down Expand Up @@ -191,6 +192,12 @@ _New in version 0.10.0_

Disables JSONPath filter expression well-typedness checks. The well-typedness of a filter expression is defined by RFC 9535.

#### `--strict`

_New in version 2.0.0_

Compile and evaluate JSONPath expressions with strict compliance with RFC 9535.

### `pointer`

Resolve a JSON Pointer against a JSON document. One of `-p`/`--pointer` or `-r`/`--pointer-file` must be given. `-p` being a JSON Pointer given on the command line as a string, `-r` being the path to a file containing a JSON Pointer.
Expand Down
31 changes: 31 additions & 0 deletions docs/convenience.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Convenience Functions

These package-level functions use the default [JSONPathEnvironment](api.md#jsonpath.JSONPathEnvironment), `jsonpath.DEFAULT_ENV` when `strict=False`, or the preconfigured strict environment, `jsonpath.STRICT_ENV` when `strict=True`.

::: jsonpath.compile

handler: python

::: jsonpath.findall

handler: python

::: jsonpath.finditer

handler: python

::: jsonpath.findall_async

handler: python

::: jsonpath.finditer_async

handler: python

::: jsonpath.match

handler: python

::: jsonpath.query

handler: python
2 changes: 1 addition & 1 deletion docs/functions.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Filter Functions

A filter function is a named function that can be called as part of a [filter selector](syntax.md#filters-expression) expression. Here we describe built-in filters. You can [define your own function extensions](advanced.md#function-extensions) too.
A filter function is a named function that can be called as part of a [filter selector](syntax.md#filter-selector). Here we describe built in filters. You can [define your own function extensions](advanced.md#function-extensions) too.

## `count()`

Expand Down
10 changes: 9 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

JSONPath is a mini language for selecting values from data formatted in JavaScript Object Notation, or equivalent Python objects, like dictionaries and lists.

Python JSONPath is a non-evaluating, read-only implementation of JSONPath, suitable for situations where JSONPath query authors are untrusted. We follow most of [RFC 9535](https://datatracker.ietf.org/doc/html/rfc9535). See [Notable differences](syntax.md#notable-differences) for a list of areas where we deviate from the standard.
Python JSONPath is a non-evaluating, read-only implementation of JSONPath, suitable for situations where JSONPath query authors are untrusted. We follow [RFC 9535](https://datatracker.ietf.org/doc/html/rfc9535) and test against the [JSONPath Compliance Test Suite](https://github.com/jsonpath-standard/jsonpath-compliance-test-suite).

We also include implementations of [JSON Pointer](pointers.md) ([RFC 6901](https://datatracker.ietf.org/doc/html/rfc6901)) and [JSON Patch](api.md#jsonpath.JSONPatch) ([RFC 6902](https://datatracker.ietf.org/doc/html/rfc6902)), plus methods for converting a [JSONPathMatch](api.md#jsonpath.JSONPathMatch) to a `JSONPointer`.

Expand Down Expand Up @@ -32,6 +32,14 @@ Or from [conda-forge](https://anaconda.org/conda-forge/python-jsonpath):
conda install -c conda-forge python-jsonpath
```

### Optional dependencies

By default, and without any additional dependencies, the syntax supported by Python JSONPath is **very close** to RFC 9535. For strict compatibility with the specification, install [regex](https://pypi.org/project/regex/) and [iregexp-check](https://pypi.org/project/iregexp-check/) packages too.

With these two packages installed, the [`match()`](functions.md#match) and [`search()`](functions.md#search) filter functions will use [regex](https://pypi.org/project/regex/) instead of `re` from the standard library, and will validate regular expression patterns against [RFC 9485](https://datatracker.ietf.org/doc/html/rfc9485).

See the [syntax guide](syntax.md) for more information about strict compatibility with RFC 9535 and extensions to the specification.

## Example

```python
Expand Down
2 changes: 1 addition & 1 deletion docs/pointers.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ JSON Pointers are a fundamental component of JSON Patch ([RFC 6902](https://data

We have extended RFC 6901 to support:

- Interoperability with the JSONPath [keys selector](syntax.md#keys-or) (`~`)
- Interoperability with the JSONPath [keys selector](syntax.md#keys-selector) (`~`)
- A special non-standard syntax for targeting **keys or indices themselves**, used in conjunction with [Relative JSON Pointer](#torel)

**Keys Selector Compatibility**
Expand Down
12 changes: 6 additions & 6 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@ This page gets you started using JSONPath, JSON Pointer and JSON Patch wih Pytho

## `findall(path, data)`

Find all values matching a JSONPath expression using [`jsonpath.findall()`](api.md#jsonpath.JSONPathEnvironment.findall).
Find all values matching a JSONPath query using [`jsonpath.findall()`](convenience.md#jsonpath.findall).

This function takes two arguments:

- `path`: a JSONPath expression as a string (e.g., `"$.users[*].name"`)
- `path`: a JSONPath query as a string (e.g. `"$.users[*].name"`)
- `data`: the JSON document to query

It always returns a **list** of matched values, even if the path resolves to a single result or nothing at all.
It **always** returns a list of matched values, even if the path resolves to a single result or nothing at all.

The `data` argument can be:

- A Python [`Mapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) (e.g., `dict`) or [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence) (e.g., `list`)
- A Python [`Mapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) (e.g. `dict`) or [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence) (e.g. `list`)
- A JSON-formatted string
- A file-like object containing JSON

Expand Down Expand Up @@ -65,7 +65,7 @@ with open("users.json") as fd:

## `finditer(path, data)`

Use [`jsonpath.finditer()`](api.md#jsonpath.JSONPathEnvironment.finditer) to iterate over instances of [`jsonpath.JSONPathMatch`](api.md#jsonpath.JSONPathMatch) for every object in _data_ that matches _path_. It accepts the same arguments as [`findall()`](#findallpath-data), a path string and data from which to select matches.
Use [`jsonpath.finditer()`](convenience.md#jsonpath.finditer) to iterate over instances of [`jsonpath.JSONPathMatch`](api.md#jsonpath.JSONPathMatch) for every object in _data_ that matches _path_. It accepts the same arguments as [`findall()`](#findallpath-data), a query string and data from which to select matches.

```python
import jsonpath
Expand Down Expand Up @@ -109,7 +109,7 @@ The selected object is available from a [`JSONPathMatch`](api.md#jsonpath.JSONPa

## `compile(path)`

When you have a JSONPath that needs to be matched against different data repeatedly, you can _compile_ the path ahead of time using [`jsonpath.compile()`](api.md#jsonpath.JSONPathEnvironment.compile). It takes a path as a string and returns a [`JSONPath`](api.md#jsonpath.JSONPath) instance. `JSONPath` has `findall()` and `finditer()` methods that behave similarly to package-level `findall()` and `finditer()`, just without the `path` argument.
When you have a JSONPath query that needs to be matched against different data repeatedly, you can compile the path ahead of time using [`jsonpath.compile()`](convenience.md#jsonpath.compile). It takes a query as a string and returns an instance of [`JSONPath`](api.md#jsonpath.JSONPath). `JSONPath` has `findall()` and `finditer()` methods that behave similarly to package-level `findall()` and `finditer()`, just without the `path` argument.

```python
import jsonpath
Expand Down
Loading
Loading