Subject: Re: [Jsonpath] Jsonpath Digest, Vol 5, Issue 16

From: Daniel P <[email-masked]>
Date: Wed, 18 Nov 2020 08:59:09 -0500

> From: Marko Mikulicic <[email-masked]>

> It was my impression that we were talking about duplicated nodes not duplicated values:
>
Okay. Terms are being used in ways that are not clear (to me), hence
my question.

> Given the array [10,20,30]
>
> $..[0,1,0]
>
> Would yield only two results [10, 20]
>
> (Not that I'm advocating for removing duplicates, personally I think we shouldn't)
>
>
It's hard to see how a standardized JSONPath could, as my understanding is that
most legacy implementations do not.

An exception may be npm jsonpath, https://www.npmjs.com/package/jsonpath. The
section "Other Minor Differences [from Stefan Goessner's original
implementation]"
includes the text: "unions now yield real unions with no duplicates
rather than concatenated
results".

In npm jsonpath, given the array [10,20,30,30], $..[0,1,0] yields

[10,20]

and $..[1,2,3] yields

[20,30,30]

Daniel

---

From: Marko Mikulicic <[email-masked]>
Date: Wed, 18 Nov 2020 17:02:34 +0100

The mental model that leads to omitting duplicate nodes in the output is
"selection": if you take an input array and select nodes with index 0,1 or
0, you get only 2 results (since selecting an index twice has no effect).

OTOH, if you opt for a "collect" model, whenever you encounter a node that
matches that query you add it to the result stream, thus the same nodes can
be present multiple times in the result.

I have a slight preference for the "collect" model, because the general
case in jsonpath is to collect things that appear at various points in the
json tree. For example:

{"a": {"b": 1, "c": 2}, "d": 3},  $.a.b yields [1] and not {"a":{"b":1}}
(i.e. jsonpath is not a filter and view operation but a pick and gather
operation)

On Wed, 18 Nov 2020 at 14:59, Daniel P <[email-masked]> wrote:

> --
> Jsonpath mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jsonpath
>

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 18 Nov 2020 21:06:42 +0100

> On 2020-11-18, at 17:02, Marko Mikulicic <[email-masked]> wrote:
> 
> {"a": {"b": 1, "c": 2}, "d": 3},  $.a.b yields [1] and not {"a":{"b":1}} (i.e. jsonpath is not a filter and view operation but a pick and gather operation)

Actually it yields /a/b (in RFC 6901 format) or $.a.b (in JSONpath format).

(Which of these is true depends on the output format, which is kind of unrelated to the query model.)

Grüße, Carsten

---

From: Marko Mikulicic <[email-masked]>
Date: Thu, 19 Nov 2020 00:28:07 +0100

Ah nice, thanks.

(I'm just trying to understand, I don't have strong opinions)

If I understand correctly the idea is to conceptually split the effective
evaluation in two parts:

1. Produce an array/set of "selected paths"
2. (Optionally) resolve the resulting paths into their pointed values.

Of these two, only the first one is relevant to query model.

The second part however is relevant to discussions around effective use
cases (e.g. whether we must allow implementations to fuse (1) and (2) and
to directly yield values)

Let's focus on the query model:

We still have the choice of how to model the results: it can be ordered or
unordered, it can contain unique paths or allow duplicate paths.

Furthermore the order can be:

* fully non-deterministic
* partially determined by input (e.g. array order preserved, but paths
derived from object fields unordered)
* fully determined by input (e.g. using a canonical ordering of field names
in objects; as commonly done when signing jsons)
* or fully determined by output (e.g. sorting all the resulting paths)

It's not clear to me how would we navigate this choice.

One one hand we have user expectations (based in use cases, end-to-end
experience, ...).
On the other we have implementations concerns (e.g. avoid sorting, avoid
extra data structures to dedup).
Ah, and we also have to attempt minimizing the disruption of tons of
existing implementations.

Do we even have to make a choice?

Could we limit the scope of the query model to just define which elements
are selected and leave the definition on how the selection is conveyed to
the consumer to another specification layer?

I'm not talking about complete freedom to implementations. Perhaps we could
define a few standard resultset types. Implementations could implement one
or more of them, but must advertise which one(s) it implements.

WDYT?

Cheers,
Marko

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 19 Nov 2020 00:34:18 +0100

> Do we even have to make a choice?

If we can achieve not having to make a choice, this approach would definitely be worth it.

> I'm not talking about complete freedom to implementations. Perhaps we could define a few standard resultset types. Implementations could implement one or more of them, but must advertise which one(s) it implements.

If we can achieve pushing (most of) the variability into the output processing model, we have a big win.

Now back to the hundreds of test cases and seeing if this works with (the majority of) current implementations.

Grüße, Carsten