Subject: [Jsonpath] January 2022 interim meeting and agenda

From: James G <[email-masked]>
Date: Mon, 3 Jan 2022 13:27:59 +0100

Folks,
This is a gentle reminder that our next interim meeting is in approximately two weeks on Tuesday, January 18th at 10:00 UTC.

https://datatracker.ietf.org/meeting/interim-2022-jsonpath-01/session/jsonpath

We haven't finalised (or published) the agenda yet, so if there’s any subjects or issues you’d like to be part of it please email myself or Tim and we can include it.

- J

---

From: Glyn Normington <[email-masked]>
Date: Mon, 3 Jan 2022 13:01:50 +0000

Thanks James. I would like to discuss the following:

https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/pull/143

Happy New Year everyone!

On Mon, 3 Jan 2022 at 12:28, James G <[email-masked]> wrote:

> Folks,
> This is a gentle reminder that our next interim meeting is in
> approximately two weeks on Tuesday, January 18th at 10:00 UTC.
>
>
> We haven't finalised (or published) the agenda yet, so if there’s any
> subjects or issues you’d like to be part of it please email myself or Tim
> and we can include it.
>
> - J
> --
> JSONpath mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jsonpath
>

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 3 Jan 2022 14:54:17 +0100

> https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/pull/143

Right.

I’ll continue the discussion here (instead of github), as we need to make a (somewhat non-technical) WG decision.

We know that there is no easy way to generate a specification for a standard form of regular expressions (RE) (there is no such standard form).  Taking it from there, options include:

(1) not having RE support in jsonpath-base
(2) defining a lowest-common-denominator syntax and including that in jsonpath-base
(3) having RE syntax but not defining what that means (“permissive”) and not providing a way to identify which syntax variant is being used

Within (2), there are multiple suboptions. Editorially:

(a) define the LCD RE syntax and semantics in jsonpath-base
(b) define the syntax and semantics in a separate document

(This is not a very important distinction, but a decision will have to be made.)

Note that we have to define both the RE syntax itself and the decoration that allows its use in jsonpath.

Within the selection process for those two, we’ll have to make subdivisions, of which I happen to have at the front of my mind:
(i) use a matching syntax (XSD-style) or a parsing syntax (PCRE-style)
(ii) make the outermost syntax look like existing syntax, failing many instances that don’t happen to presume the same syntax, but magically supporting some other instances that do happen to fit
(iii) use a regular expression subset that is actually based on regular expressions (Chomsky type 3 languages), or one that provides PCRE-style additions that don’t allow DFA transformation (and thus efficient, attack-resistant implementation)

So much for laying out the decisions.

Here is my opinion, based on a few decades of trying to make standards work:

Clearly, I think 3 is not OK for interoperability.  1 is OK by postponing the decision (as long as the extension point is announced); 2 is more “batteries included”.

Within 2, I favor 2b.
With respect to the technical subdecisions:

(i):
A matching syntax is actually what is needed.
Most programmers learn a parsing syntax first, because one use case for REs is writing shoddy parsers — something that we cannot use in JSONpath.
I wouldn’t mind making the decision for a matching syntax highly visible, also because (ii):
We don’t want to make existing REs “just work”, unless we can make sure that they are interoperable.

(iii):
I believe that actual RE-based syntaxes are a good subset decision anyway.  The only candidate here that is actually standardized in XSD, but that has some unwanted baggage (character class subtraction — which is great, but not widely offered by other RE syntaxes our LCD would need to translate into).

There are further detail decisions, e.g., what does \d or \w actually mean; we might simplify our lives by using a LCD subset that poses as few questions of this kind as possible.

Iregexp is a proposal for a separate document (2b) that plays along these lines.
I think I would like to develop this proposal further towards the goal expressed in the previous paragraph.

JSONpath of course is not the only area of application for RE syntaxes inside the IETF.  Notably, YANG and CDDL have chosen full XSD syntax, with striking acceptance problems on the YANG side (CDDL users seem to be more prudent here, in particular now with RFC 9165 available for complex cases; CDDL also has a defined extension point for dropping in multiple syntaxes).  This experience strengthens my preference for a LCD subset.

I understand that a world would be more likable where we could just drop in REs from a common, well-understood usage.  Unfortunately, we don’t have this world. Making decisions based on wishful thinking we did is not doing the right thing.

I also understand that my perspective may be somewhat different from other WG members, as I have already spent a significant amount of time researching the RE conundrum in different contexts.  So I’m very open to explaining this perspective in some more detail.

Grüße, Carsten

---

From: Glyn Normington <[email-masked]>
Date: Mon, 3 Jan 2022 15:16:31 +0000

Thanks Carsten. I agree with everything you wrote there.

I'm keen to complete the JSONPath draft within a reasonable time frame and
I fear that regexp support may take quite a while to finalise. So I'd like
us to consider finalising an *interim draft* without regexp support (i.e.
option 1). Such a draft would be a useful checkpoint and holding position
until iregexp is complete.

The other main benefit of finalising such an interim draft would be to
flush out any remaining changes that are needed in addition to regexp
support.

An interim draft would also be useful input toward a compliance test suite
and reference implementation (assuming someone will find time to work on
either or both of these!). The rest of the spec still being in flux is a
disincentive to anyone investing in building a CTS and RI.

On Mon, 3 Jan 2022, 13:54 Carsten Bormann, <[email-masked]> wrote:

> We know that there is no easy way to generate a specification for a
> standard form of regular expressions (RE) (there is no such standard
> form).  Taking it from there, options include:
>
> (3) having RE syntax but not defining what that means (“permissive”) and
> not providing a way to identify which syntax variant is being used
>
> Within the selection process for those two, we’ll have to make
> subdivisions, of which I happen to have at the front of my mind:
> (ii) make the outermost syntax look like existing syntax, failing many
> instances that don’t happen to presume the same syntax, but magically
> supporting some other instances that do happen to fit
> (iii) use a regular expression subset that is actually based on regular
> expressions (Chomsky type 3 languages), or one that provides PCRE-style
> additions that don’t allow DFA transformation (and thus efficient,
> attack-resistant implementation)
>
> Clearly, I think 3 is not OK for interoperability.  1 is OK by postponing
> the decision (as long as the extension point is announced); 2 is more
> “batteries included”.
>
> Within 2, I favor 2b.
> Most programmers learn a parsing syntax first, because one use case for
> REs is writing shoddy parsers — something that we cannot use in JSONpath.
> I believe that actual RE-based syntaxes are a good subset decision
> anyway.  The only candidate here that is actually standardized in XSD, but
> that has some unwanted baggage (character class subtraction — which is
> great, but not widely offered by other RE syntaxes our LCD would need to
> translate into).
>
> There are further detail decisions, e.g., what does \d or \w actually
> mean; we might simplify our lives by using a LCD subset that poses as few
> questions of this kind as possible.
>
> JSONpath of course is not the only area of application for RE syntaxes
> inside the IETF.  Notably, YANG and CDDL have chosen full XSD syntax, with
> striking acceptance problems on the YANG side (CDDL users seem to be more
> prudent here, in particular now with RFC 9165 available for complex cases;
> CDDL also has a defined extension point for dropping in multiple
> syntaxes).  This experience strengthens my preference for a LCD subset.
>
> I understand that a world would be more likable where we could just drop
> in REs from a common, well-understood usage.  Unfortunately, we don’t have
> this world. Making decisions based on wishful thinking we did is not doing
> the right thing.
>
> I also understand that my perspective may be somewhat different from other
> WG members, as I have already spent a significant amount of time
> researching the RE conundrum in different contexts.  So I’m very open to
> explaining this perspective in some more detail.
>
> Grüße, Carsten
>
>

---

From: Stefan Gössner <[email-masked]>
Date: Mon, 3 Jan 2022 19:56:19 +0100

Happy new year from me also to everyone.

I already committed to sympathise with Carsten's proposal 2b.

Yet I also strongly agree to Glyn's point of completing the JSONPath 
draft in a reasonable time frame.

So from Carsten's background of few decades of standards work 
experience, I would like to ask: how can we estimate the chance of 
finishing here at approximately the same time frame with both drafts?

If Iregexp presumably needs significantly more time for finishing, I 
favour the 'finalising an interim draft' approach according to proposal 
1, with respect to the benefits Glyn has stated.

But I must confess, that I don't know, what an 'interim draft' exactly 
is and if 'extension points' are necessary then.

// ================================

I would like to also discuss filter-selectors regarding ...

* Comparisons are restricted to primitive values (such as number,
       string, true, false, null).  Comparisons with complex values will
       fail, i.e. no selection occurs.

There seems to be a majority consense to allow structured (complex ?) 
values with comparisons. We should discuss that technically and try to 
strive for agreement here.

Thanks
--
sg

Am 03.01.2022 um 16:16 schrieb Glyn Normington:
> I'm keen to complete the JSONPath draft within a reasonable time frame 
> and I fear that regexp support may take quite a while to finalise. So 
> I'd like us to consider finalising an *interim draft* without regexp 
> support (i.e. option 1). Such a draft would be a useful checkpoint and 
> holding position until iregexp is complete.
>
> An interim draft would also be useful input toward a compliance test 
> suite and reference implementation (assuming someone will find time to 
> work on either or both of these!). The rest of the spec still being in 
> flux is a disincentive to anyone investing in building a CTS and RI.
>
>     I’ll continue the discussion here (instead of github), as we need
>     to make a (somewhat non-technical) WG decision.
>
>     We know that there is no easy way to generate a specification for
>     a standard form of regular expressions (RE) (there is no such
>     standard form).  Taking it from there, options include:
>
>     (3) having RE syntax but not defining what that means
>     (“permissive”) and not providing a way to identify which syntax
>     variant is being used
>
>     Note that we have to define both the RE syntax itself and the
>     decoration that allows its use in jsonpath.
>
>     (ii) make the outermost syntax look like existing syntax, failing
>     many instances that don’t happen to presume the same syntax, but
>     magically supporting some other instances that do happen to fit
>     (iii) use a regular expression subset that is actually based on
>     regular expressions (Chomsky type 3 languages), or one that
>     provides PCRE-style additions that don’t allow DFA transformation
>     (and thus efficient, attack-resistant implementation)
>
>     Clearly, I think 3 is not OK for interoperability.  1 is OK by
>     postponing the decision (as long as the extension point is
>     announced); 2 is more “batteries included”.
>
>     Within 2, I favor 2b.
>     Most programmers learn a parsing syntax first, because one use
>     case for REs is writing shoddy parsers — something that we cannot
>     use in JSONpath.
>     anyway.  The only candidate here that is actually standardized in
>     XSD, but that has some unwanted baggage (character class
>     subtraction — which is great, but not widely offered by other RE
>     syntaxes our LCD would need to translate into).
>
>     There are further detail decisions, e.g., what does \d or \w
>     actually mean; we might simplify our lives by using a LCD subset
>     that poses as few questions of this kind as possible.
>
>     I think I would like to develop this proposal further towards the
>     goal expressed in the previous paragraph.
>
>     JSONpath of course is not the only area of application for RE
>     syntaxes inside the IETF.  Notably, YANG and CDDL have chosen full
>     XSD syntax, with striking acceptance problems on the YANG side
>     (CDDL users seem to be more prudent here, in particular now with
>     RFC 9165 available for complex cases; CDDL also has a defined
>     extension point for dropping in multiple syntaxes).  This
>     experience strengthens my preference for a LCD subset.
>
>     I understand that a world would be more likable where we could
>     just drop in REs from a common, well-understood usage. 
>     Unfortunately, we don’t have this world. Making decisions based on
>     wishful thinking we did is not doing the right thing.
>
>     I also understand that my perspective may be somewhat different
>     from other WG members, as I have already spent a significant
>     amount of time researching the RE conundrum in different
>     contexts.  So I’m very open to explaining this perspective in some
>     more detail.
>
>     Grüße, Carsten
>
>
> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> 
> 	Virenfrei. www.avg.com 
> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> 
>
>
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>

---

From: Brian Rosen <[email-masked]>
Date: Mon, 3 Jan 2022 11:13:59 -0800

FWIW:

I agree, 2b makes the most sense.  There are, as Carsten points out, other needs for a regexp for json, and I’m fine with starting with the lregexp doc.

“Interim Draft” has no meaning in the IETF.  We can process a draft that has a normative reference to another draft,  That can take it all the way to the RFC Editor’s queue.  It won’t be published until the referenced draft is published.  That’s what we want to do here I think.  

Brian

> On Jan 3, 2022, at 10:56 AM, Stefan Gössner <[email-masked]> wrote:
> 
> Yet I also strongly agree to Glyn's point of completing the JSONPath draft in a reasonable time frame.
> 
> So from Carsten's background of few decades of standards work experience, I would like to ask: how can we estimate the chance of finishing here at approximately the same time frame with both drafts?
> 
> If Iregexp presumably needs significantly more time for finishing, I favour the 'finalising an interim draft' approach according to proposal 1, with respect to the benefits Glyn has stated.
> 
> But I must confess, that I don't know, what an 'interim draft' exactly is and if 'extension points' are necessary then.
> 
> There seems to be a majority consense to allow structured (complex ?) values with comparisons. We should discuss that technically and try to strive for agreement here.
> 
> Thanks
> --
> sg
> 
> 
>> I'm keen to complete the JSONPath draft within a reasonable time frame and I fear that regexp support may take quite a while to finalise. So I'd like us to consider finalising an *interim draft* without regexp support (i.e. option 1). Such a draft would be a useful checkpoint and holding position until iregexp is complete.
>> 
>> The other main benefit of finalising such an interim draft would be to flush out any remaining changes that are needed in addition to regexp support.
>> 
>> An interim draft would also be useful input toward a compliance test suite and reference implementation (assuming someone will find time to work on either or both of these!). The rest of the spec still being in flux is a disincentive to anyone investing in building a CTS and RI.
>> 
>> On Mon, 3 Jan 2022, 13:54 Carsten Bormann, <[email-masked] <mailto:[email-masked]>> wrote:
>> On 2022-01-03, at 14:01, Glyn Normington <[email-masked] <mailto:[email-masked]>> wrote:
>> > 
>> > https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/pull/143 <https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/pull/143>
>> 
>> Right.
>> 
>>  <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>	Virenfrei. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>               
>>  <x-msg://12/#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> 
> -- 
> JSONpath mailing list
> [email-masked]

---

From: Glyn Normington <[email-masked]>
Date: Mon, 3 Jan 2022 19:38:25 +0000

By "interim draft" I meant just one of our usual drafts which the WG had
agreed to stop developing, except perhaps for critical fixes, until iregexp
was ready for consumption. I don't think it would be necessary to start
turning such a draft into a RFC, unless there would be some advantage in
doing so.

On Mon, 3 Jan 2022, 19:14 Brian Rosen, <[email-masked]> wrote:

> FWIW:
>
> I agree, 2b makes the most sense.  There are, as Carsten points out, other
> needs for a regexp for json, and I’m fine with starting with the lregexp
> doc.
>
> “Interim Draft” has no meaning in the IETF.  We can process a draft that
> has a normative reference to another draft,  That can take it all the way
> to the RFC Editor’s queue.  It won’t be published until the referenced
> draft is published.  That’s what we want to do here I think.
>
> Brian
>
> So from Carsten's background of few decades of standards work experience,
> I would like to ask: how can we estimate the chance of finishing here at
> approximately the same time frame with both drafts?
>
> favour the 'finalising an interim draft' approach according to proposal 1,
> with respect to the benefits Glyn has stated.
>
> But I must confess, that I don't know, what an 'interim draft' exactly is
> and if 'extension points' are necessary then.
>
>> I also understand that my perspective may be somewhat different from
>> other WG members, as I have already spent a significant amount of time
> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Virenfrei.
> www.avg.com

---

From: Brian Rosen <[email-masked]>
Date: Mon, 3 Jan 2022 12:28:08 -0800

Oh, I think pushing all the way through is a good idea.  We get all the
critical reviews from outside people who inevitably find those “how did we
miss that?” things. Sometimes that slows down progress so starting early is
good. The only reason not to is if we think there will be dependencies
other than the obvious. I don’t think that will be the case here.

Brian

On Mon, Jan 3, 2022 at 11:38 AM Glyn Normington <
[email-masked]> wrote:

>> I agree, 2b makes the most sense.  There are, as Carsten points out,
>> other needs for a regexp for json, and I’m fine with starting with the
>> lregexp doc.
>>
>> and I fear that regexp support may take quite a while to finalise. So I'd
>> like us to consider finalising an *interim draft* without regexp support
>> (i.e. option 1). Such a draft would be a useful checkpoint and holding
>> position until iregexp is complete.
>>
>> suite and reference implementation (assuming someone will find time to work
>> on either or both of these!). The rest of the spec still being in flux is a
>>> I’ll continue the discussion here (instead of github), as we need to
>>> make a (somewhat non-technical) WG decision.
>>>
>>> postponing the decision (as long as the extension point is announced); 2 is
>>> more “batteries included”.
>>>
>>> Within 2, I favor 2b.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 3 Jan 2022 21:56:22 +0100

Clearly, we want to push through with the rest of the document.
We don’t need to decide now whether it’s 1 or 2b; we still have a couple of weeks for that.

So for now my question is:

> On 2022-01-03, at 20:38, Glyn Normington <[email-masked]> wrote:
> 
> until iregexp was ready for consumption

What do we need to do to get there?

Clearly, we want to get reviews, but before that:

My main remaining question is whether we want to have the complexity emanating from \d, \w, and friends.  Taking this out creates a rather universal and simple subset, but also takes away a little convenience in constructs like (RFC 6991)…

\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?

…which under the often found ASCII-only understanding then becomes…

[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}(\.[0-9]+)?

(Note that RFC 6991 is probably broken here:  YANG uses XSD patterns, so …

٢٠٢٢-٠١-٠٣T٢١:٤٠:٠٠

…matches that pattern, as does...

২০২২-০১-০৩T২১:৪০:০০

… because XSD interprets \d as a shorthand for \p{Nd}, so the pattern actually is:

\p{Nd}{4}-\p{Nd}{2}-\p{Nd}{2}T\p{Nd}{2}:\p{Nd}{2}:\p{Nd}{2}(\.\p{Nd}+)?

).  We would be free to define \d as we want, at the possible cost of no longer being a strict subset of XSD, or to simply leave out \d and its friends.

Grüße, Carsten

---

From: Stefan Gössner <[email-masked]>
Date: Tue, 4 Jan 2022 09:03:18 +0100

... as a notorious minimalist I clearly vote here for the rather 
universal and simple subset without those convenient but problematic 
parts. Especially as it promises to be finished faster.

Am 03.01.2022 um 21:56 schrieb Carsten Bormann:
>> On 2022-01-03, at 20:38, Glyn Normington<[email-masked]>  wrote:
>>

---

From: Glyn Normington <[email-masked]>
Date: Tue, 4 Jan 2022 08:46:45 +0000

I'd be happy with that, but I suspect agreeing on a simple subset could
take a while (unless it's the empty set ;-) ).

On Tue, 4 Jan 2022 at 08:03, Stefan Gössner <[email-masked]> wrote:

> ... as a notorious minimalist I clearly vote here for the rather universal
> and simple subset without those convenient but problematic parts.
> Especially as it promises to be finished faster.
>
>
> On 2022-01-03, at 20:38, Glyn Normington <[email-masked]> <[email-masked]> wrote:
>

---

From: Stefan Gössner <[email-masked]>
Date: Tue, 18 Jan 2022 10:55:06 +0100

hmm ... trying to login into meetecho by

https://meetings.conf.meetecho.com/interim/?short=39034cb3-f4b8-4c58-911b-8822065381f5

i get the message "This session is over" ?

--
sg

---

From: Tim Bray <[email-masked]>
Date: Tue, 18 Jan 2022 01:56:56 -0800

Yes, me too.  Hmmm…

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 18 Jan 2022 10:57:13 +0100

Same for me.  Alerted meetecho and secretariat.

> On 2022-01-18, at 10:55, Stefan Gössner <[email-masked]> wrote:
>

---

From: Glyn Normington <[email-masked]>
Date: Tue, 18 Jan 2022 09:59:16 +0000

You are welcome to use my Zoom if that helps:

Glyn Normington is inviting you to a scheduled Zoom meeting.

Topic: JSONPath interim meeting
Time: Jan 18, 2022 10:00 AM London

Join Zoom Meeting
https://us02web.zoom.us/j/89635618195?pwd=bFZXaVNoNVdSV2N1MFZyTU1aeDVKZz09

Meeting ID: 896 3561 8195
Passcode: 995455