Subject: [Jsonpath] JSONPath meeting at IETF 111

From: Tim Bray <[email-masked]>
Date: Tue, 20 Jul 2021 10:50:51 -0700

This is happening next Monday at 4PM Pacific time, which means in the
middle of the night for the bulk of our EuroPeople; sorry about that.

We have a good track record of settling issues at our meetings and interims
so far, so I'm optimistic we can succeed again.  This would be a good time
for WG participants to suggest issues they think are near the front of the
queue.

Thanks in advance!

---

From: Glyn Normington <[email-masked]>
Date: Fri, 23 Jul 2021 15:13:44 +0100

I'd like the meeting to decide
https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/103.
It would be great if people could express their opinions in the issue
beforehand to save time at the meeting and make it easier to gauge the
consensus.

I'd like to discuss next steps for regexps in filters in case that
feature is stalling. Attempting to define a subset of an existing
standard is laudable and good for interoperability, but I'm concerned
that, compared to being more open-ended and allowing the use of any
"reasonable" regexp implementation (with consequent reduction in
interoperability for many regexps), this will increase the cost of
implementing JSONPath and put implementers off.

I'd like to chat round the following new issue too as I'm hopeful
there will be a quick consensus:
https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/110.

On Tue, 20 Jul 2021 at 18:51, Tim Bray <[email-masked]> wrote:
>
> This is happening next Monday at 4PM Pacific time, which means in the middle of the night for the bulk of our EuroPeople; sorry about that.
>
> We have a good track record of settling issues at our meetings and interims so far, so I'm optimistic we can succeed again.  This would be a good time for WG participants to suggest issues they think are near the front of the queue.
>
> Thanks in advance!
>
>
> --
> Jsonpath mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/jsonpath

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 23 Jul 2021 16:44:36 +0200

> implementing JSONPath and put implementers off.

I think the iregexp approach can only work if we have running code for

— checking a supplied regexp against iregexp rules
— translating an iregexp into one of several popular platform regexps

So coming up with that code would be part of the development project for me.  That should help the implementers.

Being somewhat (but not too much) different from the standard regexp soup out there could actually help interoperability.  A little pain at the start, but portable expressions after that.

Grüße, Carsten

---

From: Greg Dennis <[email-masked]>
Date: Fri, 23 Jul 2021 21:51:33 +0000 (UTC)

I would expect prescribed RegEx support would be as near to an intersection of various implementations so as to only prescribe common ground.
Greg

Grüße, Carsten

---

From: Michael Kay <[email-masked]>
Date: Fri, 23 Jul 2021 23:21:25 +0100

> On 23 Jul 2021, at 22:51, Greg Dennis <gregsdennis=[email-masked]> wrote:
> 

That's roughly what XSD tried to do; XPath 2.0 and then 3.1 followed in its footsteps, with a modicum of success. (Some vendors, notably Microsoft, took the easy route and implemented their own regex dialect ignoring the standard.) So since the design objectives are the same, one approach would be to use the XPath 3.1 regex dialect and use it unchanged. It's not perfect but at least it's rather more thoroughly specified than most (regex specifications on the whole are notorious for their lack of precision).

A big part of the problem here is that most regex dialects predate Unicode, and include all kinds of stuff that isn't very pretty in a Unicode world. The XSD/XPath dialect tried to fix that.

Implementing this dialect by translation to a different regex engine is possible but not easy. James Clark wrote and published a translator from XSD regexes to JDK regexes - that was before Java changed so that a surrogate pair is one character rather than two. Probably the hardest thing to get right with that kind of implementation is the XPath rule for case-independent matching (the "i" flag) which is based on Unicode character properties. Doing the Unicode blocks and categories is fairly straightforward to implement but potentially very inefficient. In Saxon I used a translation approach initially, but eventually decided to implement my own regex engine (by modifying an open-source one). Contrariwise, for Saxon-JS we're translating to JavaScript regular expressions, somewhat imperfectly.

Michael Kay
Saxonica

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 24 Jul 2021 00:44:45 +0200

On 2021-07-24, at 00:21, Michael Kay <[email-masked]> wrote:
> 
> That's roughly what XSD tried to do

Yeah, except they have this character class subtraction that nobody else has, so essentially you have to profile this out.

Not sure why we would want to go beyond XSD into XPath land if we want to keep this simple.  (We need matching, not parsing regexps.)

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 24 Jul 2021 17:18:45 +0200

> This is happening next Monday at 4PM Pacific time, which means in the middle of the night for the bulk of our EuroPeople; sorry about that.

Apparently I didn’t pay enough attention, since I only just noticed that I have a conflict with IOTOPS in that slot.

I think we need to have a slightly more frequent interim cadence.

Grüße, Carsten

---

From: Stefan Gössner <[email-masked]>
Date: Mon, 26 Jul 2021 17:49:54 +0200

How will it be possible to attend the meeting tonight ... technically ?  
WebEx or something ?
--
sg

Am 20.07.2021 um 19:50 schrieb Tim Bray:
> We have a good track record of settling issues at our meetings and 
> interims so far, so I'm optimistic we can succeed again.  This would 
> be a good time for WG participants to suggest issues they think are 
> near the front of the queue.
>
> Thanks in advance!
>
>

---

From: Glyn Normington <[email-masked]>
Date: Mon, 26 Jul 2021 17:01:52 +0100

Hi Stefan

Tonight's meeting is on MeetEcho as part of IETF 111 which requires
you to register (see https://www.ietf.org/how/meetings/111/).

Once you've registered, click this link to access meetecho:
https://meetings.conf.meetecho.com/ietf111/?group=jsonpath&short=&item=1

Hope you can make it!

Regards,
Glyn

On Mon, 26 Jul 2021 at 16:50, Stefan Gössner <[email-masked]> wrote:
>
> How will it be possible to attend the meeting tonight ... technically ?  WebEx or something ?
> --
> sg
>
>

---

From: Stefan Gössner <[email-masked]>
Date: Mon, 26 Jul 2021 18:04:10 +0200

thanks ... I'll try ...
--
stefan

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 26 Jul 2021 18:10:30 +0200

In the discussion of 
https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/103, it became apparent that we weren’t quite sure what levels of compliance language we should be using.  Particularly, I would think that a SHOULD is not very useful except when discussing corner cases (or non-functional requirements like “SHOULD NOT blow up”).

So are we ready to have multiple compliance levels?
A core set to which we expect everyone to comply and set of modules/features that can be added to the compliance?  No regular expressions in the core but an optional compliance module/feature for iregexps?

I don’t want to overthink that and propose a complicated set of modules where a single monolithic core would be sufficient, but stumbling into these modules without having planned for them is probably also not a good idea.

Grüße, Carsten

> On 2021-07-24, at 17:18, Carsten Bormann <[email-masked]> wrote:
>

---

From: Greg Dennis <[email-masked]>
Date: Mon, 26 Jul 2021 19:23:58 +0000 (UTC)

I will not be able to make it as in heading into work now.
Greg

  On Tue, 27 Jul 2021 at 4:04 am, Stefan Gössner<[email-masked]> wrote:   -- 
Jsonpath mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/jsonpath

---

From: Glyn Normington <[email-masked]>
Date: Mon, 26 Jul 2021 20:32:47 +0100

Carsten: for the avoidance of uncertainty, which meeting will you be
attending, JSONPath or IOTOPS?

Greg can't make it.

Stefan needs to register, which could be a bit tight, e.g. if he needs a
fee waiver.

I am planning to attend, but if the Chairs think we won't be able to make
much progress given the above, shall we reschedule to an interim meeting in
the next week or two?

Regards,
Glyn

On Sat, 24 Jul 2021, 16:18 Carsten Bormann, <[email-masked]> wrote:

> Apparently I didn’t pay enough attention, since I only just noticed that I
> have a conflict with IOTOPS in that slot.
>

---

From: Tim Bray <[email-masked]>
Date: Mon, 26 Jul 2021 12:41:23 -0700

We should also think about other IETFers who haven't been active but have
arranged their schedule to come to our long-scheduled meeting and might be
grumpy if it's suddenly canceled.  My feeling is that we should gather, see
how many people show up in a few minutes, then decide whether to proceed.

On Mon, Jul 26, 2021 at 12:33 PM Glyn Normington <
[email-masked]> wrote:

>> Apparently I didn’t pay enough attention, since I only just noticed that
>> I have a conflict with IOTOPS in that slot.
>>

---

From: Glyn Normington <[email-masked]>
Date: Mon, 26 Jul 2021 20:42:29 +0100

Ok, thanks Tim!

---

From: Tim Bray <[email-masked]>
Date: Mon, 26 Jul 2021 12:44:37 -0700

And BTW, if we don't get to any useful work, agreed on another interim
before too long.

On Mon, Jul 26, 2021 at 12:42 PM Glyn Normington <
[email-masked]> wrote:

>>> I am planning to attend, but if the Chairs think we won't be able to
>>> make much progress given the above, shall we reschedule to an interim
>>> meeting in the next week or two?
>>>
>>> Regards,
>>> Glyn
>>>
>>>
>>>> Apparently I didn’t pay enough attention, since I only just noticed
>>>> that I have a conflict with IOTOPS in that slot.
>>>>

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 26 Jul 2021 21:50:16 +0200

On 2021-07-26, at 21:32, Glyn Normington <[email-masked]> wrote:
> 
> Carsten: for the avoidance of uncertainty, which meeting will you be attending, JSONPath or IOTOPS?

IOTOPS.  I think I have just cleared my SECDISPATCH engagement.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 26 Jul 2021 21:55:27 +0200

> IOTOPS.  I think I have just cleared my SECDISPATCH engagement.

… and, if you summon me (e.g., on Jabber), I’ll jump over briefly.

Grüße, Carsten

---

From: Stefan Gössner <[email-masked]>
Date: Tue, 27 Jul 2021 00:51:46 +0200

hmm ... do I need to pay a fee to register ?

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 27 Jul 2021 00:54:10 +0200

You can use the fee waiver program

> On 2021-07-27, at 00:51, Stefan Gössner <[email-masked]> wrote:
> 
>> you to register (see 
>> https://www.ietf.org/how/meetings/111/
>> ).
>> 
>> On Mon, 26 Jul 2021 at 16:50, Stefan Gössner 
>> <[email-masked]>
>>  wrote:
>>

---

From: Stefan Gössner <[email-masked]>
Date: Tue, 27 Jul 2021 00:58:53 +0200

ok .. I did use it.

Now it tells me, that my option is being processed and can take up two 
business days to issue a fee waiver.

blöd ...

---

From: Tim Bray <[email-masked]>
Date: Mon, 26 Jul 2021 16:02:02 -0700

Alternately, you could pay, and deduct it as a business expense. IETF is a
good cause.

On Mon, Jul 26, 2021 at 3:58 PM Stefan Gössner <[email-masked]> wrote:

> On 2021-07-27, at 00:51, Stefan Gössner <[email-masked]> <[email-masked]> wrote:
>
> On Mon, 26 Jul 2021 at 16:50, Stefan Gössner <[email-masked]> <[email-masked]>
>  wrote:
>
>
> [email-masked]://www.ietf.org/mailman/listinfo/jsonpath
>
> --
> Jsonpath mailing [email-masked]://www.ietf.org/mailman/listinfo/jsonpath
>
>
>

---

From: Tim Bray <[email-masked]>
Date: Mon, 26 Jul 2021 16:02:44 -0700

But probably don't, because we have very poor turn-out and may not have a
meeting.

---

From: Stefan Gössner <[email-masked]>
Date: Tue, 27 Jul 2021 01:15:05 +0200

Well I got an answer ... see below.

Now I need to pay 2 USD, regardless of the coupon, which I would do 
(need to choose a payment method), if you tell me that we will meet.

--
stefan

Thank you for taking the fee waiver option for IETF 111. Please feel 
free to register for IETF 111 with the following link:
https://registration.ietf.org/ <https://registration.ietf.org/>

After registering, on the final payment page there will be a box under 
the final total where you can apply the following coupon code to cover 
your registration:

w6jsGIgp

If you have any questions, please do not hesitate to contact me.Regards,

Jasmine Magallanes, CMP
Project Manager

Am 27.07.2021 um 01:02 schrieb Tim Bray:
> But probably don't, because we have very poor turn-out and may not 
> have a meeting.
>
> On Mon, Jul 26, 2021 at 4:02 PM Tim Bray <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Alternately, you could pay, and deduct it as a business expense.
>     IETF is a good cause.
>
>     On Mon, Jul 26, 2021 at 3:58 PM Stefan Gössner
>     <[email-masked] <mailto:[email-masked]>> wrote:
>
>         ok .. I did use it.
>
>         Now it tells me, that my option is being processed and can
>         take up two business days to issue a fee waiver.
>
>         blöd ...
>
>
>
>>>         On 2021-07-27, at 00:51, Stefan Gössner<[email-masked]>  <mailto:[email-masked]>  wrote:
>>>
>>>>         https://www.ietf.org/how/meetings/111/  <https://www.ietf.org/how/meetings/111/>
>>>>         ).
>>>>
>>>>         https://meetings.conf.meetecho.com/ietf111/?group=jsonpath&short=&item=1  <https://meetings.conf.meetecho.com/ietf111/?group=jsonpath&short=&item=1>
>>>>
>>>>
>>>>         Hope you can make it!
>>>>
>>>>         Regards,
>>>>         Glyn
>>>>
>>>>         <[email-masked]>  <mailto:[email-masked]>
>>>>           wrote:
>>>>
>>>>>         [email-masked]  <mailto:[email-masked]>
>>>>>         https://www.ietf.org/mailman/listinfo/jsonpath  <https://www.ietf.org/mailman/listinfo/jsonpath>
>>>         -- 
>>>         Jsonpath mailing list
>>>         [email-masked]  <mailto:[email-masked]>
>>>         https://www.ietf.org/mailman/listinfo/jsonpath  <https://www.ietf.org/mailman/listinfo/jsonpath>
>
>
> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> 
> 	Virenfrei. www.avg.com 
> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> 
>
>
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>

---

From: Glyn Normington <[email-masked]>
Date: Tue, 27 Jul 2021 08:01:00 +0100

My aim was to require iregexp for interoperability, but allow
alternate regexp implementations with potentially degraded
interoperability.

Could we achieve something similar using compliance
levels/modules/features? For example, could an implementer add (to the
regexp-free subset) an alternative regexp module/feature for their
preferred (non-compliant) regexp implementation?

I guess the semantics would be that JSONPath implementations which use
interoperable regexp implementations will interoperate. So, for
example, JSONPath/iregexp implementations would interoperate with each
other as would JSONPath/RE2 implementations. But, for any regexp
implementations x and y, JSONPath/x would only interoperate with
JSONPath/y to the extent that x and y interoperate.

I quite like that this approach separates regexp interoperation from
the JSONPath spec and makes it a matter for regexp.

---

From: Stefan Gössner <[email-masked]>
Date: Tue, 27 Jul 2021 09:35:31 +0200

I also like that concept of separation of regexp (features) from JSONPath spec.

---

From: Greg Dennis <[email-masked]>
Date: Tue, 27 Jul 2021 07:43:06 +0000 (UTC)

That's the approach that JSON Schema uses, citing ECMA-262 as the preferred flavor.
https://json-schema.org/draft/2020-12/json-schema-validation.html#rfc.section.6.3.3

  On Tue, 27 Jul 2021 at 7:01 pm, Glyn Normington<[email-masked]> wrote:   My aim was to require iregexp for interoperability, but allow
alternate regexp implementations with potentially degraded
interoperability.

---

From: Glyn Normington <[email-masked]>
Date: Tue, 27 Jul 2021 09:38:35 +0100

Thanks Greg. The JSON Schema text is straightforward:

"This string SHOULD be a valid regular expression, according to the
ECMA-262 regular expression dialect."

Couldn't we do something similar? What would be the downside of adopting
ECMA-262 regular expressions for this purpose rather than depending on
developing iregexp?

On Tue, 27 Jul 2021, 08:43 Greg Dennis, <[email-masked]> wrote:

> On Tue, 27 Jul 2021 at 7:01 pm, Glyn Normington
> <[email-masked]> wrote:
> > https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/issues/103,
> it became apparent that we weren’t quite sure what levels of compliance
> language we should be using.  Particularly, I would think that a SHOULD is
> not very useful except when discussing corner cases (or non-functional
> requirements like “SHOULD NOT blow up”).
> >
> > A core set to which we expect everyone to comply and set of
> modules/features that can be added to the compliance?  No regular
> expressions in the core but an optional compliance module/feature for
> iregexps?
> >
> > I don’t want to overthink that and propose a complicated set of modules
> where a single monolithic core would be sufficient, but stumbling into
> these modules without having planned for them is probably also not a good
> idea.
> >
> > Grüße, Carsten
> >
> >

---

From: Michael Kay <[email-masked]>
Date: Tue, 27 Jul 2021 10:02:00 +0100

>What would be the downside of adopting ECMA-262 regular expressions for this purpose rather than depending on developing iregexp?

The main one is that you'd be perpetuating the default processing mode where you're matching 16-bit code units rather than Unicode codepoints, which is really only wanted for legacy reasons: similarly other legacy behavior like the strange case-folding semantics of the "i" flag.

If you're going to go for ECMA-262, make it ECMA-262 in Unicode ("u") mode.

Michael Kay
Saxonica

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 27 Jul 2021 11:29:07 +0200

On 2021-07-27, at 10:38, Glyn Normington <[email-masked]> wrote:
> 
> "This string SHOULD be a valid regular expression, according to the ECMA-262 regular expression dialect."

ECMA-262 is a moving target, the there is not really a single “ECMA-262 regular expression dialect”; but that is easy to solve by referring to an ECMA-262 edition of a specific year.

For a few years, the functionality supported by ECMA-262 regular expressions has expanded significantly.  With today’s extent, supporting ECMA-262 regular expressions would require months of implementation effort.

This is easy to do if your code runs in a recent JavaScript environment.  It is mostly impractical (dwarfing any JSONpath effort) if your platform does not already have an ECMA-262 regexp implementation around.

So this is one of those SHOULDs that really mean “SHOULD, but we know you won’t”.  Why write something like this into a standard?  This only makes sense if interoperability is not the main goal.

Every competent engineer can implement XSD regexps (and more so the iregexp subset) in less than a week.  So if we care about interoperability, this is more likely to yield it.

By the way, json-schema.org had a more practical approach 8 months ago:

https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-00#section-6.4

This isn’t really specified very well, but if one sat down and did, it would be closer to the iregexp approach at least in aspiration (it is still littered with the useless kind of SHOULD I mentioned above).

The iregexp approach uses the XSD choice of anchoring both sides of the expression, because that is always what you want in matching operation (it is easy to add .* if it is not).  ECMA-262 of course has parsing regular expressions, so you have to add ^ and $ for essentially all expressions.

It may be a useful approach to support different regexp modules by explicitly indicating the dialect at hand, with iregexp as the Rosetta Stone that can work everywhere.  This works great both for JSONPath users that don’t care about interoperability (because they are writing their JSONPath for a specific server implementation) and for ones that need interoperability.

Grüße, Carsten

---

From: Greg Dennis <[email-masked]>
Date: Tue, 27 Jul 2021 09:45:58 +0000 (UTC)

To be clear, I wasn't suggesting ECMA-262 as the implementation. Rather I was suggesting the wording in the spec that included and referenced it as an example of an approach we could use to reference whatever we wanted to use.
Greg

Grüße, Carsten

---

From: Glyn Normington <[email-masked]>
Date: Tue, 27 Jul 2021 11:06:35 +0100

On Tue, 27 Jul 2021, 10:29 Carsten Bormann, <[email-masked]> wrote:

> ECMA-262 is a moving target, the there is not really a single “ECMA-262
> regular expression dialect”; but that is easy to solve by referring to an
> ECMA-262 edition of a specific year.
>
> For a few years, the functionality supported by ECMA-262 regular
> expressions has expanded significantly.  With today’s extent, supporting
> ECMA-262 regular expressions would require months of implementation effort.
>
> This is easy to do if your code runs in a recent JavaScript environment.
> It is mostly impractical (dwarfing any JSONpath effort) if your platform
> does not already have an ECMA-262 regexp implementation around.
>
> So this is one of those SHOULDs that really mean “SHOULD, but we know you
> won’t”.  Why write something like this into a standard?  This only makes
> sense if interoperability is not the main goal.
>
> Every competent engineer can implement XSD regexps (and more so the
> iregexp subset) in less than a week.  So if we care about interoperability,
> this is more likely to yield it.
>
> This isn’t really specified very well, but if one sat down and did, it
> would be closer to the iregexp approach at least in aspiration (it is still
> littered with the useless kind of SHOULD I mentioned above).
>
> The iregexp approach uses the XSD choice of anchoring both sides of the
> expression, because that is always what you want in matching operation (it
> is easy to add .* if it is not).  ECMA-262 of course has parsing regular
> expressions, so you have to add ^ and $ for essentially all expressions.
>
> It may be a useful approach to support different regexp modules by
> explicitly indicating the dialect at hand, with iregexp as the Rosetta
> Stone that can work everywhere.  This works great both for JSONPath users
> that don’t care about interoperability (because they are writing their
> JSONPath for a specific server implementation) and for ones that need
> interoperability.
>

Another class of JSONPath users who don't care about regexp
interoperability is those who don't use regexps.

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 27 Jul 2021 12:41:06 +0200

> If you're going to go for ECMA-262, make it ECMA-262 in Unicode ("u") mode.

Of course!
Sorry, I didn’t even think that needed mentioning.

Grüße, Carsten

---

From: Stefan Gössner <[email-masked]>
Date: Tue, 27 Jul 2021 13:29:34 +0200

Addressing ...

Users: They only want to know, if the JSONPath implementation used 
conforms to the spec regarding regexp ... if they are interested in 
regexps at all.

Implementors: They need to know, if the regexp library they want to use 
with their JSONPath implementation conforms to the spec.

So I think, having with I-Regexp a reference to an interoperable flavor 
of regexps is a good thing.
--
sg

---

From: Glyn Normington <[email-masked]>
Date: Tue, 27 Jul 2021 12:40:20 +0100

I agree the anticipated interoperability of iregexp will be valuable. I was
trying to explore how tightly coupled the JSONPath spec needs to be to a
regexp spec. It seems the conclusion we are coming to is: pretty tightly
coupled.

On Tue, 27 Jul 2021, 12:29 Stefan Gössner, <[email-masked]> wrote:

> conforms to the spec regarding regexp ... if they are interested in regexps
> at all.
>
> On 2021-07-27, at 10:38, Glyn Normington <[email-masked]> <[email-masked]> wrote:
>
>