Subject: [Json] JSON Concluded? Well, maybe not

From: Anders Rundgren <[email-masked]>
Date: Sun, 31 Dec 2017 06:24:45 +0100

Congratulations everybody to the revised JSON RFC!

Does this mean that JSON is "done" for good?

Probably not because the concept I have mentioned from time to time, the ability adding a digital signature to a JSON object (in contrast to signing arbitrary Base64Url-encoded data), is still very much alive.  In fact, there is an I-D in preparation aiming at reducing the current proliferation of "DIY-standards" for dealing with this highly requested feature.  The only real challenge is agreeing on a suitable way "normalizing" JSON data during parsing and serialization.  Such a scheme will be like an extended version of I-JSON (RFC7493), potentially having an impact on "ordinary" uses of JSON as well.

Happy New [and optionally signed] JSON Year
Anders Rundgren

---

From: Richard Gibson <[email-masked]>
Date: Sun, 31 Dec 2017 13:43:11 -0500

On the topic of JSON normalization, I believe that canonicaljson-spec
<http://gibson042.github.io/canonicaljson-spec/> covers all cases while
respecting prior art like
https://tools.ietf.org/html/draft-staykov-hu-json-canonical-form-00 and RFC
7638. I'd like to get it published as an RFC to handle scenarios like those
mentioned by Anders Rundgren, but am not sure how to go about doing so. Is
this a good place to ask for assistance?

On Sun, Dec 31, 2017 at 12:24 AM, Anders Rundgren <
[email-masked]> wrote:

> Probably not because the concept I have mentioned from time to time, the
> ability adding a digital signature to a JSON object (in contrast to signing
> arbitrary Base64Url-encoded data), is still very much alive.  In fact,
> there is an I-D in preparation aiming at reducing the current proliferation
> of "DIY-standards" for dealing with this highly requested feature.  The
> only real challenge is agreeing on a suitable way "normalizing" JSON data
> during parsing and serialization.  Such a scheme will be like an extended
> version of I-JSON (RFC7493), potentially having an impact on "ordinary"
> uses of JSON as well.
>
> _______________________________________________
> json mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/json
>

---

From: Tim Bray <[email-masked]>
Date: Sun, 31 Dec 2017 10:50:12 -0800

If I were going to make an argument for keeping the WG alive, it would
require someone to get energized and submit an I-D for a plausible JSON
schema system, or maybe even a much-needed formalization of JSONPath. We
use JSONPath a lot at AWS.

On Sun, Dec 31, 2017 at 10:43 AM, Richard Gibson <[email-masked]>
wrote:

> respecting prior art like https://tools.ietf.org/html/
> draft-staykov-hu-json-canonical-form-00 and RFC 7638. I'd like to get it
> published as an RFC to handle scenarios like those mentioned by Anders
> Rundgren, but am not sure how to go about doing so. Is this a good place to
> ask for assistance?
>

-- 
- Tim Bray (If you’d like to send me a private message, see
https://keybase.io/timbray)

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sun, 31 Dec 2017 14:28:00 -0500

+1

As a practical matter, the group would need to recharter and what is
proposed is a completely different application and one that many folk,
myself included are skeptical of having been burned on it with XML
Signature.

While it is possible that XML Signature canonicalization is covfefe because
of the convoluted XML semantics, ASN.1 has reasonably precise semantics but
I have yet to find an actual use case for DER. Seriously, VeriSign signed
every certificate using BER for years until someone noticed.

The idea that you can take a signed object, parse it and then re-emit it
from the parsed object is not inherently flawed but I have yet to hear a
good reason to do this. It is one of those terrible ideas that people keep
insisting is 'essential' and cannot give a non referential reason for doing.

We did not do canonicalization in JOSE and not for lack of people
suggesting it.

On Sun, Dec 31, 2017 at 1:50 PM, Tim Bray <[email-masked]> wrote:

> On Sun, Dec 31, 2017 at 10:43 AM, Richard Gibson <[email-masked]
> > wrote:
>
>> respecting prior art like https://tools.ietf.org/html/dr
>> aft-staykov-hu-json-canonical-form-00 and RFC 7638. I'd like to get it

---

From: Erik Wilde <[email-masked]>
Date: Sun, 31 Dec 2017 11:40:56 -0800

hello.

> use JSONPath a lot at AWS.

for the schema aspect:

https://github.com/json-schema-org/json-schema-spec/issues/526 makes the 
relatively small team behind JSON schema (which started out as a 
continuation of the long-abandoned effort by kris zyp	) aware of this 
possibility. i am not involved enough to decide whether they are 
interested in making this step, but at least it's an effort that has 
been pretty active, and it might be an interesting candidate for a JSON 
schema language.

for the JSON path aspect: +1 as well, but i am not aware of something as 
active as the schema language effort above that could be pulled in.

for kicks and giggles, i'd also like to throw in that it would be great 
to have some reusable way of how various vocabularies can exist 
alongside in JSON. one could mutter the words "XML namespaces" here, but 
most people have negative views of them. but still: in APIs, there often 
is the need to support extensibility and openness, and it seems like 
every JSON-based API design has to come up with its own way of doing it.

cheers (and happy new year to everybody!),

dret.

-- 
erik wilde | mailto:[email-masked] |
            | http://dret.net/netdret    |
            | http://twitter.com/dret    |

---

From: Anders Rundgren <[email-masked]>
Date: Sun, 31 Dec 2017 22:34:30 +0100

For normalization it turns out that JSON.parse() and JSON.stringify() as
specified by ECMA and already supported by the most widely available JSON
tools is all you need!

Anders on mobile

---

From: Richard Gibson <[email-masked]>
Date: Sun, 31 Dec 2017 18:39:04 -0500

> specified by ECMA and already supported by the most widely available JSON

That is definitely not true. A trivial counterexample:
https://jsbin.com/luyoveyuqu/edit?js,console,output

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sun, 31 Dec 2017 19:49:13 -0500

I have a schema language that I use to generate Web Service clients and
services. The implementation is open source. It also generates reference
sections for an Internet Draft:

http://www.prismproof.org/Documents/draft-hallambaker-mesh-reference.html

Right now I have three separate schema languages that were written at
different times to target XML, JSON, ASN.1 and TLS style encodings. 90% of
them are the same.

What I think really commends JSON is the lack of extraneous information in
the schema. Let us say we are serializing an integer. In the TLS schema I
have to specify whether it is an 8, 16, or 32 bit, signed or unsigned, etc.
In JSON its just an integer.

I just want to think about my application, I want the serialization to be
as straightforward as possible. What I don't need or want is the type of
capabilities XML Schema gives me, the ability to specify that an integer is
greater then 4 and less than 23. That is not information I have ever found
the need to express in a schema and I have been using this approach to
generate protocols for 25 years.

What I have now is adequate for my needs right now. If there is interest in
a standard, I would probably want to clean up the approach and make it more
regularized.

---

From: Anders Rundgren <[email-masked]>
Date: Mon, 1 Jan 2018 07:10:32 +0100

On 2018-01-01 00:39, Richard Gibson wrote:
> On Sun, Dec 31, 2017 at 4:34 PM, Anders Rundgren <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     For normalization it turns out that JSON.parse() and JSON.stringify() as specified by ECMA and already supported by the most widely available JSON tools is all you need!
> 
> 
> That is definitely not true. A trivial counterexample: https://jsbin.com/luyoveyuqu/edit?js,console,output

Right, JSON.stringify() doesn't not canonicalize but preserve property order [1] + normalize data which for "crypto-safe" JSON applications is entirely sufficient.

If you go back to your example you can see another nicety of the EMCA parse/stringify scheme: JSON data is actually generated in the order specified by the programmer.

Or as I like to put it: JSON grew out of JavaScript, now it has come home again :-)

Cheers,
Anders

1] Extract from an I-D in preparation:

For JavaScript optimization reasons, ECMAScript's JSON.parse()
internally rearranges order of properties with names expressed
as integers, making a parsed JSON string like
'{"2":"First","A":"Next","1":"Last"}' actually serialize as
'{"1":"Last","2":"First","A":"Next"}'.  Due to this fact,
signature creators must (in the hopefully rather unusual case
cross platform applications would mandate numeric property names),
in an here unspecified way, "emulate" this scheme since this
behavior is not intended to be an additional requirement to
support by JSON tools in general in order to use this specification.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 1 Jan 2018 01:27:19 -0500

The use case that I believe gives rise to the belief canonicalization is
necessary is the need to sign a data object in the same JSON document. I
have this need frequently:

{ {A = [... JSON Data...]} { Signature (A) } }

The way I solve this using JSON is to BASE-64 encode:

{ { A = base64(... JSON Data...)} { Signature (A) } }

Which is brutal but entirely effective and only expands the data by 33%

For an efficient encoding, I use JSON-B which is a strict superset of JSON
with additional code points that allow binary data to be spliced in.

JSON only uses bytes in the range 1-127 for tagging. Bytes in the range
128-255 can only occur inside strings. Which means all the other values can
be repurposed

In JSON-B, the binary sequence 01 02 03 can be encoded as either "AQID" or

88 03 01 02 03

The first byte says that this is a binary value with an 8 bit length and
the final block of the sequence (i.e. there is only one block in this
sequence).

The second byte is the length.

The rest is the data.

I would much rather extend JSON to support efficient encoding of any binary
value than develop a canonicalization hack that could only ever work for
JSON.

http://www.prismproof.org/Documents/draft-hallambaker-jsonbcd.html

---

From: Anders Rundgren <[email-masked]>
Date: Mon, 1 Jan 2018 07:43:46 +0100

On 2018-01-01 07:27, Phillip Hallam-Baker wrote:
> necessary is the need to sign a data object in the same JSON document.

Yes, but this is indeed only a belief.  The following in-object signature would validate in any JSON system compliant with ECMA:

{
   "now": "2017-04-16T11:23:06Z",
   "escapeMe": "\u20ac$\u000F\u000aA'\u0042\u0022\u005c\\\"\/",
   "numbers": [1e+30,4.5,6],
   "signature": {
     "alg": "ES256",
     "jwk": {
       "kty": "EC",
       "crv": "P-256",
       "x": "_gow8fcS3Dx9z6j57U5q8tunnRBdrgLU9A7CZTYCnqU",
       "y": "bdfJGraBVL5aPj38TG4tHwxpU2VKwG1XBp0wQfCLOFQ"
     },
     "val": "OfTUed3gOO58ZPcP5ZXpL6OA9iyHskfo0zKI60H6XriGNNNb5sugGz80nqfyJ25jXPRYGAWcjH-1KGy5_eQuJQ"
   }
}

   // Perform normalization
   var clone = Object.assign({}, signedObject.signature);     // Clone "signature" child object
   var signature = decodeBase64URL(clone.val);                // Get signature value
   delete signedObject.signature.val;                         // Remove signature value property from signed object
   var data = convertToUTF8(JSON.stringify(signedObject));    // Get normalized JSON string (signed data)
   signedObject.signature = clone;                            // Restore signed object
   // Do the crypto now

That's clearly not rocket science.

Anders

> I have this need frequently:
> 
> For an efficient encoding, I use JSON-B which is a strict superset of JSON with additional code points that allow binary data to be spliced in.
> 
> JSON only uses bytes in the range 1-127 for tagging. Bytes in the range 128-255 can only occur inside strings. Which means all the other values can be repurposed
> 
> The first byte says that this is a binary value with an 8 bit length and the final block of the sequence (i.e. there is only one block in this sequence).
> 
> I would much rather extend JSON to support efficient encoding of any binary value than develop a canonicalization hack that could only ever work for JSON.
>