Subject: [core] draft-ietf-core-coap-09: Artificial option count and length limitations

From: "Kovatsch Matthias" <[email-masked]>
Date: Mon, 26 Mar 2012 13:52:07 +0000

Dear working group

draft-ietf-core-coap-09 fixed the 15 options limitation with a special rule, the End-of-options Marker. We have a second special rule to concatenate Proxy-Uri options that exceed the length of a single option. The length limitation for options in general, however, is still there! Although hitting a limit of 270 seems more unlikely than hitting 15, it is still there. With that, CoAP would also be the first RFC that introduces a limitation for URI segments.

I suggest a solution that removes the artificial limitation and the special handling for option count and Proxy-Uri.

Starting from a 4-bit length field, an additional 1-byte field of length information is  added whenever all bits of a field are set to ones:

     0                   1                   2                   3

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   |OC/Del.|1 1 1 1|  Length - 15  | Option Value ...

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     0                   1                   2                   3

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   |OC/Del.|1 1 1 1|1 1 1 1 1 1 1 1| Length - 270  | Option Value...

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

This unary numeral system might become inefficient for really large numbers, but they are unlikely and not even supported in the current draft. It is efficient for the expected lengths, as the encoding length adapts.

The biggest pain will be that the CoAP header will need restructuring to move the option count to the end.

To keep the alignment for code and MID, we should also move version and type:

     0                   1                   2                   3

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   |      Code     |          Message ID           |Ver| T |  OC   |

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

I do not know, if anyone used the version field to filter out garbage datagrams early, but a valid code might even be more effective for that.

Although this restructuring appears radical and we just now had a CoAP conformance test with the "old format," I really think we should fix the limitation we would introduce for URIs and benefit from a simpler, unified handling of length information.

The changes in the draft text would be as follows:

3.1. Message Format

"Option Count (OC): 4-bit unsigned integer.  Indicates the number of options after the header (0-14).  If set to 0, there are no options and the payload (if any) immediately follows the header.

If the number of options is greater than 14, OC is set to 15 and another byte is added as an 8-bit unsigned integer whose value is added to the 15, allowing a number of 15-269 options. This procedure can be repeated by setting a byte to 255. Another byte is added to allow a number of 270-524 and so forth."

3.2. Option Format

"Length: Indicates the length of the Option Value, in bytes. Normally Length is a 4-bit unsigned integer allowing value lengths of 0-14 bytes.  When the Length field is set to 15, another byte is added as an 8-bit unsigned integer whose value is added to the 15, allowing option value lengths of 15-269 bytes. This procedure can be repeated like for the Option Count field (see Section 3.1) until the option length can be represented.

Note that the same mechanism can be used to de-/encode the Option Count and the option length field."

The paragraphs "Alternatively..." and "End-of-options Marker..." would be dropped.

5.10.3. Proxy-Uri

"The option value is an absolute-URI ([RFC3986], Section 4.3)." can be moved to the first paragraph and the second and third would be dropped.

I hope I will find enough support for these changes, as I am quite unhappy with the current solution in draft 09.

Best regards

Matthias

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 26 Mar 2012 17:23:35 +0200

Hi Matthias,

thank you for this WGLC comment.
Quick reaction (no WG chair hats in sight):

I like your idea of getting rid of the 270 limit for options, because the current limit made us introduce the irregular handling of uri-proxy which we really need to get rid of, and there seems to be very little downside to fixing this.

I don't understand what makes you think we need to develop a new, non-compatible way to solve the OC=15 problem.  We just had a rather successful interoperability event with over 20 implementations supporting the current format.  There are also good technical reasons to have the version field in front.  The whole thing strikes me as a gratuitous change with zero benefits.

So I would like to have us discuss separately:

1 -- removing the 270 limit, enabling us to fix long Uri-Proxy values.

2 -- going back to the drawing board on the message format to solve a problem you don't state (except that you are "unhappy").

Grüße, Carsten

---

From: Guido Moritz <[email-masked]>
Date: Mon, 26 Mar 2012 17:56:47 +0200

Just to clarify: The current limitation to 270 Byte in the draft also
affects the URI-options, not only the proxy URI?!

Without reading the draft in depth again: Wouldn't it be reasonable to
remove the limitation from all the URI options?! I would appreciate! As
Matthias already mentioned, in other protocols there is no limitation as
well. It may become terribly inefficient to use longer URIs. But it will
become even worse when putting all the stuff from the URI in the payload due
to the URI limitation.

URIs are such an important concept of REST, we shouldn't scare implementers
due to such limitations. 

My2Cents,
Guido

> -----Ursprüngliche Nachricht-----
> Von: [email-masked] [mailto:[email-masked]] Im Auftrag von
> Carsten Bormann
> Gesendet: Montag, 26. März 2012 17:24
> An: Kovatsch Matthias
> Cc: core WG
> Betreff: Re: [core] draft-ietf-core-coap-09: Artificial option count and
length
> limitations
> I like your idea of getting rid of the 270 limit for options, because the
current
> limit made us introduce the irregular handling of uri-proxy which we
really
> need to get rid of, and there seems to be very little downside to fixing
this.
> 
> I don't understand what makes you think we need to develop a new, non-
> compatible way to solve the OC=15 problem.  We just had a rather
successful
> interoperability event with over 20 implementations supporting the current
> format.  There are also good technical reasons to have the version field
in
> front.  The whole thing strikes me as a gratuitous change with zero
benefits.
> 2 -- going back to the drawing board on the message format to solve a
problem
> you don't state (except that you are "unhappy").
> 
> Grüße, Carsten
> 
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 26 Mar 2012 18:02:32 +0200

On Mar 26, 2012, at 17:56, Guido Moritz wrote:

> remove the limitation from all the URI options?! 

Hi Guido,

welcome back to the mailing list :-)

the 270 byte limitation we have in -09 applies to each single option; in the case of URIs these are already split up into component options to eliminate percent-encoding processing at the servers.  So you can have pretty long URIs, just not with segments longer than 270 bytes.
Matthias' proposal 1 fixes that limitation, too.
It also allows us to get rid of a special case that provided for longer Uri-Proxy URIs by splitting them up into 270-byte fragments (ouch).

Grüße, Carsten

---

From: "Kovatsch Matthias" <[email-masked]>
Date: Wed, 28 Mar 2012 11:28:03 +0000

So, will there be a ticket for '1?' :)
Carsten Bormann <[email-masked]> wrote:

Hi Guido,

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 28 Mar 2012 13:47:08 +0200

> So, will there be a ticket for '1?' :)

Yes!

I plan to make tickets from the notes of the first WG meeting in the course of the afternoon.
(I just want to make sure I capture things like repairing Uri-Proxy when this is fixed in the same ticket...)

Grüße, Carsten