Subject: [core] Apparent race condition in blockwise transfer draft

From: Robert Quattlebaum <[email-masked]>
Date: Mon, 24 Dec 2012 10:55:19 -0500

Hello everyone,

I just wanted to bring up a race condition apparently present in
the current blockwise transfer draft. However, even if you are not
interested in draft-ietf-core-block, you may want to read over this
email anyway: Similar problems may be present in protocols which
need the server to associate multiple messages as being a part of
the same logical request. For now I'll limit the discussion to
draft-ietf-core-block-10.

Previously, the token field (as well as the source IP address and
port) could be used by the server to correlate multiple messages
that were a part of the same logical request ("token-affinity").
In this sense the token field was not really opaque because the
server was relying on the value of the token to not change between
subsequent messages in the same logical request. However, this
semantic-entanglement had the benefit of ensuring truly atomic
behavior of requests which consist of multiple messages, without
depending on artificial limitations on how the protocol should be
used or the behavior of a specific transport layer.

At some point over the past year (Forgive me, I haven't been paying
attention), the semantics of the token parameter was clarified to
be entirely opaque (meaning that only the client's address/port
could be used for determining which messages are a part of the same
blocked request). This was a reasonable change because it allows
client implementations to be more flexible. However, without further
clarification or additional mechanisms, this change leads to
undefined/unreliable behavior in certain cases.

Consider the case where you have a LoWPAN where nodes regularly
dump gathered information to a CoAP server in "the cloud" via a
CoAP-CoAP proxy. If two (or more) nodes try to perform a large POST
(using Block1) to the same CoAP URL thru the proxy at the same time,
the destination machine in the cloud has no way to differentiate
the messages as being two independent atomic requests.

In a previous discussion off-list, Carsten suggested that the proxy
could identify the start of subsequent requests as potentially
conflicting and use a different source port for each conflicting
request sent from the proxy, but this technique only works if the
cloud-side transport layer uses the UDP/IP transport---this technique
would not work if the proxy was sending requests to the cloud via
CoAP-over-SMS, as described in section 14 of
draft-becker-core-coap-sms-gprs-02.

I think the root of this problem is the reliance on information
from the transport layer in determining which individual messages
belong to the same logical request. Doing this just happens to work
out in most cases. However, because these edge cases are transient
in nature, diagnosing such problems in the field may be very difficult
and frustrating. Sure, there are work-arounds, but they have specific
disadvantages which may not be obvious.

The bottom line is that by refining the semantics of the token field
to no longer allow the server to use token-affinity to associate
multiple messages to logical requests, a race condition has been
introduced into the protocol. This race condition was hidden by the
fact that, by using the address/port information from the transport
layer, things *usually* work just fine. Usually, but not always.

All that being said, I don't necessarily believe that reverting the
token behavior is necessarily the right solution to this problem:
Allowing the client to use the token to keep track of state on a
message-per-message basis is useful. The trick is giving the same
flexibility to the server: If we can't rely entirely on the transport
layer information to associate logically-related messages (and we
can't use the token) then what should be used?

We could always simply document this class of problems in the draft
so that implementors are aware that they may need to work around
it, but I worry that this may be the first of a class of similar
race conditions if this issue is not addressed directly.

Thoughts?

__________________
Robert Quattlebaum
eMail:  [email-masked]
www:    http://www.deepdarc.com/