Subject: Doubletree RST 'BOF' meeting notes

From: IETF user <[email-masked]>
Date: Wed, 2 Aug 2000 16:07:44 -0400 (EDT)

Attendees:  Ian Heavens ("chair"/minutes,Spider Software), William Gilliam (HP),
            Joe Touch (ISI), Bernie Volz (IPworks), Tom Lane (sss?), Scott Lawrence (Agranat)

I hope these notes reflect the discussion - please add/correct if not!
It was (as always at the IETF) a spirited discussion and I may have
not always reflected the consensus accurately. 

1.  Error codes with RSTs

Two approaches to this were apparent - useful diagnostic tool (just need to 
see these on the wire) - and whether the receiver could do anything useful
with them.  As pointed out on the mailing list, the RST data field is
not appropriate for the latter  - no API, and standardising this might (or will)
overlap with other current uses.  Joe Touch pointed out that what is
really needed is a generic "Instrumentation Option" that can be carried
in any segment.  However, the "send RST error code" option would have
to default to on (send it unless the receiver NAKs the option request),
or the current usefulness of using the data field would be lost (until
receiver TCP implementations supported the option).  Instrumentation option
could be passed to the application as SACK option implementation supports this.
[the SACK people would like this option too - Matt Matthis told me afterwards
however that currently SACK options must be sent with the SYN as some implementations
fall over if the first TCP option is sent with a non-SYN segment.  So this
must be solved before the instrumentation option is generally useful.

Two conclusions: [1] Instrumentation option generally useful and the only mechanism
by which RST error codes can be standardised [2] RST error code taxonomy
can still be published as an Informational RFC as it will be needed for
use by the instrumentation option (and TCP implementations that send data
with RSTs in this way can - at least - use agreed codes) [is anyone arguing
that it is bad to use RST data in this manner?  Or just that it cannot be
reserved for this usage?]. 

I don't know if anyone wants to follow up the instrumentation option issue??
HP/Mentat/Spider/(Sun?) will complete and submit my RST error codes draft. 
           
2. TIME-WAIT and RSTs

There is no TIME-WAIT after RST (first submitted in expired draft by myself
in 1994 I think..) and so no protection against acceptance of stale data
by a new connection.  Solutions discussed in that draft and Joe Touch's paper
on using RSTs to transfer responsibility for TIME-WAIT from servers to clients.
Basically TIME-WAIT after sending a RST (in ESTABLISHED and half-closed states)
with 'soft and hard TIME-WAIT' indicating how RSTs are treated in TIME-WAIT
(stemming from RFC 1337 and Joe's paper).

We discussed the implementation issues in supporting many TIME-WAITS - as this
was perceived at the time as an obstacle to solving the issue, since TIME-WAIT
after RST will increase the number of connections in TIME-WAIT state: 
these are well documented and understood now.
 
I will submit the "Problem Statement" again as an Informational RFC (to underline
that application writers should not abort connections with a RST to circumvent
TIME-WAIT: Scott Bradner suggested a few years ago that it could be a Best Current
Practices to reinforce this).  Joe/myself/Ted Faber will submit the solution as an 
internet draft (whether Informational or standards track will depend on whether
it is perceived that the solution is preferable to the status quo).  This will
strongly reference work on implementation issues.

3.  Miscellaneous application and API issues

Tom  Lane has highlighted a usage of RSTs that conflicts with RFC 1122 half duplex
close recommendation (and emphasised in RFC 2525 - contributed by me).  Bernie Volz 
and Tom discussed this after so I don't know whether the conclusion was that the 
application should avoid sockets close() and use shutdown() - thereby circumventing 
RFC 1122 completely (it only applying to use of APIs that close in both directions ).
To be pursued on the list if appropriate, with possible clarification of RFC 1122
and RFC 2525.

The Transport Area WG is appropriate for discussion of the above drafts and the
authors will bring them to a future IETF meeting...

Thanks to all those who participated

ian (about to duck :-)

PS I will put together a web page with the missing references above

---

From: Tom Lane <[email-masked]>
Date: Wed, 02 Aug 2000 18:41:52 -0400

Ian Heavens wrote:
> Tom Lane has highlighted a usage of RSTs that conflicts with RFC 1122
> half duplex close recommendation (and emphasised in RFC 2525 -
> contributed by me).  Bernie Volz and Tom discussed this after so I
> don't know whether the conclusion was that the application should
> avoid sockets close() and use shutdown() - thereby circumventing RFC
> 1122 completely (it only applying to use of APIs that close in both
> directions ).  To be pursued on the list if appropriate, with possible
> clarification of RFC 1122 and RFC 2525.

[ Hi folks.  I just joined the list --- am not a TCP implementor,
but got interested in this particular issue for reasons about to be
stated. ]

The situation that I am interested in is an application where data can
flow asynchronously in both directions.  Actually, it's mostly a client
request/server response protocol, but the server can also send
unsolicited messages at any time.  In particular the server application
can send an error message and immediately close the connection (indeed,
it quits completely) without waiting around for the far-end app to get
that message.  Question for the list is, is this application misusing
the TCP protocol, and if so why?  Seems to me that once the TCP stack
has accepted data to be sent, it ought to make a reasonable effort to
deliver that data, whether the sending app is holding the connection
open or not.

The problem case arises when the far-end app is busy sending a new
request just as the near-end app emits its final message and closes
its socket FD.  RFC 1122 (4.2.2.13) says

            A host MAY implement a "half-duplex" TCP close sequence, so
            that an application that has called CLOSE cannot continue to
            read data from the connection.  If such a host issues a
            CLOSE call while received data is still pending in TCP, or
            if new data is received after CLOSE is called, its TCP
            SHOULD send a RST to show that data was lost.

(See also 2.16 and 2.17 in RFC 2525.)

Some, but not all, stacks appear to issue such an RST when they see
incoming data from the far end, *even though they still have un-acked
outbound data to transmit*.  This results in loss of the outbound data.
A variant is that even if the data has been transmitted and acked but
not actually delivered to the far-end client app, the far-end TCP may
discard the data from its buffer upon receiving the RST.  Still a third
variant (which seems to be what the current Linux stack does, at least
when both ends are on the same machine) is that the far-end client may
receive an ECONNRESET error from its next read() attempt, though not
all the data has been delivered to it.  If the client is loony enough
to think that ECONNRESET is a soft error, it will indeed discover that
it can read more data by trying again!

I claim that all these behaviors are broken, and that the right thing
to do if the TCP has noplace to deliver incoming data to is to ack and
discard the data.  It would be OK to send an RST to discourage the other
end from sending more data *only* if there is no un-ACKed data remaining
to be sent from this end.  Without this, half-duplex connections are
useless.  TCPs should also be dissuaded from dropping buffered inbound
data when they see inbound RST or delivering a report of incoming RST
out-of-sequence from the data.

The discussion at the Doubletree mostly focused on whether the app
should deal with this situation by holding open its socket for some
timeout period after sending its final outgoing message, so as to
read and discard any data that might happen to arrive.  That is
certainly the only available answer if one needs an answer today,
but I still think that it's a workaround for broken TCP behavior.
Essentially, this is asking the app to take on the connection-shutdown
duties that TCP ought to handle for it.  (To take just one example,
how should the app know what an appropriate timeout is?  Seems like
that's a concern the TCP layer should deal with, not the app.  The app
will also be unable to detect when the last outbound data has been
ACKed, and will therefore have to hold the connection open much longer
than would otherwise be necessary in most cases.)

Anyway, that's my story.  Is my app broken, or is the recommendation
for issuing an RST in this scenario misguided?

			regards, tom lane

---

From: Craig Partridge <[email-masked]>
Date: Thu, 03 Aug 2000 09:21:27 -0400

>HP/Mentat/Spider/(Sun?) will complete and submit my RST error codes draft. 

Some of the email suggested the error codes should follow the SMTP/FTP
convention (3 digit numbers where the different digits indicate the
severity of error).  I thought that was a nice idea.  Is that still the
plan?

Thanks!

Craig

---

From: Ian Heavens <[email-masked]>
Date: Thu, 3 Aug 2000 14:57:41 +0100

> severity of error).  I thought that was a nice idea.  Is that still the

Sounds a good approach...the only thing is I'm not sure how many
severity levels one could define for a RST....  

cheers

ian

---

From: Craig Partridge <[email-masked]>
Date: Thu, 03 Aug 2000 10:31:31 -0400

>severity levels one could define for a RST....  

Well the existing scheme envisions two types: transient and permanent
errors and then a bunch of subcodes.

Now I'm not sure there's any transient condition that provokes an RST,
but perhaps...

Craig