Subject: strange duplicate ack behavior

From: Stefan Savage <[email-masked]>
Date: Thu, 20 Jan 2000 13:36:08 -0800

A few months ago I released a tool for estimating one-way loss by exploiting
TCP's fast retransmit behavior (see
http://www.cs.washington.edu/homes/savage/sting/ for details).  Most of my
users have been quite happy with it, but I've received one report of
spurious results that is stumping me.  It appears that under certain
configurations, modern Solaris TCP stacks (nmap suggests between 2.5 and
2.7) will generate up to TWO duplicate acknowledgements per out-of-order
packet.  I say "up to" because if the out-of-order packets are not spaced
far enough apart, only a single duplicate ack is sent.  For this reason I
suspect an interaction with the delayed ack timer.  Here's an excerpt from a
tcpdump to show what I mean:

13:11:01.023694 A.11155 > B.http: S 4096623726:4096623726(0) win 0 (DF)
13:11:01.221263 B.http > A.11155: S 3198574868:3198574868(0) ack 4096623727
win 4096 <mss 1460>
13:11:01.224463 A.11155 > B.http: P 4:5(1) ack 1 win 0 (DF)
13:11:01.422993 B.http > A.11155: . ack 1 win 4096
13:11:01.536986 B.http > A.11155: . ack 1 win 4096
13:11:02.233622 A.11155 > B.http: P 5:6(1) ack 1 win 0 (DF)
13:11:02.431472 B.http > A.11155: . ack 1 win 4096
13:11:02.535080 B.http > A.11155: . ack 1 win 4096
13:11:03.233471 A.11155 > B.http: P 6:7(1) ack 1 win 0 (DF)
13:11:03.432267 B.http > A.11155: . ack 1 win 4096
13:11:03.540995 B.http > A.11155: . ack 1 win 4096
13:11:04.233312 A.11155 > B.http: P 7:8(1) ack 1 win 0 (DF)
13:11:04.433672 B.http > A.11155: . ack 1 win 4096
13:11:04.535917 B.http > A.11155: . ack 1 win 4096
13:11:05.233163 A.11155 > B.http: P 8:9(1) ack 1 win 0 (DF)
13:11:05.439115 B.http > A.11155: . ack 1 win 4096
13:11:05.534234 B.http > A.11155: . ack 1 win 4096

Its pretty suspicious that the second ack is 100ms after the first...  

Two other clues: This does not occur with all services (for instance, if I
go to port 25 on the same box I will see normal behavior) so presumably it
may reflect some interaction between the delayed ack timer and select.
Second, this doesn't seem to happen with all solaris boxes... only some of
them (so it may reflect a combination of application characteristics, stack
and configuration options).  

Unfortunately, I don't have Solaris source code or I'd check this out
myself.  Has anyone seen this before or have any insight into what's
happening?  Thanks.

- Stefan