Subject: Re: TCP RDMA option to accelerate NFS, CIFS, SCSI, etc.

From: [email-masked]
Date: Fri, 25 Feb 2000 11:54:25 +0200

Message boundaries are only part of the proposal - and they don't imply
additional work at the client. Doing zero copy based only on the
information in current headers is certainly possible at low speed. Over 1
Gb/s it requires some innovation and lots of silicon. The RDMA option makes
it possible at a far lower price. And the zero copy it enables might go
deep into the application space as it is only an annotation on packets.
It certainly makes sense on all the new applications (NFS4, SCSI, etc) and
the retrofit into existing ones is not that difficult either.
And placing it over TCP puts it on a safer ground that having to use at
higher speed completely new and unproven protocols (VIA, NGIO etc.).

Julo
Julian Satran - IBM Research

David Robinson <[email-masked]> on 25/02/2000 03:42:05

Please respond to David Robinson <[email-masked]>

To:   [email-masked]
cc:   [email-masked] (bcc: Julian Satran/Haifa/IBM)
Subject:  Re: TCP RDMA option to accelerate NFS, CIFS, SCSI, etc.

To efficently determine boundaries within a packet stream work
must be done somewhere.  In the RDMA proposal it is up to the
clients to do the work to make the server's job easier.  In traditional
intelligent NIC cards the server does the work by parsing the headers.

It seems that the design of RDMA is backwards as it relies on changes
to the many clients to enable efficiency on the server. A traditional
intelligent NIC card with a modest amount of hardware/firmware
can handle 99+% of requests from unmodified clients.  The existence
proof is checksumming NICs and NFS accelerator boards.

For an efficient IP storage device it will have to deal with legacy IP
client stacks (no RDMA) and a competitive IP storage vendor will
implement the smart NIC described above. Why is RDMA more compelling?

     -David

---

From: Alan Cox <[email-masked]>
Date: Fri, 25 Feb 2000 13:17:05 +0000 (GMT)

> deep into the application space as it is only an annotation on packets.

I am not convinced the amount of silicon changes between the two. The
RDMA id make be faked by an attacker so must still be verified. 

Va Jacobson proposed and to an extent implemented a system where the user
context does all the TCP work. In that sort of situation and with a more
sensible API than the BSD socket one you dont appear to need a lot of silicon,
in fact the worst case is the wildcard.

Alan

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 25 Feb 2000 08:19:10 -0600

As I recall, Van's solution required an unconventional I/O system where DMA went
straight into user memory right off the LAN chip. Not too many people make
computers like that.

  Brian

---

From: Alan Cox <[email-masked]>
Date: Fri, 25 Feb 2000 14:30:58 +0000 (GMT)

> straight into user memory right off the LAN chip. Not too many people make

Actually it doesn't, although every PC today is your 'unconventional' system.
Many 10Mbit and almost all the 100Mbit cards do scatter gather DMA.  To do
zero copy means landing the buffer into memory that can become accessible to
the user, the rest of the benefits for single copy/checksum/user come with
DMA landing in purely kernel controlled space