Subject: Re: Any successor to the sockets APIs ???

From: Chuck Lever <[email-masked]>
Date: Tue, 23 May 2000 10:39:41 -0400 (EDT)

hi ramesh & al. -

our research shows that providing a more efficient poll() interface scales
as well as the new RT signals.  niels provos implemented /dev/poll in
Linux (see our Freenix track paper for Usenix 2000).  we benchmarked
thttpd using /dev/poll against phhttpd, an experimental web server that
uses Posix RT signals.

RT signals are still a young programming model, so there are issues with
using them:  portability among operating systems, how to share them among
libraries, lack of support for sharing signal queues among threads, and so
on.  of great concern to us is the use of poll() for handling an RT signal
queue overflow: poll() must be much more efficient to be an effective
overflow remedy.

at this point, i don't think we understand how to use RT signals most
effectively.  but poll is well-understood, so it's easy to change
applications to use /dev/poll and see a significant scalability
improvement.

	- Chuck Lever
--
corporate:	<[email-masked]>
personal:	<[email-masked]>

The Linux Scalability project:
	http://www.citi.umich.edu/projects/linux-scalability/

Date: Tue, 23 May 2000 10:28:31 +0200
From: Andi Kleen <[email-masked]>
To: Ramesh Shankar <[email-masked]>
Cc: [email-masked]
Subject: Re: Any successor to the sockets APIs ???

On Tue, May 23, 2000 at 05:25:56AM +0200, Ramesh Shankar wrote:
> I am curious to know whether there is any work in progress on a successor to the BSD (or for that matter Winsock) sockets API. The following are the reasons for the question:
> 
> - The BSD sockets API does not have a really powerful interface for use with event driven programming. select() and poll() are available, but the performance issues associated with it have been known for sometime now. Winsock has some interesting APIs which seem to help somewhat better than BSD sockets APIs.

Single Unix defines a replacement: real time signals that carry fd event
notifications. They unfortunately require a non standard extension to set
the SIGIO signal for a fd to be really useful (or another non standard
extension for queued SIGIO). At least in modern Linux it is supported.
poll() is still needed as fallback when signals get lost.
>From first tests queued SIGIO with a sigwaitinfo() main loop scales
very nicely for web servers.

- -Andi

---

From: "Howard Berkey" <[email-masked]>
Date: Tue, 23 May 2000 10:42:38 PDT

>> - The BSD sockets API does not have a really powerful interface for 
use with event driven programming. select() and poll() are available, 
but the performance issues associated with it have been known for 
sometime now. Winsock has some interesting APIs which seem to help 
somewhat better than BSD sockets APIs.
>
>Linux supports event based real time signalling (kind of vaguely akin 
to the
>VMS AST setup).  There are plenty of people who find poll/select scale 
nicely.
>Under load your syscalls/task count drops which materially improves 
your
>throughput. Thus as the load goes up degradation is good.
>
>Point a big internet client simulation (not crap like specweb) at a 
threaded
>or multiprocess server like apache and then try it with thttpd. Its 
not the
>simple set of equations some people think.
>
>[Apache btw suffers, thttpd just keeps going. apache is faster for 
small loads]

Two issues here cause select and poll to appear bad.  The first is 
crappy OS implementations of select;  the second is programmers that 
don't know how to properly use select in a multithreaded environment.

Alan is spot on in that if you know what you are doing, select can be 
quite scalable, provided your OS doesn't have a brain-dead 
implementation (and BSD and Linux have pretty good ones).  If you just 
throw several hundred sockets into select() and sit on it with a single 
thread, of course it will be a poor performer.

FWIW, the select() in the networking implementation currently shipping 
in BeOS is very, very poor.  It's one of the reasons we have rewritten 
the OS networking for the next release.

It also seems to me that programming correctly to select() and poll() 
is far easier and more elegant than doing it asynchronously via SIGIO.  
That opens lots of potential cans of worms. 

Howard