Subject: [codec] #3: 2.2. Conferencing: Support of binaural audio?

From: "codec issue tracker" <[email-masked]>
Date: Tue, 23 Mar 2010 17:33:18 -0000

#3: 2.2.  Conferencing: Support of binaural audio?
------------------------------------+---------------------------------------
 Reporter:  hoene@…                 |       Owner:     
     Type:  enhancement             |      Status:  new
 Priority:  major                   |   Milestone:     
Component:  requirements            |     Version:     
 Severity:  -                       |    Keywords:     
------------------------------------+---------------------------------------
 The mono mixing in teleconferences is somewhat technically outdated. It
 would make more sense to state the requirements for stereo mixing, which
 greatly helps to quality of teleconferencing.
 Then, if participants are placed at particular positions, they can be
 identified and understood much better, especially if multiple persons
 speak at the same time.

 Thus, I would suggest to give the low complexity mono mixing a low
 priority and the stereo mixing a higher priority.

-- 
Ticket URL: <http://tools.ietf.org/wg/codec/trac/ticket/3>
codec <http://tools.ietf.org/codec/>

---

From: "Slava Borilin" <[email-masked]>
Date: Tue, 23 Mar 2010 20:53:32 +0300

I agree 100% that the conference should be stereo, however, I am not sure that CODEC should solve еры positioning task.
There are other ways like sending the spatial position toward the client and let the client-side to the positioning job.
(remember, that in today's conference, like H.264SVC, assume that each participant MAY HAVE DIFFERENT LAYOUT of participants on the screen, therefore, spatial position can be different for different listeners).
So its not probably wise to mix in stereo on the server...

Slava Borilin

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of codec issue tracker
Sent: Tuesday, March 23, 2010 8:33 PM
To: [email-masked]
Cc: [email-masked]
Subject: [codec] #3: 2.2. Conferencing: Support of binaural audio?

_______________________________________________
codec mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/codec

---

From: "Christian Hoene" <[email-masked]>
Date: Tue, 23 Mar 2010 11:04:21 -0700

Hi Slava,

>I agree 100% that the conference should be stereo, however, I am not sure that CODEC should solve еры

No, this is not the task of the codec but the conferencing application

>(remember, that in today's conference, like H.264SVC, assume that each participant MAY HAVE DIFFERENT
>LAYOUT of participants on the screen, therefore, spatial position can be different for different
>listeners).

Actually, here are two ways of implementing:
a) Mixing all streams at all clients (requires N clients to N clients mono distribution).
b) Mixing at a central conferencing server (requires N clients to 1 server (mono) and 1 server to N clients (stereo) distribution)

Currently, I do not understand, why (b) is unwise. 

(a) does not have any impact on the requirements but in case of (b) codec requirements are the support of stereo speech transmission and support for efficient mixing.

With best regards,
 Christian

---

From: Gregory Maxwell <[email-masked]>
Date: Tue, 23 Mar 2010 11:59:24 -0700

> (a) does not have any impact on the requirements but in case of (b) codec requirements are the support of stereo speech transmission and support for efficient mixing.

I agree that (b) requires support for stereo. 

Since you're coming from mono you're going to do the panning/auralization to  virtually position participants at different locations.   Physiologically/acoustically correct positioning is likely going to defeat any mixing short-cuts that a codec provides.     E.g. with CELT (and presumably other codecs)  you can use the independent frame mechanisms  to eliminate or minimize  conferencing server computation when hard switching between sources,  but you can't do this if you're converting a mono stream to positioned stereo.  So I don't think that it requires any more than support for stereo. 

I suppose we could ask that codecs support a mode where a mono stream is sent with a positioning wrapper, instead of using stereo over the link... But for that case, I think that could be provided equally or better by another protocol or wrapper also running on RTP in a codec independent manner.

Of course, minimizing the codec computational burden is helpful for scaling conferencing systems.

Can anyone share some typical conference scaling numbers which they consider interesting?    It's been my view that all comers are fast enough that conferencing on commercial scale isn't much of an issue:  That, yes, it might take some serious processing grunt to handle 10,000 users on a conferencing system, but if you're running at that kind of scale you could afford the required hardware.   I've not see any numbers to suggest that this isn't the case.

---

From: "Slava Borilin" <[email-masked]>
Date: Wed, 24 Mar 2010 00:02:22 +0300

Sure.
	
I see that web-conferencing applications are different - as typical business case (take for Webex as example) is that provider charge only ONE user (host) for the conference, but then you have to mix 10 people (participants) in that conference, consuming "10" resources and getting only 1 money.

System for 1000 paid accounts like webex requires having 10 000 ports in the conferencing server.

Therefore, mixing (or muxing) efficiency of conferencing servers are nowadays MORE important then before.

So webconfrencing requires much better scaling of the conferencing that old "all-per-minute" telco models.
This is important both for audio and video.

For video finally we have 264SVC, and for audio we (again) need layered codec.

с уважением,
Вячеслав Борилин

---

From: Stefan Sayer <[email-masked]>
Date: Tue, 23 Mar 2010 22:04:42 +0100

> I agree that (b) requires support for stereo. 
I, too, agree on that, and I think stereo encoding support is required, 
and for spatial audio conferencing with centralized mixing it is 
sufficient. The question here would be: Can the codec optimize stereo 
encoding, if it is known to the codec that the stereo signal contains 
sources which are spatially rendered with known angles (because the two 
signals are strongly correlated with a known time difference)? Or will 
the usual methods employed in stereo encoding already catch this?

It is also my view that for decentralized mixing, a positioning 
mechanism should not be part of the codec.

Note also that support for stereo and multi-channel is also explicitly 
mentioned in applications 2.3 Telepresence and 2.6  Live distributed 
music performances / Internet music lessons.

> Of course, minimizing the codec computational burden is helpful for scaling conferencing systems.
definitely

> 
> Can anyone share some typical conference scaling numbers which they consider interesting?    
 > It's been my view that all comers are fast enough that conferencing 
on commercial scale isn't
 > much of an issue:  That, yes, it might take some serious processing 
grunt to handle 10,000 users on a conferencing system,
 > [...]

for PSTN quality (G.711) conferencing, today a good PC based server 
without any extra DSP hardware can handle 5000 users, but that drops to 
about 60% of it if you use GSM codec, and about 23% for iLBC or speex 
codec. so, minimizing the computational burden is indeed helpful.

Regards
Stefan