Subject: [codec] possible issues to track

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 25 Mar 2010 14:12:53 -0700

During the WG session, I kept notes about some issues that we might want
to track (on list or in the tracker -- I'll leave it up to the
continuing chairs to decide how). They are:

- delay is not *constant* on the Internet
- add reference to ITU G.107 regarding echo
- better explain echo and algorithmic delay < 10ms
- avoid phrasing subjective comparision in relation to ITU codecs
- clarify whether each requirement is nice or necessary
- clarify the measurability of each requirement
- clarify requirement about bit-exactness
- mention DTMF in requirements
- mention synchronization of audio with other media

If I missed anything, post to the list.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 25 Mar 2010 17:32:42 -0400

> - avoid phrasing subjective comparision in relation to ITU codecs

I was actually surprised to see that one come up. As far as I know, most
ITU terms of reference include comparisons like "better than G.722 at rate
X". Does the ITU phrase this differently then we did?

As for the way of discussing the issues, it seems to me like the mailing
list is just simpler. Most people so far have commented on the list anyway.

Cheers,

	Jean-Marc

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 25 Mar 2010 14:35:55 -0700

The comment was from me, and it was addressed towards the sensitivities of
cross-SDO relationships.  It is one thing if the ITU states that something
is better than ITU-Rec xxx, but it is another thing if another body states
this.  Certainly without having received appropriate input from the ITU
after requesting it through a liaison.
Stephan

On 3.25.2010 14:32 , "Jean-Marc Valin" <[email-masked]> wrote:

> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 25 Mar 2010 17:46:42 -0400

> after requesting it through a liaison.

So do you suggest we drop the ITU codecs from the list? I don't really
mind, considering that I would not expect that part of the requirement to
be a limiting factor.

	Jean-Marc

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 25 Mar 2010 14:49:12 -0700

No, I just suggest to attempt being just a little bit diplomatic.
Stephan

---

From: Michael Knappe <[email-masked]>
Date: Thu, 25 Mar 2010 14:52:48 -0700

Thanks Peter! Some notes inline...

> - add reference to ITU G.107 regarding echo
[MEK] we need to consider delay impairment factors due to both echo and
absolute latency
> - better explain echo and algorithmic delay < 10ms
[MEK] need to understand at what point a reduction in algorithmic delay
along with achievable network latencies can significantly improve perception
of echo (e.g. Reducing noticeability of acoustic echo, and/or pulling
round-trip delays in to the point that an echo moves from being perceptively
discrete to an integrated sidetone signal)
> - clarify requirement about bit-exactness
[MEK] we will need to specify whether our final reference code is fully bit
exact, or whether some or all elements can be bit compatible. We should also
specify whether any portions of said algorithms could be 'upgradeable' (e.g.
Having a stock PLC algorithm that could be replaced with a better PLC in the
future)

> - mention DTMF in requirements
[MEK] some simple talk-off tests should be done to determine if in-band DTMF
(and potentially other call progress tones) are carried in-band without
sufficient degradation as to cause a DTMF detector misread.

> - mention synchronization of audio with other media
[MEK] further investigation is needed to determine if there are any possible
implications or dependencies for a codec's use in video applications

> If I missed anything, post to the list.

- multi-channel carriage: not sure if we mentioned this during the WG
session, but we need to decide if we will support any kind of joint channel
coding (current consensus so far is to keep all channels separable). We also
should also reference multi-channel packing within RTP (RFC 3551 sec 4.1).

---

From: "Benjamin M. Schwartz" <[email-masked]>
Date: Thu, 25 Mar 2010 18:01:35 -0400

Michael Knappe wrote:
> we need to decide if we will support any kind of joint channel
> coding (current consensus so far is to keep all channels separable).

Could you clarify this?  I think we've had several discussions where
people wanted jointly coded (i.e. non-separable) stereo.

--Ben

---

From: Michael Knappe <[email-masked]>
Date: Thu, 25 Mar 2010 14:57:30 -0700

The mailing list is definitely easier for discussing the issues, but my
suggestion is that any open items that come up in discussion (as well as the
current status of that item - e.g. Who's working on resolving it) be tracked
somewhere (an official issue tracker or even on the wiki) so that we have
some easily accessible collective memory of our progress.

Mike

---

From: stephen botzko <[email-masked]>
Date: Thu, 25 Mar 2010 15:04:21 -0700

Perhaps something as simple as saying "not worse" instead of "better".

Stephen Bozko

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 25 Mar 2010 18:11:49 -0400

> Perhaps something as simple as saying "not worse" instead of "better".

Agreed.

	Jean-Marc

> 
> Stephen Bozko
> 
> On Thu, Mar 25, 2010 at 2:49 PM, Stephan Wenger <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     On 3.25.2010 14:46 , "Jean-Marc Valin" <[email-masked]
>     <mailto:[email-masked]>> wrote:
> 
>     > Stephan Wenger wrote:
>     >> On 3.25.2010 14:32 , "Jean-Marc Valin"
>     <[email-masked] <mailto:[email-masked]>>
>     wrote:
>     >>
>     >>> Peter Saint-Andre wrote:
>     >>> [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 25 Mar 2010 15:15:06 -0700

> No, I just suggest to attempt being just a little bit diplomatic.

Yes, you phrased it more diplomatically in the meeting, too. I don't
remember the exact wording, but it was something like "it might make the
job of our liaison more difficult if we compare our work only to ITU-T
codecs".

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 25 Mar 2010 18:19:28 -0400

> Having a stock PLC algorithm that could be replaced with a better PLC in the

There is already some text addressing that, but it's part of the guidelines
rather than the requirements. I guess it's possible to move it to the
requirements, but I'm undecided as to whether it's a good idea or not.
See section 4 of the guidelines draft: "Requirements Conformance" in
http://tools.ietf.org/html/draft-valin-codec-guidelines-04

It addresses bit-exactness, PLC, future improvements, ... Let me know what
you think.

> sufficient degradation as to cause a DTMF detector misread.

Definitely agree here; that was an omission from the current draft. More
precisely, we should probably select a minimum bit-rate after which DTMF
should be handled. I'm not sure we want to mandate that DTMF work down to 6
kb/s, but at 20 kb/s it should definitely work. Where we put the bar is an
open question.

> should also reference multi-channel packing within RTP (RFC 3551 sec 4.1).

Well, any codec always has the option of doing independent channels
(provided that the RTP payload format supports it). On top of that, I tend
to think that "joint coding" is a nice-to-have feature -- as long as it
improves the coding efficiency of course.

Cheers,

	Jean-Marc

---

From: Michael Knappe <[email-masked]>
Date: Thu, 25 Mar 2010 15:09:58 -0700

Thanks Ben, my understanding had come from a discussion with Jean-Marc and
some others involved in that discussion a few days back, apologies if that
misrepresented consensus on this topic.

Jean-Marc, can you discuss your thoughts with respect to joint channel
coding?

Ben, how would you approach channel counts larger than 2? Joint coding
between any two L-R pairs, with other channels (like the center channel)
left singular?

Cheers,

Mike

---

From: Michael Knappe <[email-masked]>
Date: Thu, 25 Mar 2010 15:22:19 -0700

Thanks Jean-Marc, guess I misunderstood your preference on joint coding
during our discussion, thanks Ben for calling that out.

So, in putting together our 'must-have' vs 'nice-to-have' requirements list,
we'll put joint coding down as a 'nice-to-have'?

Mike

---

From: stephen botzko <[email-masked]>
Date: Thu, 25 Mar 2010 15:30:30 -0700

I am thinking that we really ought not have a requirement for joint
encoding.  Instead it would be better to have quality requirements for
multi-channel.  If a codec candidate achieves better quality though joint
encoding, then that is great.  If it achieves the desired quality through
independent channel encoding, than that is fine also.

Ideally, we should avoid creating requirements that force candidates to use
specific technology or techniques.  I think its better if we stick with
defining the quality, delay, birates, etc that is needed for the
application.

Stephen Botzko

---

From: "Benjamin M. Schwartz" <[email-masked]>
Date: Thu, 25 Mar 2010 18:54:06 -0400

> between any two L-R pairs, with other channels (like the center channel)

I agree with Stephen Botzko that the mechanism should not be part of the
requirements.  However, we will need to agree on an interface, i.e. the
abstraction barriers.  I think:

1. The codec should provide an interface for coding a pair of closely
coupled channels with the presumption that they represent something like a
stereo pair.

2.  The format should provide an efficient way to pack multiple
independent synchronized codec channels into a single stream.

3. The format should provide a way to specify N named output channels as
linear combinations of the coded channels.

Advantages:
 - Stereo can be specially optimized.  There are mature techniques for
efficient coding of stereo, and plenty of test samples.  CELT can already
code joint stereo more efficiently than independent coding or a simple
mid-side split.
 - Surround sound can support all the various configurations (e.g. 5.1,
Ambisonics, custom speaker sets) without any major work within the spec.
 - The codec does not need to know anything about specific surround-sound
configurations, saving development time and effort.

Disadvantages:
 - This approach cannot achieve maximally efficient coding of
surround-sound, as can be achieved using techniques like those mentioned
in [1] and [2].

--Ben
[1] http://people.xiph.org/~xiphmont/surround/demo.html
[2] http://people.xiph.org/~xiphmont/surround/demo2.html

---

From: "Christian Hoene" <[email-masked]>
Date: Thu, 25 Mar 2010 16:08:56 -0700

>I agree with Stephen Botzko that the mechanism should not be part of the requirements.  However, we
>will need to agree on an interface, i.e. the abstraction barriers.  I think:
>
>1. The codec should provide an interface for coding a pair of closely coupled channels with the
>presumption that they represent something like a stereo pair.
>
>2.  The format should provide an efficient way to pack multiple independent synchronized codec
>channels into a single stream.
>
>3. The format should provide a way to specify N named output channels as linear combinations of the
>coded channels.

Later would make the mixing in a conference server also quite easy. Just pack the encoded active streams and set the linear equation
according to the placement of the participants. Very efficient!

Christian

---

From: stephen botzko <[email-masked]>
Date: Thu, 25 Mar 2010 16:54:42 -0700

Just to follow up.  i think that several of the bullets in the current
requirements draft reference specific techniques.  Though its not bad to
keep a list of potentially useful approaches somewhere, I don't think they
are actually requirements.  Time-stretching is perhaps one example, there
are several others.

Stephen Botzko

On Thu, Mar 25, 2010 at 3:54 PM, Benjamin M. Schwartz <
[email-masked]> wrote:

> [1] http://people.xiph.org/~xiphmont/surround/demo.html<http://people.xiph.org/%7Exiphmont/surround/demo.html>
> [2] http://people.xiph.org/~xiphmont/surround/demo2.html<http://people.xiph.org/%7Exiphmont/surround/demo2.html>
>
>

---

From: Jean-Marc Valin <[email-masked]>
Date: Thu, 25 Mar 2010 20:08:11 -0400

Just to clarify my last email, I think we're actually mostly in 
agreement here. We already know that we can do stereo just by using 
independent channels, so that would be the basic requirement. From 
there, it should just be up to the working group to decide whether any 
proposed stereo (or even >2 channel) coupling is worth the extra 
complexity it adds. If there's a way to get large savings from a simple 
enough coupling, then we should use it. If it turns out that we can't 
save more than a few bits, we just drop the idea. Makes sense?

	Jean-Marc

On 2010-03-25 18:30, stephen botzko wrote:
> multi-channel.  If a codec candidate achieves better quality though
> joint encoding, then that is great.  If it achieves the desired quality
> through independent channel encoding, than that is fine also.
>
> Ideally, we should avoid creating requirements that force candidates to
> use specific technology or techniques.  I think its better if we stick
> with defining the quality, delay, birates, etc that is needed for the
> application.
>
> Stephen Botzko
>
> On Thu, Mar 25, 2010 at 3:09 PM, Michael Knappe <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     On 3/25/10 3:01 PM, "Benjamin M. Schwartz" <[email-masked]
>     <mailto:[email-masked]>> wrote:
>
>      > Michael Knappe wrote:

---

From: "Benjamin M. Schwartz" <[email-masked]>
Date: Fri, 26 Mar 2010 16:58:41 -0400

> linear combinations of the coded channels.

On closer inspection, I think (3) is best left out of this work for now.
Instead, channels should be named according to their significance.
Applications that perform complex manipulations on the channels, such as
arbitrary mixing, can select appropriate names for their coded channels.
This is the approach taken in the CELT RTP encapsulation draft [1]

Positioning of speakers in multi-party conference streams should be
performed by the client in accordance with its user interface.

--Ben

[1] http://tools.ietf.org/html/draft-valin-celt-rtp-profile-02#page-13