Subject: Re: [codec] I-D Action: draft-ietf-codec-oggopus-00.txt

From: Jean-Marc Valin <[email-masked]>
Date: Fri, 29 Mar 2013 17:39:03 -0400

Hi Tim,

Here's a revised patch that describes guidelines for encoders. These
guidelines are related to gapless support, but I'm sure there's other
recommendations we should give encoder implementers. The general idea is
to minimize the number of broken or suboptimal encoders.

Cheers,

	Jean-Marc

On 11/19/2012 06:00 PM, Timothy B. Terriberry wrote:
> [email-masked] wrote:
>> A New Internet-Draft is available from the on-line Internet-Drafts
>> directories.
>>   This draft is a work item of the Internet Wideband Audio Codec
>> Working Group of the IETF.
> 
> The (other) chairs declared there was consensus to adopt this draft as a
> WG item and asked me to upload a new version of it.
> 
> I have made a few minor edits from draft-terriberry-oggopus-01:
> 1) Fixed up the references.
> 2) Fixed a couple of typos.
> 3) Replaced a "should" with a "SHOULD" in Section 4.1.
> 4) Clarified the language around the starting granule position in the
> case that a) there is more audio in packets that complete on the first
> audio data page with a completed packet than the granule position
> indicates and b) the EOS flag is set on the same page (in this case you
> should not count forwards from 0, but should work backwards to figure
> out the real starting granule position).
> 5) Updated the acknowledgments.
> 
> I have not incorporated any of the proposals that have been made to the
> list since the initial draft was posted (e.g., seamless chaining,
> replaygain tags, etc.).
> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec
> 
> 

diff --git a/doc/draft-ietf-codec-oggopus.xml b/doc/draft-ietf-codec-oggopus.xml
index ab20be5..9c23cbf 100644
--- a/doc/draft-ietf-codec-oggopus.xml
+++ b/doc/draft-ietf-codec-oggopus.xml
@@ -1138,6 +1138,74 @@ An implementation could reasonably choose any of these numbers for its internal
 </t>
 </section>
 
+<section anchor="encoder" title="Encoder Guidelines">
+<t>
+When encoding Opus files, Ogg encoders should take into account the
+algorithmic delay of the Opus encoder. In encoders derived from the reference
+implementation, the number of samples can be queried with:
+opus_encoder_ctl(encoder_state, OPUS_GET_LOOKAHEAD, &amp;samples_delay);
+To achieve good quality in the very first samples of a stream, the Ogg encoder
+MAY use LPC extrapolation to generate at least 120 extra samples
+(extra_samples) at the beginning to avoid the Opus encoder having to encode
+a discontinuous signal.
+For an input file containing length samples, the Ogg encoder, SHOULD set the
+preskip header flag to samples_delay+extra_samples, encode at least
+length+samples_delay+extra_samples samples, and set the granulepos of the last
+page to length+samples_delay+extra_samples.
+This ensures that the encoded file has the same duration as the original, with
+no time offset. The best way to pad the end of the stream is to also use LPC
+extrapolation, but zero-padding is also acceptable.
+</t>
+
+<section anchor="lpc" title="LPC Extrapolation">
+<t>
+The first step in LPC extrapolation is to compute linear prediction coefficients.
+When extending the end of the signal, order-N (typically with N ranging from 8
+to 40) LPC analysis is performed on a window near the end of the signal. The last
+N samples are used as memory to an infinite impulse response (IIR) filter. The
+filter is then applied on a zero input to extrapolate the end of the signal. Let
+a(k) be the kth LPC coefficient and x(n) be the nth sample of the signal, each
+new sample past the end of the signal is computed as:
+<artwork align="center"><![CDATA[
+        N
+       ---
+x(n) = \   a(k)*x(n-k)
+       /
+       ---
+       k=1
+]]></artwork>
+The process is repeated independently for each channel. It is possible to extend
+the beginning of the signal by applying the same process backward in time. When
+extending the beginning of the signal, it is best to apply a "fade in" to the
+extrapolated signal, e.g. by multiplying it by a half-Hanning window.
+</t>
+
+</section>
+
+<section anchor="continuous_chaining" title="Continuous Chaining">
+<t>
+In some applications, such as Internet radio, it is desirable to cut a long
+streams into smaller chains, e.g. so the comment header can be updated.
+This can be done simply by separating the input streams into segments and
+encoding each segment independenty.
+The ony drawback of this approach is that it creates a small discontinuity
+at the boundary due to the lossy nature of Opus.
+An encoder MAY avoid this discontinuity by using the following procedure:
+<list style="numbers">
+<t>On the last frame of the first stream, encoding an independent frame by
+turning off all forms of inter-frame prediction (de-emphasis is allowed).</t>
+<t>setting the granulepos of the past page to a point near the end of the last
+frame.</t>
+<t>Beginning the new stream with a copy of the last frame of the first
+stream.</t>
+<t>Setting the preskip flag of the second stream in such a way as to properly
+join the two streams.</t>
+<t>Continuing the encoding process normally from there, without any reset to
+the encoder.</t>
+</list>
+</t>
+</section>
+
 <section anchor="implementation" title="Implementation Status">
 <t>
 What follows is a brief summary of major implementations of this

---

From: Ron <[email-masked]>
Date: Sat, 30 Mar 2013 17:58:00 +1030

Hi Jean-Marc,

A few quick "first-reading" comments on this ...

> +extrapolated signal, e.g. by multiplying it by a half-Hanning window.

If this is going to be generally recommended, I wonder if we shouldn't provide
a 'reference implementation' of it in libopus, and also make some reference to
that here?

In the following section, I wonder if we need to provide a few more specifics
(and also possibly some API support in libopus), since this draft primarily
is focussed on an encapsulation format, and many people implementing it will
likely be using a separate encoder library, quite possibly without having, or
otherwise needing, a very detailed knowledge of the Opus spec itself or the
internals of the encoder they are using.

The issues you raise here, while important, do kind of blur the lines between
"encapsulating the output of an encoder" and "doing additional encoding above
and beyond what the encoder you may be using does".  And they aren't specific
to the Ogg encapsulation, other mechanisms would surely get the same benefits
from them too.

> +encoding each segment independenty.

s/independenty/independently/

> +turning off all forms of inter-frame prediction (de-emphasis is allowed).</t>

How do people turn that off?

> +<t>setting the granulepos of the past page to a point near the end of the last

How do they determine where that point should be?

> +join the two streams.</t>

What such ways would ensure this is done properly?

> +<t>Continuing the encoding process normally from there, without any reset to

The answers to these questions may be obvious to someone who has studied the
Opus spec and the encoder that they are using in detail, but in the context
of this draft I suspect we should probably offer slightly stronger guidance
about how to get this right, if we are going to include these concerns here.

 Cheers,
 Ron

---

From: Jean-Marc Valin <[email-masked]>
Date: Sat, 30 Mar 2013 21:20:43 -0400

Hi Ron,

> a 'reference implementation' of it in libopus, and also make some reference to

I'm not sure this really belongs in libopus -- maybe in libopusfile once
it gets encoder support?

> internals of the encoder they are using.

I'd rather minimize references to the libopus API because it's not
normative and it may be different many years from now.

> to the Ogg encapsulation, other mechanisms would surely get the same benefits

It's not specific to Ogg, but it's still useful knowledge for someone
generating an Ogg file. In theory, you should be able to create an
encoder using just this draft and the Opus and Ogg RFCs.

> How do people turn that off?

That's actually something that needs to be cleanly exposed in the
libopus API. The functionality is already there and you can probably get
it to work by specifying 100% packet loss, but it needs to be a
dedicated setting.

> How do they determine where that point should be?

I mean that any point near the end would work. Any thoughts on how to
make this clearer?

> What such ways would ensure this is done properly?

Well, properly as in avoiding hole or duplicated samples, i.e. the
preskip should be such that the first sample of the second stream is the
one that follows the last sample of the first stream. Any suggestion on
the wording?

Cheers,

	Jean-Marc