Subject: [codec] RTP Payload Format and File Storage Format for Opus Speech and Audio Codec - draft-spittka-payload-rtp-opus-00

From: "Julian Spittka" <[email-masked]>
Date: Tue, 5 Jul 2011 02:06:02 +0200

All,

Please find a first draft "draft-spittka-payload-rtp-opus-00" for the "RTP
Payload Format and File Storage Format for Opus Speech and Audio Codec"
uploaded at
https://datatracker.ietf.org/doc/draft-spittka-payload-rtp-opus/. 

Any feedback, questions, or comments are highly welcome.

Best Regards,

Julian.

---
Julian Spittka
Skype

see & chat: julian_spittka
write: [email-masked]

---

From: Gregory Maxwell <[email-masked]>
Date: Tue, 5 Jul 2011 07:01:59 -0700

> Any feedback, questions, or comments are highly welcome.

Regarding the file format part—

Ah— Why does the world need another multimedia container format?

Much less one which is unseekable (without doing a prior read of the whole file),
unable to recover from damage (no capture sequence, no corruption detection),
and high overhead (minimum overhead at 64kbit/sec of 2.5%, without dtx).

I suggest dropping the file format part— either completely or replacing it with
stubs with recommendations/parameters for encapsulating it in other more widely
implemented container formats (e.g. MKV, Ogg (RFC3533), RIFF/AVI)

People are going to do this in any case because of the aforementioned
limitations, so adding another one will just promote incompatibility if it
is used at all.

---

From: "Kevin P. Fleming" <[email-masked]>
Date: Tue, 05 Jul 2011 10:35:54 -0500

> Ah— Why does the world need another multimedia container format?

You beat me to that post :-) I'm also not quite understanding how 
definition of a file storage format would fall within the IETF's purview 
at all...

> implemented container formats (e.g. MKV, Ogg (RFC3533), RIFF/AVI)

MKA would be the container to use for this, I'd think, since it's 
audio-only.

-- 
Kevin P. Fleming
Digium, Inc. | Director of Software Technologies
Jabber: [email-masked] | SIP: [email-masked] | Skype: kpfleming
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at www.digium.com & www.asterisk.org

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Tue, 05 Jul 2011 09:08:49 -0700

>> implemented container formats (e.g. MKV, Ogg (RFC3533), RIFF/AVI)

+1

> MKA would be the container to use for this, I'd think, since it's

There's no reason you couldn't use Opus with video (and indeed, in 
WebRTC this is the expected use-case). But that's another advantage of 
providing a mapping for an established container format: you get muxing 
with video for free.

---

From: "Benjamin M. Schwartz" <[email-masked]>
Date: Tue, 05 Jul 2011 12:38:57 -0400

> Any feedback, questions, or comments are highly welcome.

Thank you for doing this work.  I think your proposed RTP and SDP formats
are very nice and a good step for us.

I have a few comments.

1.  I think the description of Opus should be removed.  There's no need to
duplicate the contents of the Opus RFC.

2.  I think the payload specification should make (almost) no mention of
Opus frames, modes, etc.  These aspects of Opus are entirely internal to
the payload, which is a "black box".  As far as the RTP specification is
concerned, Opus consists of (almost-)arbitrary-duration "packets" and
that's it.

3.  To avoid confusion, I think we should be very careful and clear about
the distinction between audio bandwidth and samplerate, as their
relationship in Opus is far from simple.

I have implemented these ideas, along with a few minor changes, as a patch
to the draft .xml, attached.  Please feel free to discard this patch; I
mean it only as a way to communicate my suggestions more clearly.

--Ben
--- draft-spittka-payload-rtp-opus-00.xml	2011-07-04 15:22:13.000000000 -0400
+++ draft-mod.xml	2011-07-05 12:27:02.717863975 -0400
@@ -126,246 +126,6 @@
       </t>
     </section>
 
-    <section title='Opus Codec'>
-      <t>
-        The Opus speech and audio codec has been developed to encode speech
-        signals as well as audio signals. Two different modes, a voice mode 
-        or an audio mode, may be chosen to allow the most efficient coding 
-        dependent on the type of input signal, the sampling frequency of the 
-        input signal, and the specific application.
-      </t>
-
-      <t>
-        The voice mode allows to efficiently encode voice signals at lower bit
-        rates while the audio mode is optimized for audio signals at medium and 
-        higher bit rates. 
-      </t>
-
-      <t>
-        The Opus speech and audio codec is highly scalable in terms of audio
-        bandwidth, network bit rate, and complexity. Further, Opus allows to
-        transmit stereo signals.
-      </t>
-
-      <t>
-        The Opus speech and audio codec is based on the SILK codec [SILK] 
-        and the CELT codec [CELT]. For more detailed information on how Opus 
-        operates, also refer to [Opus].
-      </t>
-
-      <section title='Modes'>
-
-        <t>
-          Opus supports five different audio bandwidths, 8000, 12000, 16000,
-          24000, and 48000 Hz sampling frequency, for the voice mode and four
-          different audio bandwidths, 8000, 16000, 24000, and 48000 Hz 
-          sampling frequency, for the audio mode.
-        </t>
-
-        <section title='Audio Mode'>
-          <t>
-            For low bit rate applications transmitting mostly speech signals
-            the voice mode of Opus SHOULD be used. The voice mode allows to 
-            encode voice signals at 8000, 12000, 16000, 24000, and 48000 Hz 
-            sampling frequency.
-          </t>
-          <t>
-            A sampling rate of 8000 Hz SHOULD only
-            be used to interface to PSTN networks or on low end devices that do 
-            not support greater than 8000 Hz sampling frequency. A sampling rate 
-            of 12000 Hz SHOULD be used for lower end devices that do not support 
-            greater than 12000 Hz sampling frequency or are under severe network 
-            bandwidth constrains (e.g. wireless devices). A sampling rate of 
-            16000 Hz SHOULD be used for all-IP platforms that do not support 
-            greater than 16000 Hz sampling frequency. Higher sampling rates are
-            recommended for all devices that support those high sampling rates
-            and desire full-bandwidth speech at medium bit rates.
-          </t>
-        </section>
-
-        <section title='Audio Mode'>
-          <t>
-            For applications desiring very low delay speech transmission as well
-            as music transmission in trade off to a higher bit rate, the 
-            audio mode SHOULD be used. This mode supports audio sampling 
-            rates of 8000, 16000, 24000, and 48000 Hz.
-          </t>
-        </section>
-
-      </section>
-
-      <section title='Network Bandwidth'>
-          <t>
-            The network bit rate is adaptive within the
-            range specified in <xref target='bitrate_by_bandwidth'/> for corresponding
-            modes and audio sampling rates. The average target network bit rate can be 
-            defined and modified in real-time while the actual bit rate will be dependent 
-            on the settings of Opus and the input signal and may change over time.
-          </t>
-
-          <texttable anchor='bitrate_by_bandwidth'>
-            <ttcol align='center'>Mode</ttcol>
-            <ttcol align='center'>fs (Hz)</ttcol>
-            <ttcol align='center'>BR (kbps)</ttcol>
-            <c>voice</c>
-            <c>8000</c>
-            <c>6 - 20</c>
-            <c>voice</c>
-            <c>12000</c>
-            <c>7 - 25</c>
-            <c>voice</c>
-            <c>16000</c>
-            <c>8 - 30</c>
-            <c>voice</c>
-            <c>24000</c>
-            <c>18 - 28</c>
-            <c>voice</c>
-            <c>48000</c>
-            <c>24 - 32</c>
-            <c>audio</c>
-            <c>8000</c>
-            <c>20 - 28</c>
-            <c>audio</c>
-            <c>16000</c>
-            <c>24 - 32</c>
-            <c>audio</c>
-            <c>24000</c>
-            <c>28 - 40</c>
-            <c>audio</c>
-            <c>48000</c>
-            <c>32 - 128</c>
-            <postamble>
-              Mode specifies the Opus mode of operation; fs specifies the audio 
-              sampling frequency in Hertz (Hz); BR specifies
-              the network bit rate range in kilobits per second (kbps).
-            </postamble>
-          </texttable>
-
-        <section title='Variable versus Constant Bit Rate'  anchor='variable-vs-constant-bitrate'>
-          <t>
-            The voice mode will always use a variable bit rate at audio sampling rates 
-            of 8000, 12000, and 16000 Hz. The average target
-            bit rate can be adjusted at any point in time. To avoid congestion of the
-            connection the average target bit rate SHOULD be adjusted to the available
-            network bandwidth. If no target bit rate is specified the average bit rate 
-            may go up to the highest bit rate specified in 
-            <xref target='bitrate_by_bandwidth'/>.
-          </t>
-
-          <t>
-            In voice mode at audio sampling rates higher than 16000 Hz, i.e. 24000, 
-            and 48000 Hz, and audio mode Opus can be operated in both variable and 
-            constant bit rate. The target bit rate can be adjusted at any point in time.
-          </t>
-            
-        </section>
-
-        <section title='Discontinuous Transmission (DTX)'>
-
-          <t>
-            The Opus codec may, as described in <xref target='variable-vs-constant-bitrate'/>, 
-            be operated with an adaptive bit rate. In that case, the bit rate 
-            will automatically be reduced for certain input signals like periods 
-            of silence. During continuous transmission the bit rate will be 
-            reduced, when the input signal allows to do so, but the transmission 
-            to the receiver itself will never be interrupted. Therefore, the 
-            received signal will maintain the same high level of quality over the 
-            full duration of a transmission while minimizing the average bit 
-            rate over time.
-          </t>
-
-          <t>
-            In cases where the bit rate of Opus needs to be reduced even
-            further or in cases where only constant bit rate is available, 
-            the Opus encoder may be set to use discontinuous
-            transmission (DTX), where parts of the encoded signal that
-            correspond to periods of silence in the input speech or audio signal
-            are not transmitted to the receiver.
-          </t>
-
-          <t>
-            On the receiving side, the non-transmitted parts will be handled by a
-            frame loss concealment unit in the Opus decoder which generates a
-            comfort noise signal to replace the non transmitted parts of the
-            speech or audio signal.
-          </t>
-
-          <t>
-            The DTX mode of Opus will have a slightly lower speech or audio
-            quality than the continuous mode. Therefore, it is RECOMMENDED to
-            use Opus in the continuous mode unless restraints on network
-            bandwidth are severe. The DTX mode can be engaged for operation
-            in both adaptive or constant bit rate.
-          </t>
-
-        </section>
-        
-        </section>
-
-      <section title='Complexity'>
-
-        <t>
-          Complexity can be scaled to optimize for CPU resources in real-time, mostly in
-          trade-off to network bit rate. Also, different modes of Opus have different complexity.
-        </t>
-        
-      </section>
-
-      <section title="Forward Error Correction (FEC)">
-
-        <t>
-          The voice mode of Opus allows for "in-band" forward error correction (FEC)
-          data to be embedded into the bit stream of Opus. This FEC scheme adds
-          redundant information about the previous packet (n-1) to the current 
-          output packet n. For
-          each frame, the encoder decides whether to use FEC based on (1) an
-          externally-provided estimate of the channel's packet loss rate; (2) an
-          externally-provided estimate of the channel's capacity; (3) the
-          sensitivity of the audio or speech signal to packet loss; (4) whether
-          the receiving decoder has indicated it can take advantage of "in-band"
-          FEC information. The decision to send "in-band" FEC information is
-          entirely controlled by the encoder and therefore no special precautions
-          for the payload or storage format have to be taken.
-        </t>
-
-        <t>
-          On the receiving side, the decoder can take advantage of this
-          additional information when, in case of a packet loss, the next packet
-          is available.  In order to use the FEC data, the jitter buffer needs
-          to provide access to payloads with the FEC data.  The decoder API function
-          has a flag to indicate that a FEC frame rather than a regular frame should
-          be decoded.  If no FEC data is available for the current frame, the decoder
-          will consider the frame lost and invokes the frame loss concealment.
-        </t>
-
-        <t>
-          If the FEC scheme is not implemented on the receiving side, FEC
-          SHOULD NOT be used, as it leads to an inefficient usage of network
-          bandwidth. Decoder support for FEC SHOULD be indicated at the time a
-          session is set up.
-        </t>
-      
-      </section>
-
-      <section title='Stereo Operation'>
-
-        <t>
-          Opus allows for transmission of stereo audio signals. This operation
-          will be signaled in the Opus payload and no special arrangements
-          have to be made in the payload format. Any implementation of the Opus
-          decoder MUST be capable to receive stereo signals.
-        </t>
-        <t>
-          If a decoder can not take advantage of the benefits of a stereo signal 
-          this SHOULD be indicated at the time a session is set up. In that case
-          the sending side SHOULD NOT send stereo signals as it leads to an 
-          inefficient usage of network bandwidth.
-        </t>
-
-      </section>
-
-    </section>
- 
     <section title='Opus RTP Payload Format' anchor='opus-rtp-payload-format'>
       <t>The payload format for Opus consists of the RTP header and Opus payload
       data.</t>
@@ -388,42 +148,16 @@
         packets. Only one of those payloads MUST be provided to the Opus decoder
         for decoding and others MUST be discarded.</t>
 
-        <t>Opus supports 5 different sampling rates which may be adjusted during
-        the duration of a call. The RTP timestamp clock frequency is defined as 
-        the highest supported sampling frequency of Opus, i.e. 48000 Hz, for all 
+        <t>The RTP timestamp clock frequency is defined as 48000 Hz, for all 
         modes and sampling rates of Opus. The unit
         for the timestamp is samples. The RTP timestamp corresponds to the
-        sample time of the first encoded sample in the encoded frame. For sampling
-        rates lower than 48000 Hz the number of samples has to be multiplied with 
-        a multiplier according to <xref target="fs-upsample-factors"/> to determine 
-        the RTP timestamp.</t>
-
-        <texttable anchor='fs-upsample-factors'>
-          <ttcol align='center'>fs (Hz)</ttcol>
-          <ttcol align='center'>Multiplier</ttcol>
-          <c>8000</c>
-          <c>6</c>
-          <c>12000</c>
-          <c>4</c>
-          <c>16000</c>
-          <c>3</c>
-          <c>24000</c>
-          <c>2</c>
-          <c>48000</c>
-          <c>1</c>
-          <postamble>
-            fs specifies the audio sampling frequency in Hertz (Hz); Multiplier is the 
-            value that the number of samples have to be multiplied with to calculate 
-            the RTP timestamp.
-          </postamble>
-        </texttable>
+        sample time of the first encoded sample in the encoded frame.</t>
+
       </section>
 
       <section title='Payload Structure'>
         <t>
-          The Opus encoder can be set to output encoded frames representing 2.5, 5, 10, 20,
-          40, or 60 ms of speech or audio data. Further, an arbitrary number of frames can be
-          combined into a packet. The maximum packet length is limited to the amount of encoded
+          The maximum packet length is limited to the amount of encoded
           data representing 120 ms of speech or audio data. The packetization of encoded data
           is purely done by the Opus encoder and therefore only one packet output from the Opus
           encoder MUST be used as a payload.
@@ -441,146 +175,8 @@
            ]]>
           </artwork>
         </figure>
-
-        <t>
-          <xref target='opus-packetization'/> shows supported frame sizes for different modes
-          and sampling rates of Opus and how the timestamp needs to be incremented for 
-          packetization.
-        </t>
-
-        <texttable anchor='opus-packetization'>
-            <ttcol align='center'>Mode</ttcol>
-            <ttcol align='center'>fs</ttcol>
-            <ttcol align='center'>2.5</ttcol>
-            <ttcol align='center'>5</ttcol>
-            <ttcol align='center'>10</ttcol>
-            <ttcol align='center'>20</ttcol>
-            <ttcol align='center'>40</ttcol>
-            <ttcol align='center'>60</ttcol>
-            <c>ts incr</c>
-            <c>all</c>
-            <c>120</c>
-            <c>240</c>
-            <c>480</c>
-            <c>960</c>
-            <c>1920</c>
-            <c>2880</c>
-            <c>voice</c>
-            <c>8000/12000/16000/24000/48000</c>
-            <c></c>
-            <c></c>
-            <c>x</c>
-            <c>x</c>
-            <c>x</c>
-            <c>x</c>
-            <c>audio</c>
-            <c>8000/16000/24000/48000</c>
-            <c>x</c>
-            <c>x</c>
-            <c>x</c>
-            <c>x</c>
-            <c></c>
-            <c></c>
-            <postamble>
-              Mode specifies the Opus mode of operation; fs specifies the audio sampling 
-              frequency in Hertz (Hz); 2.5, 5, 10, 20, 40, and 60 represent the duration of 
-              encoded speech or audio data in a packet; ts incr specifies the 
-              value the timestamp needs to be incremented for the representing packet size.
-              For multiple frames in a packet these values have to be multiplied with the 
-              respective number of frames.
-            </postamble>
-          </texttable>
-
-      </section>
-
-    </section>
-
-    <section title='Opus Storage Format' anchor='opus-storage-format'>
-      <t>The Opus storage format allows to store Opus encoded data into e.g. a
-      file or an email attachment. The storage format consists of a header and a
-      series of blocks containing encoded speech or audio frames. The storage
-      format closely mimics the real-time payload format and allows to easily 
-      convert packets, e.g. received by a voicemail system, into a storage format 
-      and vice versa and therefore allowing maximum flexibility and low overhead. 
-      Please note that this storage format is not meant to be a robust storage format,
-      nor the most efficient storage format. For a robust storage format that 
-      allows advanced functionality like e.g. seeking,  a more advanced container
-      format should be used.</t>
-
-      <t><xref target='opus-storage-format-figure'/> shows an example of an Opus
-      encoded file. Note that due to the potentially adaptive bit rate the packet length
-      may be variable and no fixed block size can be defined for
-      blocks containing encoded data.</t>
-
-       <figure anchor="opus-storage-format-figure" 
-               title="Example of Opus file storage format showing different block
-                      lengths due to potentially adaptive bit rate of Opus">
-
-           <artwork><![CDATA[
-+------------------+
-| Header           |
-+-----------+------+
-| block 1   |
-+-----------+--+
-| block 2      |
-+--------------+--+
-: ...             :
-+--------------+--+
-| block n         |
-+-----------------+
-           ]]></artwork>
-       </figure>
-
-      <section title='Storage Header Structure'>
-        <t>An Opus storage header contains the following ASCII character string
-        as a magic number:</t>
-
-        <t>"#!opus\n" (hexadecimal: 0x23 0x21 0x6f 0x70 0x75
-        0x73 0x0A)</t>
-
       </section>
 
-      <section title='Storage Block Structure'>
-        <t>Following the storage header, blocks of encoded data are stored in
-        consecutive order in time according to <xref
-        target='opus-storage-format-figure'/>. Each block contains a block
-        header followed by a payload according to <xref
-        target='storage-block-header'/>.</t>
-
-        <t>The block header contains information that, for an RTP-based session,
-        can be derived from the IP and RTP headers: The number
-        of octets contained in the subsequent payload and the RTP timestamp.</t>
-
-        <t>The number of octets in the payload is represented by 16
-        bits and the timestamp is specified by 32 bits. For the first block, the
-        timestamp MAY be a random number. For the following blocks, the
-        timestamp MUST be incremented according to the way timestamps are
-        incremented when Opus payloads are transmitted over RTP.</t>
-
-
-       <figure anchor="storage-block-header"
-               title="Storage block header structure">
-
-           <artwork><![CDATA[
-0                   16                           48
-+-------------------+----------------------------+-----------------
-|    # of octets    |        Timestamp           |  Payload
-+-------------------+----------------------------+-----------------
-           ]]></artwork>
-       </figure>
-
-       <t>The payload of each block in <xref target='opus-storage-format-figure'/> represents 
-       one packet of Opus encoded data the way as originally encoded by the Opus encoder.
-       Information about frame size representing the duration of encoded speech or audio data, 
-       number of encoded frames, stereo information, and DTX is embedded into the payload of
-       Opus and not subject to the storage format. It can be extracted from the payload during
-       decoding of the encoded data.</t>
-
-       <t>During the usage of DTX no blocks are stored when the channel is
-       inactive. Timestamps MUST be used to reassemble the decoded signal in a
-       time-aligned way.</t>
-
-      </section>
     </section>
 
     <section title='Congestion Control'>
@@ -588,17 +184,9 @@
       <t>The adaptive nature of the Opus codec allows for an efficient
       congestion control.</t>
 
-      <t>
-        The voice mode of Opus at audio sampling rates of 8000, 12000, and 16000 
-        always runs with a variable bit rate. The average bit
-        rate in that mode is dependent on the input signal and will especially
-        decrease during silent periods. The voice mode at audio sampling rates of 
-        24000 and 48000 Hz and the audio mode may run 
-        at a variable or constant bit rate. In either way, the target bit rate 
-        of Opus can be adjusted at any point in time and thus allowing for an 
-        efficient congestion control.</t>
-      
-      <t>Furthermore, the amount of encoded speech or audio data encoded in a 
+      <t>For any packet duration, an Opus encoder may produce a range of output
+         sizes.  When congestion is detected, the encoder may reduce the output
+         frame size. Furthermore, the duration of audio data encoded in a 
       single packet can be used for congestion control since the transmission 
       rate is inversely proportional to these frame sizes. A lower packet 
       transmission rate reduces the amount of header overhead but at the same 
@@ -624,19 +212,18 @@
           <t><list style="hanging">
             <t hangText="rate:"> RTP timestamp clock rate is incremented with 
             48000 Hz clock rate for all modes of Opus and all sampling 
-            frequencies. For audio sampling rates other than 48000 Hz the rate
-            has to be adjusted to 48000 Hz according to <xref target="fs-upsample-factors"/>.
+            frequencies.
           </t>
           </list></t>
 
           <t>Optional parameters:</t>
           
           <t><list style="hanging">
-            <t hangText="maxcodedaudiobandwidth:">
-              the decoder's maximum sampling frequency specified in Hertz (Hz) that the 
+            <t hangText="decodesamplerate:">
+              The decoder's maximum sampling rate specified in Hertz (Hz) that the 
               application can take advantage of. The decoder MUST be capable to receive 
-              any allowed sampling frequency but due to hardware limitations only signals 
-              up to the specified sampling frequency can be processed. Sending signals 
+              any allowed sampling frequency but the signal will be downsampled
+              to this rate during the decode process. Sending signals 
               with higher sampling frequency may result in higher than necessary network 
               bandwidth and encoding complexity. Possible values are 8000, 12000, 16000, 
               24000, 48000.
@@ -647,14 +234,10 @@
             milliseconds rounded up to the next full integer value represented 
             by the media in a packet that can be
             encapsulated in a received packet according to Section 6 of
-            <xref target="RFC4566"/>. Possible values are 3, 5, 10, 20, 40, 
-            and 60 or an arbitrary multiple of Opus frame sizes rounded up to 
-            the next full integer value up to a maximum value of 120 as 
-            defined in <xref target='opus-rtp-payload-format'/> and <xref
-            target='opus-storage-format'/> of this document. If no value is
+            <xref target="RFC4566"/>. Possible values are 3 to 120. If no value is
               specified, 120 is assumed as default. This value is a recommendation 
-              by the decoding side to ensure the best
-              performance for the decoder. The decoder MUST be
+              by the decoding side indicating that longer packets will harm
+              perceived performance. The decoder MUST be
               capable to accept any allowed packet sizes to
               ensure maximum compatibility.
               <vspace blankLines='1'/></t>
@@ -663,16 +246,11 @@
             milliseconds rounded up to the next full integer value represented 
             by the media in a packet according to
             Section 6 of <xref target="RFC4566"/>. Possible values are
-            3, 5, 10, 20, 40, or 60 or an arbitrary multiple of Opus frame sizes 
-            rounded up to the next full integer value up to a maximum 
-            value of 120 as defined in <xref
-            target='opus-rtp-payload-format'/> and <xref
-            target='opus-storage-format'/> of this document. If no value is
+            3 to 120. If no value is
               specified, 20 is assumed as default. If ptime is greater than
               maxptime, ptime MUST be ignored. This parameter MAY be changed
               during a session. This value is a recommendation by the decoding 
-              side to ensure the best
-              performance for the decoder. The decoder MUST be
+              side that this duration provides good performance. The decoder MUST be
               capable to accept any allowed packet sizes to
               ensure maximum compatibility.
               <vspace blankLines='1'/></t>
@@ -681,14 +259,11 @@
             milliseconds rounded up to the next full integer value represented 
             by the media in a packet that SHOULD
             be encapsulated in a received packet according to Section 6 of <xref
-            target="RFC4566"/>. Possible values are 3, 5, 10, 20, 40, and 60 
-            or an arbitrary multiple of Opus frame sizes rounded up to the next 
-            full integer value up to a maximum value of 120
-            as defined in <xref target='opus-rtp-payload-format'/> and <xref
-            target='opus-storage-format'/> of this document. If no value is
+            target="RFC4566"/>. Possible values are 3 to 120.
+              If no value is
               specified, 3 is assumed as default. This value is a recommendation 
-              by the decoding side to ensure the best
-              performance for the decoder. The decoder MUST be
+              by the decoding side that shorter durations will not improve
+              performance. The decoder MUST be
               capable to accept any allowed packet sizes to
               ensure maximum compatibility.
               <vspace blankLines='1'/></t>
@@ -699,9 +274,8 @@
             characteristics of the media in a packet. Note that the maximum
             average bit rate MAY be modified dynamically during a session. Any
             positive integer is allowed but values outside the range between 
-            6000 and 510000 SHOULD be ignored. If no value is specified, the 
-            maximum value specified in <xref target='bitrate_by_bandwidth'/>
-            for the corresponding mode of Opus and corresponding clock rate 
+            6000 and 510000 SHOULD be ignored. If no value is specified, then 
+            510000
             will be the default.<vspace blankLines='1'/></t>
 
             <t hangText="stereo:">
@@ -757,13 +331,6 @@
           </list></t>
 
           <t>Additional information:</t>
-          <t><list style="hanging">
-            <t>For storage transfer methods the following applies:<vspace blankLines='1'/></t>
-            <t>Magic number:"#!opus\n" (hexadecimal: 0x23 0x21 0x6f 0x70 0x75 0x73 0x0A)<vspace blankLines='1'/></t>
-            <t>File extension(s): ops, OPS<vspace blankLines='1'/></t>
-            <t>Macintosh file type code(s): "opus"</t>
-          </list></t>
-          
           <t>Person & email address to contact for further information:</t>
           <t><list style="hanging">
             <t>SILK Support [email-masked]</t>
@@ -777,8 +344,7 @@
           <t><list style="hanging">
             <t>For transfer over RTP, the RTP payload format (<xref
             target='opus-rtp-payload-format'/> of this document) SHALL be
-            used. For storage usage, the storage format (<xref target='opus-storage-format'/>
-            of this document) SHALL be used.</t>
+            used.</t>
           </list></t>
           
           <t>Author:</t>
@@ -819,7 +385,7 @@
 
         <t>Below are some examples of SDP session descriptions for Opus:</t>
 
-        <t>Example 1: Standard session with 48000 Hz clock rate</t>
+        <t>Example 1: Standard session with 48000 Hz decoder sample rate</t>
           <figure>  
             <artwork>
               <![CDATA[
@@ -830,7 +396,7 @@
           </figure>

-        <t>Example 2: 16000 Hz clock rate, maximum packet size of 40 ms,
+        <t>Example 2: 16000 Hz decoder sample rate, maximum packet size of 40 ms,
         recommended packet size of 40 ms, maximum average bit rate of 20000 bps,
         stereo signals are preferred, FEC is allowed, DTX is not allowed</t>
 
@@ -839,7 +405,7 @@
             <![CDATA[
     m=audio 54312 RTP/AVP 101
     a=rtpmap:101 opus/48000
-    a=fmtp:101 maxcodedaudiobandwidth=16000; maxaveragebitrate=20000; 
+    a=fmtp:101 decodesamplerate=16000; maxaveragebitrate=20000; 
     stereo=1; useinbandfec=1; usedtx=0
     a=ptime:40
     a=maxptime:40
@@ -855,8 +421,8 @@
 
           <t><list style="symbols">
 
-            <t>Opus supports several clock rates. For signaling purposes only
-            the highest, i.e. 48000, is used. The actual clock rate of the 
+            <t>For signaling purposes, Opus always has a clock rate of 48000 Hz.
+            The internal sample rate of the 
             corresponding media is signaled inside the payload and is not 
             subject to this payload format description. The decoder MUST be
             capable to decode every received clock rate. An example
@@ -889,12 +455,12 @@
             </t>
 
             <t>
-              The parameter "maxcodedaudiobandwidth" is a unidirectional receive-only
+              The parameter "decodesamplerate" is a unidirectional receive-only
               parameter that reflects limitations of the local receiver. The sender
               of the other side SHOULD NOT send with a sampling rate higher than
-              "maxcodedaudiobandwidth" as it represents an inefficient use of network bandwidth
+              "decodesamplerate" as it represents an inefficient use of network bandwidth
               resources and CPU cycles on the encoding side. The parameter 
-              "maxcodedaudiobandwidth" typically will not compromise interoperability; however, 
+              "decodesamplerate" typically will not compromise interoperability; however, 
               dependent on the set value of the parameter the performance of the 
               application may suffer and should be set with care.
             </t>
@@ -908,8 +474,8 @@
             the parameter the performance of the application may suffer and should
             be set with care.</t>
             
-            <t>If the parameter "maxaveragebitrate" is below the range specified
-            in <xref target='bitrate_by_bandwidth'/> the session MUST be rejected.</t>
+            <t>If the parameter "maxaveragebitrate" is below 6000
+            the session MUST be rejected.</t>
 
             <t>
               The parameter "stereo" is a unidirectional receive-only
@@ -941,7 +507,7 @@
 
         <t><list style="symbols">
 
-          <t>The values for "maxptime", "ptime", "minptime", "maxcodedaudiobandwidth", and 
+          <t>The values for "maxptime", "ptime", "minptime", "decodesamplerate", and 
           "maxaveragebitrate" should be selected carefully to ensure that a 
           reasonable performance can be achieved for the participants of a session.</t>

---

From: "Roni Even" <[email-masked]>
Date: Wed, 6 Jul 2011 00:15:21 +0300

Hi,
I find it difficult to address all you comments being in XML patch format.
In general the payload specification should provide enough information that
will enable the reader to create or parse the RTP payload without
understanding the codec payload based on the payload specification. As such
it is useful to provide information that will explain the general concepts
of the codec and the optional parameters. So I find sections 1 and 3 useful
and they should be in the document
Roni Even

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Benjamin M. Schwartz
> Sent: Tuesday, July 05, 2011 7:39 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [codec] RTP Payload Format and File Storage Format for
> Opus Speech and Audio Codec - draft-spittka-payload-rtp-opus-00
> 
> Thank you for doing this work.  I think your proposed RTP and SDP
> formats are very nice and a good step for us.
> 
> I have a few comments.
> 
> 1.  I think the description of Opus should be removed.  There's no need
> to duplicate the contents of the Opus RFC.
> 
> 2.  I think the payload specification should make (almost) no mention
> of Opus frames, modes, etc.  These aspects of Opus are entirely
> internal to the payload, which is a "black box".  As far as the RTP
> specification is concerned, Opus consists of (almost-)arbitrary-
> duration "packets" and that's it.
> 
> 3.  To avoid confusion, I think we should be very careful and clear
> about the distinction between audio bandwidth and samplerate, as their
> I have implemented these ideas, along with a few minor changes, as a
> patch to the draft .xml, attached.  Please feel free to discard this
> patch; I mean it only as a way to communicate my suggestions more
> clearly.
> 
> --Ben

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 6 Jul 2011 10:43:45 -0600

I would prefer to see the file format separated into a separate draft for two reason. One, it allows an implementation to say they do RFC XXXX and have that be clear if the mean the RTP payload or the file format. Two, I suspect the file format conversation might take longer to resolve. 

Cullen

PS - this discussion and draft needs to happen in the payloads WG, feel free to CC the [email-masked] mailing list.

On Jul 4, 2011, at 6:06 PM, Julian Spittka wrote:

> _______________________________________________
> codec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/codec

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 7 Jul 2011 12:53:30 +0200

> PS - this discussion and draft needs to happen in the payloads WG, feel free to CC the [email-masked] mailing list.

Cullen, I assume you mean CODEC WG rather than the RTCWEB WG.

Cheer

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 7 Jul 2011 08:29:17 -0600

> Cullen, I assume you mean CODEC WG rather than the RTCWEB WG.

yes - thank you. too much RTCweb on my mind and was just typing while on a call. Always bad to do

---

From: "Benjamin M. Schwartz" <[email-masked]>
Date: Sat, 09 Jul 2011 00:30:16 -0400

> and they should be in the document

Despite having removed Section 3 in my suggested edit, I agree.  The
payload document should present relevant information about Opus, such as
the available frame sizes and the significance of the low-bitrate
redundancy feature.  However, it should not discuss non-normative
behaviors of the reference coder, or internal details such as subsampling
and coding modes.

The most important thing to understand about Opus coding modes is that the
encoder can, and does, dynamically switch among them on a packet-by-packet
basis in order to optimize quality.  The draft seems to have been written
under the assumption that a stream, once started in a particular mode, may
not be able to switch to other modes.   This is no longer true, as
explained in the latest Opus draft.

--Ben