Subject: [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

From: Adam Fineberg <[email-masked]>
Date: Wed, 17 Jul 2013 17:09:46 -0700

Hi,

I'm working on WebRTC services and have found that while developing 
services that forward VP8 video streams if we want to take advantage of 
the VP8 temporal scaling we must get the temporal layer information from 
the RTP header which requires us to decrypt the SRTP packets. This is 
undesirable both because the middle-box needs to have access to the keys 
as well as the because of the added overhead of the decrypt/encrypt 
cycle. This draft proposes an RTP header extension that will allow us to 
use the VP8 temporal layer information included in the header extension 
and therefore do forwarding without SRTP decryption. Comments welcome.

Regards,
Adam Fineberg
fineberg at vline.com

-------- Original Message --------
Subject: 	New Version Notification for 
draft-fineberg-avtext-temporal-layer-ext-00.txt
Date: 	Tue, 09 Jul 2013 10:02:05 -0700
From: 	internet-drafts at ietf.org
To: 	Adam Fineberg<fineberg at vline.com>

A new version of I-D, draft-fineberg-avtext-temporal-layer-ext-00.txt
has been successfully submitted by Adam Fineberg and posted to the
IETF repository.

Filename:	 draft-fineberg-avtext-temporal-layer-ext
Revision:	 00
Title:		 A Real-Time Transport Protocol (RTP) Header Extension for VP8 Temporal Layer Information
Creation date:	 2013-07-08
Group:		 Individual Submission
Number of pages: 6
URL:http://www.ietf.org/internet-drafts/draft-fineberg-avtext-temporal-layer-ext-00.txt
Status:http://datatracker.ietf.org/doc/draft-fineberg-avtext-temporal-layer-ext
Htmlized:http://tools.ietf.org/html/draft-fineberg-avtext-temporal-layer-ext-00

Abstract:
    This document defines a mechanism by which packets of Real-Time
    Tranport Protocol (RTP) video streams encoded with the VP8 codec can
    indicate, in an RTP header extension, the temporal layer information
    about the frame encoded in the RTP packet.  This information can be
    used in a middlebox performing bandwidth management of streams
    without requiring it to decrypt the streams.

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 17 Jul 2013 20:26:58 -0700

Since the need is not codec specific (e.g. it arises with any codec supporting temporal, spatial and quality scalability), why 
 a VP8-specific RTP extension? 
 
Date: Wed, 17 Jul 2013 17:09:46 -0700
From: [email-masked]
To: [email-masked]
Subject: [rtcweb] Fwd: New Version Notification for	draft-fineberg-avtext-temporal-layer-ext-00.txt

  
  
    Hi,
    
    I'm working on
      WebRTC services and have found that while developing services that
      forward VP8 video streams if we want to take advantage of the VP8
      temporal scaling we must get the temporal layer information from
      the RTP header which requires us to decrypt the SRTP packets. This
      is undesirable both because the middle-box needs to have access to
      the keys as well as the because of the added overhead of the
      decrypt/encrypt cycle. This draft proposes an RTP header extension
      that will allow us to use the VP8 temporal layer information
      included in the header extension and therefore do forwarding
      without SRTP decryption. Comments welcome.
    
    Regards,
    Adam Fineberg
    fineberg at vline.com

      -------- Original Message --------

            Subject:
            New Version Notification for
              draft-fineberg-avtext-temporal-layer-ext-00.txt

            Date:
            Tue, 09 Jul 2013 10:02:05 -0700

            From:
            internet-drafts

                at ietf.org

            To:
            Adam Fineberg <fineberg at
                vline.com>

      

Filename:	 draft-fineberg-avtext-temporal-layer-ext
Revision:	 00
Title:		 A Real-Time Transport Protocol (RTP) Header Extension for VP8 Temporal Layer Information
Creation date:	 2013-07-08
Group:		 Individual Submission
Number of pages: 6
URL:             http://www.ietf.org/internet-drafts/draft-fineberg-avtext-temporal-layer-ext-00.txt
Status:          http://datatracker.ietf.org/doc/draft-fineberg-avtext-temporal-layer-ext
Htmlized:        http://tools.ietf.org/html/draft-fineberg-avtext-temporal-layer-ext-00

Abstract:

_______________________________________________
rtcweb mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 18 Jul 2013 10:16:50 +0200

On 07/18/2013 02:09 AM, Adam Fineberg wrote:
> services that forward VP8 video streams if we want to take advantage 
> of the VP8 temporal scaling we must get the temporal layer information 
> from the RTP header which requires us to decrypt the SRTP packets. 
> This is undesirable both because the middle-box needs to have access 
> to the keys as well as the because of the added overhead of the 
> that will allow us to use the VP8 temporal layer information included 
> in the header extension and therefore do forwarding without SRTP 
> decryption. Comments welcome.

Just checking - you named your draft with the -avtext- convention; this 
means that you want comments to go to avtext, yes?

---

From: Adam Fineberg <[email-masked]>
Date: Thu, 18 Jul 2013 08:43:07 -0700

Yes, I apologize for not adding avtext as a cc.

Regards,
Adam

On 7/18/13 1:16 AM, Harald Alvestrand wrote:
>> of the VP8 temporal scaling we must get the temporal layer 
>> information from the RTP header which requires us to decrypt the SRTP 
>> packets. This is undesirable both because the middle-box needs to 
>> have access to the keys as well as the because of the added overhead 
>> of the decrypt/encrypt cycle. This draft proposes an RTP header 
>> extension that will allow us to use the VP8 temporal layer 
>> information included in the header extension and therefore do 
>> forwarding without SRTP decryption. Comments welcome.
>
> Just checking - you named your draft with the -avtext- convention; 
> this means that you want comments to go to avtext, yes?
>
>
>>
>> Regards,
>> Adam Fineberg
>> fineberg at vline.com
>>

-- 
Regards,
Adam

---

From: Adam Fineberg <[email-masked]>
Date: Thu, 18 Jul 2013 08:45:38 -0700

Bernard,

Good question.  I'm not familiar enough with the parameter requirements 
of all other scalable codecs to be able to generalize. If you'd like to 
help specify them, I'd be fine revising the draft to generalize.

Regards,
Adam

On 7/17/13 8:26 PM, Bernard Aboba wrote:
> Since the need is not codec specific (e.g. it arises with any codec 
> supporting temporal, spatial and quality scalability), why
> ------------------------------------------------------------------------
> Subject: [rtcweb] Fwd: New Version Notification for 
> fineberg at vline.com <mailto:fineberg%20at%20vline.com>
>
> <mailto:internet-drafts%20at%20ietf.org>
> <mailto:fineberg%20at%20vline.com>
>
>
>
> _______________________________________________ rtcweb mailing list 
> [email-masked] https://www.ietf.org/mailman/listinfo/rtcweb

-- 
Regards,
Adam

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 18 Jul 2013 19:40:07 -0700

I think it may be possible to generalize this.  For example, for H.264/SVC which can support temporal, spatial and quality scalability, you would need the quality_id and dependency_id in addition to the temporal_id (what you call the temporal layer index).    

Date: Thu, 18 Jul 2013 08:45:38 -0700
From: [email-masked]
To: [email-masked]
CC: [email-masked]; [email-masked]
Subject: Re: [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

  
    Bernard,

    Good question.  I'm not familiar enough with the parameter
    requirements of all other scalable codecs to be able to generalize. 
    If you'd like to help specify them, I'd be fine revising the draft
    to generalize.

    Regards,

    Adam

    On 7/17/13 8:26 PM, Bernard Aboba
      wrote:

      
      Since the need is not codec specific (e.g. it
        arises with any codec supporting temporal, spatial and quality
        scalability), why 

        

          From: [email-masked]

          To: [email-masked]

          Hi,
          
          I'm working on WebRTC
            services and have found that while developing services that
            forward VP8 video streams if we want to take advantage of
            the VP8 temporal scaling we must get the temporal layer
            information from the RTP header which requires us to decrypt
            the SRTP packets. This is undesirable both because the
            middle-box needs to have access to the keys as well as the
            because of the added overhead of the decrypt/encrypt cycle.
            This draft proposes an RTP header extension that will allow
            us to use the VP8 temporal layer information included in the
            header extension and therefore do forwarding without SRTP
            decryption. Comments welcome.
          
          Regards,
          Adam Fineberg
          fineberg
              at vline.com

            -------- Original Message --------

                  Subject:

                  Date:

                  From:
                  internet-drafts at ietf.org

                  To:
                  Adam Fineberg <fineberg at vline.com>

            

Abstract:

    

    -- 
Regards,
Adam

---

From: Adam Fineberg <[email-masked]>
Date: Thu, 18 Jul 2013 20:34:16 -0700

Bernard,

Are there other codecs you are thinking should be supported?  If it's 
generalized I would think we want to be able to cover all known scalable 
codecs. I'll look into the H264/SVC fields to see how to encode them in 
a generalized header.

Regards,
Adam

On 7/18/13 7:40 PM, Bernard Aboba wrote:
> I think it may be possible to generalize this.  For example, for 
> H.264/SVC which can support temporal, spatial and quality scalability, 
> you would need the quality_id and dependency_id in addition to the 
> temporal_id (what you call the temporal layer index).
>
> Subject: Re: [rtcweb] Fwd: New Version Notification for 
> If you'd like to help specify them, I'd be fine revising the draft to 
> generalize.
>
> Regards,
> Adam
>
>     Since the need is not codec specific (e.g. it arises with any
>     codec supporting temporal, spatial and quality scalability), why
>     From: [email-masked] <mailto:[email-masked]>
>     To: [email-masked] <mailto:[email-masked]>
>     I'm working on WebRTC services and have found that while
>     developing services that forward VP8 video streams if we want to
>     take advantage of the VP8 temporal scaling we must get the
>     temporal layer information from the RTP header which requires us
>     to decrypt the SRTP packets. This is undesirable both because the
>     middle-box needs to have access to the keys as well as the because
>     of the added overhead of the decrypt/encrypt cycle. This draft
>     proposes an RTP header extension that will allow us to use the VP8
>     temporal layer information included in the header extension and
>     therefore do forwarding without SRTP decryption. Comments welcome.
>
>     Regards,
>     Adam Fineberg
>     _______________________________________________ rtcweb mailing
>     list [email-masked] <mailto:[email-masked]>

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 18 Jul 2013 22:56:12 -0700

If we can support VP8/9 as well as H.264/5 SVC
that would be a start. It seems doable to me.

On Jul 18, 2013, at 8:34 PM, "Adam Fineberg" <[email-masked]> wrote:

> Bernard,
> 
> Are there other codecs you are thinking should be supported?  If it's generalized I would think we want to be able to cover all known scalable codecs. I'll look into the H264/SVC fields to see how to encode them in a generalized header.
> 
> Regards,
> Adam
> 
>> Good question.  I'm not familiar enough with the parameter requirements of all other scalable codecs to be able to generalize.  If you'd like to help specify them, I'd be fine revising the draft to generalize.
>> 
>> Regards,
>> Adam
>> 
>> I'm working on WebRTC services and have found that while developing services that forward VP8 video streams if we want to take advantage of the VP8 temporal scaling we must get the temporal layer information from the RTP header which requires us to decrypt the SRTP packets. This is undesirable both because the middle-box needs to have access to the keys as well as the because of the added overhead of the decrypt/encrypt cycle. This draft proposes an RTP header extension that will allow us to use the VP8 temporal layer information included in the header extension and therefore do forwarding without SRTP decryption. Comments welcome.
>> 
>> Regards,
>> Adam Fineberg
>> fineberg at vline.com
>> 
>> Subject:	New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt
>> To:	Adam Fineberg <fineberg at vline.com>
>> 
>> _______________________________________________ rtcweb mailing list [email-masked] https://www.ietf.org/mailman/listinfo/rtcweb
>> 
>> -- 
>> Regards,
>> Adam
>

---

From: Justin Uberti <[email-masked]>
Date: Fri, 19 Jul 2013 09:32:23 -0400

Agree those are the right codecs to design for. Since in each case there
are fairly low limits on the number of supported layers (i.e. 3 spatial
layers for SVC), I think it should be possible to pack the temporal,
spatial, quality layer ids into 16 bits.

On Fri, Jul 19, 2013 at 1:56 AM, Bernard Aboba <[email-masked]>wrote:

> codecs. I'll look into the H264/SVC fields to see how to encode them in a
> generalized header.
>
> Regards,
> Adam
>
> I think it may be possible to generalize this.  For example, for H.264/SVC
> which can support temporal, spatial and quality scalability, you would need
> the quality_id and dependency_id in addition to the temporal_id (what you
> call the temporal layer index).
>
>  ------------------------------
> Good question.  I'm not familiar enough with the parameter requirements of
> all other scalable codecs to be able to generalize.  If you'd like to help
> specify them, I'd be fine revising the draft to generalize.
>
> Regards,
> Adam
>
>  ------------------------------
> services that forward VP8 video streams if we want to take advantage of the
> VP8 temporal scaling we must get the temporal layer information from the
> RTP header which requires us to decrypt the SRTP packets. This is
> undesirable both because the middle-box needs to have access to the keys as
> well as the because of the added overhead of the decrypt/encrypt cycle.
> This draft proposes an RTP header extension that will allow us to use the
> VP8 temporal layer information included in the header extension and
> -------- Original Message --------  Subject: New Version Notification for
> draft-fineberg-avtext-temporal-layer-ext-00.txt  Date: Tue, 09 Jul 2013
> 10:02:05 -0700  From: internet-drafts at ietf.org  To: Adam Fineberg <fineberg
> at vline.com> <fineberg%20at%20vline.com>
>

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 19 Jul 2013 06:42:38 -0700

That's the set of codecs that seems interesting to most people right now

On Jul 18, 2013, at 10:56 PM, Bernard Aboba <[email-masked]> wrote:

> avtext mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avtext

---

From: Stephan Wenger <[email-masked]>
Date: Fri, 19 Jul 2013 15:33:02 +0000

Hi,
I also believe that 16 bits should be enough.  For H.264 and VP8 that has already been demonstrated.  For H.265, some initial thoughts below.  Apologies for the word-count.

The scalable version of H265 (called SHVC) is currently under development.  The current working draft can be found here: http://phenix.int-evry.fr/jct/doc_end_user/documents/13_Incheon/wg11/JCTVC-M1008-v3.zip.  Therein, the options for defining layering structures are considerably more complex.  To start, we have 3 bits for the temporal ID in the NAL unit header of the H.265 version 1 (HEVC) base specification (temporal scalability is already nicely supported in version 1).  Just like in SVC.  In the scalable extension, the NAL unit header contains a six bit field that points into a data structure known as "Video Parameter Set" (VPS).  Inside the VPS, those six bits are mapped to to a position in a directed graph (specified through "dimension_id[][]"), which tells you about the reference relationship of the layer in question and its parent layer.  One can recursively follow the graph to determine what used to be called dependency_id, quality_id, view_id, and whatnot.  The six bit pointer field can (or: is to be when possible) organized by the encoder such that it is prudent for a middle box to throw away NAL units (belonging to layers) with higher values of the six bit field first, before throwing away NAL units with lower values.  Relying on this feature, 3+6 bits == 9 bits should be fine for the header extension.

That said, the ordering by the encoder is just a recommendation, and there may well be cases where different pruning strategies may be advisable.  For example, a layering structure could be constructed that expands into two branches, one using 2D scalable tools only, the other including view_id for multi view coding.  By looking at the six bit field alone, a middle box will not be able to meaningfully remove NAL units belonging to one of the branches completely while pruning the other branch.  In order to meaningfully deal with that scenario, there would be two options: one to represent the dimension_id[][] (and associated control info) in the header extension, or require the middle box to have access to the VPS and be able to interpret its content.  The further could take considerably more than 16 bits and we would be talking about a variable length data structure.  The latter requires the middle box to have state and a mechanism to intercept the VPS (through signaling—as the encrypted in-band VPS would not be useful under the assumption that the middle box does not have the key to the media—which is the motivation of the draft in the first place).  I personally don't mind at all the second mechanism, as I'm a big fan of out-of-band parameter set transmission and any middle box must be in the signaling path anyway to meaningfully manipulate RTP.  I do not like the first option due to its variable, and possibly substantial, overhead.

Stephan

From: Justin Uberti <[email-masked]<mailto:[email-masked]>>
Date: Friday, 19 July, 2013 06:32
To: Bernard Aboba <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

Agree those are the right codecs to design for. Since in each case there are fairly low limits on the number of supported layers (i.e. 3 spatial layers for SVC), I think it should be possible to pack the temporal, spatial, quality layer ids into 16 bits.

On Fri, Jul 19, 2013 at 1:56 AM, Bernard Aboba <[email-masked]<mailto:[email-masked]>> wrote:
If we can support VP8/9 as well as H.264/5 SVC
that would be a start. It seems doable to me.

On Jul 18, 2013, at 8:34 PM, "Adam Fineberg" <[email-masked]<mailto:[email-masked]>> wrote:

Bernard,

Regards,
Adam

________________________________
Date: Thu, 18 Jul 2013 08:45:38 -0700
From: [email-masked]<mailto:[email-masked]>
To: [email-masked]<mailto:[email-masked]>
CC: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

Bernard,

Regards,
Adam

________________________________
Date: Wed, 17 Jul 2013 17:09:46 -0700
From: [email-masked]<mailto:[email-masked]>
To: [email-masked]<mailto:[email-masked]>
Subject: [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

Hi,

Regards,
Adam Fineberg
fineberg at vline.com<mailto:fineberg%20at%20vline.com>

-------- Original Message --------
Subject:        New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt
Date:   Tue, 09 Jul 2013 10:02:05 -0700
From:   internet-drafts at ietf.org<mailto:internet-drafts%20at%20ietf.org>
To:     Adam Fineberg <fineberg at vline.com><mailto:fineberg%20at%20vline.com>

Abstract:

_______________________________________________ rtcweb mailing list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/rtcweb

--
Regards,
Adam

_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Adam Fineberg <[email-masked]>
Date: Fri, 19 Jul 2013 15:05:39 -0700

Bernard,

I'll take a look at the other codecs to see what fields are needed and 
how to encode them in the header.  Any help is appreciated as I'm not 
that familiar with H.264 and not at all familiar with H.265.

Regards,
Adam

On 7/18/13 10:56 PM, Bernard Aboba wrote:
>> generalized I would think we want to be able to cover all known 
>> scalable codecs. I'll look into the H264/SVC fields to see how to 
>> encode them in a generalized header.
>>
>> Regards,
>> Adam
>>
>>> H.264/SVC which can support temporal, spatial and quality 
>>> scalability, you would need the quality_id and dependency_id in 
>>> addition to the temporal_id (what you call the temporal layer index).
>>>
>>>     to decrypt the SRTP packets. This is undesirable both because
>>>     the middle-box needs to have access to the keys as well as the
>>>     because of the added overhead of the decrypt/encrypt cycle. This
>>>     draft proposes an RTP header extension that will allow us to use
>>>     the VP8 temporal layer information included in the header
>>>     extension and therefore do forwarding without SRTP decryption.
>>>     Comments welcome.
>>>
>>>     Regards,
>>>     Adam Fineberg

-- 
Regards,
Adam

---

From: Adam Fineberg <[email-masked]>
Date: Fri, 19 Jul 2013 15:12:49 -0700

Stephan,

Thanks for the info and the reference.  I'm not sure I follow as I'm not 
at all familiar with H.265.  I'll review the reference and see what I 
can figure.  It seems though to me that you are suggesting that except 
in the simple case, that the data for H.265 would not be well suited to 
a header extension, am I understanding you correctly?  There is no 
reason the middlebox couldn't get out of band signaling of the VPS as 
you mention but that would not be within the scope of this header extension.

Any insights are appreciated.

Regards,
Adam

On 7/19/13 8:33 AM, Stephan Wenger wrote:
> Hi,
> I also believe that 16 bits should be enough.  For H.264 and VP8 that 
> has already been demonstrated.  For H.265, some initial thoughts 
> below.  Apologies for the word-count.
>
> The scalable version of H265 (called SHVC) is currently under 
> development.  The current working draft can be found here: 
> http://phenix.int-evry.fr/jct/doc_end_user/documents/13_Incheon/wg11/JCTVC-M1008-v3.zip. 
>  Therein, the options for defining layering structures are 
> considerably more complex.  To start, we have 3 bits for the temporal 
> ID in the NAL unit header of the H.265 version 1 (HEVC) base 
> specification (temporal scalability is already nicely supported in 
> version 1).  Just like in SVC.  In the scalable extension, the NAL 
> unit header contains a six bit field that points into a data structure 
> known as "Video Parameter Set" (VPS).  Inside the VPS, those six bits 
> are mapped to to a position in a directed graph (specified through 
> "dimension_id[][]"), which tells you about the reference relationship 
> of the layer in question and its parent layer.  One can recursively 
> follow the graph to determine what used to be called dependency_id, 
> quality_id, view_id, and whatnot.  The six bit pointer field can (or: 
> is to be when possible) organized by the encoder such that it is 
> prudent for a middle box to throw away NAL units (belonging to layers) 
> with higher values of the six bit field first, before throwing away 
> NAL units with lower values.  Relying on this feature, 3+6 bits == 9 
> bits should be fine for the header extension.
>
> That said, the ordering by the encoder is just a recommendation, and 
> there may well be cases where different pruning strategies may be 
> advisable.  For example, a layering structure could be constructed 
> that expands into two branches, one using 2D scalable tools only, the 
> other including view_id for multi view coding.  By looking at the six 
> bit field alone, a middle box will not be able to meaningfully remove 
> NAL units belonging to one of the branches completely while pruning 
> the other branch.  In order to meaningfully deal with that scenario, 
> there would be two options: one to represent the dimension_id[][] (and 
> associated control info) in the header extension, or require the 
> middle box to have access to the VPS and be able to interpret its 
> content.  The further could take considerably more than 16 bits and we 
> would be talking about a variable length data structure.  The latter 
> requires the middle box to have state and a mechanism to intercept the 
> VPS (through signaling---as the encrypted in-band VPS would not be 
> useful under the assumption that the middle box does not have the key 
> to the media---which is the motivation of the draft in the first 
> place).  I personally don't mind at all the second mechanism, as I'm a 
> big fan of out-of-band parameter set transmission and any middle box 
> must be in the signaling path anyway to meaningfully manipulate RTP. 
>  I do not like the first option due to its variable, and possibly 
> substantial, overhead.
>
> Stephan
>
>
> From: Justin Uberti <[email-masked] <mailto:[email-masked]>>
> Cc: "[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>, "[email-masked] <mailto:[email-masked]>" 
> <[email-masked] <mailto:[email-masked]>>
> Agree those are the right codecs to design for. Since in each case 
> there are fairly low limits on the number of supported layers (i.e. 3 
> spatial layers for SVC), I think it should be possible to pack the 
> temporal, spatial, quality layer ids into 16 bits.
>
>
> On Fri, Jul 19, 2013 at 1:56 AM, Bernard Aboba 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>>     Are there other codecs you are thinking should be supported?  If
>>     it's generalized I would think we want to be able to cover all
>>     known scalable codecs. I'll look into the H264/SVC fields to see
>>     how to encode them in a generalized header.
>>
>>     Regards,
>>     Adam
>>
>>>     To: [email-masked] <mailto:[email-masked]>
>>>     CC: [email-masked] <mailto:[email-masked]>; [email-masked]
>>>     <mailto:[email-masked]>
>>>     requirements of all other scalable codecs to be able to
>>>     generalize.  If you'd like to help specify them, I'd be fine
>>>     revising the draft to generalize.
>>>
>>>     Regards,
>>>     Adam
>>>
>>>         Since the need is not codec specific (e.g. it arises with
>>>         any codec supporting temporal, spatial and quality
>>>         scalability), why
>>>         developing services that forward VP8 video streams if we
>>>         want to take advantage of the VP8 temporal scaling we must
>>>         get the temporal layer information from the RTP header which
>>>         requires us to decrypt the SRTP packets. This is undesirable
>>>         both because the middle-box needs to have access to the keys
>>>         as well as the because of the added overhead of the
>>>         decrypt/encrypt cycle. This draft proposes an RTP header
>>>         information included in the header extension and therefore
>>>         do forwarding without SRTP decryption. Comments welcome.
>>>
>>>         Regards,
>>>         Adam Fineberg
>>>         _______________________________________________ rtcweb
>>>         mailing list [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

-- 
Regards,
Adam

---

From: Stephan Wenger <[email-masked]>
Date: Fri, 19 Jul 2013 22:44:12 +0000

Hi,

From: Adam Fineberg <[email-masked]<mailto:[email-masked]>>
Date: Friday, 19 July, 2013 15:12
To: Stephan Wenger <[email-masked]<mailto:[email-masked]>>
Cc: Justin Uberti <[email-masked]<mailto:[email-masked]>>, Bernard Aboba <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [avtext] [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

Stephan,

Thanks for the info and the reference.  I'm not sure I follow as I'm not at all familiar with H.265.  I'll review the reference and see what I can figure.

StW: Good luck :-)

It seems though to me that you are suggesting that except in the simple case, that the data for H.265 would not be well suited to a header extension, am I understanding you correctly?  There is no reason the middlebox couldn't get out of band signaling of the VPS as you mention but that would not be within the scope of this header extension.

StW: well, if you would copy the layer_id into your header extension (just as you need to do for the simple case), a really smart middle box could use this information just as a decoder uses it, assuming that it intercepted the VPS in the first place.  Insofar, I wouldn't rule out the second option on technical grounds.  Whether any of the actual products would bother to do that, ever, is another question.  I think the case ought to be documented, though.  I can help drafting text.
While we are at it: doing this right could mean that you need multiple specs.  First, a generic header extension mechanism dedicated to side information required for pruning of RTP packet streams—ideally not only for scalable video, although that is the main customer today.  And second, for each "payload" (at present we are talking about H.264/SVC, H.265v1 (HEVC), H.265v2 (including scalable and 3D extensions, which are not yet finalized), VP8, and VP9 (I know little about the latter), plus Daala, and whatnot) a mapping of the bits available in the generic header extension to the bits in the payload itself (NAL header and VPS in case of H.265, NALU header in SVC, and the fields you mention for VP8).

Stephan

Regards,
Adam

Stephan

Bernard,

Regards,
Adam

Bernard,

Regards,
Adam

Hi,

Regards,
Adam Fineberg

-------- Original Message --------
Subject:        New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt
Date:   Tue, 09 Jul 2013 10:02:05 -0700
From:   internet-drafts at ietf.org<mailto:internet-drafts%20at%20ietf.org>
To:     Adam Fineberg <fineberg at vline.com><mailto:fineberg%20at%20vline.com>

Abstract:

--
Regards,
Adam

_______________________________________________
avtext mailing list
[email-masked]<mailto:[email-masked]>https://www.ietf.org/mailman/listinfo/avtext

--
Regards,
Adam

---

From: Adam Fineberg <[email-masked]>
Date: Fri, 19 Jul 2013 15:53:37 -0700

Stephan,

On 7/19/13 3:44 PM, Stephan Wenger wrote:
> Hi,
>
> From: Adam Fineberg <[email-masked] <mailto:[email-masked]>>
> To: Stephan Wenger <[email-masked] <mailto:[email-masked]>>
> Cc: Justin Uberti <[email-masked] <mailto:[email-masked]>>, 
> Bernard Aboba <[email-masked] 
> <mailto:[email-masked]>>, "[email-masked] 
> <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>, 
> "[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>
> Subject: Re: [avtext] [rtcweb] Fwd: New Version Notification for 
> Thanks for the info and the reference.  I'm not sure I follow as I'm 
> not at all familiar with H.265.  I'll review the reference and see 
> what I can figure.
>
> StW: Good luck :-)
>
> It seems though to me that you are suggesting that except in the 
> simple case, that the data for H.265 would not be well suited to a 
> header extension, am I understanding you correctly?  There is no 
> StW: well, if you would copy the layer_id into your header extension 
> (just as you need to do for the simple case), a really smart middle 
> box could use this information just as a decoder uses 
> it, assuming that it intercepted the VPS in the first place.  Insofar, 
> I wouldn't rule out the second option on technical grounds.  Whether 
> any of the actual products would bother to do that, ever, is another 
> question.  I think the case ought to be documented, though.  I can 
> help drafting text.

Any help is appreciated.

> While we are at it: doing this right could mean that you need multiple 
> specs.  First, a generic header extension mechanism dedicated to side 
> information required for pruning of RTP packet streams—ideally not 
> only for scalable video, although that is the main customer today. 
>  And second, for each "payload" (at present we are talking about 
> H.264/SVC, H.265v1 (HEVC), H.265v2 (including scalable and 3D 
> extensions, which are not yet finalized), VP8, and VP9 (I know little 
> about the latter), plus Daala, and whatnot) a mapping of the bits 
> available in the generic header extension to the bits in the payload 
> itself (NAL header and VPS in case of H.265, NALU header in SVC, and 
> the fields you mention for VP8).

I was just thinking about that since it seems like the extension payload 
maybe completely specific and therefore not suitable for the same spec.  
The problem in my mind with taking the hierarchical a[[roach you mention 
is that I'm not sure what if any common data is suitable accross all 
codecs and since there is already a spec for the general header 
extension, the generic one you mention might be fairly redundant.

I'm certainly open to your thoughts though.

Regards,
Adam

>> unit header contains a six bit field that points into a data 
>> structure known as "Video Parameter Set" (VPS).  Inside the VPS, 
>> those six bits are mapped to to a position in a directed graph 
>> (specified through "dimension_id[][]"), which tells you about the 
>> reference relationship of the layer in question and its parent layer. 
>>  One can recursively follow the graph to determine what used to be 
>> called dependency_id, quality_id, view_id, and whatnot.  The six bit 
>> pointer field can (or: is to be when possible) organized by the 
>> encoder such that it is prudent for a middle box to throw away NAL 
>> units (belonging to layers) with higher values of the six bit field 
>> first, before throwing away NAL units with lower values.  Relying on 
>> this feature, 3+6 bits == 9 bits should be fine for the header extension.
>>
>> there would be two options: one to represent the dimension_id[][] 
>> (and associated control info) in the header extension, or require the 
>> content.  The further could take considerably more than 16 bits and 
>> we would be talking about a variable length data structure.  The 
>> latter requires the middle box to have state and a mechanism to 
>> intercept the VPS (through signaling—as the encrypted in-band VPS 
>> would not be useful under the assumption that the middle box does not 
>> have the key to the media—which is the motivation of the draft in the 
>> first place).  I personally don't mind at all the second mechanism, 
>> as I'm a big fan of out-of-band parameter set transmission and any 
>> middle box must be in the signaling path anyway to meaningfully 
>> manipulate RTP.  I do not like the first option due to its variable, 
>> and possibly substantial, overhead.
>>
>> Stephan
>>
>>
>>>>         get the temporal layer information from the RTP header
>>>>         which requires us to decrypt the SRTP packets. This is
>>>>         undesirable both because the middle-box needs to have
>>>>         access to the keys as well as the because of the added
>>>>         overhead of the decrypt/encrypt cycle. This draft proposes
>>>>         an RTP header extension that will allow us to use the VP8
>>>>         temporal layer information included in the header extension
>> [email-masked]://www.ietf.org/mailman/listinfo/avtext
>
> -- 
> Regards,
> Adam

-- 
Regards,
Adam

---

From: "Wang, Ye-Kui" <[email-masked]>
Date: Sat, 20 Jul 2013 15:39:58 +0000

Just to mention that the family of 3D codecs based on H.264, incl. MVC, MVC+D, 3D-AVC, and even MFC, developed or being developed, should also be taken into consideration.

BR, YK

On Jul 19, 2013, at 15:54, "Adam Fineberg" <[email-masked]<mailto:[email-masked]>> wrote:

Stephan,

Stephan,

StW: Good luck :-)

Any help is appreciated.

I was just thinking about that since it seems like the extension payload maybe completely specific and therefore not suitable for the same spec.  The problem in my mind with taking the hierarchical a[[roach you mention is that I'm not sure what if any common data is suitable accross all codecs and since there is already a spec for the general header extension, the generic one you mention might be fairly redundant.

Regards,
Adam

Stephan

Regards,
Adam

Stephan

Bernard,

Regards,
Adam

Bernard,

Regards,
Adam

Hi,

Regards,
Adam Fineberg

-------- Original Message --------
Subject:        New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt
Date:   Tue, 09 Jul 2013 10:02:05 -0700
From:   internet-drafts at ietf.org<mailto:internet-drafts%20at%20ietf.org>
To:     Adam Fineberg <fineberg at vline.com><mailto:fineberg%20at%20vline.com>

Abstract:

--
Regards,
Adam

--
Regards,
Adam

--
Regards,
Adam

---

From: Adam Fineberg <[email-masked]>
Date: Tue, 23 Jul 2013 08:53:45 -0700

I've been thinking about this and given the ease at which RFC5285 allows 
for the specification of a header extension and the complexity 
introduced by trying to generalize the header extension to support all 
forms of scalability for all codecs that the generalization might not be 
the best approach.  I'm not sure what we really gain by trying to 
capture all this in a single header extension rather than one per that 
can succinctly explain the fields without the complexity of multiplexing 
the bits.

Thoughts?

Regards,
Adam

-- 
Regards,
Adam

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 23 Jul 2013 12:07:16 -0700

I do not think it is necessary to "support all forms of scalability for all codecs".   In fact, I would make that an explicit "non-goal".  All that was suggested is to try to create a single extension that supports a few common cases.   If it is possible to handle VP8, VP9 and H.264/SVC in a single extension that would be sufficient.  So why not limit it to that? 

Date: Tue, 23 Jul 2013 08:53:45 -0700
From: [email-masked]
To: [email-masked]
CC: [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [avtext] [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

  
    I've been thinking about this and given the ease at which RFC5285
    allows for the specification of a header extension and the
    complexity introduced by trying to generalize the header extension
    to support all forms of scalability for all codecs that the
    generalization might not be the best approach.  I'm not sure what we
    really gain by trying to capture all this in a single header
    extension rather than one per that can succinctly explain the fields
    without the complexity of multiplexing the bits.

    Thoughts?

    Regards,

    Adam

    On 7/19/13 3:44 PM, Stephan Wenger
      wrote:

      
      Hi,

          From: Adam Fineberg
          <[email-masked]>

          Date: Friday, 19 July,
          2013 15:12 

          To: Stephan Wenger <[email-masked]>

          Cc: Justin Uberti <[email-masked]>,
          Bernard Aboba <[email-masked]>,
          "[email-masked]"
          <[email-masked]>,
          "[email-masked]"
          <[email-masked]>

          Subject: Re: [avtext]
          [rtcweb] Fwd: New Version Notification for
          draft-fineberg-avtext-temporal-layer-ext-00.txt

        
          Stephan,

            Thanks for the info and the reference.  I'm not sure I
            follow as I'm not at all familiar with H.265.  I'll review
            the reference and see what I can figure. 

      
      StW: Good luck :-)  

          It seems though to me
            that you are suggesting that except in the simple case, that
            the data for H.265 would not be well suited to a header
            extension, am I understanding you correctly?  There is no
            reason the middlebox couldn't get out of band signaling of
            the VPS as you mention but that would not be within the
            scope of this header extension.

      
      StW:
            well, if you would copy the layer_id into your header
            extension (just as you need to do for the simple case), a
            really smart middle box could use this information just as a
            decoder uses it, assuming that it intercepted the VPS in the
            first place.  Insofar, I wouldn't rule out the second option
            on technical grounds.  Whether any of the actual products
            would bother to do that, ever, is another question.  I think
            the case ought to be documented, though.  I can help
            drafting text.
      While
            we are at it: doing this right could mean that you need
            multiple specs.  First, a generic header extension mechanism
            dedicated to side information required for pruning of RTP
            packet streams—ideally not only for scalable video, although
            that is the main customer today.  And second, for each
            "payload" (at present we are talking about H.264/SVC,
            H.265v1 (HEVC), H.265v2 (including scalable and 3D
            extensions, which are not yet finalized), VP8, and VP9 (I
            know little about the latter), plus Daala, and whatnot) a
            mapping of the bits available in the generic header
            extension to the bits in the payload itself (NAL header and
            VPS in case of H.265, NALU header in SVC, and the fields you
            mention for VP8).

      Stephan

            Regards,

            Adam

            On 7/19/13 8:33 AM, Stephan
              Wenger wrote:

              Hi,
              I also believe that 16 bits should be enough.  For
                H.264 and VP8 that has already been demonstrated.  For
                H.265, some initial thoughts below.  Apologies for the
                word-count.

              The scalable version of H265 (called SHVC) is
                currently under development.  The current working draft
                can be found here: http://phenix.int-evry.fr/jct/doc_end_user/documents/13_Incheon/wg11/JCTVC-M1008-v3.zip.
                 Therein, the options for defining layering structures
                are considerably more complex.  To start, we have 3 bits
                for the temporal ID in the NAL unit header of the H.265
                version 1 (HEVC) base specification (temporal
                scalability is already nicely supported in version 1).
                 Just like in SVC.  In the scalable extension, the NAL
                unit header contains a six bit field that points into a
                data structure known as "Video Parameter Set" (VPS).
                 Inside the VPS, those six bits are mapped to to a
                position in a directed graph (specified through
                "dimension_id[][]"), which tells you about the reference
                relationship of the layer in question and its parent
                layer.  One can recursively follow the graph to
                determine what used to be called dependency_id,
                quality_id, view_id, and whatnot.  The six bit pointer
                field can (or: is to be when possible) organized by the
                encoder such that it is prudent for a middle box to
                throw away NAL units (belonging to layers) with higher
                values of the six bit field first, before throwing away
                NAL units with lower values.  Relying on this feature,
                3+6 bits == 9 bits should be fine for the header
                extension.

              That said, the ordering by the encoder is just a
                recommendation, and there may well be cases where
                different pruning strategies may be advisable.  For
                example, a layering structure could be constructed that
                expands into two branches, one using 2D scalable tools
                only, the other including view_id for multi view coding.
                 By looking at the six bit field alone, a middle box
                will not be able to meaningfully remove NAL units
                belonging to one of the branches completely while
                pruning the other branch.  In order to meaningfully deal
                with that scenario, there would be two options: one to
                represent the dimension_id[][] (and associated control
                info) in the header extension, or require the middle box
                to have access to the VPS and be able to interpret its
                content.  The further could take considerably more than
                16 bits and we would be talking about a variable length
                data structure.  The latter requires the middle box to
                have state and a mechanism to intercept the VPS (through
                signaling—as the encrypted in-band VPS would not be
                useful under the assumption that the middle box does not
                have the key to the media—which is the motivation of the
                draft in the first place).  I personally don't mind at
                all the second mechanism, as I'm a big fan of
                out-of-band parameter set transmission and any middle
                box must be in the signaling path anyway to meaningfully
                manipulate RTP.  I do not like the first option due to
                its variable, and possibly substantial, overhead.

              Stephan   

              

                  From: Justin
                  Uberti <[email-masked]>

                  Date: Friday,
                  19 July, 2013 06:32 

                  To: Bernard
                  Aboba <[email-masked]>

                  Cc: "[email-masked]"
                  <[email-masked]>,
                  "[email-masked]"
                  <[email-masked]>

                  Subject: Re:
                  [rtcweb] Fwd: New Version Notification for
                  draft-fineberg-avtext-temporal-layer-ext-00.txt

                
                  
                    Agree those are the right codecs to
                      design for. Since in each case there are fairly
                      low limits on the number of supported layers (i.e.
                      3 spatial layers for SVC), I think it should be
                      possible to pack the temporal, spatial, quality
                      layer ids into 16 bits.

                        On Fri, Jul 19, 2013 at
                          1:56 AM, Bernard Aboba 
                            <[email-masked]>
                          wrote:

                              If we can support VP8/9 as well as
                                H.264/5 SVC
                              that would be a start. It seems
                                doable to me.

                                    On Jul 18, 2013, at 8:34 PM, "Adam
                                    Fineberg" <[email-masked]>
                                    wrote:

                                      Bernard,

                                        Are there other codecs you are
                                        thinking should be supported? 
                                        If it's generalized I would
                                        think we want to be able to
                                        cover all known scalable codecs.
                                        I'll look into the H264/SVC
                                        fields to see how to encode them
                                        in a generalized header.

                                        Regards,

                                        Adam

                                        On 7/18/13 7:40 PM, Bernard
                                        Aboba wrote:

                                        I think it may be
                                          possible to generalize this. 
                                          For example, for H.264/SVC
                                          which can support temporal,
                                          spatial and quality
                                          scalability, you would need
                                          the quality_id and
                                          dependency_id in addition to
                                          the temporal_id (what you call
                                          the temporal layer index).   

                                            
                                            Date: Thu, 18 Jul 2013
                                            08:45:38 -0700

                                            From: [email-masked]

                                            To: 
                                              [email-masked]

                                            CC: [email-masked];
                                            [email-masked]

                                            Subject: Re: [rtcweb] Fwd:
                                            New Version Notification for
draft-fineberg-avtext-temporal-layer-ext-00.txt

                                            Bernard,

                                            Good question.  I'm not
                                            familiar enough with the
                                            parameter requirements of
                                            all other scalable codecs to
                                            be able to generalize.  If
                                            you'd like to help specify
                                            them, I'd be fine revising
                                            the draft to generalize.

                                            Regards,

                                            Adam

                                            On 7/17/13 8:26 PM,
                                              Bernard Aboba wrote:

                                              Since the
                                                need is not codec
                                                specific (e.g. it arises
                                                with any codec
                                                supporting temporal,
                                                spatial and quality
                                                scalability), why

                                                 a VP8-specific RTP
                                                extension? 

                                                  
                                                  Date: Wed, 17 Jul 2013
                                                  17:09:46 -0700

                                                  From: [email-masked]

                                                  To: [email-masked]

                                                  Subject: [rtcweb] Fwd:
                                                  New Version
                                                  Notification for

                                                  Hi,
                                                  
                                                  I'm
                                                    working on WebRTC
                                                    services and have
                                                    found that while
                                                    developing services
                                                    that forward VP8
                                                    video streams if we
                                                    want to take
                                                    advantage of the VP8
                                                    temporal scaling we
                                                    must get the
                                                    temporal layer
                                                    information from the
                                                    RTP header which
                                                    requires us to
                                                    decrypt the SRTP
                                                    packets. This is
                                                    undesirable both
                                                    because the
                                                    middle-box needs to
                                                    have access to the
                                                    keys as well as the
                                                    because of the added
                                                    overhead of the
                                                    decrypt/encrypt
                                                    cycle. This draft
                                                    proposes an RTP
                                                    header extension
                                                    that will allow us
                                                    to use the VP8
                                                    temporal layer
                                                    information included
                                                    in the header
                                                    extension and
                                                    therefore do
                                                    forwarding without
                                                    SRTP decryption.
                                                    Comments welcome.
                                                  
                                                  Regards,
                                                  Adam
                                                    Fineberg
                                                  fineberg
                                                      at vline.com

                                                    -------- Original
                                                    Message --------

                                                          Subject:
                                                          New
                                                          Version
                                                          Notification
                                                          for

                                                          Date:
                                                          Tue, 09
                                                          Jul 2013
                                                          10:02:05 -0700

                                                          From:
                                                          internet-drafts
                                                          at ietf.org

                                                          To:
                                                          Adam
                                                          Fineberg <fineberg at vline.com>

                                                    

Abstract:

                                                  

                                            -- 
Regards,
Adam

                                    

                            rtcweb mailing list

                            [email-masked]

                          

                

            -- 
Regards,
Adam

    -- 
Regards,
Adam

---

From: Adam Fineberg <[email-masked]>
Date: Wed, 24 Jul 2013 10:08:18 -0700

Bernard,

I apologize if I come across as being difficult here but I stil am not 
seeing the benefits.  Since the fields are not the same for the codecs, 
we will be multiplexing the bits and that seems to me to add complexity 
rather than add clarity.  Also, I can't find an IETF VP9 document for 
the payload format to reference.  If the group thinks generalization is 
the right approach I would appreciate some collaboration on getting the 
right bit definitions for the other codecs.

Regards,
Adam

On 7/23/13 12:07 PM, Bernard Aboba wrote:
> I do not think it is necessary to "support all forms of scalability 
> for all codecs".   In fact, I would make that an explicit "non-goal".  
> All that was suggested is to try to create a single extension that 
> supports a few common cases. If it is possible to handle VP8, VP9 and 
> H.264/SVC in a single extension that would be sufficient.  So why not 
> limit it to that?
>
> allows for the specification of a header extension and the complexity 
> forms of scalability for all codecs that the generalization might not 
> be the best approach.  I'm not sure what we really gain by trying to 
> can succinctly explain the fields without the complexity of 
> multiplexing the bits.
>
> Thoughts?
>
> Regards,
> Adam
>
>     Cc: Justin Uberti <[email-masked]
>     <mailto:[email-masked]>>, Bernard Aboba
>     <[email-masked] <mailto:[email-masked]>>,
>     "[email-masked] <mailto:[email-masked]>" <[email-masked]
>     <mailto:[email-masked]>>, "[email-masked]
>     <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>
>     Thanks for the info and the reference.  I'm not sure I follow as
>     I'm not at all familiar with H.265.  I'll review the reference and
>     see what I can figure.
>
>     StW: Good luck :-)
>
>     reason the middlebox couldn't get out of band signaling of the VPS
>     as you mention but that would not be within the scope of this
>     header extension.
>
>     StW: well, if you would copy the layer_id into your header
>     extension (just as you need to do for the simple case), a really
>     smart middle box could use this information just as a decoder uses
>     it, assuming that it intercepted the VPS in the first place.
>      Insofar, I wouldn't rule out the second option on technical
>     grounds.  Whether any of the actual products would bother to do
>     that, ever, is another question.  I think the case ought to be
>     documented, though.  I can help drafting text.
>     While we are at it: doing this right could mean that you need
>     dedicated to side information required for pruning of RTP packet
>     streams—ideally not only for scalable video, although that is the
>     main customer today.  And second, for each "payload" (at present
>     we are talking about H.264/SVC, H.265v1 (HEVC), H.265v2 (including
>     scalable and 3D extensions, which are not yet finalized), VP8, and
>     VP9 (I know little about the latter), plus Daala, and whatnot) a
>     mapping of the bits available in the generic header extension to
>     the bits in the payload itself (NAL header and VPS in case of
>     H.265, NALU header in SVC, and the fields you mention for VP8).
>
>     Stephan
>
>
>         I also believe that 16 bits should be enough.  For H.264 and
>         VP8 that has already been demonstrated.  For H.265, some
>         initial thoughts below.  Apologies for the word-count.
>
>         considerably more complex.  To start, we have 3 bits for the
>         temporal ID in the NAL unit header of the H.265 version 1
>         (HEVC) base specification (temporal scalability is already
>         nicely supported in version 1).  Just like in SVC.  In the
>         scalable extension, the NAL unit header contains a six bit
>         field that points into a data structure known as "Video
>         Parameter Set" (VPS).  Inside the VPS, those six bits are
>         mapped to to a position in a directed graph (specified through
>         relationship of the layer in question and its parent layer.
>          One can recursively follow the graph to determine what used
>         to be called dependency_id, quality_id, view_id, and whatnot.
>          The six bit pointer field can (or: is to be when possible)
>         organized by the encoder such that it is prudent for a middle
>         box to throw away NAL units (belonging to layers) with higher
>         values of the six bit field first, before throwing away NAL
>         units with lower values.  Relying on this feature, 3+6 bits ==
>         9 bits should be fine for the header extension.
>
>         recommendation, and there may well be cases where different
>         pruning strategies may be advisable.  For example, a layering
>         structure could be constructed that expands into two branches,
>         one using 2D scalable tools only, the other including view_id
>         for multi view coding.  By looking at the six bit field alone,
>         a middle box will not be able to meaningfully remove NAL units
>         belonging to one of the branches completely while pruning the
>         other branch.  In order to meaningfully deal with that
>         scenario, there would be two options: one to represent the
>         dimension_id[][] (and associated control info) in the header
>         extension, or require the middle box to have access to the VPS
>         and be able to interpret its content.  The further could take
>         considerably more than 16 bits and we would be talking about a
>         variable length data structure.  The latter requires the
>         middle box to have state and a mechanism to intercept the VPS
>         (through signaling—as the encrypted in-band VPS would not be
>         useful under the assumption that the middle box does not have
>         the key to the media—which is the motivation of the draft in
>         the first place).  I personally don't mind at all the second
>         mechanism, as I'm a big fan of out-of-band parameter set
>         transmission and any middle box must be in the signaling path
>         anyway to meaningfully manipulate RTP.  I do not like the
>         first option due to its variable, and possibly substantial,
>         overhead.
>
>         Stephan
>
>
>         Cc: "[email-masked] <mailto:[email-masked]>"
>         <[email-masked] <mailto:[email-masked]>>, "[email-masked]
>         <mailto:[email-masked]>" <[email-masked]
>         Agree those are the right codecs to design for. Since in each
>         case there are fairly low limits on the number of supported
>         layers (i.e. 3 spatial layers for SVC), I think it should be
>         possible to pack the temporal, spatial, quality layer ids into
>         16 bits.
>
>
>             On Jul 18, 2013, at 8:34 PM, "Adam Fineberg"
>             <[email-masked] <mailto:[email-masked]>> wrote:
>
>                 Bernard,
>
>                 Are there other codecs you are thinking should be
>                 supported?  If it's generalized I would think we want
>                 to be able to cover all known scalable codecs. I'll
>                 look into the H264/SVC fields to see how to encode
>                 them in a generalized header.
>
>                 Regards,
>                 Adam
>
>                     I think it may be possible to generalize this. 
>                     For example, for H.264/SVC which can support
>                     temporal, spatial and quality scalability, you
>                     would need the quality_id and dependency_id in
>                     <mailto:[email-masked]>
>                     CC: [email-masked] <mailto:[email-masked]>;
>                     [email-masked] <mailto:[email-masked]>
>                     Good question.  I'm not familiar enough with the
>                     parameter requirements of all other scalable
>                     codecs to be able to generalize.  If you'd like to
>                         arises with any codec supporting temporal,
>                         spatial and quality scalability), why
>                         I'm working on WebRTC services and have found
>                         that while developing services that forward
>                         VP8 video streams if we want to take advantage
>                         of the VP8 temporal scaling we must get the
>                         temporal layer information from the RTP header
>                         which requires us to decrypt the SRTP packets.
>                         This is undesirable both because the
>                         middle-box needs to have access to the keys as
>                         well as the because of the added overhead of
>                         the decrypt/encrypt cycle. This draft proposes
>                         an RTP header extension that will allow us to
>                         use the VP8 temporal layer information
>                         included in the header extension and therefore
>         [email-masked]  <mailto:[email-masked]>https://www.ietf.org/mailman/listinfo/avtext
>
>
>     -- 
>     Regards,
>     Adam
>
>
> -- 
> Regards,
> Adam

-- 
Regards,
Adam

---

From: "Wang, Ye-Kui" <[email-masked]>
Date: Wed, 24 Jul 2013 22:32:01 +0000

If the group is to specify something generic, naturally it should be generic enough to cover at least all existing standard scalable video codecs if possible. And I personally think that is possible and not difficult at all. Thus, why limit to only a few scalable video codecs?

I could provide some help here too if needed.

BR, YK

From: [email-masked] [mailto:[email-masked]] On Behalf Of Adam Fineberg
Sent: Wednesday, July 24, 2013 10:08 AM
To: Bernard Aboba
Cc: [email-masked]; [email-masked]; Justin Uberti
Subject: Re: [avtext] [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

Bernard,

I apologize if I come across as being difficult here but I stil am not seeing the benefits.  Since the fields are not the same for the codecs, we will be multiplexing the bits and that seems to me to add complexity rather than add clarity.  Also, I can't find an IETF VP9 document for the payload format to reference.  If the group thinks generalization is the right approach I would appreciate some collaboration on getting the right bit definitions for the other codecs.

Regards,
Adam
On 7/23/13 12:07 PM, Bernard Aboba wrote:
I do not think it is necessary to "support all forms of scalability for all codecs".   In fact, I would make that an explicit "non-goal".  All that was suggested is to try to create a single extension that supports a few common cases.   If it is possible to handle VP8, VP9 and H.264/SVC in a single extension that would be sufficient.  So why not limit it to that?
________________________________
Date: Tue, 23 Jul 2013 08:53:45 -0700
From: [email-masked]<mailto:[email-masked]>
To: [email-masked]<mailto:[email-masked]>
CC: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [avtext] [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

I've been thinking about this and given the ease at which RFC5285 allows for the specification of a header extension and the complexity introduced by trying to generalize the header extension to support all forms of scalability for all codecs that the generalization might not be the best approach.  I'm not sure what we really gain by trying to capture all this in a single header extension rather than one per that can succinctly explain the fields without the complexity of multiplexing the bits.

Thoughts?

Regards,
Adam

Stephan,

StW: Good luck :-)

StW: well, if you would copy the layer_id into your header extension (just as you need to do for the simple case), a really smart middle box could use this information just as a decoder uses it, assuming that it intercepted the VPS in the first place.  Insofar, I wouldn't rule out the second option on technical grounds.  Whether any of the actual products would bother to do that, ever, is another question.  I think the case ought to be documented, though.  I can help drafting text.
While we are at it: doing this right could mean that you need multiple specs.  First, a generic header extension mechanism dedicated to side information required for pruning of RTP packet streams-ideally not only for scalable video, although that is the main customer today.  And second, for each "payload" (at present we are talking about H.264/SVC, H.265v1 (HEVC), H.265v2 (including scalable and 3D extensions, which are not yet finalized), VP8, and VP9 (I know little about the latter), plus Daala, and whatnot) a mapping of the bits available in the generic header extension to the bits in the payload itself (NAL header and VPS in case of H.265, NALU header in SVC, and the fields you mention for VP8).

Stephan

Regards,
Adam

That said, the ordering by the encoder is just a recommendation, and there may well be cases where different pruning strategies may be advisable.  For example, a layering structure could be constructed that expands into two branches, one using 2D scalable tools only, the other including view_id for multi view coding.  By looking at the six bit field alone, a middle box will not be able to meaningfully remove NAL units belonging to one of the branches completely while pruning the other branch.  In order to meaningfully deal with that scenario, there would be two options: one to represent the dimension_id[][] (and associated control info) in the header extension, or require the middle box to have access to the VPS and be able to interpret its content.  The further could take considerably more than 16 bits and we would be talking about a variable length data structure.  The latter requires the middle box to have state and a mechanism to intercept the VPS (through signaling-as the encrypted in-band VPS would not be useful under the assumption that the middle box does not have the key to the media-which is the motivation of the draft in the first place).  I personally don't mind at all the second mechanism, as I'm a big fan of out-of-band parameter set transmission and any middle box must be in the signaling path anyway to meaningfully manipulate RTP.  I do not like the first option due to its variable, and possibly substantial, overhead.

Stephan

Regards,
Adam

Bernard,

Regards,
Adam

Hi,

Regards,
Adam Fineberg

-------- Original Message --------
Subject:

New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

Date:

From:

internet-drafts at ietf.org<mailto:internet-drafts%20at%20ietf.org>

To:

Adam Fineberg <fineberg at vline.com><mailto:fineberg%20at%20vline.com>

IETF repository.

Revision:         00

Number of pages: 6

Abstract:

--

Regards,

Adam

avtext mailing list

--

Regards,

Adam

--

Regards,

Adam

--

Regards,

Adam

---

From: Adam Fineberg <[email-masked]>
Date: Wed, 24 Jul 2013 15:54:42 -0700

YK,

I would appreciate your collaboration.  Which codecs are you referring 
to when you say "all existing standard scalable video codecs"?

Regards,
Adam

On 7/24/13 3:32 PM, Wang, Ye-Kui wrote:
>
> If the group is to specify something generic, naturally it should be 
> generic enough to cover at least all existing standard scalable video 
> codecs if possible. And I personally think that is possible and not 
> difficult at all. Thus, why limit to only a few scalable video codecs?
>
> *From:[email-masked] [mailto:[email-masked]] *On 
> Behalf Of *Adam Fineberg
> *Sent:* Wednesday, July 24, 2013 10:08 AM
> *To:* Bernard Aboba
> *Cc:* [email-masked]; [email-masked]; Justin Uberti
> *Subject:* Re: [avtext] [rtcweb] Fwd: New Version Notification for 
> seeing the benefits.  Since the fields are not the same for the 
> codecs, we will be multiplexing the bits and that seems to me to add 
> complexity rather than add clarity.  Also, I can't find an IETF VP9 
> document for the payload format to reference.  If the group thinks 
> generalization is the right approach I would appreciate some 
> collaboration on getting the right bit definitions for the other codecs.
>
> Regards,
> Adam
>
>     I do not think it is necessary to "support all forms of
>     scalability for all codecs".   In fact, I would make that an
>     explicit "non-goal".  All that was suggested is to try to create a
>     single extension that supports a few common cases.   If it is
>     possible to handle VP8, VP9 and H.264/SVC in a single extension
>     that would be sufficient.  So why not limit it to that?
>
>     To: [email-masked] <mailto:[email-masked]>
>     CC: [email-masked] <mailto:[email-masked]>;
>     [email-masked] <mailto:[email-masked]>;
>     [email-masked] <mailto:[email-masked]>; [email-masked]
>     <mailto:[email-masked]>
>     generalization might not be the best approach.  I'm not sure what
>     we really gain by trying to capture all this in a single header
>     extension rather than one per that can succinctly explain the
>     fields without the complexity of multiplexing the bits.
>
>     Thoughts?
>
>     Regards,
>     Adam
>
>         *From: *Adam Fineberg <[email-masked]
>         <mailto:[email-masked]>>
>         *Date: *Friday, 19 July, 2013 15:12
>         *To: *Stephan Wenger <[email-masked] <mailto:[email-masked]>>
>         *Cc: *Justin Uberti <[email-masked]
>         <[email-masked]
>         <mailto:[email-masked]>" <[email-masked]
>         *Subject: *Re: [avtext] [rtcweb] Fwd: New Version Notification
>         for draft-fineberg-avtext-temporal-layer-ext-00.txt
>
>         Stephan,
>
>         Thanks for the info and the reference.  I'm not sure I follow
>         as I'm not at all familiar with H.265.  I'll review the
>         reference and see what I can figure.
>
>         StW: Good luck :-)
>
>         It seems though to me that you are suggesting that except in
>         the simple case, that the data for H.265 would not be well
>         suited to a header extension, am I understanding you
>         correctly? There is no reason the middlebox couldn't get out
>         of band signaling of the VPS as you mention but that would not
>         be within the scope of this header extension.
>
>         the case ought to be documented, though.  I can help drafting
>         text.
>
>         packet streams---ideally not only for scalable video, although
>         "payload" (at present we are talking about H.264/SVC, H.265v1
>         (HEVC), H.265v2 (including scalable and 3D extensions, which
>         are not yet finalized), VP8, and VP9 (I know little about the
>         latter), plus Daala, and whatnot) a mapping of the bits
>         available in the generic header extension to the bits in the
>         payload itself (NAL header and VPS in case of H.265, NALU
>         header in SVC, and the fields you mention for VP8).
>
>         Stephan
>
>
>
>             I also believe that 16 bits should be enough.  For H.264
>             and VP8 that has already been demonstrated.  For H.265,
>             some initial thoughts below.  Apologies for the word-count.
>
>             The scalable version of H265 (called SHVC) is currently
>             under development.  The current working draft can be found
>             here:
>             considerably more complex.  To start, we have 3 bits for
>             the temporal ID in the NAL unit header of the H.265
>             version 1 (HEVC) base specification (temporal scalability
>             is already nicely supported in version 1).  Just like in
>             SVC.  In the scalable extension, the NAL unit header
>             contains a six bit field that points into a data structure
>             known as "Video Parameter Set" (VPS).  Inside the VPS,
>             those six bits are mapped to to a position in a directed
>             graph (specified through "dimension_id[][]"), which tells
>             you about the reference relationship of the layer in
>             question and its parent layer.  One can recursively follow
>             the graph to determine what used to be called
>             dependency_id, quality_id, view_id, and whatnot.  The six
>             bit pointer field can (or: is to be when possible)
>             organized by the encoder such that it is prudent for a
>             middle box to throw away NAL units (belonging to layers)
>             with higher values of the six bit field first, before
>             throwing away NAL units with lower values.  Relying on
>              By looking at the six bit field alone, a middle box will
>             not be able to meaningfully remove NAL units belonging to
>             one of the branches completely while pruning the other
>             branch.  In order to meaningfully deal with that scenario,
>             there would be two options: one to represent the
>             dimension_id[][] (and associated control info) in the
>             header extension, or require the middle box to have access
>             to the VPS and be able to interpret its content.  The
>             further could take considerably more than 16 bits and we
>             would be talking about a variable length data structure.
>              The latter requires the middle box to have state and a
>             mechanism to intercept the VPS (through signaling---as the
>             encrypted in-band VPS would not be useful under the
>             assumption that the middle box does not have the key to
>             the media---which is the motivation of the draft in the
>             first place).  I personally don't mind at all the second
>             transmission and any middle box must be in the signaling
>             path anyway to meaningfully manipulate RTP.  I do not like
>             the first option due to its variable, and possibly
>             substantial, overhead.
>
>             Stephan
>
>             *From: *Justin Uberti <[email-masked]
>             <mailto:[email-masked]>>
>             *Date: *Friday, 19 July, 2013 06:32
>             *To: *Bernard Aboba <[email-masked]
>             <mailto:[email-masked]>>
>             *Cc: *"[email-masked] <mailto:[email-masked]>"
>             <[email-masked] <mailto:[email-masked]>>,
>             "[email-masked] <mailto:[email-masked]>"
>             *Subject: *Re: [rtcweb] Fwd: New Version Notification for
>             Agree those are the right codecs to design for. Since in
>             each case there are fairly low limits on the number of
>             supported layers (i.e. 3 spatial layers for SVC), I think
>             it should be possible to pack the temporal, spatial,
>             quality layer ids into 16 bits.
>
>             <[email-masked]
>             <mailto:[email-masked]>> wrote:
>
>                     I think it may be possible to generalize this. For
>                     example, for H.264/SVC which can support temporal,
>                     spatial and quality scalability, you would need
>                     the quality_id and dependency_id in addition to
>                     the temporal_id (what you call the temporal layer
>                     index).
>
>                         *Subject:*
>                         *Date:*
>                         *From:*
>                         *To:*

-- 
Regards,
Adam

---

From: "Wang, Ye-Kui" <[email-masked]>
Date: Thu, 25 Jul 2013 07:28:40 +0000

Adam,

I meant the scalable codecs mentioned so far by everyone, including H.265/HEVC and its extensions, and H.264/AVC and its extensions.

BR, YK

From: Adam Fineberg [mailto:[email-masked]]
Sent: Wednesday, July 24, 2013 3:55 PM
To: Wang, Ye-Kui
Cc: Bernard Aboba; [email-masked]; [email-masked]; Justin Uberti
Subject: Re: [avtext] [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

YK,

I would appreciate your collaboration.  Which codecs are you referring to when you say "all existing standard scalable video codecs"?

Regards,
Adam

BR, YK

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Adam Fineberg
Sent: Wednesday, July 24, 2013 10:08 AM
To: Bernard Aboba
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; Justin Uberti
Subject: Re: [avtext] [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

Bernard,

Regards,
Adam

Thoughts?

Regards,
Adam

Stephan,

StW: Good luck :-)

Stephan

Regards,
Adam

Stephan

Regards,
Adam

Bernard,

Regards,
Adam

Hi,

Regards,
Adam Fineberg

-------- Original Message --------
Subject:

Date:

From:

To:

IETF repository.

Revision:         00

Number of pages: 6

Abstract:

--

Regards,

Adam

avtext mailing list

--

Regards,

Adam

--

Regards,

Adam

--

Regards,

Adam

--

Regards,

Adam

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 25 Jul 2013 07:44:50 +0000

Hi,
I thought about this a bit more.  I think that one sensible document structure may be as follows:

(1) generic structure of scalability/pruning information to an RTP header extension (to be worked on in avtext).
(2) mapping of codec-specific features to the generic structure.  As a location, I would suggest that (for new codecs) the RTP payload specs are the right document.  For codecs with existing RTP payload specs, a short doc that updates the payload spec would be the right place, or a revision of the payload spec itself if that needs to be done anyway.  This would bundle all the codec-specifics to one document.

This document structure, and the philosophy behind it, would allow the design of a generic RTP stack that could take advantage of scalability features independently of the scalable codec actually in use.  I believe that such an advantage outweighs the possible advantage of quick standardization of a more specific approach as proposed in Adam's draft.  That said, we need to get the design of the generic structure right such that it is indeed useful, ideally for all present and hopefully for a large subset of future codecs.

As for the specs involved:
-H.264 SVC, H.265, and VP8 have payload format RFCs or reasonably stable drafts.
-No one uses H.263 and MPEG-2 scalability, so we can ignore those two.
-As Adam, I'm also not aware of a VP9 payload spec draft.  However, my understanding is that VP9 will include a number of scalability features enabled by multiple reference pictures and reference picture resampling.  The latter is a somewhat different approach compared to the traditional scalability implementations.  I think that google planned to freeze the VP9 bitstream a few weeks go, but I don't know whether that has happened.  Without a stable spec and/or input from google/webm folks it will indeed be hard to address VP9's specific needs, if any.
-There are a bunch of audio codecs that claim to be scalable.  We should have a scope discussion on whether those need to be included here.

Stephan

From: Adam Fineberg <[email-masked]<mailto:[email-masked]>>
Date: Thursday, 25 July, 2013 00:54
To: "Wang, Ye-Kui" <[email-masked]<mailto:[email-masked]>>
Cc: Bernard Aboba <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Justin Uberti <[email-masked]<mailto:[email-masked]>>
Subject: Re: [avtext] [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

YK,

Regards,
Adam

BR, YK

From:[email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Adam Fineberg
Sent: Wednesday, July 24, 2013 10:08 AM
To: Bernard Aboba
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; Justin Uberti
Subject: Re: [avtext] [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

Bernard,

Regards,
Adam

Thoughts?

Regards,
Adam

Stephan,

StW: Good luck :-)

Stephan

Regards,
Adam

Stephan

Regards,
Adam

Bernard,

Regards,
Adam

Hi,

Regards,
Adam Fineberg

-------- Original Message --------
Subject:

Date:

From:

To:

IETF repository.

Revision:         00

Number of pages: 6

Abstract:

--

Regards,

Adam

avtext mailing list

--

Regards,

Adam

--

Regards,

Adam

--

Regards,

Adam

--
Regards,
Adam

---

From: Adam Fineberg <[email-masked]>
Date: Fri, 26 Jul 2013 06:14:25 -0700

Stephan,

This seems like a reasonable approach.  I like your point about 
designing the RTP stack to generically handle scalability features.  My 
concern is complexity and overhead for what should be a very lightweight 
process however generalizing seems to be the consensus so that should be 
the direction forward.

Regards,
Adam

On 7/25/13 12:44 AM, Stephan Wenger wrote:
> (1) generic structure of scalability/pruning information to an RTP 
> header extension (to be worked on in avtext).
> (2) mapping of codec-specific features to the generic structure.  As a 
> location, I would suggest that (for new codecs) the RTP payload specs 
> are the right document.  For codecs with existing RTP payload specs, a 
> short doc that updates the payload spec would be the right place, or a 
> revision of the payload spec itself if that needs to be done anyway. 
>  This would bundle all the codec-specifics to one document.
>
> This document structure, and the philosophy behind it, would allow the 
> design of a generic RTP stack that could take advantage of scalability 
> features independently of the scalable codec actually in use.  I 
> believe that such an advantage outweighs the possible advantage of 
> quick standardization of a more specific approach as proposed in 
> Adam's draft.  That said, we need to get the design of the generic 
> structure right such that it is indeed useful, ideally for all present 
> and hopefully for a large subset of future codecs.
>
> -As Adam, I'm also not aware of a VP9 payload spec draft.  However, my 
> understanding is that VP9 will include a number of scalability 
> features enabled by multiple reference pictures and reference picture 
> resampling.  The latter is a somewhat different approach compared to 
> the traditional scalability implementations.  I think that google 
> planned to freeze the VP9 bitstream a few weeks go, but I don't know 
> whether that has happened.  Without a stable spec and/or input from 
> google/webm folks it will indeed be hard to address VP9's specific 
> needs, if any.
> -There are a bunch of audio codecs that claim to be scalable.  We 
> should have a scope discussion on whether those need to be included here.
>
> Stephan
>
>
>
> Cc: Bernard Aboba <[email-masked] 
> <mailto:[email-masked]>>, Justin Uberti <[email-masked] 
>> I apologize if I come across as being difficult here but I stil am 
>> not seeing the benefits.  Since the fields are not the same for the 
>>     explicit "non-goal".  All that was suggested is to try to create
>>     a single extension that supports a few common cases.   If it is
>>     complexity introduced by trying to generalize the header
>>     extension to support all forms of scalability for all codecs that
>>     the generalization might not be the best approach.  I'm not sure
>>     what we really gain by trying to capture all this in a single
>>     header extension rather than one per that can succinctly explain
>>     the fields without the complexity of multiplexing the bits.
>>
>>     Thoughts?
>>
>>     Regards,
>>     Adam
>>
>>         *Subject: *Re: [avtext] [rtcweb] Fwd: New Version
>>         Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt
>>
>>         Stephan,
>>
>>         of band signaling of the VPS as you mention but that would
>>         not be within the scope of this header extension.
>>
>>             under development.  The current working draft can be
>>             found here:
>>             contains a six bit field that points into a data
>>             structure known as "Video Parameter Set" (VPS).  Inside
>>             the VPS, those six bits are mapped to to a position in a
>>             directed graph (specified through "dimension_id[][]"),
>>             which tells you about the reference relationship of the
>>             layer in question and its parent layer.  One can
>>             recursively follow the graph to determine what used to be
>>             called dependency_id, quality_id, view_id, and whatnot.
>>              The six bit pointer field can (or: is to be when
>>             possible) organized by the encoder such that it is
>>             prudent for a middle box to throw away NAL units
>>             (belonging to layers) with higher values of the six bit
>>             field first, before throwing away NAL units with lower
>>             values.  Relying on this feature, 3+6 bits == 9 bits
>>             should be fine for the header extension.
>>
>>             branch.  In order to meaningfully deal with that
>>             scenario, there would be two options: one to represent
>>             the dimension_id[][] (and associated control info) in the
>>             header extension, or require the middle box to have
>>             access to the VPS and be able to interpret its content.
>>              The further could take considerably more than 16 bits
>>             and we would be talking about a variable length data
>>             structure.  The latter requires the middle box to have
>>             state and a mechanism to intercept the VPS (through
>>             all the second mechanism, as I'm a big fan of out-of-band
>>             parameter set transmission and any middle box must be in
>>             the signaling path anyway to meaningfully manipulate RTP.
>>              I do not like the first option due to its variable, and
>>             possibly substantial, overhead.
>>
>>             Stephan
>>
>>                         <mailto:[email-masked]>
>>                         VP8 video streams if we want to take
>>                         advantage of the VP8 temporal scaling we must
>>                         get the temporal layer information from the
>>                         RTP header which requires us to decrypt the
>>                         SRTP packets. This is undesirable both
>>                         because the middle-box needs to have access
>>                         to the keys as well as the because of the
>>                         added overhead of the decrypt/encrypt cycle.
>>                         This draft proposes an RTP header extension
>>                         that will allow us to use the VP8 temporal
>>                         layer information included in the header
>>                         extension and therefore do forwarding without
>>                         SRTP decryption. Comments welcome.
>>
>>                         Regards,
>>                         Adam Fineberg
>>
>>                         fineberg at vline.com
>>                         rtcweb mailing list [email-masked]
>>                         <mailto:[email-masked]>

---

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Wed, 31 Jul 2013 15:12:01 +0000

Hi Adam,

Why do you include PictureID? I can understand how the "Y" bit for base layer sync can be useful to include. But how does including PictureID help identify temporal layers?

Regards,
Mo

________________________________________
Date: Wed, 17 Jul 2013 17:09:46 -0700
From: [email-masked]
To: [email-masked]
Subject: [rtcweb] Fwd: New Version Notification for draft-fineberg-avtext-temporal-layer-ext-00.txt

Hi,

Regards,
Adam Fineberg
fineberg at vline.com

-------- Original Message -------- 
Subject:

Abstract:

-- 
Regards,
Adam