Subject: [pm-dir] request for an RFC 6390 review of http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-jb-10.txt

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 8 Apr 2013 14:14:08 +0000

Hi PMDIR,

As a co-chair of XRBLOCK WG, I would like to request the an RFC 6390 review of http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-jb-10.txt. This document is in WGLC until 4/22.

Thanks and Regards,

Dan

---

From: "MORTON JR., ALFRED C (AL)" <[email-masked]>
Date: Mon, 8 Apr 2013 11:02:18 -0400

Can a Directorate member *volunteer* to perform the review by Friday, April 12,
with deadline for completion as Dan noted (the 22nd), please?

The current review assignments are here:
https://docs.google.com/spreadsheet/ccc?key=0AmKrqWIOBsprdGZqMnB6dmx5bFJvVUhta3VLSjl3SkE#gid=0

Al
pm-dir-admin

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Romascanu, Dan (Dan)
> Sent: Monday, April 08, 2013 10:14 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]; Gonzalo Camarillo
> Subject: [pm-dir] request for an RFC 6390 review of
> http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-jb-10.txt
> 
> 
> Hi PMDIR,
> 
> As a co-chair of XRBLOCK WG, I would like to request the an RFC 6390
> review of http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-jb-10.txt.
> This document is in WGLC until 4/22.
> 
> Thanks and Regards,
> 
> Dan
> 
> 
> 
> _______________________________________________
> pm-dir mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/pm-dir

---

From: Vinayak Hegde <[email-masked]>
Date: Tue, 9 Apr 2013 07:24:32 +0530

Al / Dan,

I can do the review by the 22nd and send any comments to the list.

Regards
Vinayak

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 22 Apr 2013 12:14:18 +0000

Hi PM-DIR,

The 2nd WGLC for this document ends today. Any news concerning the RFC 6390 review? 

Thanks and Regards,

Dan

> Sent: Monday, April 08, 2013 5:14 PM
> To: [email-masked]

---

From: Benoit Claise <[email-masked]>
Date: Tue, 23 Apr 2013 15:16:29 +0200

Hi Dan,

According to 
https://docs.google.com/spreadsheet/ccc?key=0AmKrqWIOBsprdGZqMnB6dmx5bFJvVUhta3VLSjl3SkE#gid=0,
  Vinajak is the point of contact for this draft.
Vinajak, any news?

Regards,

---

From: Vinayak Hegde <[email-masked]>
Date: Wed, 24 Apr 2013 07:06:24 +0530

Hi PM Directorate / Authors / Others,

I reviewed the draft on the basis of RFC 6390 as per Benoit's suggestion
and found some missing information.
The draft makes several references to RFC 3611 which I read to get enough
context.

I am ccing the authors from the mail addresses in the RFC so they can make
the suggested changes.

My comments on the drafts are below:

Section specific comments
----------------------------------------

Section 4.2

What is NJB - First occurence of an acronym and it is not defined.

Section 4.2

When Interval duration bit is set, it is not clear how the interval is
communicated between the sender and receiver. Need to specify this
explicitly.

Security considerations section
The security considerations section refers to RFC 3611 which is more
generic (though contains sufficient infomation). Only the sections relevant
to this draft can be replicated here (such as the reaction to DDoS attacks).

Overall comments
--------------------------
According to RFC 6390, there should be some sources of erroneous
measurement that should be identified. One possible source of errors is
clock synchronization between sender and receiver to measure delay. This is
not mentioned in the RFC.

Another one could be clock skew when the interval period for measurement is
large such as a day of live broadcast television. Errors could easily creep
in due to clock skew if the end-point machines are not synchronized
sufficiently regularly.

Is there an implementation of this say in some opensource player such as
VLC ? This could make the basis of the RFC stronger.

The authors can elaborate a little bit on the relationship between this
metric and perceived and real user metric as well as dependence on other
factors such as encoder glitches or delays (alluded to in some places in
the draft)

Regards
Vinayak

On Tue, Apr 23, 2013 at 6:46 PM, Benoit Claise <[email-masked]> wrote:

> Hi Dan,
>
> According to https://docs.google.com/**spreadsheet/ccc?key=**
> 0AmKrqWIOBsprdGZqMnB6dmx5bFJvV**Uhta3VLSjl3SkE#gid=0<https://docs.google.com/spreadsheet/ccc?key=0AmKrqWIOBsprdGZqMnB6dmx5bFJvVUhta3VLSjl3SkE#gid=0>
> ,
>>> From: [email-masked] [mailto:pm-dir-bounces@ietf.**org<[email-masked]>]
>>> On Behalf
>>> Of Romascanu, Dan (Dan)
>>> http://www.ietf.org/id/draft-**ietf-xrblock-rtcp-xr-jb-10.txt<http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-jb-10.txt>
>>>
>>>
>>> Hi PMDIR,
>>>
>>> review of http://www.ietf.org/id/draft-**ietf-xrblock-rtcp-xr-jb-10.txt<http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-jb-10.txt>
>>> **.
>>> ______________________________**_________________
>>> pm-dir mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/**listinfo/pm-dir<https://www.ietf.org/mailman/listinfo/pm-dir>
>>>
>>
>>
> ______________________________**_________________
> pm-dir mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/pm-dir<https://www.ietf.org/mailman/listinfo/pm-dir>
>

---

From: Varun Singh <[email-masked]>
Date: Wed, 24 Apr 2013 08:56:52 +0300

Hi Vinayak,

Thanks for the review, comments are inline. 

Regards,
Varun

----
http://www.netlab.tkk.fi/~varun

On 24.4.2013, at 4.36, Vinayak Hegde <[email-masked]> wrote:

> I reviewed the draft on the basis of RFC 6390 as per Benoit's suggestion and found some missing information.
> The draft makes several references to RFC 3611 which I read to get enough context.
> 
> I am ccing the authors from the mail addresses in the RFC so they can make the suggested changes.
> 

This is a placeholder constant and will be replaced by an 8-bit value assigned by IANA. There are rfc editor notes in the document asking them on replace it on document approval. 

> Section 4.2 
> 
> When Interval duration bit is set, it is not clear how the interval is communicated between the sender and receiver. Need to specify this explicitly.

This is communicated in an additional block defined in rfc6776. I think section 4.2 is missing a reference to that document. 

We'll update the text to appropriately reference the rfc. 

> The security considerations section refers to RFC 3611 which is more generic (though contains sufficient infomation). Only the sections relevant to this draft can be replicated here (such as the reaction to DDoS attacks).
> 
> Overall comments
> According to RFC 6390, there should be some sources of erroneous measurement that should be identified. One possible source of errors is clock synchronization between sender and receiver to measure delay. This is not mentioned in the RFC.
> 

Explicitly, the senders and the receives don't need to be synchronized. This is because we are comparing the inter packet interval at the sender and the inter packet interval at the receiver (of the same packet).  

> Another one could be clock skew when the interval period for measurement is large such as a day
> of live broadcast television. Errors could easily creep in due to clock skew if the end-point machines are not synchronized sufficiently regularly.
> 

Yes this is a possible source of error, as the clock at either endpoint can drift from its own reference clock and introduce errors. 

We can make this more explicit in the text. 

> Is there an implementation of this say in some opensource player such as VLC ? This could make the basis of the RFC stronger.
> 
> The authors can elaborate a little bit on the relationship between this metric and perceived and real user metric as well as dependence on other factors such as encoder glitches or delays (alluded to in some places in the draft)

I think we can add something similar to the following in the introduction of the document:

If packets miss their Playout time, they are discarded from the queue and depending on the concealment method will affect the QoE. 

>> According to https://docs.google.com/spreadsheet/ccc?key=0AmKrqWIOBsprdGZqMnB6dmx5bFJvVUhta3VLSjl3SkE#gid=0,

---

From: Qin Wu <[email-masked]>
Date: Wed, 24 Apr 2013 07:46:02 +0000

Hi,Vinayak:
Thank for your thoughtful review, please see my reply inline below.

Regards!
-Qin

From: Vinayak Hegde [mailto:[email-masked]]
Sent: Wednesday, April 24, 2013 9:36 AM
To: Benoit Claise
Cc: Romascanu, Dan (Dan); [email-masked]; [email-masked]; Gonzalo Camarillo; [email-masked]; [email-masked]; Qin Wu
Subject: Re: [pm-dir] request for an RFC 6390 review of http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-jb-10.txt

Section 4.2

What is NJB - First occurence of an acronym and it is not defined.
[Qin]:  Just to clarify, "NJB" is just used to stand for a block type value. It is not necessary to define it since "NJB" will be replaced with IANA allocated value when this draft has been published.
See IANA allocation for new block type :
http://www.iana.org/assignments/rtcp-xr-block-types/rtcp-xr-block-types.xml
BTW:  NJB is the abbreviation of Network Jitter Buffer.

Section 4.2

When Interval duration bit is set, it is not clear how the interval is communicated between the sender and receiver. Need to specify this explicitly.
[Qin]: Just to clarify, this block will be sent together with measurement information block defined in [RFC6776]. Measurement information block will carry
Measurement interval.
See section 4, 1st paragraph:
"
  Instances of this Metrics Block refer by
   SSRC to the separate auxiliary Measurement Information block
   [RFC6776] which describes the measurement interval in use.  This
   Metrics Block relies on the measurement interval in the Measurement
   Information block indicating the span of the report and MUST be
   sent in the same compound RTCP packet as the measurement information
   block.  If the measurement interval is not received in the same
   compound RTCP packet as this Metrics Block, this Metrics Block MUST
   be discarded.
"

Security considerations section
The security considerations section refers to RFC 3611 which is more generic (though contains sufficient infomation). Only the sections relevant to this draft can be replicated here (such as the reaction to DDoS attacks).
[Qin]: Just to clarify, regarding reaction to DDos attack, it has already been clarified in the last paragraph of section 7,RFC3611. It said:
"
   The extensions also make certain denial of service attacks easier.
   This is because of the potential to create RTCP packets much larger
   than average with the per packet reporting capabilities of the Loss
   RLE, Duplicate RLE, and Timestamp Report Blocks.  Because of the
   automatic bandwidth adjustment mechanisms in RTCP, if some session
   participants are sending large RTCP packets, all participants will
   see their RTCP reporting intervals lengthened, meaning they will be
   able to report less frequently.  To limit the effects of large
   packets, even in the absence of denial of service attacks,
   applications SHOULD place an upper limit on the size of the XR report
   blocks they employ.  The "thinning" techniques described in Section<http://tools.ietf.org/html/rfc3611#section-4.1>
   4.1<http://tools.ietf.org/html/rfc3611#section-4.1> permit the packet-by-packet report blocks to adhere to a
   predefined size limit.
"
So since we reference RFC3611, section 7, I believe the current text in the security consideration section suffices.

Overall comments

Another one could be clock skew when the interval period for measurement is large such as a day of live broadcast television. Errors could easily creep in due to clock skew if the end-point machines are not synchronized sufficiently regularly.

[Qin]: We did add a statement in the last paragraph of section 3.1 to say:
"
   Note that this idealized implementation assumes that the sender's RTP
   clock is synchronized to the clock in the receiver which is used to
   timestamp packet arrivals.  If there is no such inherent
   synchronization, the system may need to use an adaptive jitter buffer
   or other techniques to ensure reliable reception.
"
I think this statement suffices and we don't need to deep dig into it to get the bottom since that is not the focus of this draft.

Is there an implementation of this say in some opensource player such as VLC ? This could make the basis of the RFC stronger.
[Qin]: We have already mentioned fixed jitter buffer implementation and adaptive jitter buffer implementation from the general perspective in this draft, but not give specific example like VLC
Since In practice jitter buffer implementations vary considerably.

The authors can elaborate a little bit on the relationship between this metric and perceived and real user metric as well as dependence on other factors such as encoder glitches or delays (alluded to in some places in the draft)
[Qin]: I think it is better to focus on the terminal related factors we defined in this draft, i.e., jitter buffer metrics, rather than mention other factors like encoder glitches, but you are right, we may
Need some text to clarify the relationship between this metric and QoE metrics. I can add a sentence at the end of section 1.4 to say:
"
Note that terminal-related factors are used as
inputs to calculate QoE metrics.
"

Regards
Vinayak

On Tue, Apr 23, 2013 at 6:46 PM, Benoit Claise <[email-masked]<mailto:[email-masked]>> wrote:
Hi Dan,

Regards,

Hi PM-DIR,

Thanks and Regards,

Dan

-----Original Message-----
From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf
Of Romascanu, Dan (Dan)
Sent: Monday, April 08, 2013 5:14 PM
To: [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; Gonzalo Camarillo
Subject: [pm-dir] request for an RFC 6390 review of
http://www.ietf.org/id/draft-ietf-xrblock-rtcp-xr-jb-10.txt

Hi PMDIR,

Thanks and Regards,

Dan

_______________________________________________
pm-dir mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/pm-dir

_______________________________________________
pm-dir mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/pm-dir

---

From: Qin Wu <[email-masked]>
Date: Wed, 24 Apr 2013 08:31:09 +0000

>>The authors can elaborate a little bit on the relationship between this metric and perceived and real user metric as well as dependence on other factors such as encoder glitches >>or delays (alluded to in some places in the draft)

>If packets miss their Playout time, they are discarded from the queue and depending on the concealment method will affect the QoE.

[Qin]: I think section 3 1st paragraph and second paragraph have already talked about this. It said:
“
Section 3.1
  If packets are delayed
   excessively by the network they may be discarded if they miss their
   playout time.

   Overall user perceived delay = network round trip delay + local
   (jitter buffer (nominal) delay + encoder serialization delay) +
   remote (jitter buffer (nominal) delay + encoder serialization delay)

”
We may not need to emphasize or repeat this in the introduction section. Regarding this comment from Vinayak, I think we can propose to add some text in the section 1.4 to clarify the relationship between the metrics defined In this draft and end user QoE, I have given my proposed change in the separate email.

Regards
Vinayak

Regards,

Hi PM-DIR,

Thanks and Regards,

Dan

-----Original Message-----

Hi PMDIR,

Thanks and Regards,

Dan