Subject: [pm-dir] Benoit Claise's Discuss on draft-ietf-xrblock-rtcp-xr-burst-gap-discard-13: (with DISCUSS and COMMENT)

From: "Benoit Claise" <[email-masked]>
Date: Wed, 24 Apr 2013 05:53:28 -0700

Benoit Claise has entered the following ballot position for
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-13: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to http://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

No problem with the publication of this document. 
However, before doing so, I have 2 points I want to address: I'm missing
something, and I don't know what.

1. 
In this sentence, I wonder which jitter calculation you were speaking
about:
   The new block type
   supports the reporting of the proportion of packets discarded by the
   receiver due to jitter.  The discards during discard bursts are
   reported, together with the number of bursts.  This block is intended
   to be used in conjunction with [DISCARD] which provides the total
   packets discarded, and on which this block therefore depends.
   However the metric in [DISCARD] may be used independently of the
   metrics in this block.

I know of the two methods [RFC 5481]
      4.1. IPDV: Inter-Packet Delay Variation ........................11
      4.2. PDV: Packet Delay Variation ...............................11
Or maybe the results are independent of the jitter calculation method, in
which case you want to clearly mention it.
Maybe it's explained with this sentence, but I don't know how to
interpret it:

   To accommodate the range of jitter buffer
   algorithms and packet discard logic that may be used by implementors,
   the method used to distinguish between bursts and gaps may be an
   equivalent method to that defined in[RFC3611].

So it "may be an equivalent method to that defined in[RFC3611]."
What if it's not the case?

2.
You define "Discarded" as:
      A packet that arrives within
      this time window but is too early or late to be played out or
      thrown away before playout due to packet duplication or redundancy
      shall be regarded as discarded. 

I wonder: what's the point to include the discarded duplicated packet.
Those don't affect the quality.
On top of that, it's inconsistent with "Discard" definition of RFC 3611.
You wrote in the draft:
   The definitions of Burst, Gap, Loss and Discard are consistent with
   definitions in [RFC3611].

And RFC 3611 Discard mentions:
   discard rate: 8 bits
         The fraction of RTP data packets from the source that have been
         discarded since the beginning of reception, due to late or
         early arrival, under-run or overflow at the receiving jitter
         buffer.  This value is expressed as a fixed point number with
         the binary point at the left edge of the field.  It is
         calculated by dividing the total number of packets discarded
         (excluding duplicate packet discards) by the total number of
         packets expected, multiplying the result of the division by
         256, limiting the maximum value to 255 (to avoid overflow), and
         taking the integer part.

So you want to report the "Packets discarded in bursts", i.e. "The total
number of packets discarded during discard bursts."

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

EDITORIAL
-
   This block provides information on transient IP problems.  Burst/Gap
   metrics are typically used in Cumulative reports, however they also
   may be used in Interval reports. 

Cumulative -> cumulative 
Interval -> interval 

Unless those are definitions ... in which case they need a reference.
Reading further, I understand that you refer to:
         I=10: Interval Duration - the reported value applies to the
         most recent measurement interval duration between successive
         metrics reports.

         I=11: Cumulative Duration - the reported value applies to the
         accumulation period characteristic of cumulative measurements. 

You need a reference
OLD:
   This block provides information on transient IP problems.  Burst/Gap
   metrics are typically used in Cumulative reports, however they also
   may be used in Interval reports. 

NEW:
   This block provides information on transient IP problems.  Burst/Gap
   metrics are typically used in Cumulative Duration reports, however
they also
   may be used in Interval Duration reports (see the Interval Metric flag
in section 3.2).

Note: there are many instances of capitalized words, for which I'm not
too sure if we deal with a definition, or if it's just a bad habit to
capitalize term in this industry. Example

  If Voice Activity Detection is used, the Burst and Gap Duration shall
   be determined as if silence packets had been sent, i.e. a period of
   silence in excess of Gmin packets will terminate a burst condition.

   The recommended value for the threshold Gmin in [RFC3611] results in
   a Burst being a period of time during which the call quality is
   degraded to a similar extent to a typical Pulse-Code Modulation(PCM)
   Severely Errored Second.

Please be consistent.

-
"The definitions of Burst, Gap, Loss and Discard are consistent with
definitions in [RFC3611]."
This sentence should be in the terminology section, and not section 1.1

-

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |    BT=NBGD    | I |   resv.   |      block length = 3         |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                        SSRC of Source                         |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |   Threshold   |         Packets Discarded in Bursts           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |       Total Packets expected in bursts        |   Reserved.   |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Bursts and burst in the same picture. Pick one.

- To finish on the positive note, thanks for the RFC6390 template.

---

From: Gonzalo Camarillo <[email-masked]>
Date: Wed, 24 Apr 2013 21:15:06 +0300

Hi Benoit,

I am glad you are happy with the use of the RFC6390 template this time ;-)

I won't be able to join the telechat tomorrow. So, let's try and resolve
your discuss via email (Qin will answer you as soon as he is back online).

Cheers,

Gonzalo

---

From: Benoit Claise <[email-masked]>
Date: Thu, 25 Apr 2013 11:57:56 +0200

Dear all,

In order to speed up the process, and to avoid exchanging many emails, I 
had a quick call with Qin. An efficient call.
See in line.
> Hi, Benoit:
> Thank for your valuable review, please see my reply inline below.
>
> Regards!
> -Qin
> -----Original Message-----
> From: Benoit Claise [mailto:[email-masked]]
> Sent: Wednesday, April 24, 2013 8:53 PM
> To: The IESG
> Cc: [email-masked]; [email-masked]; [email-masked]
> Subject: Benoit Claise's Discuss on draft-ietf-xrblock-rtcp-xr-burst-gap-discard-13: (with DISCUSS and COMMENT)
>
> [Qin]: The reporting results is independent of jitter calculation method.
> We may add this sentence at the end of the second sentence in the paragraph you are referring to.
> However I doubt we need to add something to clarify this, since burst gap discards metrics
> have no any dependency to jitter metrics or packet delay variation metrics and will not be affected by
> jitter metrics or how jitter is calculated. Am I right?
[Benoit] Now understood.
There is a difference between the jitter on the wire (PDV versus IPDV), 
and the de-jitter buffer algorithm.
This draft focuses on the latter.
Here is a proposal.
OLD:

    The new block type
    supports the reporting of the proportion of packets discarded by the
    receiver due to jitter.  The discards during discard bursts are
    reported, together with the number of bursts.

NEW:

    The new block type
    supports the reporting of the proportion of packets discarded by the
    jitter buffer at the receiver, i.e. by the jitter buffer and packet discard logic
    algorithms. The discards during discard bursts are
    reported, together with the number of bursts.

> [Qin]: In the definition of Burst Gap in the section 2.1, we also said:
> "
>     Bursts and Gaps
>
>        The terms Burst and Gap are used in a manner consistent with that
>        of RTCP XR [RFC3611].
> "
> To make two places consistent, I propose to change the "may be" in the
> place you are referring to "is".
> OLD TEXT:
> "
> "
> NEW TEXT:
> "
>     the method used to distinguish between bursts and gaps is an
> "

Discussing with Qin, the intend of this paragraph is that any algorithm for jitter buffer and packet discard logic can be used at the condition that the bursts and gaps concepts are equivalent to the ones defined in [RFC 3611].

Proposal:
OLD TEXT:

NEW TEXT:

    Any algorithm for jitter buffer and packet discard logic
    can be used at the condition that the bursts and gaps
    concepts are equivalent to the ones defined in [RFC 3611].

It would even better (up to you) to put this text next to "The definitions of Burst, Gap, Loss and Discard are consistent with definitions in [RFC3611]."
Important note: see the DISCUSS part 2, because this sentence will probably have to changed.

NEW NEW TEXT:
    The definitions of Burst, Gap, Loss and Discard are consistent with
    definitions in [RFC3611]. This implies that any algorithm for jitter buffer
    and packet discard logic can be used at the condition that the bursts
    and gaps concepts are equivalent to the ones defined in [RFC 3611].

In the introduction, you should add a sentence such as (this was really a source of confusion for me):

    Reporting the specific jitter buffer and/or packet discard logic algorithms is out of scope of this draft.

> [Qin]: There was WG consensus to add this use case, I remembered it was proposed
> By Varun and Other proponent are Alan and Colin.
> Here is the pointer to one relevant discussion occured on the list when this draft was in WGLC.
> http://www.ietf.org/mail-archive/web/xrblock/current/msg00572.html
> http://www.ietf.org/mail-archive/web/xrblock/current/msg00573.html
It was discussed and agreed in the WG, fine with me.
See below.
> [Qin]: It was WG consensus to report each discard type rather report any combination of discard type using one discard count metric block.
> See the relevant discussion on the list:
> http://www.ietf.org/mail-archive/web/xrblock/current/msg00617.html

>
> So Discards draft allow reporting any combination of discard types in each
> reporting interval by including several Discard Count Metric
> Report Blocks in a single RTCP XR packet.
> So it will be not problem you want to report the total number of packets discards during discard bursts.
Here is my issue.
One one side, I see

However, this is not correct because you take "duplication" into account 
in the Discard (actually Discarded) definition in this draft

       A packet that arrives within
       this time window but is too early or late to be played out or
       thrown away before playout due to packet_duplication_or redundancy
       shall be regarded as discarded.

What I learned from Qin over the phone is that this draft build on the 
Discard definition from RFC3611, but report multiple Discard counters.
And those multiple counters are actually covered in a different draft: 
http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-discard-12 [DISCARD]

    Discard Type (DT): 2bits

       This field is used to identify the discard type used in this
       report block.  The discard type is defined as follows:

          00: Report packet discarded or being thrown away before playout
          due to packets duplication.

          01: Report packet discarded due to too early to be played out.

          10: Report packet discarded due to too late to be played out.

One way or the other, you want to clearly mention that this draft build 
on the Discard definition [RFC 3611], but extend the concept to take 
into account the duplication, and report different counters 
[draft-ietf-xrblock-rtcp-xr-discard-12]

Regards, Benoit

---

From: Qin Wu <[email-masked]>
Date: Thu, 25 Apr 2013 02:36:31 +0000

Hi, Benoit:

Regards!
-Qin
-----Original Message-----
From: Benoit Claise [mailto:[email-masked]] 
Sent: Wednesday, April 24, 2013 8:53 PM
To: The IESG
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Benoit Claise's Discuss on draft-ietf-xrblock-rtcp-xr-burst-gap-discard-13: (with DISCUSS and COMMENT)

1. 

2.

EDITORIAL
-

Cumulative -> cumulative 
Interval -> interval 

[Qin]: Okay.

You need a reference
OLD:

NEW:

[Qin]: Okay.

Please be consistent.

[Qin]: Okay and will fix this.

-

[Qin]: Okay, will move this from section 1.1 to the terminology section.
-

[Qin]: Okay.

---

From: Qin Wu <[email-masked]>
Date: Thu, 25 Apr 2013 11:18:25 +0000

Hi,Benoit:
Thank for your calling and proposed changes, I like to make additional rewording based on your proposed text.
Please see my reply below.

Regards!
-Qin

From: Benoit Claise [mailto:[email-masked]]
Sent: Thursday, April 25, 2013 5:58 PM
To: Qin Wu
Cc: The IESG; [email-masked]; [email-masked]; [email-masked]
Subject: Re: Benoit Claise's Discuss on draft-ietf-xrblock-rtcp-xr-burst-gap-discard-13: (with DISCUSS and COMMENT)

Dear all,

In order to speed up the process, and to avoid exchanging many emails, I had a quick call with Qin. An efficient call.
See in line.

Hi, Benoit:

Regards!

-Qin

-----Original Message-----

To: The IESG

Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>

DISCUSS:

1.

about:

   The new block type

   metrics in this block.

interpret it:

jitter metrics or how jitter is calculated. Am I right?
[Benoit] Now understood.
There is a difference between the jitter on the wire (PDV versus IPDV), and the de-jitter buffer algorithm.
This draft focuses on the latter.
Here is a proposal.
OLD:

   The new block type

   The new block type

[Qin]: Agree, I think jitter buffer algorithms and packet discard logic are still two different things, packet discard logic can be part of jitter buffer, so I made a little bit rewording as follows:

OLD TEXT:

“

   The new block type

”

NEW TEXT:

“

   The new block type

   jitter buffer at the receiver, using packet discard logic according to the jitter buffer algorithms.

   . The discards during discard bursts are

”

"

   Bursts and Gaps

      of RTCP XR [RFC3611].

"

OLD TEXT:

"

"

NEW TEXT:

"

"

Proposal:

OLD TEXT:

NEW TEXT:

[Qin]: Just to clarify, you ask me what the equivalent method is in the call, I revisited the first two paragraph of section  4.7.2 of RFC3611,

"

   A burst is a period during which a high proportion of packets are

   either lost or discarded due to late arrival.  A burst is defined, in

   terms of a value Gmin, as the longest sequence that (a) starts with a

   lost or discarded packet, (b) does not contain any occurrences of

   Gmin or more consecutively received (and not discarded) packets, and

   (c) ends with a lost or discarded packet.

   A gap, informally, is a period of low packet losses and/or discards.

   Formally, a gap is defined as any of the following: (a) the period

   from the start of an RTP session to the receipt time of the last

   received packet before the first burst, (b) the period from the end

   of the last burst to either the time of the report or the end of the

   RTP session, whichever comes first, or (c) the period of time between

   two bursts.

"

I think the equivalent method are referring to the first two paragraph of section 4.7.2 of RFC3611 on what the burst is and what the gap is.

In the draft-ietf-xrblock-rtcp-xr-burst-gap-discard-13, section 2.1, it said:

“

   Bursts and Gaps

      of RTCP XR [RFC3611].  RTCP XR views a RTP stream as being divided

      into bursts, which are periods during which the discard rate is

      high enough to cause noticeable quality degradation (generally

      over 5 percent discard rate), and gaps, which are periods during

      which discarded packets are infrequent and hence quality is

      generally acceptable.

”

So draft-ietf-xrblock-rtcp-xr-burst-gap-discard-13 didn’t change what burst is and what gap is, which(Term Burst and Term Gap) are defined

In RFC3611 and just use burst and gap defined in RFC3611.

So maybe we can do the following change to the paragraph you are referring to:

OLD TEXT:

“

”

NEW TEXT:

“
   To accommodate the range of jitter buffer
   algorithms and packet discard logic that may be used by implementors,
   the method used to distinguish between bursts and gaps shall use an
   equivalent method to that defined in the section 4.7.2 of [RFC3611].

”

It is equivalent to what your proposed and also emphasize this draft use the burst and gap definition in the RFC3611.

NEW NEW TEXT:

[Qin]: Make sense, in this case, I suggest we don’t move  the sentence “The definitions of Burst, Gap, Loss and Discard are consistent with

   definitions in [RFC3611].” from the introduction to the terminology section.

      And I propose the following change to section 1.1 with your proposed text:

OLD TEXT:

“

Section 1.1:

   definitions in [RFC3611].  To accommodate the range of jitter buffer

”

NEW TEXT:

“

Section 1.1:

  definitions in [RFC3611]. To accommodate the range of jitter buffer

   the method used to distinguish between bursts and gaps shall use an

   equivalent method to that defined in the section 4.7.2 of [RFC3611].

   Note that Reporting the specific jitter buffer algorithms and/or

   packet discard logic is out of scope of this draft.

”

2.

You wrote in the draft:

   discard rate: 8 bits

         taking the integer part.

   definitions in [RFC3611]."
However, this is not correct because you take "duplication" into account in the Discard (actually Discarded) definition in this draft

      shall be regarded as discarded.
What I learned from Qin over the phone is that this draft build on the Discard definition from RFC3611, but report multiple Discard counters.
And those multiple counters are actually covered in a different draft: http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-discard-12  [DISCARD]

   Discard Type (DT): 2bits

One way or the other, you want to clearly mention that this draft build on the Discard definition [RFC 3611], but extend the concept to take into account the duplication, and report different counters [draft-ietf-xrblock-rtcp-xr-discard-12]

[Qin]: Agree, I propose to add the following at the end of  the definition of “ Received, Lost and Discarded”

OLD TEXT:

“

Section 2.1

   Received, Lost and Discarded

      A packet shall be regarded as lost if it fails to arrive within an

      implementation-specific time window.  A packet that arrives within

      shall be regarded as discarded.  A packet shall be classified as

      one of received (or OK), discarded or lost.  The metric

      "cumulative number of packets lost" defined in [RFC3550] reports a

      count of packets lost from the media stream (single SSRC within

      single RTP session).  Similarly the metric "number of packets

      discarded" defined in [DISCARD] reports a count of packets

      discarded from the media stream (single SSRC within single RTP

      session) arriving at the receiver.  Another metric defined in

      [RFC5725] is available to report on packets which are not

      recovered by any repair techniques which may be in use.

”

NEW TEXT:

“

Section 2.1

Received, Lost and Discarded

      A packet shall be regarded as lost if it fails to arrive within an

      implementation-specific time window.  A packet that arrives within

      shall be regarded as discarded.  A packet shall be classified as

      one of received (or OK), discarded or lost.  The metric

      "cumulative number of packets lost" defined in [RFC3550] reports a

      count of packets lost from the media stream (single SSRC within

      single RTP session).  Similarly the metric "number of packets

      discarded" defined in [DISCARD] reports a count of packets

      discarded from the media stream (single SSRC within single RTP

      session) arriving at the receiver.  Another metric defined in

      [RFC5725] is available to report on packets which are not

      recovered by any repair techniques which may be in use.

      Note that the term discard defined here build on the Discard

      definition in [RFC 3611], but extend the concept to take into

      account the packet duplication, and report different types of discard count  metrics

      [draft-ietf-xrblock-rtcp-xr-discard-12].

”

Regards, Benoit

---

From: Benoit Claise <[email-masked]>
Date: Thu, 25 Apr 2013 17:26:00 +0200

Hi Qin,
> *From:*Benoit Claise [mailto:[email-masked]]
> *Sent:* Thursday, April 25, 2013 5:58 PM
> *To:* Qin Wu
> *Cc:* The IESG; [email-masked]; 
> [email-masked]; 
> [email-masked]
> *Subject:* Re: Benoit Claise's Discuss on 
> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-13: (with DISCUSS and 
> COMMENT)
>
> Dear all,
>
> In order to speed up the process, and to avoid exchanging many emails, 
> I had a quick call with Qin. An efficient call.
> See in line.
>
>     Hi, Benoit:
>
>     Cc:[email-masked]  <mailto:[email-masked]>;[email-masked]  <mailto:[email-masked]>;[email-masked]  <mailto:[email-masked]>
>
>     Please refer tohttp://www.ietf.org/iesg/statement/discuss-criteria.html
>
> There is a difference between the jitter on the wire (PDV versus 
> IPDV), and the de-jitter buffer algorithm.
[Benoit] Perfect.
>    the method used to distinguish between bursts and gaps *shall use* an
>
> *NEW NEW TEXT:*
> *    The definitions of Burst, Gap, Loss and Discard are consistent with*
> *    definitions in [RFC3611]. This implies that any algorithm for jitter buffer*
> *    and packet discard logic can be used at the condition that the bursts*
> *    and gaps concepts are equivalent to the ones defined in [RFC 3611].*
> *  *
> *In the introduction, you should add a sentence such as (this was really a source of confusion for me):*
> *  *
> *    Reporting the specific jitter buffer and/or packet discard logic algorithms is out of scope of this draft.*
>   
>     *Note that Reporting the specific jitter buffer algorithms and/or*
> *    packet discard logic is out of scope of this draft.*
[Benoit] perfect.
> **
> However, this is not correct because you take "duplication" into 
> account in the Discard (actually Discarded) definition in this draft
>
> One way or the other, you want to clearly mention that this draft 
> build on the Discard definition [RFC 3611], but extend the concept to 
> take into account the duplication, and report different counters 
> *       Note that the term discard defined here build on the Discard*
> *       definition in [RFC 3611], but extend the concept to take into*
> *       account the packet duplication, and report different types of discard count  metrics*
> *       [draft-ietf-xrblock-rtcp-xr-discard-12].*
> ”
>
>
Great.

Post and I'll clear my DISCUSS.

Regards, Benoit

---

From: Qin Wu <[email-masked]>
Date: Fri, 26 Apr 2013 02:58:58 +0000

Hi, Benoit:
Thanks for your confirmation.
Here is the diff:

http://www.ietf.org/rfcdiff?url2=draft-ietf-xrblock-rtcp-xr-burst-gap-discard-14

Regards!
-Qin
From: Benoit Claise [mailto:[email-masked]]
Sent: Thursday, April 25, 2013 11:26 PM
To: Qin Wu
Cc: [email-masked]; [email-masked]; [email-masked]; The IESG
Subject: Re: Benoit Claise's Discuss on draft-ietf-xrblock-rtcp-xr-burst-gap-discard-13: (with DISCUSS and COMMENT)

Hi Qin,
Hi,Benoit:

Regards!
-Qin

From: Benoit Claise [mailto:[email-masked]]
Sent: Thursday, April 25, 2013 5:58 PM
To: Qin Wu
Cc: The IESG; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: Benoit Claise's Discuss on draft-ietf-xrblock-rtcp-xr-burst-gap-discard-13: (with DISCUSS and COMMENT)

Dear all,

Hi, Benoit:

Regards!

-Qin

-----Original Message-----

To: The IESG

DISCUSS:

1.

about:

   The new block type

   metrics in this block.

interpret it:

   The new block type

   The new block type

OLD TEXT:

“

   The new block type

”

NEW TEXT:

“

   The new block type

”

[Benoit] Perfect.

"

   Bursts and Gaps

      of RTCP XR [RFC3611].

"

OLD TEXT:

"

"

NEW TEXT:

"

"

Proposal:

OLD TEXT:

NEW TEXT:

"

   two bursts.

"

“

   Bursts and Gaps

      generally acceptable.

”

OLD TEXT:

“

”

NEW TEXT:

“

”

NEW NEW TEXT:

OLD TEXT:

“

Section 1.1:

”

NEW TEXT:

“

Section 1.1:

”

2.

You wrote in the draft:

   discard rate: 8 bits

         taking the integer part.

   Discard Type (DT): 2bits

OLD TEXT:

“

Section 2.1

”

NEW TEXT:

“

Section 2.1

”

Great.

Regards, Benoit

Regards, Benoit