Subject: Re: [rtcweb] [xrblock] FW: I-DAction:draft-alvestrand-rtcweb-stats-registry-00.txt

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Thu, 27 Sep 2012 01:20:30 +0200

> I am happy to see my comments being forwarded to [email-masked] if
they
> are interested and want to more input from XRBLOCK WG. However I am
not
> sure how much of their work is related to XRBLOCK work? Do they only
> look for some basic metrics obtained from SR/RR. But I agree with you
> consistency between the metrics used by RTCP XR and RTCWEB is a good
> thing.

The work on "rtcweb-stats" is related to XRBLOCK in my opinon.
At least the "basic metrics from SR/RR" with respect to packet loss are controversial and could be misleading, and should be replaced by correspondent XR performance metric types.
The deficiency of some of these basic metrics was one reason to start work on extension reports (XR, and former HR).

Albrecht

________________________________________
Von: [email-masked] [[email-masked]] im Auftrag von Romascanu, Dan (Dan) [[email-masked]]
Gesendet: Mittwoch, 26. September 2012 11:57
An: Qin Wu; [email-masked]
Betreff: Re: [xrblock] FW:      I-DAction:draft-alvestrand-rtcweb-stats-registry-00.txt

Hi Qin,

If you subscribed to the rtcweb list it would be best that you forward
directly your comments there - does this seem OK?

Dan

> -----Original Message-----
> From: Qin Wu [mailto:[email-masked]]
> Sent: Wednesday, September 26, 2012 3:45 AM
> To: Romascanu, Dan (Dan); [email-masked]
> Subject: Re: [xrblock] FW: I-DAction:draft-alvestrand-rtcweb-stats-
> registry-00.txt
>
> Hi, Dan:
> I am happy to see my comments being forwarded to [email-masked] if
they
> are interested and want to more input from XRBLOCK WG. However I am
not
> sure how much of their work is related to XRBLOCK work? Do they only
> look for some basic metrics obtained from SR/RR. But I agree with you
> consistency between the metrics used by RTCP XR and RTCWEB is a good
> thing.
> It will be great to see metrics used by RTCP XR can be used in RTCWEB
as
> well.
>
> BTW: I have subscribed to rtcweb list and am interest to join the
> discussion as I said and expected.
>
> Regards!
> -Qin
> ----- Original Message -----
> From: "Romascanu, Dan (Dan)" <[email-masked]>
> To: "Qin Wu" <[email-masked]>; <[email-masked]>
> Sent: Tuesday, September 25, 2012 5:45 PM
> Subject: Re: [xrblock] FW: I-DAction:draft-alvestrand-rtcweb-stats-
> registry-00.txt
>
>
> > Hi Qin,
> >
> > Would you like me to forward your comments to [email-masked]? You
can
> > also subscribe to the list to follow the responses and participate
in
> > the discussions, but I should warn you that performance statistics
are
> a
> > fraction of the many problems this very active WG have on their
hands.
> >
> > Dan
> >
> >
> >
> >> -----Original Message-----
> >> From: Qin Wu [mailto:[email-masked]]
> >> Sent: Tuesday, September 25, 2012 4:17 AM
> >> To: Romascanu, Dan (Dan); [email-masked]
> >> Subject: Re: [xrblock] FW: I-D
Action:draft-alvestrand-rtcweb-stats-
> >> registry-00.txt
> >>
> >> Interesting -00 draft.
> >> Have a quick look at this draft, I have the folowing comments:
> >> 1. Jitter should also refers to PDV draft since PDV draft provide
> >> another two measurement methods with different measurement
precision
> >> which are different from "inter-arrival jitter" described in
RFC3550.
> >> In the early version of PDV draft, we also accomodate
"inter-arrival
> >> jitter" and support reporting inter-arrival jitter. However we
remove
> >> inter-arrival jitter from this draft based on WG review since
inter-
> >> arrival jitter use different measurement unit and has wider range.
> >> 2.ReceivedPacketCount
> >> Loss RLE report block can be used to report the packet received. It
> is
> >> more staightfoward to calculate ReceivedPacket Count based on  Loss
> > RLE
> >> report block defined in RFC3611 rather than based on MIB
information
> >> defined in RFC2959. Also in some case, WebClient may not support
> SNMP.
> >> So I suggest to refer to section 4.1, RFC3611.
> >> 3. RecevedOctetCount
> >> Same as RecevedPacketCount. I think Loss RLE report block also can
be
> >> used to calculate RecevedOctetCount.
> >> It is better to refer to section 4.1, RFC3611.
> >> 4. Section 4.2 Transport variable
> >> It is not clear how Transport variable is relate to Variables from
> > basic
> >> RTCP SR/RR defined in section 4.1, how transport vaiable is
> > calculated,
> >> e.g., what's the relationship between ReceivedPackets and
> >> ReceivedPacketcount? Are they the same variable?
> >> Is 4 transport variable defined in section 4.2 sufficient to
measure
> > the
> >> performance of media stream? Why not refer to some variable defined
> in
> >> XRBLOCK WG?
> >>
> >> Regards!
> >> -Qin
> >>
> >>  ----- Original Message -----
> >> From: "Romascanu, Dan (Dan)" <[email-masked]>
> >> To: <[email-masked]>
> >> Sent: Monday, September 24, 2012 7:48 PM
> >> Subject: [xrblock] FW: I-D Action:draft-alvestrand-rtcweb-stats-
> >> registry-00.txt
> >>
> >>
> >> >I believe that this I-D (individual submission targeting the
RTCWEB
> > WG)
> >> > is of interest for some of the folks who participate in XRBLOCK.
> >> > Reaching consistency between the metrics used by RTCP XR and
RTCWEB
> >> > (when relevant) seems to be desirable.
> >> >
> >> > Regards,
> >> >
> >> > Dan
> >> >
> >> >
> >> >
> >> >
> >> > -----Original Message-----
> >> > From: [email-masked]
> >> > [mailto:[email-masked]] On Behalf Of
> >> > [email-masked]
> >> > Sent: Monday, September 24, 2012 12:17 PM
> >> > To: [email-masked]
> >> > Subject: I-D Action:
draft-alvestrand-rtcweb-stats-registry-00.txt
> >> >
> >> >
> >> > A New Internet-Draft is available from the on-line
Internet-Drafts
> >> > directories.
> >> >
> >> >
> >> > Title           : A Registry for WebRTC statistics identifiers
> >> > Author(s)       : Harald T. Alvestrand
> >> > Filename        : draft-alvestrand-rtcweb-stats-registry-00.txt
> >> > Pages           : 8
> >> > Date            : 2012-09-24
> >> >
> >> > Abstract:
> >> >   This memo describes a registration procedure for statistics
> >> >   identifiers used in the WebRTC Javascript API to access
> > statistical
> >> >   information about a PeerConnection.
> >> >
> >> >   It also gives some identifiers that will, when approved, form
the
> >> >   initial content of this registry.
> >> >
> >> >
> >> >
> >> > The IETF datatracker status page for this draft is:
> >> > https://datatracker.ietf.org/doc/draft-alvestrand-rtcweb-stats-
> >> registry
> >> >
> >> > There's also a htmlized version available at:
> >> >
http://tools.ietf.org/html/draft-alvestrand-rtcweb-stats-registry-
> 00
> >> >
> >> >
> >> > Internet-Drafts are also available by anonymous FTP at:
> >> > ftp://ftp.ietf.org/internet-drafts/
> >> >
> >> > _______________________________________________
> >> > I-D-Announce mailing list
> >> > [email-masked]
> >> > https://www.ietf.org/mailman/listinfo/i-d-announce
> >> > Internet-Draft directories: http://www.ietf.org/shadow.html or
> >> > ftp://ftp.ietf.org/ietf/1shadow-sites.txt
> >> > _______________________________________________
> >> > xrblock mailing list
> >> > [email-masked]
> >> > https://www.ietf.org/mailman/listinfo/xrblock
> > _______________________________________________
> > xrblock mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/xrblock
_______________________________________________
xrblock mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/xrblock

---

From: Qin Wu <[email-masked]>
Date: Thu, 27 Sep 2012 09:34:55 +0800

Albrecht, you have a good point. So the question to rtcweb is 
for rtcweb applications, are the basic metrics for transport impairments provided in RTCP SR and RR packets sufficient?
What kind of metrics are basically needed for RTCWeb application? e.g., are the metrics like packet loss, delay, jitter basic metrics 
for rtcweb applications? 

Regards!
-Qin
----- Original Message ----- 
From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
To: "Romascanu, Dan (Dan)" <[email-masked]>; "Qin Wu" <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, September 27, 2012 7:20 AM
Subject: AW: [xrblock] FW:I-DAction:draft-alvestrand-rtcweb-stats-registry-00.txt

they
not

Albrecht

Hi Qin,

Dan

they
not
as
can
in
are
hands.
precision
RFC3550.
"inter-arrival
remove
inter-
information
be
measure
RTCWEB
RTCWEB
Internet-Drafts
the

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 28 Sep 2012 13:31:50 +0200

> The deficiency of some of these basic metrics was one reason to start work on extension reports (XR, and former HR).
Indeed. The reason I started the document with packet counts and packet 
loss from SR/RR was that I thought these would be uncontroversial and 
unambiguous; I'm seeking guidance from XR people on what other metrics 
are well known enough - and implemented widely enough - that it makes 
sense to include them.

(For reference, the WebRTC codebase seems to decode block type 7 and no 
other block type....)

---

From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
Date: Sat, 29 Sep 2012 01:51:05 +0200

Harald,

other block type....)

BT=7 would be the "VoIP Metrics Report Block" (RFC 3611).

Whereas BT=6 would be the "Statistics Summary Report Block", related also to some improved SR/BR metric.

Regards,
Albrecht

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Harald Alvestrand
Sent: Freitag, 28. September 2012 13:32
To: [email-masked]
Subject: Re: [rtcweb] [xrblock] FW: I-DAction:draft-alvestrand-rtcweb-stats-registry-00.txt

_______________________________________________
rtcweb mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Qin Wu <[email-masked]>
Date: Sat, 29 Sep 2012 11:40:26 +0800

Hi,Harald:
----- Original Message ----- 
From: "Harald Alvestrand" <[email-masked]>
To: <[email-masked]>
Sent: Friday, September 28, 2012 7:31 PM
Subject: Re: [rtcweb] [xrblock]FW: I-DAction:draft-alvestrand-rtcweb-stats-registry-00.txt

> (For reference, the WebRTC codebase seems to decode block type 7 and no 

[Qin]: For jitter, you may refer to both RFC3550 section 6.4.1 for inter-arrival jitter and
draft-ietf-xrblock-rtcp-xr-pdv for "2-point PDV" and "Absolute Packet Delay Variation 2 (MAPDV2) ".
Since jitter is also known as Packet Delay Variation.

For other metrics, I think you may also include round trip delay metric by referring to draft-ietf-rtcp-xr-delay
include discard metric by referring to draft-ietf-rtcp-xr-discard.

For measurement timing for these XR blocks, you should refer to draft-ietf-xrblock-meas-identity.

If you look into end device related metrics, you may look at jitter buffer metric,
concealment seconds metrics, loss concealment metrics developed in XRBLOCK WG, 
these metrics also affect real time appliation quality.

---

From: Qin Wu <[email-masked]>
Date: Sat, 29 Sep 2012 11:46:55 +0800

Hi,
----- Original Message ----- 
From: "Schwarz, Albrecht (Albrecht)" <[email-masked]>
To: "Harald Alvestrand" <[email-masked]>; <[email-masked]>
Sent: Saturday, September 29, 2012 7:51 AM
Subject: Re: [rtcweb] [xrblock] FW: I-DAction:draft-alvestrand-rtcweb-stats-registry-00.txt

> BT=7 would be the "VoIP Metrics Report Block" (RFC 3611).

[Qin]: If you look for metrics only for audio stream, you may look at BT =7,
However if you look for metrics not only for audio stream but also for video stream,
you may look at burst gap loss metrics, burst gap discard metrics, Delay metrics,
QoS metrics, Jitter Buffer metrics developed in XRBLOCK WG which are 
decomposed from VOIP metrics Report block but more generic metrics that can be 
applied to both audio and video stream.

---

From: Harald Alvestrand <[email-masked]>
Date: Sat, 29 Sep 2012 11:34:09 +0200

> Whereas BT=6 would be the "Statistics Summary Report Block", related also to some improved SR/BR metric.
Yep. I looked in the source code and found code that special cased the 
number 7. (This is part of the open source WebRTC release).

I guess this means there has been customer demand for the VoIP Metrics, 
but no customer demand for the Statistics Summary - which led to my 
query about which XR metrics are widely supported in others' equipment.

                     Harald