Subject: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

From: Magnus Westerlund <[email-masked]>
Date: Wed, 27 Jun 2012 09:36:42 +0200

WG,

We had a discussion at the interim if RTP Retransmission is to be
considered REQUIRED or RECOMMENDED to implement. I would like to see if
we can first have some discussion on this topic before moving on to see
if we can get a consensus here on the mailing list.

Please provide your views on this topic.

Cheers

Magnus Westerlund
(As Chair and document editor)

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: "Fabio Pietrosanti (naif)" <[email-masked]>
Date: Wed, 27 Jun 2012 09:42:00 +0200

If you use RTP over a Mobile Networks subject to strong packet loss
(especially 3G or area plenty of many mobile devices), the RTP
retransmission became a requirement.

If there is RTCP support within RTCWeb, we may define some "quality
threshold" after that the RTP Retransmition became REQUIRED.

Fabio

On 6/27/12 9:36 AM, Magnus Westerlund wrote:
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 27 Jun 2012 07:34:00 -0700

> retransmission became a requirement.

[BA] The usefulness of retransmission will depend on the RTT.  There are quite a few
mobile networks that suffer from buffer bloat to the point where retransmission is quite useless. 
Also, retransmission (and FEC) both increase bandwidth usage which is a problem for congestion 
control.

---

From: Cameron Byrne <[email-masked]>
Date: Wed, 27 Jun 2012 09:19:00 -0700

I think Bernard noted this as well, but it is not so much buffer
bloat... but the issue is indeed latency, not loss.  Consequently,
re-transmission not productive.

> retransmission became a requirement.

Please cite the packet loss you are referring to.

Here is one imperfect data source,
http://pam2012.ftw.at/papers/PAM2012paper6.pdf.

I say it is imperfect because it relies on SACK, which can infer
packet loss, but frequently for mobile it just infers delay.  So, the
mobile packet loss is overstated.

Mobile (UMTS, ...) networks have extremely robust forward error
correct and re-transmission capabilities at layer 1 and layer 2, this
induces delay but ensure packet delivery.

See: http://en.wikipedia.org/wiki/Hybrid_automatic_repeat_request

Attempting to optimize for packet delivery at higher levels (RTP
re-transmission...)  leads to various counter productive race
conditions, since the RAN (radio access network) is designed for 100%
packet delivery.

CB

---

From: "Fabio Pietrosanti (naif)" <[email-masked]>
Date: Wed, 27 Jun 2012 19:04:53 +0200

> Please cite the packet loss you are referring to.

My company does Mobile VoIP Clients for Blackberry, Nokia S60, iPhone,
Android and i know that Packet Loss is the *major cause of bad quality*
reported by my customer.

Given empirical practical experience i have from my customer:
 - In crowded areas in 3G
 - Within certain building (both 2G and 3G)

I don't know which kind of retransmission schemas you are referring to,
but i am sure that packet loss of UDP packets in a mobile network exists
and it's a concrete issue.

Fabio

---

From: "Roy, Radhika R CIV (US)" <[email-masked]>
Date: Wed, 27 Jun 2012 18:14:26 +0000

Hi, all:

Real-time protocols (Audio/Video payloads for two-way conversations) over UDP/IP were being developed based on the single thing: Retransmission delays are unacceptable (more over congestion problems especially for the high-bandwidth videos).

If this fundamental performance characteristic for the Real-time protocols (Audio/Video payloads for two-way conversations) need to be changed, the experimental results with lots of actual measurements must be produced to the technical communities.

Otherwise, someone may make some subjective statements here and there without any actual measurements will only provide evidences what those folks do not know what they are actually talking about creating technical confusions.

Best regards,
Radhika

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Cameron Byrne
Sent: Wednesday, June 27, 2012 12:19 PM
To: Fabio Pietrosanti (naif)
Cc: [email-masked]
Subject: Re: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

I think Bernard noted this as well, but it is not so much buffer bloat... but the issue is indeed latency, not loss.  Consequently, re-transmission not productive.

I say it is imperfect because it relies on SACK, which can infer packet loss, but frequently for mobile it just infers delay.  So, the mobile packet loss is overstated.

Mobile (UMTS, ...) networks have extremely robust forward error correct and re-transmission capabilities at layer 1 and layer 2, this induces delay but ensure packet delivery.

Attempting to optimize for packet delivery at higher levels (RTP
re-transmission...)  leads to various counter productive race conditions, since the RAN (radio access network) is designed for 100% packet delivery.

CB

>> considered REQUIRED or RECOMMENDED to implement. I would like to see 
>> if we can first have some discussion on this topic before moving on 
>> to see if we can get a consensus here on the mailing list.
>>
>> ---------------------------------------------------------------------
>> - Multimedia Technologies, Ericsson Research EAB/TVM
>> ---------------------------------------------------------------------
>> - Ericsson AB                | Phone  +46 10 7148287 Färögatan 6                
>> | Mobile +46 73 0949079

---

From: "Fabio Pietrosanti (naif)" <[email-masked]>
Date: Wed, 27 Jun 2012 20:22:46 +0200

> Otherwise, someone may make some subjective statements here and there without any actual measurements will only provide evidences what those folks do not know what they are actually talking about c…

It's not a subjective topic that over a mobile network there is packet
loss and that there are known condition where this lead to unacceptable
voip quality.

The Internet Measurement of VoIP on different Transport Layer Protocols
http://netarchlab.tsinghua.edu.cn/PersonalMainPage/wzl/public_html/pubs/2009_ICOIN_transport_measure.pdf

SipDroid: An opensource Mobile client using RTP Retransmission to
improve quality over 3G network with packet loss:
http://code.google.com/p/sipdroid/wiki/NewImprovedAudio

RTP Retransmission RFC (explaining how to handle dynamically
retransmission from RTCP feedback):
http://tools.ietf.org/html/rfc4588

Fabio

---

From: "Roy, Radhika R CIV (US)" <[email-masked]>
Date: Wed, 27 Jun 2012 19:03:02 +0000

Hi, Fabio:

I could not retrieve the first one, but other two links are OK.

I understand these things, but still they are NOT sufficient enough to generalize these results because of the following:

1. Experiments seem to be local without any precise measurements under all possible circumstances noting the network traffic, audio quality, video quality, and precise applications and number of subjects under test using different applications (e.g. point-to-point audio & video conf, multipoint audio & video conf, streaming audio/video applications). That is, it is not global, that means, there is NOT a wide area network (e.g. public Internet) connected with two different access networks where end-to-end one-way propagation delay is around 120-150 milliseconds, types of applications, traffic loads, subjects who were under test, and others.

2. If we do experiments over the high-speed LANs connecting users in the same LANs, we may find many things are possible including retransmissions of audios/videos. We can then extend these networking considerations unless and until we get a situation where retransmissions are no longer applicable. RFCs with retransmissions of audios/videos may be acceptable considering the trade-offs of end-to-end-delay and jitter-buffer-sizes under those limited circumstances. Each of those precise conditions can be noted generated. Again, these are very specific and case-by-case basis and cannot be generalized. 

3. Video quality measurements need to be taken in terms of real-time point-to-point and multipoint conferencing along with audio and video. Then these experimental results need to be measured separately for both audio and video separately. For example, at what point lip-synchronization is no longer possible.

Unless one can note all of those special cases for retransmissions of audio/video, it will always be technically confusing to use the retransmissions of audio and video as a generalized one.

I also know one case such as wiretapping applications. For silent recording in a distance location, audio & video retransmissions are needed because the exact things that are produced legally in the court need to be near 100% correct.

Hope this helps.

Radhika 

-----Original Message-----
From: Fabio Pietrosanti [mailto:[email-masked]] On Behalf Of Fabio Pietrosanti (naif)
Sent: Wednesday, June 27, 2012 2:23 PM
To: Roy, Radhika R CIV (US)
Cc: Cameron Byrne; [email-masked]
Subject: Re: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

> Otherwise, someone may make some subjective statements here and there without any actual measurements will only provide evidences what those folks do not know what they are actually talking about c…

It's not a subjective topic that over a mobile network there is packet loss and that there are known condition where this lead to unacceptable voip quality.

The Internet Measurement of VoIP on different Transport Layer Protocols http://netarchlab.tsinghua.edu.cn/PersonalMainPage/wzl/public_html/pubs/2009_ICOIN_transport_measure.pdf

SipDroid: An opensource Mobile client using RTP Retransmission to improve quality over 3G network with packet loss:
http://code.google.com/p/sipdroid/wiki/NewImprovedAudio

RTP Retransmission RFC (explaining how to handle dynamically retransmission from RTCP feedback):
http://tools.ietf.org/html/rfc4588

Fabio

---

From: Colin Perkins <[email-masked]>
Date: Wed, 27 Jun 2012 21:06:58 +0100

> re-transmission not productive.

This surely depends on the network being used, and the acceptable latency for the application. Making blanket statements that retransmission is, or is not, productive are not helpful, since it's usefulness is very context dependent.

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: Cameron Byrne <[email-masked]>
Date: Wed, 27 Jun 2012 13:23:56 -0700

> This surely depends on the network being used, and the acceptable latency for the application. Making blanket statements that retransmission is, or is not, productive are not helpful, since it's us…

Allow me to set context.

A comment was made that re-transmission of packets is a good idea
because mobile networks drop a lot of packets.

My comment is that re-transmission of RTP is not productive in 3GPP
wireless networks such as GSM, UMTS, or LTE.

This is because L2 mechanisms do forward error correction and PDU
acknowledgement and re-transmission.  The RAN system does not have
packet loss, generally speaking.  Packet loss may occur on the
internet prior to entering the RAN, this is a separate issue than the
specific case of mobile.

What is sometimes perceived as packet loss is more often than not,
just latency / jitter due to local L2 re-transmission.   RTP packets
that are too late are discarded today, and IMHO that is the right
approach (the late packet is no longer relevant)

RTP re-transmission from the source will not improve this situation.
In fact, if the issue is congestion (the channel is too narrow) then
re-transmission may make the problem worse (channel becomes more
narrow with higher load of re-transmitted packets)

I don't believe anyone has provided meaningful and relevant
quantitative data here that there is a problem, or a solution, for the
case of mobile and re-transmitted RTP.

Cameron

---

From: Colin Perkins <[email-masked]>
Date: Wed, 27 Jun 2012 21:31:58 +0100

Cameron,

I generally agree. Of course mobile is not the only scenario where WebRTC might be used, and the question being asked is not whether retransmission is to be used in any particular scenario, but whether it is RECOMMENDED or REQUIRED to implement.

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: "Fabio Pietrosanti (naif)" <[email-masked]>
Date: Wed, 27 Jun 2012 22:50:00 +0200

> internet prior to entering the RAN, this is a separate issue than the

You are saying that in EDGE, UMTS, HSDPA when an IP stack send an UDP
packet, it will handled from L2 like in the old age of GSM CSD and be
automatically retransmitted?

The L2 RLC can goes in:

* "acknolwedged mode" (handling retransmission)

* "unacknowledged mode" (no retransmission).

UDP streaming packets over Mobile Data Networks goes normally in
"unacknowledge mode".

As far as my professional experience VoIP over mobile can be normally be
subject to packet loss.

Fabio

p.s. however i agree that's a particular use cases of WebRTC

---

From: Cameron Byrne <[email-masked]>
Date: Wed, 27 Jun 2012 13:54:17 -0700

> automatically retransmitted?

Yes. LTE too.

Cameron

---

From: "Fabio Pietrosanti (naif)" <[email-masked]>
Date: Wed, 27 Jun 2012 23:11:29 +0200

>> automatically retransmitted?

I read (by googling cause i don't have LTE access) that also LTE uses
for IPTV and all other IP streaming applications the UM_RLC
(unacknowledged mode).

So also LTE should not do retransmission, unless being set in AM_RLC
(acknowledged mode).

Fabio

---

From: "Roni Even" <[email-masked]>
Date: Thu, 28 Jun 2012 09:24:58 +0200

Hi,
My view is that retransmission have great value for content delivery or
streaming applications, when the communication is unidirectional. 
For conversional applications like we are defining in RTCweb, retransmission
has the lower value comparing to other loss recovery options like FEC or
receiver based recovery. The latency involved with the extra roundtrips
makes it not effective comparing to typical jitter buffer sizes at the
receiver, note that receiver will take correction actions even for late
arrival and will not try to dynamically change the jitter buffer to a larger
size beyond the maximum threshold of end to end delay. 
 If we look at the typical behavior of conversional applications, they will
add to audio on the sender side FEC or even send the media twice is
subsequent packets encoded in different encoders,( of course for the
conversation itself the listener can always ask the talker to repeat since
he did not hear it clearly). On the video FEC and receiver side recovery
like motion estimation is common.
I propose that retransmission can be recommended and in my view for
conversional usage even this is strong.

Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Magnus Westerlund
Sent: 27 June, 2012 9:37 AM
To: [email-masked]
Subject: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

WG,

We had a discussion at the interim if RTP Retransmission is to be considered
REQUIRED or RECOMMENDED to implement. I would like to see if we can first
have some discussion on this topic before moving on to see if we can get a
consensus here on the mailing list.

Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 28 Jun 2012 09:49:07 +0200

Hi,

As Individual I like to state my position.

We have a video conference system developed by my colleagues used
internally at Ericsson that uses RTP Retransmission for video, not for
audio with great success. This is implemented such that we actually
allow the video to fall behind the audio when packet loss and
retransmission is not able to repair in a timely enough fashion. The
benefit is minimal overhead and still no loss induced degradations in
the video. Yes, we get degradation in form of frame display jittering
and short freezes. But those events that are truly visible are rare over
wired networks.

I am personally convinced that RTP Retransmission is great tool in the
toolbox when it comes to improve media quality in many use cases. Yes
there are scenarios where RTP retransmission is less efficient. Long
RTTs (over 200-400 ms) is the primary source of degradations. Compared
to FEC it so much more efficient from bandwidth consumption perspective.

I also think it is important that we have some mandatory to implement
tool for making the transport more robust now that we have a consensus
that we are not going for a FEC solution in the initial specification.

Thus my personal position is that RTP Retransmission should be REQUIRED
to implement.

Cheers

Magnus

-- 

Magnus Westerlund

---

From: Michael Welzl <[email-masked]>
Date: Thu, 28 Jun 2012 12:30:06 +0200

Hi,

I second that.

Here's one more reason: I believe that it should also be possible for a 
receiver to make use of retransmitted I-macroblocks from old frames that 
have been shown long ago. Future MBs may still depend on them, and so 
their quality could be improved with that information.

This is a quite theoretical idea - I don't have any proof that this 
would work, and don't know of research that has that proof (but I 
suspect one can find some, if one looks). It's just one more reason why 
retransmissions could turn out to be valuable.

Cheers,
Michael

---

From: "Roy, Radhika R CIV (US)" <[email-masked]>
Date: Thu, 28 Jun 2012 10:30:09 +0000

Hi, Magnus:

The audio/video performances for both point-to-point and multipoint conferencing must be very carefully articulated, not just what your colleagues did or did not do. All networking parameters have been discussed in list email including lip-synchronization.

So, let us see all the details of networking conditions in the published papers. We can then examine what is this all about.

The only thing that I am pointing out that these types of simplistic emails are not good enough to convincing the technical communities unless all the technical details are presented with precise measurements. Both subjective and quantitative performance parameters are involved along the many subjects who remain in the test environments. 

If anyone thinks that RTP retransmissions is a great tool, it is OK. Let them keep is private as many usages of RTP retransmissions were also discussed via emails.

BR/Radhika

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Magnus Westerlund
Sent: Thursday, June 28, 2012 3:49 AM
To: [email-masked]
Subject: Re: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

Hi,

We have a video conference system developed by my colleagues used internally at Ericsson that uses RTP Retransmission for video, not for audio with great success. This is implemented such that we actually allow the video to fall behind the audio when packet loss and retransmission is not able to repair in a timely enough fashion. The benefit is minimal overhead and still no loss induced degradations in the video. Yes, we get degradation in form of frame display jittering and short freezes. But those events that are truly visible are rare over wired networks.

I am personally convinced that RTP Retransmission is great tool in the toolbox when it comes to improve media quality in many use cases. Yes there are scenarios where RTP retransmission is less efficient. Long RTTs (over 200-400 ms) is the primary source of degradations. Compared to FEC it so much more efficient from bandwidth consumption perspective.

I also think it is important that we have some mandatory to implement tool for making the transport more robust now that we have a consensus that we are not going for a FEC solution in the initial specification.

Thus my personal position is that RTP Retransmission should be REQUIRED to implement.

Cheers

Magnus

---

From: "Roy, Radhika R CIV (US)" <[email-masked]>
Date: Thu, 28 Jun 2012 10:39:05 +0000

Hi, Roni:

Why do you want to recommend something that cannot be generalized conversional applications just because of more tools can be sold in the market?

No one is objecting to generalize the RTP retransmissions for the content delivery or streaming applications.

BR/Radhika

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roni Even
Sent: Thursday, June 28, 2012 3:25 AM
To: 'Magnus Westerlund'; [email-masked]
Subject: Re: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

Hi,
My view is that retransmission have great value for content delivery or streaming applications, when the communication is unidirectional. 
For conversional applications like we are defining in RTCweb, retransmission has the lower value comparing to other loss recovery options like FEC or receiver based recovery. The latency involved with the extra roundtrips makes it not effective comparing to typical jitter buffer sizes at the receiver, note that receiver will take correction actions even for late arrival and will not try to dynamically change the jitter buffer to a larger size beyond the maximum threshold of end to end delay. 
 If we look at the typical behavior of conversional applications, they will add to audio on the sender side FEC or even send the media twice is subsequent packets encoded in different encoders,( of course for the conversation itself the listener can always ask the talker to repeat since he did not hear it clearly). On the video FEC and receiver side recovery like motion estimation is common.
I propose that retransmission can be recommended and in my view for conversional usage even this is strong.

Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Magnus Westerlund
Sent: 27 June, 2012 9:37 AM
To: [email-masked]
Subject: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

WG,

We had a discussion at the interim if RTP Retransmission is to be considered REQUIRED or RECOMMENDED to implement. I would like to see if we can first have some discussion on this topic before moving on to see if we can get a consensus here on the mailing list.

Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 28 Jun 2012 13:34:36 +0200

On 2012-06-28 12:30, Roy, Radhika R CIV (US) wrote:
> Hi, Magnus:
> 
> The audio/video performances for both point-to-point and multipoint
> conferencing must be very carefully articulated, not just what your
> colleagues did or did not do. All networking parameters have been
> discussed in list email including lip-synchronization.
> 
> So, let us see all the details of networking conditions in the
> published papers. We can then examine what is this all about.

Yes, this would be desirable but I see as unrealistic amounts of work to
go through for each technical functionality. Also it is something that
is not possible. As WebRTC will be used over the very heterogeneous
network of networks called Internet. We also don't have any firm
definition of the service that we tries to fulfill.

I have been part of standardization work for services, like 3GPP's
Packet-based Streaming Service (PSS) where we have a much clearer
service definition and a clear network model. Then you could argue much
more firmly for how well a given tool would work in this service.

For WebRTC it is about enable the application. So the question is on
what requirement level towards the implementation RTP Retransmission
shall be available on. Then it becomes a question for the application
implementors to use the available toolbox as best suiting their
application, media requirements and the current network characteristics.

When it comes to determine what functionality that is to be included in
WebRTC we are operating on a contributor driven model where people
provide proposals for what to include and the amount of motivation they
are willing to provide. Then people provide feedback on that based on
their knowledge. In some cases the proponents have to provide additional
information in other cases consensus is achieved with less.

In this particular case I requested that we continue the discussion from
the Interim meeting on the mailing list to see if more arguments are
raised that will sway people in either direction.

> 
> The only thing that I am pointing out that these types of simplistic
> emails are not good enough to convincing the technical communities
> unless all the technical details are presented with precise
> measurements. Both subjective and quantitative performance parameters
> are involved along the many subjects who remain in the test
> environments.

I was stating my position on this. I think I have a fairly good
understanding myself of when RTP retransmission works and when it is
less efficient. I see it as a very good tool. And the main obstacle is
when the RTT becomes to high. But also in cases with higher RTT > ~300
ms there are still tricks certain service will accept that gives them
much higher quality compared to not having retransmission.

Yes, I am bad at providing references to backing papers about this. I
draw my conviction from having seen or heard about working systems using
retransmission.

> 
> If anyone thinks that RTP retransmissions is a great tool, it is OK.
> Let them keep is private as many usages of RTP retransmissions were
> also discussed via emails.

Sorry, I don't understand what you want to say with the above paragraph.

Cheers

Magnus Westerlund

---

From: "Roni Even" <[email-masked]>
Date: Thu, 28 Jun 2012 16:52:59 +0200

Hi Magnus,
So using retransmission is at the cost of losing lip synch which is very
noticeable. I am not saying that it does not work to do retransmission, my
claim that it is not practical and this is why I do not think it is
required.
Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Magnus Westerlund
Sent: 28 June, 2012 9:49 AM
To: [email-masked]
Subject: Re: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or
RECOMMENDED

Hi,

We have a video conference system developed by my colleagues used internally
at Ericsson that uses RTP Retransmission for video, not for audio with great
success. This is implemented such that we actually allow the video to fall
behind the audio when packet loss and retransmission is not able to repair
in a timely enough fashion. The benefit is minimal overhead and still no
loss induced degradations in the video. Yes, we get degradation in form of
frame display jittering and short freezes. But those events that are truly
visible are rare over wired networks.

I am personally convinced that RTP Retransmission is great tool in the
toolbox when it comes to improve media quality in many use cases. Yes there
are scenarios where RTP retransmission is less efficient. Long RTTs (over
200-400 ms) is the primary source of degradations. Compared to FEC it so
much more efficient from bandwidth consumption perspective.

I also think it is important that we have some mandatory to implement tool
for making the transport more robust now that we have a consensus that we
are not going for a FEC solution in the initial specification.

Thus my personal position is that RTP Retransmission should be REQUIRED to
implement.

Cheers

Magnus

On 2012-06-27 09:36, Magnus Westerlund wrote:
> if we can first have some discussion on this topic before moving on to 
> see if we can get a consensus here on the mailing list.
> 

-- 

Magnus Westerlund

---

From: "Roni Even" <[email-masked]>
Date: Thu, 28 Jun 2012 16:56:52 +0200

Hi,
The question was asked if it should be required or recommended. From these
two I choose  recommended which is less strong even though I think that MAY
will be better for using retransmission in conversional applications.
Roni

-----Original Message-----
From: Roy, Radhika R CIV (US) [mailto:[email-masked]] 
Sent: 28 June, 2012 12:39 PM
To: Roni Even
Cc: 'Magnus Westerlund'; [email-masked]
Subject: RE: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or
RECOMMENDED

Hi, Roni:

Why do you want to recommend something that cannot be generalized
conversional applications just because of more tools can be sold in the
market?

No one is objecting to generalize the RTP retransmissions for the content
delivery or streaming applications.

BR/Radhika

-----Original Message-----
Roni Even

Hi,

Roni

-----Original Message-----

WG,

Cheers

Magnus Westerlund

---

From: "Roy, Radhika R CIV (US)" <[email-masked]>
Date: Thu, 28 Jun 2012 14:03:05 +0000

If this is your personal opinion, it is OK.

However, like many others, I would not recommend this with this kind of blanket statement.

It is at all recommended, then the special circumstances under all those technical assumptions need to be noted. In this way, the RTP retransmission users will take their technically informed decisions and not to blame when perils occur.

BR/Radhika

-----Original Message-----
From: Roni Even [mailto:[email-masked]] 
Sent: Thursday, June 28, 2012 10:57 AM
To: Roy, Radhika R CIV (US)
Cc: 'Magnus Westerlund'; [email-masked]
Subject: RE: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

Hi,
The question was asked if it should be required or recommended. From these two I choose  recommended which is less strong even though I think that MAY will be better for using retransmission in conversional applications.
Roni

-----Original Message-----
From: Roy, Radhika R CIV (US) [mailto:[email-masked]]
Sent: 28 June, 2012 12:39 PM
To: Roni Even
Cc: 'Magnus Westerlund'; [email-masked]
Subject: RE: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

Hi, Roni:

BR/Radhika

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roni Even
Sent: Thursday, June 28, 2012 3:25 AM
To: 'Magnus Westerlund'; [email-masked]
Subject: Re: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

Hi,

Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Magnus Westerlund
Sent: 27 June, 2012 9:37 AM
To: [email-masked]
Subject: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

WG,

Cheers

Magnus Westerlund

---

From: Cameron Byrne <[email-masked]>
Date: Thu, 28 Jun 2012 07:04:40 -0700

> will be better for using retransmission in conversional applications.

+1 for MAY.

CB

> We had a discussion at the interim if RTP Retransmission is to be
considered

---

From: "Roy, Radhika R CIV (US)" <[email-masked]>
Date: Thu, 28 Jun 2012 14:12:49 +0000

Yes, like CB, I will also support "MAY."

BR/Radhika

-----Original Message-----
From: Cameron Byrne [mailto:[email-masked]] 
Sent: Thursday, June 28, 2012 10:05 AM
To: Roni Even
Cc: [email-masked]; Roy, Radhika R CIV (US)
Subject: Re: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

On Jun 28, 2012 6:57 AM, "Roni Even" <[email-masked]> wrote:
>
> Hi,
> The question was asked if it should be required or recommended. From 
> these two I choose  recommended which is less strong even though I 
> think that MAY will be better for using retransmission in conversional applications.
> Roni
>

+1 for MAY.

CB

> conversional applications just because of more tools can be sold in 
> the market?
>
> No one is objecting to generalize the RTP retransmissions for the 
> content delivery or streaming applications.
>
> BR/Radhika
>
> My view is that retransmission have great value for content delivery 
> or streaming applications, when the communication is unidirectional.
> For conversional applications like we are defining in RTCweb, 
> retransmission has the lower value comparing to other loss recovery 
> options like FEC or receiver based recovery. The latency involved with 
> the extra roundtrips makes it not effective comparing to typical 
> jitter buffer sizes at the receiver, note that receiver will take 
> correction actions even for late arrival and will not try to 
> dynamically change the jitter buffer to a larger size beyond the maximum threshold of end to end delay.
>  If we look at the typical behavior of conversional applications, they 
> will add to audio on the sender side FEC or even send the media twice 
> is subsequent packets encoded in different encoders,( of course for 
> the conversation itself the listener can always ask the talker to 
> repeat since he did not hear it clearly). On the video FEC and 
> receiver side recovery like motion estimation is common.

---

From: Colin Perkins <[email-masked]>
Date: Thu, 28 Jun 2012 15:14:51 +0100

Roni,

I don't understand how you can say this is not practical. Magnus has an implementation, and he states it works well. That seems practical to me.

There are also self-evidently network paths with low enough RTT for RTP retransmission to be used for interactive calls, if the playout buffer code knows the RTT and adjusts the buffer to allow this. Not all paths, I agree, but that doesn't mean it's not useful in some environments. 

We're defining requirements for the RTP infrastructure in WebRTC clients here, not forcing all implementations to use every feature. My personal view is that retransmission should be REQUIRED. Given that the RTP/AVPF profile is REQUIRED, implementation of retransmission is straightforward. I see no benefit in leaving it out, and very little cost in requiring it.

Colin

On 28 Jun 2012, at 15:52, Roni Even wrote:
> Hi Magnus,
> So using retransmission is at the cost of losing lip synch which is very noticeable. I am not saying that it does not work to do retransmission, my claim that it is not practical and this is why I do not think it is required.
> Roni
> 

-- 
Colin Perkins
http://csperkins.org/

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 28 Jun 2012 16:16:59 +0200

> claim that it is not practical and this is why I do not think it is

My experience with using this system several times a week for the last
two years says that it is not that noticable. The type of video image
degradations you get from unrepaired burst losses are so much worse.

I also think it is important that we consider the goals. If your goal is
a really high quality telepresence system then this approach is most
likely not usable. But, most WebRTC applications doesn't have that
quality requirements nor the user expectancies to be significantly
annoyed at a short loss of lip sync. The bit-rate savings that RTP
retransmission provides compared to FEC is much more important in my mind.

I think the Google hangout session during the interim showed the
importance of having some video being much higher than having no video
or requiring video with perfect sync with audio. And Justin said, Google
Hangout uses Retransmission.

Cheers

Magnus

-- 

Magnus Westerlund

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 28 Jun 2012 07:36:30 -0700

I think you need to separate this for audio and video and be far more specific about what type of retransmit ion you are talking about. In many cases the retransmit ion schemes for audio make the end suer experience worse.

---

From: "Roni Even" <[email-masked]>
Date: Fri, 29 Jun 2012 00:17:50 +0200

Colin,
Magnus also said that the penalty was losing lip synch, this is why it is
not practical. His solution is for proof of work which I agree it is working
but in my view not relevant for real usage. In order to use it you may need
a shorter jitter buffer and make a decision to ask for retransmit early, my
view is that larger jitter or playout buffer for late arrival will work
better.
So if there are some network path with short RTT (I assume not the Internet)
they can use retransmission, so recommended will be enough with text explain
when it make sense to support it.
Roni

-----Original Message-----
From: Colin Perkins [mailto:[email-masked]] 
Sent: 28 June, 2012 4:15 PM
To: Roni Even
Cc: 'Magnus Westerlund'; [email-masked]
Subject: Re: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or
RECOMMENDED

Roni,

I don't understand how you can say this is not practical. Magnus has an
implementation, and he states it works well. That seems practical to me.

There are also self-evidently network paths with low enough RTT for RTP
retransmission to be used for interactive calls, if the playout buffer code
knows the RTT and adjusts the buffer to allow this. Not all paths, I agree,
but that doesn't mean it's not useful in some environments. 

We're defining requirements for the RTP infrastructure in WebRTC clients
here, not forcing all implementations to use every feature. My personal view
is that retransmission should be REQUIRED. Given that the RTP/AVPF profile
is REQUIRED, implementation of retransmission is straightforward. I see no
benefit in leaving it out, and very little cost in requiring it.

Colin

> We have a video conference system developed by my colleagues used
internally at Ericsson that uses RTP Retransmission for video, not for audio
with great success. This is implemented such that we actually allow the
video to fall behind the audio when packet loss and retransmission is not
able to repair in a timely enough fashion. The benefit is minimal overhead
and still no loss induced degradations in the video. Yes, we get degradation
in form of frame display jittering and short freezes. But those events that
are truly visible are rare over wired networks.
implement.
>> -

--
Colin Perkins
http://csperkins.org/

---

From: "Roni Even" <[email-masked]>
Date: Fri, 29 Jun 2012 00:20:39 +0200

Hi Magnus,
The demo in the interim of what happens in retransmission was not in par
with my experience with any other recovery  methods including receiver based
repair without FEC.
Roni

-----Original Message-----
From: Magnus Westerlund [mailto:[email-masked]] 
Sent: 28 June, 2012 4:17 PM
To: Roni Even
Cc: [email-masked]
Subject: Re: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or
RECOMMENDED

On 2012-06-28 16:52, Roni Even wrote:
> Hi Magnus,
> So using retransmission is at the cost of losing lip synch which is 
> very noticeable. I am not saying that it does not work to do 
> retransmission, my claim that it is not practical and this is why I do 
> not think it is required.

My experience with using this system several times a week for the last two
years says that it is not that noticable. The type of video image
degradations you get from unrepaired burst losses are so much worse.

I also think it is important that we consider the goals. If your goal is a
really high quality telepresence system then this approach is most likely
not usable. But, most WebRTC applications doesn't have that quality
requirements nor the user expectancies to be significantly annoyed at a
short loss of lip sync. The bit-rate savings that RTP retransmission
provides compared to FEC is much more important in my mind.

I think the Google hangout session during the interim showed the importance
of having some video being much higher than having no video or requiring
video with perfect sync with audio. And Justin said, Google Hangout uses
Retransmission.

Cheers

Magnus

wired networks.
> RTTs (over
> 200-400 ms) is the primary source of degradations. Compared to FEC it 
> so much more efficient from bandwidth consumption perspective.
> 

-- 

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 29 Jun 2012 08:43:13 +0200

On 2012-06-28 16:36, Cullen Jennings wrote:
> 
> I think you need to separate this for audio and video and be far more
> specific about what type of retransmit ion you are talking about. In
> many cases the retransmit ion schemes for audio make the end suer
> experience worse.

I am not disagreeing unless the RTT is really low.

What I am asking the WG is if RTP retransmission is a RECOMMENDED or
REQUIRED feature in the toolbox that an WebRTC end-point supports. This
says nothing on when you select to use it and on which media. If we want
to include such recommendations we can do it. In fact the RTP usage
draft has a bit on text discussing the issue with RTT.

Cheers

Magnus
(As WG chair)

>> considered REQUIRED or RECOMMENDED to implement. I would like to
>> see if we can first have some discussion on this topic before
>> moving on to see if we can get a consensus here on the mailing
>> list.
>> 
>> Magnus Westerlund (As Chair and document editor)
>> 
>> 
>> Färögatan 6                | Mobile +46 73 0949079 SE-164 80
>> Stockholm, Sweden| mailto: [email-masked] 
>> rtcweb mailing list [email-masked] 

-- 

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 29 Jun 2012 08:50:24 +0200

> view is that larger jitter or playout buffer for late arrival will work

The loss of lip sync is one possible way of implementing retransmission
to handle situations where the RTT is large enough.

My main point is that all applications are not created equal and what
quality expectations an application has depends on its needs.

> when it make sense to support it.

I have low enough RTT within most of Europe from Stockholm over wired
access to wired access to use retransmission with minimal impact on user
experience. If I am willing to accept some degradation in the quality in
form of increased delay or jitter I can use retransmission on
intercontinental paths.

Yes, we should have a reasonable description on what to consider when
using RTP retransmission. Would you please be kind to review the current
text in the RTP usage draft and propose text or at least what should be
changed.

Best Regards

Magnus

-- 

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 29 Jun 2012 08:59:43 +0200

> with my experience with any other recovery  methods including receiver based

Agreed, but it did work. It was a session with intercontinental paths,
multiple of them in fact and I have no knowledge of where the hangouts
central node we used where located. Although the retransmission is
actually improved as the legs to and from are separated from a repair
perspective, there is some impact from the first leg onto the second
when it comes to delay.

I am also used to better quality in our system. But that is also not
trying to do things in a browser plugin.

Cheers

Magnus Westerlund

---

From: Stefan Holmer <[email-masked]>
Date: Fri, 29 Jun 2012 09:30:41 +0200

Doesn't this in the end boil down to what tools we have for baseline
resilience? Retransmissions are fairly easy to implement and will in most
cases give a much better experience than for example relying on periodic
key frames or doing key frame requests. Also it isn't necessary for the
decoder to support decoding streams with lost packets.

/Stefan

---

From: John Leslie <[email-masked]>
Date: Fri, 29 Jun 2012 10:25:26 -0400

> I am not disagreeing unless the RTT is really low.

   Actually, there is a balancing act between RTT and jitter...

> draft has a bit on text discussing the issue with RTT.

   I'd like to support Magnus here: I believe retransmission SHOULD be a
tool available to the recipient to request. There will be cases where
this is the most appropriate tool.

--
John Leslie <[email-masked]>

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 29 Jun 2012 08:11:30 -0700

Right - so on the question of it retransmission is mandatory to implement for audio codec, I am on a definitely "No". The bulk of systems today do not do it and work fine. Vendors can easily choose to do if they want in an interoperable way with out it being MTI. Why we should add a bunch of stuff in to version 1 of this that we can live without is beyond me. This is how IPv6 got big and hard, by everyone taking their favorite technology and attaching it to v6. I don't even want it as RECOMMENDED for audio - I see it as OPTIONAL. 

I probably feel differently about video.

---

From: Colin Perkins <[email-masked]>
Date: Fri, 29 Jun 2012 16:26:55 +0100

Cullen,

You're being asked what RTP tools the browser needs to implement, not what RTP tools you want use with audio or video codecs. 

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 29 Jun 2012 08:44:13 -0700

as Ted regularly points out, this is used by stuff other than browsers. So some RTCWeb endpoints will not do video. All I am saying is that I don't think things that are audio only should have to implement retransmission. 

In general my answer to what RTP tools are needed will be the minimal set that has proven to be needed to get things to work on the internet. At the Google IO conference this week, Google announced three major browser vendors plan to ship WebRTC before the end of 2012 and it would also work inside IE using chrome frame. I've also been looking at our milestones dates - we are going to need to have some long discussions about what we can safely leave till later if the IETF wants to produce standards that are done soon enough that they are not ignored.

---

From: "Roy, Radhika R CIV (US)" <[email-masked]>
Date: Fri, 29 Jun 2012 16:17:36 +0000

Yes, I support Cullen because so many problems are there for RTP-retransmissions since the inception real-time conversational applications including lip-synch.

With respect video quality, it is a question mark what you are getting at the cost of what, where, and how. Who is paying for the extra-bandwidth that may even lead to severe congestions over the networks if it is not done very carefully and selectively and if this is what all users try to do "fun" through retransmissions? Video-RTP-retransmissions are NOT well proven in the public Internet for wider use.

I even NOT want for even for video retransmissions, if it's the audio and video conversational application. If anyone likes to do this, let us keep this a private (optional) at users' discretion without publicly supporting in the IETF RFCs. At best, we can support the word "MAY."

BR/Radhika

---

From: Randell Jesup <[email-masked]>
Date: Sat, 30 Jun 2012 08:11:48 -0400

On 6/28/2012 3:49 AM, Magnus Westerlund wrote:
> retransmission is not able to repair in a timely enough fashion.

With my videophone-engineer's hat on.... I'm really surprised that this 
produces a good User Experience.  Sync failure (especially 
audio-before-video) is jarring to users.  Some classes of users may 
learn to ignore it more, or with very large displays they may prefer 
that to quality degradation, or the alternatives they've been used to 
were even worse.

I believe however that there are better ways to deal with loss.  But 
this really does come down to "what sort of artifacts do you find less 
annoying/noticeable/jarring?"  There's also a tradeoff with jitter 
buffer depths (deeper buffers allow more time for recovery, at a cost).

> The

The operative issues are: how long after expected arrival time is the 
re-xmit requested; how long does it take to get there, how long to get 
the re-transmitted packet, how much induced delay in the video occurs, 
what's the interaction with jitter buffer depth, what's the typical BW 
increase, packet size impacts, etc.

If you run longish jitter buffers (compared to actual jitter), and are 
on a fast network with low loss, it might be an improvement -- the long 
jitter buffer and fast network lets re-xmits arrive "in-time" or close 
to, minimizing induced delay and visual glitches, and total delay can 
stay in bounds.  (and this may be an overly-generous assessment)  Most 
real-world, open internet cases it will not make sense; not even close IMHO.

> to FEC it so much more efficient from bandwidth consumption perspective.

I'd also almost never want to use FEC for video, but my preference is 
for accepting temporary visual glitches and recovering as fast as 
possible with a high, constant framerate.  Don't ever freeze someone for 
more than a frame if you can avoid it.

> Thus my personal position is that RTP Retransmission should be REQUIRED

The other trick is *knowing* when to use it.  Something has to know if a 
re-xmit is likely to help instead of just wasting more bandwidth.  Is 
this under app control?  Is this logic baked into the core webrtc code? 
  How long is data held for possible retransmission?  (Perhaps the RFC 
says, I didn't check.)  How aggressive is it at asking for retransmits 
of late packets?  To make it work, do we need to artificially 
lower-bound the jitter-buffer depth?  Does the application get to decide 
to block video and let sync fail?

I understand Magnus' request, but I'm tempted to RECOMMEND.  Also: lack 
of support will not make calls fail, they'll just use "normal" 
error-recovery techniques.

-- 
Randell Jesup
[email-masked]

---

From: Randell Jesup <[email-masked]>
Date: Sun, 01 Jul 2012 02:44:06 -0400

On 6/29/2012 3:30 AM, Stefan Holmer wrote:
> resilience? Retransmissions are fairly easy to implement and will in
> most cases give a much better experience than for example relying on
> periodic key frames or doing key frame requests. Also it isn't necessary
> for the decoder to support decoding streams with lost packets.

"Better experience" incorporates a bunch of assumptions.

Assuming we're talking video only:

If you want it to be seamless, you have to be running a jitter buffer 
depth of at least 1 RTT plus time-to-decide-packet-may-be-lost plus 
random-constant.  The second part of that can be tricky and 
network-state dependent.  On corporate networks, I'd see sudden 100ms 
changes in base delay over 1-3 packets; a short time-to-decide would 
flag each of those as missing.  This is ok (the re-xmit will get ignored 
as a dup), but adds packets at an apparent bad point for the network.

That said, small RTTs certainly happen within LANs and corporations, and 
between neighbors in a single provider often.

Note that often/normally with small RTTs you also get reasonably small 
jitter, and thus the adaptive jitter buffer might run very small 
numbers, so one might need to put a lower limit on the jitter buffer in 
these cases.

The alternatives:

1) request re-xmit and freeze video queue when the jitter buffer runs 
dry.  If the packet comes in ASAP, freeze <= RTT+constant (modulo 
application scheduling delay at the sender side).  Then you can repair 
the broken frame, decode it, and then decode any following frames. 
Typically after a freeze you drop a frame or more; you might end up 
showing some of them depending on decode speed and how long the freeze 
lasted.  If the re-xmit (or the request) is lost, then delays can 
stretch much longer, since you need to wait RTT+bigger constant before 
requesting again.

In some conference apps even if the mixer just forwards video packets, 
the RTT is receiver<->mixer, not receiver<->mixer<->sender (this 
requires the mixer to buffer packets for possible retransmission).

2) request IDR (or refresh of corrupted slice).  Downside here is that 
you need to wait for the next frame encode at the sender side (so 0-33ms 
normally, or perhaps 0-100ms), and then the frame typically takes many 
packets to send (which also may be lost, and take time to receive 
especially on low-BW links), and IDR initial quality may be noticeably 
lower.  The receiver may freeze the video until the IDR is received or 
keep decoding with errors.  If there's a significant chance of the IDR 
losing a packet, a freeze could be substantial time.

3) request repair.  Note that #2 and #3 may be the same at the receiver 
side, with the sender deciding the best available repair.  In this case, 
the sender would instead of an IDR use a some other set of (smaller) 
packets to create an error-free up-to-date state at the receiver, such 
as encoding using a long-term-reference-frame, or using some other 
known-error-free frame in the receiver.  Note that repair is basically 
just a normal p-frame, albeit with somewhat lower quality and/or 
somewhat higher bandwidth used.  A big plus is that the end state is 
close to the same as re-transmit, but you don't have to speed through 
decoding any skipped frames.  Repair also allows the receiver to use 
alternate recovery mechanisms than simply freezing, including simply 
continuing to decode p-frames ignoring the lost frame.  This induces 
artifacts, but keeps motion loss to a minimum, especially in longer-RTT 
links.

Note that #3 in particular should result in similar freeze length as #1, 
perhaps 1 frame longer, if it freezes.  If it plays with errors instead, 
the freeze is typically 1 frame (the lost one).

(Others?)

My normal preference is #3. But, as discussed above, in low-RTT networks 
it may be a good option and might give 0-frames of freeze.  But, you 
still have all the complexities about how to decide when and how to use 
re-xmit; I think the 1 frame freeze is a reasonable option and a 
reasonable fallback if the source doesn't re-transmit.

Also, if retransmit is REQUIRED, and the media is gatewayed to other 
sources (non-webrtc), and they don't support retransmit (likely), then 
the gateway would need to buffer and act as a retransmit agent 
(complicating the gateway).  If it's RECOMMENDED, this is not a big deal 
and the gateway can stay simpler.
-- 
Randell Jesup
[email-masked]