Subject: [rtcweb] Welcome to a new mailing list

From: Harald Alvestrand <[email-masked]>
Date: Fri, 08 Apr 2011 13:56:03 +0200

There's always a quiet period when changing mailing lists.
Therefore, someone should send out a message that's provocative enough 
to get a few responses, so that everyone's filters get tuned.... here's 
my try.

Controversial things that need more discussions, culled from Friday's notes:

- Non-media data: RTP, UDP-without-RTP, TCP, or "something else"?
   Congestion control is obviously needed - both for this and for media, 
naturally.

- Mandatory to implement codecs: How can we figure out how to get the 
arguments documented, and make the call on whether to have them or not?

- Interim meeting:
When, with whom, and using what technology?

Enough ... that should set a few tongues wagging....

---

From: Tim Panton <[email-masked]>
Date: Fri, 8 Apr 2011 13:58:52 +0100

On 8 Apr 2011, at 12:56, Harald Alvestrand wrote:

> Therefore, someone should send out a message that's provocative enough to get a few responses, so that everyone's filters get tuned.... here's my try.
> 
>  Congestion control is obviously needed - both for this and for media, naturally.
> 
> - Mandatory to implement codecs: How can we figure out how to get the arguments documented, and make the call on whether to have them or not?
> 
> - Interim meeting:
> ________

Ok, I'll bite :-)

On the non-media data, I think it is somewhere between essential and _very_ useful.

Our experience of this is running a small java applet softphone that implements IAX.
We've been using the TEXT frame type to send data between the pbx and the browser.

IAX's TEXT (and in general non-media) frames are small (optionally encrypted), sequenced 
and reliable, but carried over UDP.

Typically it is notification of who is speaking in a conference call, a slide change, transcription text
or IVR prompts in text form. So the fact that it is sent directly and not polled is essential.
Mostly we send JSON objects, and have the page's javascript interpret them as it sees fit.

I'm not saying we should use IAX, but these properties of the TEXT frame are very useful 
in this context, so we could aim to reproduce those properties in any solution.

There are mechanisms for passing messages from one browser context to another,
see http://www.ajaxlines.com/ajax/stuff/article/using_html_postmessage.php - we should probably
be adopting that as an interface.

On the codec front, we've found SILK's adaptive nature really valuable in situations where a TDM codec
might fail - (e.g. user on the edge of a wifi network with high packet loss and variable bandwidth), so
we should aim to include a codec of that kind.

Tim.

---

From: Mary Barnes <[email-masked]>
Date: Fri, 8 Apr 2011 08:24:21 -0500

One suggestion I have for the group is to please start separate threads on
each of the items as that will make it MUCH easier to track the discussion.

In the spirit of this suggestion, I'll post my comments on the codec issue
separately.

Regards,
Mary.

On Fri, Apr 8, 2011 at 6:56 AM, Harald Alvestrand <[email-masked]>wrote:

> Therefore, someone should send out a message that's provocative enough to
> get a few responses, so that everyone's filters get tuned.... here's my try.
>
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb
>

---

From: Lorenzo Miniero <[email-masked]>
Date: Fri, 8 Apr 2011 15:24:01 +0200

Hi Harald and all,

a few comments inline.

Il giorno Fri, 08 Apr 2011 13:56:03 +0200
Harald Alvestrand <[email-masked]> ha scritto:

> Therefore, someone should send out a message that's provocative
> enough to get a few responses, so that everyone's filters get
> tuned.... here's my try.
> 

I guess that depends on what non-media is.

During the RTCWEB BOF we shared the session using a web-based interface
(some in this list may even have tried it), and it involved both media
(audio and video) and what we may call non-media (the jabber room, the
slides). All this non-media interaction, which was mostly based on
XMPP and a bit of HTTP, was just made available as a JavaScript API for
the webpage to exploit: asynchronous interaction was achieved the usual
way (one or more AJAX channels to send data, a long poll AJAX channel
to get updates) which I guess is the simpliest approach, from a
functional point of view, with respect to this kind of non-media.
Meaning I wouldn't go for a new protocol/solution or force RTP for
that as well, if avoidable.

For media itself, the approach we used so far was using another
JavaScript API to handle the SIP negotiation (masking all the SIP
details), and then relying on a Java applet to handle the RTP frames
and the encoding: I guess that the ability to handle also this last
frame of the picture within the browser would be the most
important outcome of the WG. What may also be interesting would be
debating whether or not an HTTP fallback (or something similar) should
be considered for RTP in constrained scenarios, as I seem to recall
having been mentioned during the BOF as well.

> arguments documented, and make the call on whether to have them or

This is likely to become a hot debate: HTML5 video has thought us
there's no easy way out. As someone already told during the BOF, I'd go
for a very simple (even if not optimal) common ground as mandatory, and
leave advanced codec choices up to developers, rather than saying
"this is how you can negotiate stuff, which stuff is up to you". I have
no strong sentiment towards any specific choice, but I'd go for
something that's not (too?) closed. Forcing AMR and H.264 as mandatory,
for example, would very likely cut a lot of people out of the loop,
which I don't think is what anybody wants.

> When, with whom, and using what technology?

If a virtual interim meeting is to be setup, you may count on us to
host and support it should you think it could be useful.

> Enough ... that should set a few tongues wagging....

L.

-- 
Lorenzo Miniero, COB

Meetecho s.r.l.
Web Conferencing and Collaboration Tools
http://www.meetecho.com

---

From: David Singer <[email-masked]>
Date: Fri, 08 Apr 2011 11:19:02 -0700

>  Congestion control is obviously needed - both for this and for media, naturally.

My feeling is that we should not forget we're designing something to sit in a browser or browser-like context.  We *have* text chat tools, file upload and download tools, and so on, today, working in browsers.  Yes, they typically go through a server, and going point to point might be an optimization.  But that's the rub; I don't think we should spend a lot of time optimizing tools that are ancillary to the core problem of doing real-time a/v between two users. If someone wants to start a group looking at real-time point-to-point text chat, well, OK, but I think it's a distraction for us, at least initially.

David Singer
Multimedia and Software Standards, Apple Inc.

---

From: "Elwell, John" <[email-masked]>
Date: Tue, 12 Apr 2011 09:19:49 +0200

It is always good to try to limit scope, and I think this is indeed a case where we can useful limit scope to RTP-based media (audio, video, real-time text...).

John

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of David Singer
> Sent: 08 April 2011 19:19
> To: [email-masked]
> Subject: [rtcweb] non-media data
> 
> 
> My feeling is that we should not forget we're designing 
> something to sit in a browser or browser-like context.  We 
> *have* text chat tools, file upload and download tools, and 
> so on, today, working in browsers.  Yes, they typically go 
> through a server, and going point to point might be an 
> optimization.  But that's the rub; I don't think we should 
> spend a lot of time optimizing tools that are ancillary to 
> the core problem of doing real-time a/v between two users. If 
> someone wants to start a group looking at real-time 
> point-to-point text chat, well, OK, but I think it's a 
> distraction for us, at least initially.
> 
> David Singer

---

From: Tim Panton <[email-masked]>
Date: Tue, 12 Apr 2011 09:40:20 +0100

Although we should aim to limit scope, it is also a good plan to listen to experience in the field.
Our experience is that direct, reliable, sequenced transmission of small messages between 
endpoints hugely adds to the value of a web-embedded RTC component.

Typically these messages are intimately tied to events in the media engine (change of speaker,
end of message playback, transcription or recording available). (dynamic media meta-data if you like)

They are used to change the visual state of the web page (via javascript) so that (for example) the 
web displayed IVR prompts stay in sync with the audio.

Whilst all of these are achievable via a more circuitous route (mediaserver->signalingserver->webserver->long poll-> browser)
the added latency will make the visual changes lag the audio which undermines the user experience.

The value in RTC-web isn't the realtime audio/video itself, it's the fact that it is an integral part of
the web-page, that integration is aided by non-media data.
Tim.

---

From: David Singer <[email-masked]>
Date: Tue, 12 Apr 2011 11:28:53 -0700

> the web-page, that integration is aided by non-media data.

OK, this is fine, but having arbitrary numbers of streams of arbitrary media types is different from designing chat sessions or file transfer as 'part of' the tool.

David Singer

---

From: Christopher Blizzard <[email-masked]>
Date: Tue, 12 Apr 2011 11:51:42 -0700

On 4/8/2011 11:19 AM, David Singer wrote:
> My feeling is that we should not forget we're designing something to sit in a browser or browser-like context.  We*have*  text chat tools, file upload and download tools, and so on, today, working in browsers.  Yes, they typically go through a server, and going point to point might be an optimization.  But that's the rub; I don't think we should spend a lot of time optimizing tools that are ancillary to the core problem of doing real-time a/v between two users. If someone wants to start a group looking at real-time point-to-point text chat, well, OK, but I think it's a distraction for us, at least initially.
>

I'd also like to point out that at Mozilla we want to see the browser 
change.  What you're describing is the way that browsers work today, but 
I think we're ready to move beyond that model to something that's 
app-driven, including point to point connections for data as well as 
media.  Latency isn't just an excuse to do point to point, but is an end 
unto itself.

John Elwell points out that limiting scope is important, and I agree.  
But I want to make sure that any APIs or protocols that get designed can 
handle this case as well.

--Chris

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 13 Apr 2011 10:43:47 +0200

> My feeling is that we should not forget we're designing something to sit in a browser or browser-like context.  We *have* text chat tools, file upload and download tools, and so on, today, working…
One reason it's somewhat urgent, organizationally, is that there is an 
effort being pursued by some participants in WHATWG to define a protocol 
for data transfer in the RTCWeb context.

http://www.whatwg.org/specs/web-apps/current-work/multipage/dnd.html#the-data-stream 
is the link.

I'd like to make sure we don't end up in a situation where there's a 
thought-through protocol defined by the IETF and a non-thought-through 
protocol that "everyone" implements "just because the spec was there". 
Ideally, I'd like one protocol referenced by both contexts that is well 
designed.

                                   Harald

---

From: Stefan Håkansson LK <[email-masked]>
Date: Thu, 14 Apr 2011 15:37:01 +0200

I also think a direct "small message" channel would be very useful: for enhancing conferencing-like services as discussed below as well as for gaming (where latency can be extremely important). And once it is available I'm sure developers will find other, more innovative, ways to use it!

Stefan

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Tim Panton
Sent: den 12 april 2011 10:40
To: Elwell, John
Cc: [email-masked]
Subject: Re: [rtcweb] non-media data

Although we should aim to limit scope, it is also a good plan to listen to experience in the field.
Our experience is that direct, reliable, sequenced transmission of small messages between endpoints hugely adds to the value of a web-embedded RTC component.

Typically these messages are intimately tied to events in the media engine (change of speaker, end of message playback, transcription or recording available). (dynamic media meta-data if you like)

They are used to change the visual state of the web page (via javascript) so that (for example) the web displayed IVR prompts stay in sync with the audio.

Whilst all of these are achievable via a more circuitous route (mediaserver->signalingserver->webserver->long poll-> browser) the added latency will make the visual changes lag the audio which undermines the user experience.

The value in RTC-web isn't the realtime audio/video itself, it's the fact that it is an integral part of the web-page, that integration is aided by non-media data.
Tim.

On 12 Apr 2011, at 08:19, Elwell, John wrote:

>> My feeling is that we should not forget we're designing something to 
>> sit in a browser or browser-like context.  We
>> *have* text chat tools, file upload and download tools, and so on, 
>> today, working in browsers.  Yes, they typically go through a server, 
>> and going point to point might be an optimization.  But that's the 
>> rub; I don't think we should spend a lot of time optimizing tools 
>> that are ancillary to the core problem of doing real-time a/v between 
>> two users. If someone wants to start a group looking at real-time 
>> point-to-point text chat, well, OK, but I think it's a distraction 
>> for us, at least initially.
>> 
>> David Singer

---

From: Justin Uberti <[email-masked]>
Date: Thu, 14 Apr 2011 23:37:30 -0700

I tend to think we should separate "RTP session" (i.e. PeerConnection) from
"network transport"; a RTP session will run over network transports, but I
don't know if we want to add generic transport facility to the RTP session
API. As in, if reliable message delivery is desired, do we want to go down
the road of TCP-over-RTP?

If we expose raw transport as a lower-level API, then RTP sessions and
generic sessions can make use of this transport service.

On Thu, Apr 14, 2011 at 6:37 AM, Stefan Håkansson LK <
[email-masked]> wrote:

> I also think a direct "small message" channel would be very useful: for
> enhancing conferencing-like services as discussed below as well as for
> gaming (where latency can be extremely important). And once it is available
> I'm sure developers will find other, more innovative, ways to use it!
>
> Stefan
>
> Our experience is that direct, reliable, sequenced transmission of small
> messages between endpoints hugely adds to the value of a web-embedded RTC
> component.
>
> Typically these messages are intimately tied to events in the media engine
> (change of speaker, end of message playback, transcription or recording
> available). (dynamic media meta-data if you like)
>
> They are used to change the visual state of the web page (via javascript)
> so that (for example) the web displayed IVR prompts stay in sync with the
> audio.
>
> Whilst all of these are achievable via a more circuitous route
> (mediaserver->signalingserver->webserver->long poll-> browser) the added
> latency will make the visual changes lag the audio which undermines the user
> experience.
>
> The value in RTC-web isn't the realtime audio/video itself, it's the fact
> that it is an integral part of the web-page, that integration is aided by
> non-media data.
> Tim.
>
>
> > It is always good to try to limit scope, and I think this is indeed a
> case where we can useful limit scope to RTP-based media (audio, video,
> real-time text...).
> >
> > John
> >
> >

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 15 Apr 2011 09:42:27 +0200

Justin Uberti skrev 2011-04-15 08:37:
> I tend to think we should separate "RTP session" (i.e. PeerConnection)
> from "network transport"; a RTP session will run over network
> transports, but I don't know if we want to add generic transport
> facility to the RTP session API. As in, if reliable message delivery is
> desired, do we want to go down the road of TCP-over-RTP?

I don't like all this talk about data over RTP. Generic data over RTP
isn't the most suitable model for a number of reasons which I will come
back to below. From my perspective we do want data transport that
provide a unreliable datagram based transport. That would in my view
include make available to the web application through the API this. And
let the WEBAPP decided what is put in each datagram. However, we clearly
needs some requirements on that datagram service so that it remains safe
and fair when it comes to transport. To me that means requirements such as:

- Congestion Controlled
- Confidentiality and Integrity Protected
- Providing consent to receive data prior to allow transmission of
substantial amounts of data
- Some type of source authentication to prevent man in the middles

There is a question of how much additional functionality one likes to
offer around an unreliable datagram transport. I think we need to go
through http://www.rfc-editor.org/rfc/rfc5405.txt and see what is
desired to have as part of the browser implementation and what is left
to the web application to implement.

Then there is the question if we desire to provide additional transport
properties, like reliable datagram, or reliable byte stream modes. There
there exist several possible ways of doing that. Separate protocol
stacks, reliability extension on top of an unreliable datagram transport
etc. I also think it is important that we do discuss what requirements
exist for each of the transport properties.

I promised to return to why I think TCP over RTP or more general,
generic data over RTP isn't the most suitable model.
- RTP is a protocol built towards real-time media. Yes the definition of
media can be pretty loose here. But carrying the individual datagrams of
a TCP connection over RTP is plainly stupid, when TCP over UDP will get
you all the desired functionality anyway with less overhead.
- RTP requires RTP payload formats for anything it carries. That is to
ensure that the application level framing ideas are as well captured for
a particular media encoding as possible. That optimization is not
possible for general data.
- If one can't do general optimization one goes with more general
transport solutions, thus why not use a more general data transport for
the general data?
- RTP and RTCP monitoring is also designed with more continous medias in
mind. Here we don't know how much such properties will be true. Also the
congestion controls available might actually have not the most suitable
properties for certain type of data.

> generic sessions can make use of this transport service.

I understand your thinking, but there is important to remember that the
NAT/FW traversal solution is likely needing to operate on top of the
bottom most transport protocol, most likely UDP. On top of that we have
thus may ICE's STUN servers multiplexed with one RTP session on the same
5-tuple. Isn't it likely that one will from an API perspective request
an RTP sessison to be established to the peer with a certain set of RTP
payload types to carry a particular media type for a particular purpose.
In the same way I see that a unreliable datagram transport will be
requested and as part of that request indicate what particular data one
wants to run on top, for example using media types like for the UDP and
TCP protocol identifiers in SDP, or another label scheme. But the web
application and perhaps the browser will have an understanding what a
particular flow will be used to.

When it comes to actually doing a data transport of any kind. I do think
this is an individual module that can be negotiated for availability
between two peers. However, I do think there are a number of use cases
that are so interesting that at least unreliable datagram transport
should be aimed at completing at the same time as setup and negotiation,
NAT/FW traversal, and the real-time media modules. But more use cases
and more opinions on this are desirable.

At the same time I would like to hear if anyone has use cases for
reliable data services that goes directly peer to peer, either message
based or byte stream based. So far the arguments I have seen for this
are reducing load on relays, and lower transport delay. Some views from
someone actually needs such transport services as part of the solution
would be great.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: Tim Panton <[email-masked]>
Date: Fri, 15 Apr 2011 09:40:11 +0100

> someone actually needs such transport services as part of the solution

It's true that there are always multiple routes to solve a problem, but 
take my example of sending visible IVR prompts to a browserbased softphone.

(So if you call your bank from a web-embedded softphone you get a visual menu
to go with the spoken IVR, prompts can have more explanatory help text and
chosen faster as most people can read faster than an IVR can speak. The prompts 
change as you move through the IVR tree - at some point you may get handed off to
a real person)

Ideally these messages are carried over reliable datagrams, in reasonably close
sync with the media. You don't want to miss them and you don't want them to be delivered
out of sequence. A TCP stream wouldn't be a good fit as these are messages not streams.

The case is much more like DTMF, except more generalised and hopefully without the
ugliness of the "send the same thing many times in the hope that one will get through"
that asymmetric RTP requires when doing DTMF.

This is a near-term transitional example, I'm sure that in the future, there will be more 
extreme examples say involving lip-sync animation of locally drawn (SVG?) avatars, again where
the timing is critical.

Sure you could define a new media-type for lip-sync animation, but which web coder will do that
if they can throw a JSON packet up the wire ? In this case a generalist web-friendly mechanism
is going to win out.

Tim.

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 15 Apr 2011 14:53:24 +0200

> out of sequence. A TCP stream wouldn't be a good fit as these are messages not streams.
Actually, this use case seem to me like an excellent fit with HTML-based 
menus with menu content carried over XDR. On a good day, the time 
required to replace menu content is ~100 msec; this is far faster than 
most humans can react.

It does point out (to me) a need to insert syncpoints into media streams 
so that the HTML action can be synchronized with the spoken text; the 
syncpoints may be represented as specific timestamps that are part of 
HTML content with values pointing into the media streams, or they might 
be synchronized packets carried on a data channel that arrives in sync 
with the audio.
> if they can throw a JSON packet up the wire ? In this case a generalist web-friendly mechanism
For a working example of stuff synchronized with media, check out WebVTT 
(subtitling).

http://www.whatwg.org/specs/web-apps/current-work/webvtt.html

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 15 Apr 2011 14:55:32 +0200

When I wrote draft-alvestrand-dispatch-rtcweb-01, I was trying to 
describe a datagram service with assured "permission to send", which 
could be used to underpin either RTP or a raw datagram service.

It's received relatively little feedback; we might want to pursue it or not.

On 04/15/11 08:37, Justin Uberti wrote:
> facility to the RTP session API. As in, if reliable message delivery 
> is desired, do we want to go down the road of TCP-over-RTP?
>
> On Thu, Apr 14, 2011 at 6:37 AM, Stefan Håkansson LK 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     I also think a direct "small message" channel would be very
>     useful: for enhancing conferencing-like services as discussed
>     below as well as for gaming (where latency can be extremely
>     important). And once it is available I'm sure developers will find
>     other, more innovative, ways to use it!
>
>     Stefan
>
>     From: [email-masked] <mailto:[email-masked]>
>     [mailto:[email-masked] <mailto:[email-masked]>]
>     On Behalf Of Tim Panton
>     Cc: [email-masked] <mailto:[email-masked]>
>     Our experience is that direct, reliable, sequenced transmission of
>     small messages between endpoints hugely adds to the value of a
>     web-embedded RTC component.
>
>     Typically these messages are intimately tied to events in the
>     media engine (change of speaker, end of message playback,
>     transcription or recording available). (dynamic media meta-data if
>     you like)
>
>     They are used to change the visual state of the web page (via
>     javascript) so that (for example) the web displayed IVR prompts
>     stay in sync with the audio.
>
>     (mediaserver->signalingserver->webserver->long poll-> browser) the
>     added latency will make the visual changes lag the audio which
>     undermines the user experience.
>
>     The value in RTC-web isn't the realtime audio/video itself, it's
>     the fact that it is an integral part of the web-page, that
>     integration is aided by non-media data.
>     Tim.
>
>
>     > It is always good to try to limit scope, and I think this is
>     indeed a case where we can useful limit scope to RTP-based media
>     (audio, video, real-time text...).
>     >
>     > John
>     >
>     >
>     >> From: [email-masked] <mailto:[email-masked]>
>     >> [mailto:[email-masked]
>     <mailto:[email-masked]>] On Behalf Of David Singer
>     >> To: [email-masked] <mailto:[email-masked]>
>     something to
>     >> [email-masked] <mailto:[email-masked]>
>     > [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 15 Apr 2011 15:12:41 +0200

> - Some type of source authentication to prevent man in the middles
This is a good list to keep in mind. I'll return to it.
> you all the desired functionality anyway with less overhead.
RTP is, in the extreme view, a ~12-byte header and a  set of handling 
rules. We should look at what it is, not so much what it was originally 
thought to be designed for (a component in a framework for multicast 
distribution of unidirectional media channels).
> possible for general data.
I think "general data" is a red herring. We can define a few different 
data-carrying "modes" that have different properties; if some of these 
are useful, I think it makes sense to define them.
> transport solutions, thus why not use a more general data transport for
Pushing stuff towards a "more general data transport" requires one of 
two things:
a) having an existing more general data transport that fulfils our 
requirements
b) defining a new more general data transport that fulfils our requirements
The second seems tough. The first ..... what's your preferred candidate?
> properties for certain type of data.
It's not clear to me that there *are* congestion controls available 
(other than media-specific ones).
Which ones are you thinking of?

Now, the flipside: What does RTP *have* that we will otherwise need to 
define?
 From your earlier list:

- Congestion Controlled: eh... maybe not.
- Confidentiality and Integrity Protected - done, we've got SRTP.
- Providing consent to receive data prior to allow transmission of
substantial amounts of data - done, we have ICE, and need it for media too.
- Some type of source authentication to prevent man in the middles - done, we have SRTP

So apart from congestion control, it seems that RTP's 12-byte header actually gives us a lot.

> payload types to carry a particular media type for a particular purpose.
If that's the API, we've got an RTP-level API (the RTP machinery is 
below the API). I believe Justin was talking about a lower-level API, 
where the RTP-level machinery would be above it.

Dropping the rest of the message ... this note is already long enough.

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 15 Apr 2011 16:28:39 +0200

Haralad and other interested

Snipped out more parts as I added quite a lot around possible
alternatives for you to think about. See below

> distribution of unidirectional media channels).

Well, the handling rules include RTCP and an intention to handle a
number of different topologies with multi-party handling. There are also
additional signalling issues, like how to configure the RTCP bandwidth
value, which features to use and all these things. Things you may not
need for your data transport. If you need them, you anyway should define
an RTP payload format for your data format.

And to be frank, the transport and RAI ADs has been very hesitant to
allow any RTP payload format that is defined for just being a bit pipe
over RTP. The two reasons for this has been Application level framing
principle and the lack of congestion control for RTP.

> are useful, I think it makes sense to define them.

I agree that different transport protocol provides different properties.
And that you should chose the one most appropriate. However, one usually
has to compromise on some aspects, either by needing to do specific
solution for ones data or accept some other potential limitation. On a
high level comparisons between our transport protocols I think can be
summarized like this:

TCP: unicast only, congestion controlled reliable byte stream

SCTP: unicast, multihoming, congestion controlled, reliable and
unreliable, message oriented.

DCCP: unicast, unreliable, congestion controlled sequence numbered datagram

RTP/UDP: unicast and multicast, unreliable, RTP payload of single MTU
size, medium time scale transport feedback.

UDP: unicast and multicast, unreliable, datagram

> The second seems tough. The first ..... what's your preferred candidate?

Actually the one I think is easiest from a specification point of view
and that does provide a reasonable service for unreliable datagrams are:

DTLS/DCCP/UDP

That would be the following references:
https://datatracker.ietf.org/doc/rfc4340/ (Base Spec)
https://datatracker.ietf.org/doc/rfc4341/ (TCP-Like CC)
https://datatracker.ietf.org/doc/rfc4342/ (TFRC CC)
https://datatracker.ietf.org/doc/rfc5238/ (DTLS for DCCP)
https://datatracker.ietf.org/doc/draft-ietf-dccp-udpencap/ (DCCP over UDP)

My main concern is the maturity of available implementations of DCCP.
Otherwise it provides congestion control and can actually choose which
congestion control algorithms we want supported, TCP-like or TFRC. It
also have some additional features. One is that we get a second port
space providing additional multiplexing points.

Please note that this is just a friendly suggestion. At least check it
out, and possible post some comments why it would be unsuitable.

> Which ones are you thinking of?

The one that is full defined is to run RTP over DCCP.
https://datatracker.ietf.org/doc/rfc5762/

I guess the biggest issue is the lack of experience with this and
especially how you interface the DCCP stack to work well between media
encoding, RTP stack and the DCCP congestion control.

The alternative, not standardized yet but closest available for RTP over
UDP is TFRC for RTP.
https://datatracker.ietf.org/doc/draft-gharai-avtcore-rtp-tfrc/

That was in reasonable shape prior to its abandonment due to lack of
funding from the ones driving it.

> Now, the flipside: What does RTP *have* that we will otherwise need
> to define? From your earlier list:
> 
>    - done, we have SRTP

Well, the security parts can be provided by ICE and DTLS over UDP also.
we still are short the congestion control for such a thing plus all
these things that the UDP guidelines (RFC5405) discuss. Using RTP over
UDP does drag all the RTP baggage into something that may not needed it,
but it will provide some parts of what you need. If a particular data
format needs RTP functionality, it should define its own RTP payload
format anyway.

Cheers

Magnus Westerlund

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 27 Apr 2011 14:18:43 +0200

Returning to this thread after a few weeks.... I'm still being contrary 
to make sure we have all the issues explicit where they need to be... 
and I do hope that some of the people who actually want to send data 
(instead of us types who speculate about what you possibly might want) 
can speak up!

> an RTP payload format for your data format.
If these questions have fixed answers in a specific scenario (like 
point-to-point), they can be handled by specifying a set of reasonable 
values for that scenario.
> principle and the lack of congestion control for RTP.
I do wonder about the ALF principle. In some ways, it seems reasonable 
under ALF that one should be able to use a framing mechanism that 
provides some of what we want without necessarily worrying that it 
provides more than what we want...

The congestion control is definitely an issue. With RTCWEB deployment 
likely to increase the amount of media streams encountering congestion 
significantly, I think we have to address this in a way that's 
applicable to RTP in general - but that's an AVTCORE matter, not an 
RTCWEB matter, I think.
> TCP: unicast only, congestion controlled reliable byte stream
Also: Implemented in kernels, allowed through firewalls, 
double-NAT-traversal fails frequently.
> unreliable, message oriented.
Also: Not implemented in kernels, firewalls know nothing of it (unless 
UDP encapsulated). NAT boxes know nothing about it (so UDP encapsulation 
seems a must).
> DCCP: unicast, unreliable, congestion controlled sequence numbered datagram
The "also" list here is the same as for SCTP, I think.
> UDP: unicast and multicast, unreliable, datagram
Properties of traversal of firewalls and NAT boxes are well known. 
Kernel implemented.
> out, and possible post some comments why it would be unsuitable.
In a design point spectrum where Ian Hixie's 44-byte datagram header, 
this proposal and a "data-over-RTP" proposal are the 3 candidates, we 
should have enough design points that we can start discussing the 
criteria we evaluate them against (often a good thing).
> encoding, RTP stack and the DCCP congestion control.
If we buy DCCP anyway, this is certainly worth investigating.
> funding from the ones driving it.
Does anyone want to pick it up and see what's missing?

---

From: "Christian Hoene" <[email-masked]>
Date: Wed, 27 Apr 2011 19:48:37 +0200

> (instead of us types who speculate about what you possibly might want)

[Christian Hoene] Most RTP payload formats are for audio or video. I have not seen that RTP is just for other real time data such as GPS, location, orientation, input devices (mouse, kinect, etc...) or gaming data. Thus, I do not see that data over RTP is an option that is well supported. Real-time-Data-over-RTP seems not to be in the standardization traditions of AVT. Of course, this might change at some point in future.

Until then, I would suggest to support a plain real-time-data-over-datagram option, as we do not have to follow standards anyhow to support non-media data over whatsoever. 

Spoken up,
 Christian

---

From: Colin Perkins <[email-masked]>
Date: Wed, 27 Apr 2011 19:46:09 +0100

> Until then, I would suggest to support a plain real-time-data-over-datagram option, as we do not have to follow standards anyhow to support non-media data over whatsoever. 

RTP is designed for real-time data, and is built on the assumption that the data is time sensitive, and should be processed subject to time constraints at the receiver. There are RTP payload formats for non-audio-visual media, for example text conversation [RFC2793 and RFC4103] and pointers [RFC2862]. I can envisage RTP being appropriate for various types of input device, and for some forms of data being sent in games, and for some types of sensor data, since those things are timed. It's not a general purpose data transfer protocol though, and isn't well suited for non-timed data.

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Wed, 27 Apr 2011 19:48:35 +0100

On 27 Apr 2011, at 13:18, Harald Alvestrand wrote:
> Returning to this thread after a few weeks.... I'm still being contrary to make sure we have all the issues explicit where they need to be... and I do hope that some of the people who actually want to send data (instead of us types who speculate about what you possibly might want) can speak up!
> 
> If these questions have fixed answers in a specific scenario (like point-to-point), they can be handled by specifying a set of reasonable values for that scenario.
> I do wonder about the ALF principle. In some ways, it seems reasonable under ALF that one should be able to use a framing mechanism that provides some of what we want without necessarily worrying that it provides more than what we want...

A protocol isn't just a framing format though. The semantics of RTP are very much focussed on real-time, time sensitive, media delivery. Trying to use it outside that space is likely to cause problematic semantic mismatch, to the extent that you'd be better duplicating the framing (which is the most trivial aspect of RTP) in another protocol that doesn't have all the other features of RTP that you don't want.

-- 
Colin Perkins
http://csperkins.org/

---

From: David Singer <[email-masked]>
Date: Thu, 28 Apr 2011 09:44:42 -0400

> RTP is designed for real-time data, and is built on the assumption that the data is time sensitive, and should be processed subject to time constraints at the receiver. There are RTP payload format…

Specifically, I'd say that, given the choice between getting all the data (possibly at some indefinite time in the future) or getting data on time (but possibly incomplete), TCP attempts to do the former, and RTP the latter.  That makes RTP unsuitable if loss is undesirable and/or timeliness is not needed.

David Singer

---

From: "Elwell, John" <[email-masked]>
Date: Thu, 28 Apr 2011 18:57:13 +0200

> 
> A protocol isn't just a framing format though. The semantics 
> of RTP are very much focussed on real-time, time sensitive, 
> media delivery. Trying to use it outside that space is likely 
> to cause problematic semantic mismatch, to the extent that 
> you'd be better duplicating the framing (which is the most 
> trivial aspect of RTP) in another protocol that doesn't have 
> all the other features of RTP that you don't want.

[JRE] Not to mention all the RTCP stuff comes with RTP and probably isn't relevant to non-real-time data.

John