Subject: [rtcweb] Text communication in RTCWEB sessions

From: Gunnar Hellström <[email-masked]>
Date: Mon, 12 Nov 2012 10:06:12 +0100

Real time sessions with video and audio are often accompanied with some 
form of text communication.

It is mentioned in a couple of places in RTCWEB related specifications, 
but I see a need to get the specifications of how text should be handled 
up to the level of specification of the other media.

The real-time requirements are slightly lower on text communication than 
on video and audio, so I realize that it can be carried by mechanisms 
already available in the web browser environment. But even if that will 
be the case, it needs to be considered and described, because text 
communication is part of the real-time communication concepts that 
RTCWEB belongs to.

I know three kinds of text communication:

Real-time text:

Real-time text is text transmitted while it is being typed or 
created.*The recipient can read the sender's text as it is written, 
without waiting.

*Text messaging:

Text messaging is text collected to complete messages and sent as a 
whole when completed by specific user action.***The recipient can read 
the sender's text *soon after it is transmitted.

Timed text:

Timed text is text sent together with with information about the 
intended timing of the display, often related to another medium, such as 
audio or video. The recipient can read the text presented timely to 
events in the other media.

Real-time text for conversational use has the timing requirement that 
end-to-end latency shall not be more than 2 seconds for usable text, and 
one second for good usability. This is clearly lower than the 
requirements on audio and video that traditionally is 400 ms for usable 
end-to-end latency.

Real-time text is briefly mentioned in a requirement in 
draft-ietf-rtcweb-data-channel
http://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-02

All three text communication forms are mentioned in 
http://tools.ietf.org/id/draft-aboba-rtcweb-ecrit-00.txt

I am most eager to see the use of real-time text specified in relation 
to RTCWEB, but it might be good to touch all three types.

I suggest that it starts with requirements and use cases in
http://tools.ietf.org/wg/rtcweb/draft-ietf-rtcweb-use-cases-and-requirements/

I propose to add real-time text to use case 4.2.1 so that it reads:

        4.2.1. Simple Video Communication Service

  with audio and real-time text

          4.2.1.1. Description

    Two or more users have loaded a video communication web application
    into their browsers, provided by the same service provider, and
    logged into the service it provides.  The web service publishes
    information about user login status by pushing updates to the web
    application in the browsers.  When one online user selects a peer
    online user, a 1-1 video communication session between the browsers
    of the two peers is initiated.  The invited user might accept or
    reject the session.Audio and real-time text is also available
    during the session.

    During session establishment a self-view is displayed, and once the
    session has been established the video sent from the remote peer is
    displayed in addition to the self-view.  During the session, each
    user can select to remove and re-insert the self-view as often as
    desired.  Each user can also change the sizes of his/her two video
    displays during the session.Audio from each user is presented to
    the other user. While one user types in the real-time text area, it
    is nearly immediately presented to the other user.  Each user can
    also pause sending of media (audio, video, or both) and mute incoming
    media

An alternative is to add a small additional section in 4.2
"

        4.2.x. Simple Video Communication Service with real-time text

          4.2.x.1. Description

    This use-case has the audio and video communication of the Simple
    Video Communication Service use-case (Section 4.2.1  <http://tools.ietf.org/html/draft-ietf-rtcweb-use-cases-and-requirements-09#section-4.2.1>).

    But in addition to this, the users can send and receive real-time text.
    While one user types in the real-time text area, it
    is nearly immediately presented to the other user.

"
A couple of requirements then need to be added in the A and F ranges in 
5.2 and 5.3, but I want to start with this initial proposal.

Gunnar Hellstrom

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 12 Nov 2012 13:15:26 +0100

I would prefer this to be added as a separate specification, rather than 
done at this time.
The reason being that this should be relatively easy to add on top of 
the data channel functionality, but will definitely take some time to 
specify, so should not be on the critical path for this round of 
specifications.

             Harald

On 11/12/2012 10:06 AM, Gunnar Hellström wrote:
> Real time sessions with video and audio are often accompanied with 
> some form of text communication.
>
> It is mentioned in a couple of places in RTCWEB related 
> specifications, but I see a need to get the specifications of how text 
> should be handled up to the level of specification of the other media.
>
> The real-time requirements are slightly lower on text communication 
> than on video and audio, so I realize that it can be carried by 
> mechanisms already available in the web browser environment. But even 
> if that will be the case, it needs to be considered and described, 
> because text communication is part of the real-time communication 
> concepts that RTCWEB belongs to.
>
> intended timing of the display, often related to another medium, such 
> as audio or video. The recipient can read the text presented timely to 
> end-to-end latency shall not be more than 2 seconds for usable text, 
> and one second for good usability. This is clearly lower than the 
> requirements on audio and video that traditionally is 400 ms for 
> usable end-to-end latency.
>
> A couple of requirements then need to be added in the A and F ranges 
> in 5.2 and 5.3, but I want to start with this initial proposal.
>
> Gunnar Hellstrom
>
>
>
>
>
>
> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Gunnar Hellström <[email-masked]>
Date: Mon, 12 Nov 2012 13:45:38 +0100

I think the opposite. Inclusion of real-time text from the beginning is 
crucial for successful deployment of RTCWEB.
It is so often a reason appears in a voice or video call to also convey 
something by text, that it would be a mistake to launch something 
without it.

It is not at all sure that the best alternative is to use the data 
channel for this purpose. The currently only standarized way to do 
real-time text is through RTP. It would be even easier than doing it in 
the data channel, to specify use of RTP for this purpose.

Then, interoperability with other environments, and use for emergency 
services is also easily achieved.

It is so simple and can be specified rapidly, while the gaps in other 
central RTCWEB specifications are filled.

Gunnar

----------------------

On 2012-11-12 13:15, Harald Alvestrand wrote:
> I would prefer this to be added as a separate specification, rather 
> than done at this time.
>> specifications, but I see a need to get the specifications of how 
>> text should be handled up to the level of specification of the other 
>> media.
>>
>> whole when completed by specific user action.***The recipient can 
>> read the sender's text *soon after it is transmitted.
>>
>> Timed text:
>>
>> as audio or video. The recipient can read the text presented timely 
>> to events in the other media.
>>
>>
>> I am most eager to see the use of real-time text specified in 
>> relation to RTCWEB, but it might be good to touch all three types.
>>
>>

---

From: Stefan Hakansson LK <[email-masked]>
Date: Mon, 12 Nov 2012 14:01:52 +0100

To me it seems that real-time text use can be solved using the data 
channel (with the accompanying W3C API).

I'm not sure anything needs to be standardized (except if there are 
legal requirements meaning we must).

Stefan

---

From: "Jim Barnett" <[email-masked]>
Date: Mon, 12 Nov 2012 06:04:28 -0800

I think that Stefan is right.  In the case of peer-to-peer communication, we are leaving the signaling up to the two end-points, and we can certainly expect them to figure out what to do with text (over the data channel) as well.  In the case of communication with a legacy device (i.e., one that expects to send or receive text), it will be up to the webRTC endpoint to adapt to the device's text API.  I don't see why we would need to specify anything in either case.  

If it eventually turns out that a) there are common peer-to-peer use cases involving text and b) it takes a lot of complicated JS coding to get them to work, then we can add API-level support in a future specification.  But the new API surface wouldn't enable anything that couldn't already be done over the data channel.  

- Jim

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Stefan Hakansson LK
Sent: Monday, November 12, 2012 8:02 AM
To: [email-masked]
Subject: Re: [rtcweb] Text communication in RTCWEB sessions

To me it seems that real-time text use can be solved using the data channel (with the accompanying W3C API).

I'm not sure anything needs to be standardized (except if there are legal requirements meaning we must).

Stefan

On 11/12/2012 01:45 PM, Gunnar Hellström wrote:
> I think the opposite. Inclusion of real-time text from the beginning 
> is crucial for successful deployment of RTCWEB.
> It is so often a reason appears in a voice or video call to also 
> convey something by text, that it would be a mistake to launch 
> something without it.
>
> real-time text is through RTP. It would be even easier than doing it 
> in the data channel, to specify use of RTP for this purpose.
>
>>> mechanisms already available in the web browser environment. But 
>>> even if that will be the case, it needs to be considered and 
>>> described, because text communication is part of the real-time 
>>> communication concepts that RTCWEB belongs to.
>>>
>>> intended timing of the display, often related to another medium, 
>>> such as audio or video. The recipient can read the text presented 
>>> timely to events in the other media.
>>>
>>>
>>> Real-time text for conversational use has the timing requirement 
>>> that end-to-end latency shall not be more than 2 seconds for usable 
>>> text, and one second for good usability. This is clearly lower than 
>>> the requirements on audio and video that traditionally is 400 ms for 
>>> An alternative is to add a small additional section in 4.2 "
>>>
>>>

---

From: Gunnar Hellström <[email-masked]>
Date: Mon, 12 Nov 2012 15:21:11 +0100

I do not see the logic here.
You propose that real-time text shall be left to the peers to agree on. 
Why are you then specifying audio and video, when such agreements can be 
left to the peers?

Or the opposite:
What makes the text codec in need of less standardization than audio and 
video?
It is the same need, both sides must know how to code and decode the 
medium, some presentation characteristics, and on what transport to send 
it and how it is identified when sending so that the other end will use 
the proper decoder.

What is the difference?

And why use the data channel that says that it is for non-media. 
Real-time text is a conversational medium.
( This can of course easily be amended by opening the data channel also 
for media by some changed wording in the spec, I agree that real-time 
text could be transported over that channel if you see any preference 
for that over the RTP transport. )

Gunnar

------------------

---

From: Stefan Håkansson LK <[email-masked]>
Date: Mon, 12 Nov 2012 15:38:44 +0100

> Why are you then specifying audio and video, when such agreements can be

Because, as it looks right now, among other reasons, it is not really 
feasible to handle all media processing (video coding, RTP 
encapsulation, jitter management, decoding, rate adaptation, ...) in JS 
right now. So we need to specify the exact formats so implementations 
interop.

My analogy would be that we leave it up to the application do decide how 
it uses the "raw" media streams/tracks (playing out, recording, mixing, 
....) in the same way as we leave it to the application how it uses the 
"raw" data channel (RT-text, send an image, game date, ...).

Another difference is that the app creates the text (perhaps as a result 
of user typing) to be sent over the data channel, while devices such as 
cameras and microphones creates the audio and video.

---

From: Adam Roach <[email-masked]>
Date: Mon, 12 Nov 2012 09:34:52 -0600

On 11/12/12 08:38, Stefan Håkansson LK wrote:
> encapsulation, jitter management, decoding, rate adaptation, ...) in 
> JS right now. So we need to specify the exact formats so 
> implementations interop.
>
> My analogy would be that we leave it up to the application do decide 
> how it uses the "raw" media streams/tracks (playing out, recording, 
> mixing, ....) in the same way as we leave it to the application how it 
> uses the "raw" data channel (RT-text, send an image, game date, ...).
>
> Another difference is that the app creates the text (perhaps as a 
> result of user typing) to be sent over the data channel, while devices 
> such as cameras and microphones creates the audio and video.

This is a very good explanation of what makes real-time text different 
(no processor-intensive codecs, no specialized access to hardware like 
cameras and microphones), but you didn't quite drive it all the way to 
its logical conclusion.

Actually, I'd like to throw in one more characteristic of real-time 
text: it is not as sensitive to latency as voice and video. This means 
that relaying it through a web server is eminently possible.

The key ramification of these three facts, taken together, is that _real 
time text is possible today_, without any of the WebRTC extensions. 
Trivial evidence of this is the existence of web tools like etherpad and 
typewith.me.

I'm not saying that we should ignore the work necessary to make 
real-time text go directly browser-to-browser. I *am* agreeing with 
earlier comments that it should be a lower priority. Such an effort 
would be adding a second, supplemental way of sending text in real-time. 
I'd like to see work put into getting the first and only way of sending 
audio and sending video finished before we start allocating resources 
towards specifying additional means of doing something that is already 
possible.

/a

---

From: "Eric Burger"<[email-masked]>
Date: Mon, 12 Nov 2012 10:43:19 -0500

Wild*ssed counter proposal: getting the signaling and API right with a codec  
that does not require too much codec-level negotiation, arguments about  
quality, or arguments about intellectual property, seems like an incredibly  
reasonable first step to delivering something that people can use, instead  
of writing books about.

If we cannot get RTT to work, we will never get VP8 to work.

--
Sent from my mobile device. Thanks be to lemonade!  
http://www.standardstrack.com/ietf/lemonade/

-----Original message-----
From: Adam Roach <[email-masked]>
To: [email-masked]
Sent: Mon, Nov 12, 2012 15:35:08 GMT+00:00
Subject: Re: [rtcweb] Text communication in RTCWEB sessions

---

From: Randell Jesup <[email-masked]>
Date: Mon, 12 Nov 2012 11:19:42 -0500

> I think that Stefan is right.  In the case of peer-to-peer communication, we are leaving the signaling up to the two end-points, and we can certainly expect them to figure out what to do with text…

This may help you: at Atlanta, I gave an (impromptu) discussion on 
DataChannels in rtcweb.  One thing I proposed adding to the existing 
implementation is a separate "protocol" field, similar to WebSockets.  
Most uses of DataChannels will be for application-specific protocols 
between two instances of the same app.  However, since we have 
heterogeneous applications that may end up talking to each other, having 
one open a DataChannel with label "Chat" doesn't help (in the 
hetereogeneous case) if the two apps don't know what protocol the data 
in the "Chat" channel is.

So with a protocol field, you can specify that the "Chat" channel is a 
specific chat protocol, and if both sides implement it, then you're 
set.  A Text/Chat protocol (and maybe a file transfer protocol) seem 
like an idea set to specify as an example.  Wherever possible, I'd 
prefer to specify them as "Foo over a DataChannel 
(reliable/unreliable)", to avoid re-implementing the wheel and easing 
implementation.

-- 
Randell Jesup
[email-masked]

---

From: Igor Faynberg <[email-masked]>
Date: Mon, 12 Nov 2012 14:03:27 -0500

On 11/12/2012 4:06 AM, Gunnar Hellström wrote:
> ...
> *ietf.org/mailman/listinfo/rtcweb

In this case, would that mean that when I type something and then want 
to correct it, backspaces would be transmitted, too?  If so, I doubt it 
is useful.   The second case, with the user having control over sending 
of the message, is much better.

Igor

---

From: Igor Faynberg <[email-masked]>
Date: Mon, 12 Nov 2012 14:04:27 -0500

+1

Igor

---

From: Adam Roach <[email-masked]>
Date: Mon, 12 Nov 2012 13:23:37 -0600

On 11/12/12 09:43, Eric Burger wrote:
> Wild*ssed counter proposal: getting the signaling and API right with a 
> codec that does not require too much codec-level negotiation, 
> arguments about quality, or arguments about intellectual property, 
> seems like an incredibly reasonable first step to delivering something 
> that people can use, instead of writing books about.

I look forward to the UTF-8 versus UTF-16 battle. :)

/a

---

From: Gunnar Hellström <[email-masked]>
Date: Mon, 12 Nov 2012 20:29:46 +0100

Yes, backspaces are transmitted.

I can assure that it is useful.

And it needs not to be compared to messaging. It is intended for real 
conversational use of text, without the artificially created pauses for 
composing complete messages occurring in messaging. Just let the text go 
as you type it, with slight delay for time-sampling as for audio and video.
It provides a rapid and smooth way to have the conversing parties 
synchronized in thoughts.

There is of course still scope for traditional messaging even if I am 
more interested to see the real-time text implemented together with the 
other real-time media in RTCWEB. And there is even work on the way to 
introduce correction in already sent messages in traditional messaging. 
It is in XMPP XEP-0308. http://xmpp.org/extensions/xep-0308.html    Also 
that is useful, and reduces stress, frustration and extra typing. It is 
usually when you have just completed and sent your message that you 
discover the mistake and want to correct it.

Gunnar

___________________________________________________
Gunnar Hellström
Omnitor
[email-masked]
+46708204288

On 2012-11-12 20:03, Igor Faynberg wrote:
> to correct it, backspaces would be transmitted, too?  If so, I doubt 
> it is useful.   The second case, with the user having control over 
> sending of the message, is much better.
>
> Igor
>
>

---

From: Igor Faynberg <[email-masked]>
Date: Mon, 12 Nov 2012 14:51:01 -0500

Thanks for a quick reply!  (Meanwhile, I was assured by Keith Drage that 
this feature is needed by deaf--something I  was not aware of, so I need 
to catch up on learning.  I imagine this would be used in speech-to-text 
conversion and transmission.)

Of course, thanks to my advanced age, I remember the Unix "Write" 
application that did just that.  My bad experience with it was that 22 
years ago I used that very application to communicate with my big boss, 
and I wrote  something without counting to 1,000.  I should have 
counted:  The backspaces did not help...

  So I walked to Dennis Ritchie's office, told him what happened, and 
suggested that "Write" ought to display messages only when Return is 
hit. Surprisingly, he immediately agreed with me, but System 5 was 
already out of his immediate control.  He promised to follow-up through 
the Unix Consortium, but soon that entity disappeared from our radar 
screen.  Inferno and Plan 9 though transmitted only full messages.

This is what prompted my question.

Now it has been answered!

Igor

On 11/12/2012 2:29 PM, Gunnar Hellström wrote:
> conversational use of text, without the artificially created pauses 
> for composing complete messages occurring in messaging. Just let the 
> text go as you type it, with slight delay for time-sampling as for 
> audio and video.
> more interested to see the real-time text implemented together with 
> the other real-time media in RTCWEB. And there is even work on the way 
> to introduce correction in already sent messages in traditional 
> messaging. It is in XMPP XEP-0308. 
> http://xmpp.org/extensions/xep-0308.html    Also that is useful, and 
> reduces stress, frustration and extra typing. It is usually when you 
> have just completed and sent your message that you discover the 
> mistake and want to correct it.
>
> Gunnar
>> In this case, would that mean that when I type something and then 
>> want to correct it, backspaces would be transmitted, too?  If so, I 
>> doubt it is useful.   The second case, with the user having control 
>> over sending of the message, is much better.
>>
>> Igor
>>
>>

---

From: Gunnar Hellström <[email-masked]>
Date: Mon, 12 Nov 2012 21:05:21 +0100

Igor,
Thanks for the explanation.
And I am striving for the communicating users to get back the rights to 
display their mistakes ;-)

It is true that the feature is often mentioned in relation to accessible 
communication, but it is a generally usable feature.

Gunnar

On 2012-11-12 20:51, Igor Faynberg wrote:
> Thanks for a quick reply!  (Meanwhile, I was assured by Keith Drage 
> that this feature is needed by deaf--something I  was not aware of, so 
> I need to catch up on learning.  I imagine this would be used in 
> speech-to-text conversion and transmission.)
>
> years ago I used that very application to communicate with my big 
> boss, and I wrote  something without counting to 1,000.  I should have 
> already out of his immediate control.  He promised to follow-up 
> through the Unix Consortium, but soon that entity disappeared from our 
> radar screen.  Inferno and Plan 9 though transmitted only full messages.
>
>> the other real-time media in RTCWEB. And there is even work on the 
>> way to introduce correction in already sent messages in traditional

---

From: Randell Jesup <[email-masked]>
Date: Mon, 12 Nov 2012 15:27:20 -0500

> counted:  The backspaces did not help...

Write(1) is one-way, talk(1) is two-way, but with real-time view of 
typing like Write.

On 11/12/2012 2:48 PM, Jim Barnett wrote:
> Yes, but is that a requirement on us, or on the people who roll out
> services using webRTC?  If RTT can be done via a bunch of (potentially
> ugly) JS code and the data channel,  then it's the service providers'
> problem to write that ugly JS code.  There's no requirement for the
> W3C/IETF to provide native support for RTT as long as it's possible to
> implement it on top of our standards.
>
> - Jim
> P.S.  I intend this as an argument for looking at RTT later, not for
> skipping it altogether.

I agree with Jim here - we-the-rtcweb-working-group do not have to spec 
out the solution to a specific country's legislative VRS/text 
requirements here, but should provide a mechanism for someone providing 
a service using these protocols to comply with (most) of those standards 
around the world.  We can't follow all such legal requirements, as they 
are myriad and ever-changing, which is why good, generic transports and 
programmability are an excellent answer to the issue, especially as a 
start, and encourage people building responses to those requirements to 
do so in a standard way (preferably through a standards body, even an 
appropriate IETF working group).

For example, the current VRS spec requires implementation of H.323 and 
ENUM... (and H.263 for video).

I'll note that other country-specific telecom-regulation-requirements 
fall into this bucket as well, which I have no wish to bake into the 
core spec.

> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Richard Shockey
> Sent: Monday, November 12, 2012 2:41 PM
> To: 'Dale R. Worley'; [email-masked]
> Dale is right ..you can delay the discussion but not the requirement.
> The regulators will insist on this.
>
> http://transition.fcc.gov/cgb/dro/cvaa.html
>
> http://europa.eu/legislation_summaries/information_society/legislative_f
> rame
> work/l24216a_en.htm
>

-- 
Randell Jesup
[email-masked]

---

From: Gunnar Hellström <[email-masked]>
Date: Mon, 12 Nov 2012 23:25:56 +0100

I am not talking about this for any specific requirements of a country.
And not specfically from the accessibility point of view, even if it is 
included in the picture.

All have a need for text communication occasionally or often in a call 
with voice or video.
And the less wait time it causes, the more popular it will be.

Same with the standards it uses. The more wide spread and interoperable 
it is, the more happy users it will get.
And we can be proactive. Making a global solution usable for all should 
satisfy also the accessibility side better than any national solution.

After the discussions today, I can see four solutions. It might be good 
to take them all a bit further to check pros and cons:

1. RFC 4103/T.140 in RTP/AVPF and SDP exchange in the same way as for video.

2. T.140 and something RFC4103-like in RTCWEB data channel and a 
specific protocol id.

3. XMPP XEP-0301 negotiated with SDP as an expired draft proposes, and 
transmitted by JS

4. XMPP XEP-0301 transmitted by JS / Websockets and negotiated in XMPP, 
and just specified to be used for this purpose.

All 4 are variants fulfilling the use case extending the video telephony 
use case that I proposed to be extended by audio and real-time text.  ( 
audio might have been there, but very vaguely described )

Gunnar

On 2012-11-12 21:27, Randell Jesup wrote:
>> that this feature is needed by deaf--something I  was not aware of, 
>> so I need to catch up on learning.  I imagine this would be used in 
>> application that did just that.  My bad experience with it was that 
>> 22 years ago I used that very application to communicate with my big 
>> boss, and I wrote  something without counting to 1,000.  I should 
>> have counted:  The backspaces did not help...
>
> I agree with Jim here - we-the-rtcweb-working-group do not have to 
> spec out the solution to a specific country's legislative VRS/text 
> requirements here, but should provide a mechanism for someone 
> providing a service using these protocols to comply with (most) of 
> those standards around the world.  We can't follow all such legal 
> requirements, as they are myriad and ever-changing, which is why good, 
> generic transports and programmability are an excellent answer to the 
> issue, especially as a start, and encourage people building responses 
> to those requirements to do so in a standard way (preferably through a 
> standards body, even an appropriate IETF working group).
>

---

From: Silvia Pfeiffer <[email-masked]>
Date: Tue, 13 Nov 2012 10:30:51 +1100

I'd like to enter WebVTT or more generically the TextTrackCue API of HTML
into this discussion as a format for specifying timed text snippets - in
particular for captions/subtitles, but also generally for any timed text
requirements. They are being supported by all browsers, so should be
relevant to WebRTC.

The way I can see this being used is that you can look at a TextTrackCue as
a "frame" of text with time stamps that can be transferred as a text entity
over something like XMPP.

In fact, Apple's HLS [1] is using WebVTT to transport captions.

If we're starting to talk about baseline formats for timed text/data in
WebRTC, WebVTT sounds like a pretty good candidate to me. Combined with a
protocol that includes backspace and other live text editing features, I
think it can provide for the transfer of richer text and not just plain
text.

Regards,
Silvia.

[1] http://tools.ietf.org/html/draft-pantos-http-live-streaming-10

On Tue, Nov 13, 2012 at 9:25 AM, Gunnar Hellström <
[email-masked]> wrote:

> Same with the standards it uses. The more wide spread and interoperable it
> is, the more happy users it will get.
> After the discussions today, I can see four solutions. It might be good to
> take them all a bit further to check pros and cons:
>
> 2. T.140 and something RFC4103-like in RTCWEB data channel and a specific
> protocol id.
>
>>> this feature is needed by deaf--something I  was not aware of, so I need to
>>> catch up on learning.  I imagine this would be used in speech-to-text
>>> years ago I used that very application to communicate with my big boss, and
>>> I wrote  something without counting to 1,000.  I should have counted:  The
>>> backspaces did not help...
>>>
>>
>> out the solution to a specific country's legislative VRS/text requirements
>> here, but should provide a mechanism for someone providing a service using
>> these protocols to comply with (most) of those standards around the world.
>>  We can't follow all such legal requirements, as they are myriad and
>> ever-changing, which is why good, generic transports and programmability
>> are an excellent answer to the issue, especially as a start, and encourage
>> people building responses to those requirements to do so in a standard way
>> (preferably through a standards body, even an appropriate IETF working
>> group).
>>
>> I'll note that other country-specific telecom-regulation-**requirements
>> fall into this bucket as well, which I have no wish to bake into the core
>> spec.
>>
>>> From: [email-masked] [mailto:rtcweb-bounces@ietf.**org<[email-masked]>]
>>> On Behalf
>>> Of Richard Shockey
>>> http://transition.fcc.gov/cgb/**dro/cvaa.html<http://transition.fcc.gov/cgb/dro/cvaa.html>
>>>
>>> http://europa.eu/legislation_**summaries/information_society/**
>>> legislative_f<http://europa.eu/legislation_summaries/information_society/legislative_f>
>>> rame
>>> work/l24216a_en.htm
>>>
>>>
>>
> ______________________________**_________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/rtcweb<https://www.ietf.org/mailman/listinfo/rtcweb>
>

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 13 Nov 2012 16:58:17 +0900

> I look forward to the UTF-8 versus UTF-16 battle. :)

The IETF is very strongly UTF-8. The W3C is very strongly UTF-8 on the 
wire, but of course UTF-16 in JavaScript. So these battles are over.

Regards,    Martin.

---

From: "Olle E. Johansson" <[email-masked]>
Date: Tue, 13 Nov 2012 09:55:24 +0100

12 nov 2012 kl. 13:15 skrev Harald Alvestrand <[email-masked]>:

> I would prefer this to be added as a separate specification, rather than done at this time.
> The reason being that this should be relatively easy to add on top of the data channel functionality, but will definitely take some time to specify, so should not be on the critical path for this round of specifications.
T.140 is a codec in the  RTP flow and is implemented in Asterisk and a couple of Video SIP phones.
I see no reason to move it to the data channel, that would limit interoperability.

Much easier - and faster - to implement in the RTP subsystem as it is already covered by SDP offer/answer.

/O

>> Real time sessions with video and audio are often accompanied with some form of text communication.
>> 
>> It is mentioned in a couple of places in RTCWEB related specifications, but I see a need to get the specifications of how text should be handled up to the level of specification of the other media.
>> 
>> The real-time requirements are slightly lower on text communication than on video and audio, so I realize that it can be carried by mechanisms already available in the web browser environment. But even if that will be the case, it needs to be considered and described, because text communication is part of       the real-time communication concepts that RTCWEB belongs to.
>> 
>> Real-time text is text transmitted while it is being typed or created. The recipient can read the sender's text as it is written, without waiting.
>> 
>> Text messaging: 
>> 
>> Text messaging is text collected to complete messages and sent as a whole when completed by specific user action. The recipient can read the sender's text soon after it is transmitted.
>> 
>> Timed text:
>> 
>> Timed text is text sent together with with information about the intended timing of the display, often related to another medium, such as audio or video. The recipient can read the text presented timely to events in the other media.
>> 
>> 
>> Real-time text for conversational use has the timing requirement that end-to-end latency shall not be more than 2 seconds for usable text, and one second for good usability. This is clearly lower than the requirements on audio and video that traditionally is 400 ms for usable end-to-end latency.
>> 
>> Real-time text is briefly mentioned in a requirement in draft-ietf-rtcweb-data-channel
>> All three text communication forms are mentioned in http://tools.ietf.org/id/draft-aboba-rtcweb-ecrit-00.txt
>> 
>> I am most eager to see the use of real-time text specified in relation to RTCWEB, but it might be good to touch all three types.
>> 
>> 
>> 4.2.1.  Simple Video Communication Service with audio and real-time text
>> 
>> 4.2.1.1.  Description
>> 
>>    reject the session. Audio and real-time text is also available
>>    during the session.
>> 
>>    displays during the session.  Audio from each user is presented to 
>>    Video Communication Service use-case (Section 4.2.1).
>> 
>> A couple of requirements then need to be added in the A and F ranges in 5.2 and 5.3, but I want to start with this initial proposal.
>> 
>> Gunnar Hellstrom
>> 
>> 
>> 
>> 
>> 
>>

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 13 Nov 2012 10:55:11 +0100

On 11/13/2012 09:55 AM, Olle E. Johansson wrote:
>
> 12 nov 2012 kl. 13:15 skrev Harald Alvestrand <[email-masked] 
> <mailto:[email-masked]>>:
>
Anything is possible, if someone is willing to do it.

Olle and Gunnar, can you undertake to write a complete specification for 
how to use T.140 with RTCWEB, including how it should fit in with the 
API specification, and whether or not it supports the needs that Gunnar 
has claimed for text services?

I don't understand T.140 - so I can't do the work. Someone who wants it 
should.

---

From: Gunnar Hellström <[email-masked]>
Date: Tue, 13 Nov 2012 11:20:03 +0100

I am of course willing to contribute.

Gunnar

On 2012-11-13 10:55, Harald Alvestrand wrote:
>>> The reason being that this should be relatively easy to add on top 
>>> of the data channel functionality, but will definitely take some 
>>> time to specify, so should not be on the critical path for this 
>>> round of specifications.
> Olle and Gunnar, can you undertake to write a complete specification 
> for how to use T.140 with RTCWEB, including how it should fit in with 
> the API specification, and whether or not it supports the needs that 
> Gunnar has claimed for text services?
>
> I don't understand T.140 - so I can't do the work. Someone who wants 
> it should.
>
>
>

---

From: [email-masked] (Dale R. Worley)
Date: Tue, 13 Nov 2012 10:47:03 -0500 (EST)

> From: "Martin J. DC<rst" <[email-masked]>
> 

RFC 4103 specifies that text/t140 is encoded in UTF-8.  Of course, the
API can present it to the Javascript in whatever way it wants.  There
are a bunch of CJK extension sets above 0x10000, though, so UTF-16 may
be problematic.

Dale

---

From: Adam Roach <[email-masked]>
Date: Tue, 13 Nov 2012 10:33:54 -0600

> wire, but of course UTF-16 in JavaScript. So these battles are over.

Yeah, the smiley face was supposed to convey that the the comment was 
meant as tongue-in-cheek.

/a

---

From: Gunnar Hellström <[email-masked]>
Date: Sun, 18 Nov 2012 00:33:38 +0100

I have proposals for adding real-time text to the use cases document.
I hope it is OK with a word file with changemarks to show the change 
proposals. (attached )

The new medium is proposed to be added straight into current use cases.

/Gunnar

---

From: Eric Rescorla <[email-masked]>
Date: Sat, 17 Nov 2012 15:46:19 -0800

> I hope it is OK with a word file with changemarks to show the change

That's really pretty hard to use. Can you provide some sort of text diff
instead?

-Ekr

> The reason being that this should be relatively easy to add on top of the
> data channel functionality, but will definitely take some time to specify,
> so should not be on the critical path for this round of specifications.
>
> how to use T.140 with RTCWEB, including how it should fit in with the API
> specification, and whether or not it supports the needs that Gunnar has
> claimed for text services?
>
> rtcweb mailing [email-masked]://www.ietf.org/mailman/listinfo/rtcweb
>
>
>
>
> rtcweb mailing [email-masked]://www.ietf.org/mailman/listinfo/rtcweb
>
>
>

---

From: Gunnar Hellström <[email-masked]>
Date: Sun, 18 Nov 2012 08:40:04 +0100

On 2012-11-18 00:46, Eric Rescorla wrote:
>
> On Sat, Nov 17, 2012 at 3:33 PM, Gunnar Hellström 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     I hope it is OK with a word file with changemarks to show the
>     change proposals. (attached )
>
>
How about the attached version as a start.
You can do a diff against the -09 version.

The main changes are in
-The definitions
-Use case simple video
-Use case multi-part video
-Use case multi-part video conference
-Requirements F39 - F42
-Requirement A27
-Many requirement lists in use cases.

/Gunnar
>>>>     12 nov 2012 kl. 13:15 skrev Harald Alvestrand
>>>>     <[email-masked] <mailto:[email-masked]>>:
>>>>
>>>>>     The reason being that this should be relatively easy to add on
>>>>>     top of the data channel functionality, but will definitely
>>>>>     take some time to specify, so should not be on the critical
>>>>>     path for this round of specifications.
>>>>     T.140 is a codec in the  RTP flow and is implemented in
>>>>     Asterisk and a couple of Video SIP phones.
>>>>     Much easier - and faster - to implement in the RTP subsystem as
>>>>     it is already covered by SDP offer/answer.
>>>>
>>>     Olle and Gunnar, can you undertake to write a complete
>>>     specification for how to use T.140 with RTCWEB, including how it
>>>     should fit in with the API specification, and whether or not it
>>>     supports the needs that Gunnar has claimed for text services?
>>>
>>>     I don't understand T.140 - so I can't do the work. Someone who
>>>     wants it should.
>>>
>>>
>>>
>>>     [email-masked]  <mailto:[email-masked]>
>>     [email-masked]  <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Göran Eriksson AP <[email-masked]>
Date: Sun, 18 Nov 2012 09:00:39 +0000

A diff would have been better but I'll look into the proposal.

One note though- You have added the real time text into the basic video chat example. The way this was structured was to move from basic-atomic- use cases, adding complexity such as more/other media&data streams in separate use cases.

I would like to stick to that approach. Any thoughts about that.

Regards
Göran

Sent from my iPad

On 18 nov 2012, at 08:40, "Gunnar Hellström" <[email-masked]<mailto:[email-masked]>> wrote:

On Sat, Nov 17, 2012 at 3:33 PM, Gunnar Hellström <[email-masked]<mailto:[email-masked]>> wrote:
I have proposals for adding real-time text to the use cases document.
I hope it is OK with a word file with changemarks to show the change proposals. (attached )

The main changes are in
-The definitions
-Use case simple video

/Gunnar
-Ekr

/Gunnar

Gunnar

12 nov 2012 kl. 13:15 skrev Harald Alvestrand <[email-masked]<mailto:[email-masked]>>:

Olle and Gunnar, can you undertake to write a complete specification for how to use T.140 with RTCWEB, including how it should fit in with the API specification, and whether or not it supports the needs that Gunnar has claimed for text services?

I don't understand T.140 - so I can't do the work. Someone who wants it should.

_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

<draft-ietf-rtcweb-use-cases-and-requirements-09 plus rtt integrated.txt>
_______________________________________________
rtcweb mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Gunnar Hellström <[email-masked]>
Date: Sun, 18 Nov 2012 11:20:38 +0100

For SDP, changes in line with the example below are proposed to get 
real-time text included in draft-nandakumar-rtcweb-sdp

----------------------

5.  WebRTC Session Description Examples

    A typical web based real-time multimedia communication session can be
    characterized as below:

       It has zero or more Media
       streams with  any combination of Audio, Video and Real-time text
       MAY contain zero or more non-media data streams
       All the streams are secured with DTLS/SRTP
       ICE processing for NAT Traversal
       Sessions over IPv4-only, IPv6-only, dual-stack based clients.
....
5.1.  Secure Two-Way Audio,Video,Real-time text  and Data with RTCP Feedback

    This use-case allows two users to participate in a two-way
    communication session securely on their WebRTC enabled Web browsers.

    title WebRTC Session - 2-Way Secure Audio,Video,RTT with RTCP Feedback
    Alice->Bob: Offer(Audio:G.711,Opus,iLBC Video:H.264,VP8  RTT:T.140)
    Bob->Alice: Answer(Audio:Opus,DTMF Video:H.264  RTT:T.140)
    Alice->Bob: Two-way Opus Audio, H.264 Video,  T.140 Real-time text
    note right of Alice
      Session also suports RTP/RTCP Mux, RTCP feedback (nack,pli)
    end note

    More specifically, this use-case demonstrates following aspects of a
    WebRTC session
       SRTP with DTLS based encryption
       RTP and RTCP Muxing
       RTCP based feedback and reduced size support
       ICE processing for NAT Traversal
       Audio Codec Offered :  PCMU, Opus, iLBC
       Audio Codec Answered :  Opus
       Video Codecs Offered:  H.264, VP8
       Video Codecs Answered:  H.264
       Real-time text Offered: T.140
       Real-time text Answered: T.140
       Data Channel Support
    The tables (4.1 and 4.2) below capture in detail, the initial SDP
    Offer and Answer messages exchanged.

    +------------------------------------------+------------------------+
    | SDP Contents                             | RFC#/Notes             |
    +------------------------------------------+------------------------+
/------Add the following after video SDP and before SCTP SDP-----/

    | m=text 60109 RTP/SAVPF 105 104           | [RFC4566]              |
    | c= IN IP4 24.23.204.141                  | [RFC4566]              |
    | a=rtpmap:105 RED/1000                    | [RFC4102]              |
    | a=rtpmap:104 T140/1000                   | [RFC4103]              |
    | a=fmtp:105 104/104/104                   | [RFC4103]              |
    | a=sendrecv                               | [RFC3264] - Alice can  |
    |                                          | send and recv rtt      |
    | a=rtcp-mux                               | [RFC5761] - Alice can  |
    |                                          | perform RTP/RTCP       |
    |                                          | Muxing on port 54609   |
    | a=candidate:0 1 UDP 2113667327           | [RFC5245] - Host ICE   |
    | 192.168.1.4 60109 typ host               | Candidate              |
    | a=candidate:1 1 UDP 694302207            | [RFC5245] - Server     |
    | 24.23.204.141 60109 typ srflx raddr      | Reflexive ICE          |
    | 192.168.1.4 rport 60109                  | Candidate for the      |
    |                                          | above host candidate   |
    | a=candidate:0 2 UDP 2113667326           | [RFC5245] - Second     |
    | 192.168.1.4 61778 typ host               | Host Candidate         |
    | a=candidate:1 2 UDP 1694302206           | [RFC5245] -Server      |
    | 24.23.204.141 61778 typ srflx raddr      | Reflexive Candidate    |
    | 192.168.1.4 rport 61778                  | for the Second Host    |
    |                                          | Candidate              |
    | a=rtcp-fb:105 nack                       | [RFC5104] - Indicates  |
    |                                          | NACK RTCP feedback     |
    |                                          | support                |
/-------End of addition for real-time text---------------------/
    +------------------------------------------+------------------------+

                           Table 1: 4.1 SDP Offer

And similarly for the SDP answer and the other examples.

If that is the accepted principle, we can go ahead and propose the complete changes.

/Gunnar

-----------------------------------------

---

From: Gunnar Hellström <[email-masked]>
Date: Sun, 18 Nov 2012 12:34:27 +0100

On 2012-11-18 10:00, Göran Eriksson AP wrote:
> One note though- You have added the real time text into the basic 
> video chat example. The way this was structured was to move from 
> basic-atomic- use cases, adding complexity such as more/other 
> media&data streams in separate use cases.

Yes, I prepared also the "sequential" variant,  just adding the 
real-time text in a short clause among the other atomic additions.

But I realized the risk for it to be seen as an addition for special 
cases, rather than one of the three important natural components of 
human conversation. You talk, you show, you write to each other. Both 
face-to-face and remotely.

So, I preferred the upfront integrated variant, and sent that to the group.

/Gunnar

---

From: Göran Eriksson AP <[email-masked]>
Date: Sun, 18 Nov 2012 12:13:29 +0000

What is "most natural" can be discussed and we could add a lot of stuff to capture that, :-), but real time text chat for the hearing impaired is a relevant use case in its own right as is a video and text chat case.

Having audio, video and text chat is not the most basic use case- audio only, video only or text chat only is. Hence, single stream cases are basic. Perhaps not most natural, but still essential use cases to secure.

Concerning what most important or natural, the use case draft is not governing that. The draft was mainly intended for providing a common set of different kinds of architecture, including supporting the spilt into API related requirements and protocol requirements, which is useful when trying to coordinate the w3c API work and that of IETF.

For instance the objective of making an API that is easy to use for the ordinary web developer who just want to enrich his web page with an 1-1 audio or video chat component is one reason for having the "atomic" use case approach. Another is when we move towards test spec time in the w3c, where many test cases will be very basic, :-).

So I would still like to keep this approach, adding the cases You describe where appropriate, be that in the existing use case or as new ones.

Would You like me to have a go at it, drafting a proposal?

Regards
Göran

Sent from my iPad

On 18 nov 2012, at 12:34, "Gunnar Hellström" <[email-masked]> wrote:

> Yes, I prepared also the "sequential" variant,  just adding the real-time text in a short clause among the other atomic additions.
> 
> But I realized the risk for it to be seen as an addition for special cases, rather than one of the three important natural components of human conversation. You talk, you show, you write to each other. Both face-to-face and remotely.
>

---

From: Gunnar Hellström <[email-masked]>
Date: Sun, 18 Nov 2012 13:40:16 +0100

Yes, please have a go on it.
You can use my initial effort on the atomic variant as a base. Attached.

Gunnar

---

From: Göran Eriksson AP <[email-masked]>
Date: Sun, 18 Nov 2012 12:43:43 +0000

Ok.

Göran

18 nov 2012 kl. 13:40 skrev "Gunnar Hellström" <[email-masked]>:

> <draft-ietf-rtcweb-use-cases-and-requirements-09 plus rtt sequentially.txt>

---

From: Gunnar Hellström <[email-masked]>
Date: Sun, 18 Nov 2012 20:55:02 +0100

After a rapid browse, this is my view of what needs to be done to 
specify real-time text in RTCWEB and WebRTC.

1. draft-ietf-rtcweb-overview
      Make definition of Media to include real-time text
     Define the short form rtt
     Add rtt to descriptions in e.g. sections 2.1 and 2.2
     non-critical

2. draft-ietf-rtcweb-use-cases-and-requirements
     add a couple of rtt use cases
     add requirements for rtt
     add api requirements for rtt
     important, initiated

3. draft-nandakumar-rtcweb-sdp
     add codec description for rtt
     add sdp for rtt for the use cases
     important, initiated

4. draft-x-rtcweb-rtt
     rtt specifics for rtt codec and rtp usage
     ( take inspiration from draft-ietf-rtcweb-audio )
     new needed draft - not initiated

5. draft-ietf-rtcweb-jsep
     add rtt to API MediaHints  clause 6.1.1
     add rtt to examples in chapter 7.
    important

6. draft-ietf-rtcweb-rtpusage
     mention rtt
     add rtt to mixer discussion
     non-critical

7. draft-thomson-rtcweb-api-req
     mention rtt
     non-critical

8. draft-aboba-rtcweb-ecrit
     adjust rtt discussion for emergency calls to current status
     non critical

9. w3c.webrtc
     add rtt API, e.g. to Network Stream API
     important, urgent

10. w3c getusermedia
     add rtt to GetUserMedia API
     important, urgent

Any more?
Any less?

/Gunnar

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 19 Nov 2012 02:09:45 +0100

>     new needed draft - not initiated
This is the one that will have to make the compelling case that we know 
what we're talking about.
So far, I'm not compelled - but as I said before, I don't understand 
T.140 or the case for picking that particular solution over the 5-6 
other solutions proposed in this discussion.

(I sure hope we're not going to revisit the RFC 4103 / RFC 4351 
discussion; I don't know what it was, but I recognize the tracks of 
battles when I see them. Gunnar is an author on both documents, so I 
assume he can explain if needed.)

             Harald

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 19 Nov 2012 10:19:33 +0900

On 2012/11/14 0:47, Dale R. Worley wrote:

>> From: "Martin J. DC<rst"<[email-masked]>
>>
> RFC 4103 specifies that text/t140 is encoded in UTF-8.

Great to know.

> Of course, the

Just to avoid misunderstandings: UTF-16 can represent all Unicode 
codepoints, even U+10000 and above. For these, it needs 2 16-bit code 
units, so certain string manipulations have to be done with care.

Regards,   Martin.

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 18 Nov 2012 23:48:29 -0800

Harald said:

"This is the one that will have to make the compelling case that we
    know what we're talking about."

[BA] To understand what we're talking about, some uses cases would be helpful, in order to understand which of the following requirements apply:

1. Need to interoperate with existing RTT implementations (e.g. XEP-0301 or RFC 4103).   
2. Need to support "real-time text" capabilities as defined in M376 or the Access Board requirements 2.0 (both still in draft). 
3. Need to support "lip sync" between audio/video/text (as opposed to simply replication of inter-character spacing).

Requirement #1 can be supported in a JS library with an appropriate gateway (e.g. BOSH/Websockets for XEP-0301, an SCTP to RTP gateway for RFC 4103) without any support in the browser. 

Requirement #2 can be supported via the data channel without a standard, or by extracting parts of a standard (e.g. XEP-0301 RTT support)

Requirement #3 may require native support, but a clarification of the use case would be helpful.  For example, in realtime conference with captioning, there is a time lag, so "lip syncing" of text to the audio/video (as is possible in RFC 4103) may not in practice perform that much better than replication of inter-character spacing as is supported in XEP-0301, though it would be considerably more complex to implement.

---

From: Gunnar Hellström <[email-masked]>
Date: Mon, 19 Nov 2012 09:54:08 +0100

On 2012-11-19 02:09, Harald Alvestrand wrote:
> This is the one that will have to make the compelling case that we 
> know what we're talking about.

Yes, now after the overview of what would be needed in RTCWEB and WebRTC 
to include RTT I agree that starting with "draft-x-rtcweb-rtt" would be 
good for clarifying requirements and alternatives and draw conclusions.

For your info, all real-time text methods in IP environments transmit 
time-sampled text, with sample time of 300 -700 ms.

1. RFC 4103 uses RTP, with redundancy to get reliability, and declares 
the stream in SDP as text medium. T.140 is the text codec, just saying 
that text shall be transmitted by UTF-8 and specifying some control 
codes. There are two payload types combined RED for simple RFC 2198 type 
redundancy and T140 for the text itself. The recommended sample time is 
300 ms. This is in use in SIP applications and also specified in 
emergency service specifications.

2. RFC4351 also uses RTP in a very similar way to send RTT. But it has 
SDP declaration as audio and is transmitted multiplexed with audio 
codecs in a similar way as RFC 4733. The main use was intended to be for 
reliable transport of legacy real-time text through IP between VoIP 
gateways, where the extra ports needed for a separate text RTP stream 
through separate ports would possibly be expensive. RFC4351 was given 
Historic status already from its approval in order to discourage from 
any other use. Transmission is UTF-8-coded.

3. XEP-0301 is a draft XMPP extension, that transmits time sampled text 
in XMPP Messages, but not as Body, but as RTT XML elements, so it can be 
sent and presented as a gradually built up message, and then replaced by 
the message when completed. The recommended sample time is 700 ms, and 
in order to avoid jerky impression, the presentation of keystrokes are 
timed within these transmission intervals. Transmission is UTF-8.

Gunnar

---

From: Randell Jesup <[email-masked]>
Date: Mon, 19 Nov 2012 08:10:30 -0500

>     add rtt to GetUserMedia API

W3C-ish stuff:
This one would be complex, and probably shouldn't be in getUserMedia 
(though it could).  A more equivalent API would be mediastream = 
video_element.captureStreamUntilEnded() - create a MediaStream from a 
video (or audio) element; you could do something like that.

To implement this, there needs to be a DOM element to capture it via.  
The main options would be an <input> element or a media element.  But 
the are more issues, especially with a media element as a source, and 
serious UI issues if specified in them.

A better question would be "What's the minimal (and most generic) API 
that covers the use cases, and do these abilites already exist?"

I'll make an assertion: existing facilities for access to keystroke data 
(and certainly for text display) are sufficient, and the only API 
*needed* (and which covers all sorts of other use cases) would be the 
ability to insert/receive text data from a MediaStream from JS.  E.g 
textstream = <whatever>; textstream.insert(key) (or keys).  And the same 
in the other direction: textstream.ondata(function(keys, time) { do 
whatever with the keys });  I'd assert for text in a MediaStream that's 
a far more natural JS API - and so happens to also be very close to the 
API of a simple keystream over WebSocket or DataChannel.

Another thing to note is that WebSockets and DataChannels are virtually 
identical (on purpose), so any work to standardize text chat over 
DataChannels would apply equally well to chat over WebSockets, which 
would have lots of real-world applications.

This assumes that a MediaStream is the correct transfer mechanism (and 
correct API at the W3 level), which is definitely something I'd agree to 
at this point.  I also wouldn't rule it out (I see some arguments in 
favor, as well as against) - but this needs to occur in 
public-webrtc@w3, and would do so before going too far down the path 
here assuming a MediaStream is the mechanism.

For this part of the conversation, I strongly suggest it occur on 
public-webrtc, so reply there please.

-- 
Randell Jesup
[email-masked]

---

From: Randell Jesup <[email-masked]>
Date: Mon, 19 Nov 2012 08:27:36 -0500

On 11/19/2012 3:54 AM, Gunnar Hellström wrote:
>
> Yes, now after the overview of what would be needed in RTCWEB and 
> WebRTC to include RTT I agree that starting with "draft-x-rtcweb-rtt" 
> would be good for clarifying requirements and alternatives and draw 
> conclusions.
>
> codes. There are two payload types combined RED for simple RFC 2198 
> type redundancy and T140 for the text itself. The recommended sample 
> time is 300 ms. This is in use in SIP applications and also specified 
> in emergency service specifications.
>
> codecs in a similar way as RFC 4733. The main use was intended to be 
> for reliable transport of legacy real-time text through IP between 
> VoIP gateways, where the extra ports needed for a separate text RTP 
> stream through separate ports would possibly be expensive. RFC4351 was 
> given Historic status already from its approval in order to discourage 
> from any other use. Transmission is UTF-8-coded.
>
> 3. XEP-0301 is a draft XMPP extension, that transmits time sampled 
> text in XMPP Messages, but not as Body, but as RTT XML elements, so it 
> can be sent and presented as a gradually built up message, and then 
> replaced by the message when completed. The recommended sample time is 
> 700 ms, and in order to avoid jerky impression, the presentation of 
> keystrokes are timed within these transmission intervals. Transmission 
> is UTF-8.

I'd state (from this very short description) that XEP is a better match 
for web and webapps (and WebRTC), though XML is not necessarily an ideal 
encoding format for JS apps.  But if interoperability with existing 
T.140-based infrastructure is important (and realize gateways will be 
involved!), then something easily transformed into RFC 4103/4351 might 
be preferred (if XEP can't be).  (But this does not necessarily mean 
transmission over RTP is a *requirement* between a rtcweb UA and the 
gateway.)  (Cue Hadriel's normal statement about gateways not always 
having access to the media channels.... ;-) Though he hasn't spoken 
about DataChannel gatewaying; it may make his head explode - sorry Hadriel.)

-- 
Randell Jesup
[email-masked]