Subject: [rtcweb] Use cases: summary of status

From: Stefan Håkansson LK <[email-masked]>
Date: Thu, 4 Aug 2011 09:57:32 +0200

All,
 
sorry for cross-posting.
 
As one of the editors for the use case doc, I would like to summarize the use case discussion a bit. A and B below is my understanding of the status regarding use cases after attending the webrtc meeting Sat July 23rd and the first rtcweb session of IETF 81, and after trying to digest the mail lists. C discusses how we could move forward, D where we should discuss and finally E my opinion as an individual.

Input/feedback is appreciated.
 
A. Existing (old) use cases 
=================
The use cases listed in http://datatracker.ietf.org/doc/draft-ietf-rtcweb-use-cases-and-requirements/ where not questioned, all survived. However, the document type should be changed.

B. New use cases
===========
I noted (as not immediately dismissed) the following proposals:
 
    1. Distributed music band, discussed at the webrtc meeting
    2. Use cases regarding different situations when being invited to a “session”, e.g. browser open, browser open but another tab active, browser open but active in session, browser closed, …. (Matthew Kaufman); discussed at webrtc meeting
    3. Different TURN provider scenarios (Cullen Jennings); discussed at the webrtc meeting
    4. More challenging NAT case (Matthew Kaufman); UDP blocked http://www.ietf.org/mail-archive/web/rtcweb/current/msg00468.html
    5. E911 (Paul Beaumont) http://www.ietf.org/mail-archive/web/rtcweb/current/msg00525.html
    6. Local Recording (John Ewell) at webrtc meeting
    7. Remote recording (John)  http://www.ietf.org/mail-archive/web/rtcweb/current/msg00472.html
    8. Emergency access for disabled (Bernard Aboba) http://www.ietf.org/mail-archive/web/rtcweb/current/msg00478.html
    9. Clue use cases (Roni Even) http://tools.ietf.org/html/draft-ietf-clue-telepresence-use-cases-01
    10. Rohan red cross (Cullen Jennings); proposed a bit earlier http://www.ietf.org/mail-archive/web/rtcweb/current/msg00323.html

Probably I have missed a few.
 
C. Way forward
==========
The proper way forward would probably be to have people draft text describing these use cases and derive requirements they add, and then have a discussion on whether they should be included for version one of the standard or not.
 
In some cases the description exists already (Rohan red cross, emergency access for disabled, clue use cases) but not requirements.
 
Are there volunteers (maybe the ones proposing them)?

D. Where to discuss
=============
This mail is sent to both public-webrtc at w3.org and to rtcweb at ietf.org. We have discussed that there should be only one use case document that is common for the webrtc and the rtcweb WGs, possibly the current ietf use case doc (http://datatracker.ietf.org/doc/draft-ietf-rtcweb-use-cases-and-requirements/). Does this mean that it would be OK to discuss use cases on the rtcweb list only, or should we cross post?

Some of the proposed new use cases (2, 6) have relatively little bearing on IETF IMO; they are more W3C related, perhaps they should be documented and discussed in a webrtc context only.

E. Opinion as individual
===============
My personal opinion is that we at this stage should focus on the basic use cases, and solve those in a good way. To me that would mean that we should add 3 (to allow different providers) and 4 (if you can’t connect no use case can be supported). I think that these use cases can be added as twists of/extensions to the first use case (Simple Video Communication Service).
 
In my view it also means that we should add text and requirements for 2 to make sure that the communication capabilites we are adding to the browser is usable in everyday scenarios. Presumable this would lead requirements (e.g. background execution capability) that are transferred to other W3C WGs - not to requirements in the webrtc/rtcweb space.
 
>From a W3C perspective it could also make sense to add a recording use case (as it seems that other W3C WGs rely on webrtc to provide an API for recording).

The rest of the new proposed use cases, IMO, could be looked into in a second stage when we've established the basics.

==================================== 
 
Comments?
 
Stefan

---

From: Randell Jesup <[email-masked]>
Date: Thu, 04 Aug 2011 11:03:48 -0400

> B. New use cases

Would this cover all "voicemail" and "videomail" cases?  I.e. having a client
accept connections in the background if the call is not answered, optionally
play a message, and record incoming audio/video, and allow the remote user to
interact with it.  Also remote playback of messages.  If not (and if it's not
covered by the current document, we need to add this usecase.

Note that there are two variants: one local (local client handles it, which
has more security issues around camera access and pre-approval), and one for
remote recording (after some time with no answer or if not "registered" call
is forwarded to a message server that answers it).  Note that there are
security identity issues here (key chains, etc).

> Probably I have missed a few.

I'd add:

11. Remote assistance (ala VNC or RDP) - User is helping another user on
their computer with either view-only or view-with-control, either of just
the browser of the the entire screen.  Computer audio is optionally sent
as well.  They are optionally talking and/or viewing each other while this
is occurring.  They may be transferring files over a reliable data
connection during this session.

Commentary: How often have you talked to your father/mother/brother/etc
and tried to debug their computer problems remotely?  And getting them to
install a specific remote-assistance package, and to use it, and get it to
work through firewalls, can be very painful.  (There are other uses of this
ability to copilot as well, including classroom instruction, distance learning,
etc.)  This use-case enables someone to build a JS app to provide this
functionality.  (Note that for the W3 and browser vendors there will be
significant security issues to resolve.)  I think this is actually a fairly
important use-case.

Additional resulting requirements:

* Need to be able to select a "video" source other than a camera (window or
screen) (W3 issue)

* Need to be able to send multiple video and audio streams from different
sources (probably already covered, mostly W3 issue)

* Need to be able to use a reliable data connection (for mouse/keyboard/etc
control, plus file transfer)

12. Security camera/baby monitor usage - use a persistent or temporary
connection to provide basic security camera operation, including switching
cameras or mics, or with the ability to remotely request review of recent
data recorded.  Should be able to switch to 2-way audio and/or video remotely.

Additional requirements:

* Need to be able to select a "video" source other than a camera (file)
(W3 issue)

* Remote control of camera/mic selection and enabling (W3 issue) - may
require reliable data connection

* May need control from JS over codecs used, at least voice vs audio, etc,
and max video framerate/resolution/bandwidth (W3 issue largely?)

> E. Opinion as individual

I'd very much want to include the "remote assistance" case, and I think the
voicemail cases could be very important in overall acceptance and utility, especially
given the issues over whether someone's machine is running and accepting calls.

Security cam/baby-monitor is less important, but highlights some controls that
we may want to expose to the JS over maximum bitrate/framerate/resolution/etc,
and of course a slew of security issues for W3 to think about.

-- 
Randell Jesup
[email-masked]

---

From: Randell Jesup <[email-masked]>
Date: Thu, 04 Aug 2011 11:37:10 -0400

> significant security issues to resolve.)  I think this is actually a fairly

Other uses this use-case covers:

Meetecho - ability to use as a video source a window on your computer (i.e.
presentation), while also optionally exchanging or sending audio and/or video,
and optionally having a (reliable) data channel for side-chat ala the meetecho
chat window.  Note that this is most useful if the window can be any window, not
just the browser.

Very useful for business/sales presentations as well (could help avoid a whole
bunch of travel by sales engineers and the like).

I don't believe this would add any new requirements.

-- 
Randell Jesup
[email-masked]

---

From: Sohel Khan <[email-masked]>
Date: Sat, 6 Aug 2011 15:19:09 -0700 (PDT)

Thanks Randell.
In my opinion, both local and remote "voicemail" and videomail" use cases will be helpful.
 
Regards,
Sohel Khan

________________________________
From: Randell Jesup <[email-masked]>
To: [email-masked]; [email-masked]
Sent: Thursday, August 4, 2011 11:03 AM
Subject: Re: [rtcweb] Use cases: summary of status

...<deleted>...

....<deleted>....
-- 
Randell Jesup
[email-masked]

_______________________________________________
rtcweb mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/rtcweb

---

From: "Elwell, John" <[email-masked]>
Date: Fri, 19 Aug 2011 09:34:01 +0200

Sorry for lateness catching up on this thread.

I did indeed propose recording use cases. I see that Randell has applied that to mail (voicemail / videomail) use cases. I would like to try to provide some separation here.

The recording use cases I had in mind were cases where audio/video are captured/rendered locally (through camera/mic/display/speaker) and IN ADDITION are recorded, either locally or remotely. I tend to think of voicemail/videomail as being a replacement for local capture/rendering, or to put it another way, the storage device acts as the capture/rendering device. So in the voicemail/videomail case, recording is INSTEAD OF conventional capture/rendering.

Voicemail/videomail can be local or remote. In the remote case, the inbound call is diverted to a remote mailbox device. This is a signalling plane matter, and outside the scope of RTC-Web, I believe.

So the interesting case is local voicemail/videomail. It simply means that a mailboxes or files replace the conventional capture rendering devices. So received audio and video can be sent to files, and files can be used as the source of prompts or other information to be fed back to a caller. The basic requirement seems to be to use files as sources and sinks of media sent/received over RTP. One additional requirement is some means by which the caller can control the mailbox - this is conventionally done by DTMF or voice recognition - in fact these are the only methods available when a call comes from PSTN. So this could also lead to a requirement for receiving DTMF.

Local recording, on the other hand, means that media captured from local devices and sent via RTP and media received via RTP and rendered at local devices are also copied to local files.

Remote recording means that media captured from local devices and sent via RTP and media received via RTP and rendered at local devices are also sent via RTP to a remote recording device.

Basically, all these use cases, if we choose to progress them, have a general impact on requirements - some sort of flexibility in terms of where media is sourced and sunk, including the ability to use a file as a source/sink and the ability to have multiple sinks. It seems that a well chosen API architecture could accommodate these, but leaving these requirements until later might mean that the chosen API architecture is not sufficiently flexible to accommodate these requirements later. DTMF reception, if we decide we need it, would be a separate requirement.

So I would welcome feedback on which of these use cases are interesting (I have already heard some support for recording use cases), and I could propose text if necessary.

John

John Elwell
Tel: +44 1908 817801 (office and mobile)
Email: [email-masked]
http://www.siemens-enterprise.com/uk/

Siemens Enterprise Communications Limited.
Registered office: Brickhill Street, Willen Lake, Milton Keynes, MK15 0DJ.
Registered No: 5903714, England.

Siemens Enterprise Communications Limited is a Trademark Licensee of Siemens AG.

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Randell Jesup
> Sent: 04 August 2011 16:04
> >      2. Use cases regarding different situations when being 
> invited to a "session", e.g. browser open, browser open but 
> another tab active, browser open but active in session, 
> browser closed, .. (Matthew Kaufman); discussed at webrtc meeting
> >      3. Different TURN provider scenarios (Cullen 
> Jennings); discussed at the webrtc meeting
> >      4. More challenging NAT case (Matthew Kaufman); UDP 
> blocked 
> http://www.ietf.org/mail-archive/web/rtcweb/current/msg00468.html
> >      5. E911 (Paul Beaumont) 
> http://www.ietf.org/mail-archive/web/rtcweb/current/msg00525.html
> >      7. Remote recording (John)  
> http://www.ietf.org/mail-archive/web/rtcweb/current/msg00472.html
> >      8. Emergency access for disabled (Bernard Aboba) 
> http://www.ietf.org/mail-archive/web/rtcweb/current/msg00478.html
> >      9. Clue use cases (Roni Even) 
> http://tools.ietf.org/html/draft-ietf-clue-telepresence-use-cases-01
> >      10. Rohan red cross (Cullen Jennings); proposed a bit 
> earlier 
> http://www.ietf.org/mail-archive/web/rtcweb/current/msg00323.html
> >
> > My personal opinion is that we at this stage should focus 
> on the basic use cases, and solve those in a good way. To me 
> that would mean that we should add 3 (to allow different 
> providers) and 4 (if you can't connect no use case can be 
> supported). I think that these use cases can be added as 
> twists of/extensions to the first use case (Simple Video 
> Communication Service).
> >
> > In my view it also means that we should add text and 
> requirements for 2 to make sure that the communication 
> capabilites we are adding to the browser is usable in 
> everyday scenarios. Presumable this would lead requirements 
> (e.g. background execution capability) that are transferred 
> to other W3C WGs - not to requirements in the webrtc/rtcweb space.
> >
> > > From a W3C perspective it could also make sense to add a 
> recording use case (as it seems that other W3C WGs rely on 
> webrtc to provide an API for recording).
> >
> > The rest of the new proposed use cases, IMO, could be 
> looked into in a second stage when we've established the basics.
>

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 19 Aug 2011 10:08:59 +0200

John,

I was just uploading an updated version of the use case doc when receiving this, so it is not taken into account in that version.

Stefan
________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Elwell, John [[email-masked]]
Sent: Friday, August 19, 2011 9:34 AM
To: [email-masked]; [email-masked]
Subject: [rtcweb] Use cases - recording and voicemail

John

John Elwell

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 19 Aug 2011 11:49:56 +0200

John,

this was a good way sort things up.

I my view we should definitely support local recording of streams (regardless of if they are generated by local devices or received via RTP), and this could be done in parallel to rendering them or not (up to the app).

The recorded media should also be possible to render locally (be the source for a video element).

I'm less sure about that the recorded media should also be an RTP source - couldn't you just as well send the file over and then play it at the remote end?

Stefan

John

John Elwell

---

From: "Elwell, John" <[email-masked]>
Date: Fri, 19 Aug 2011 14:20:06 +0200

Stefan,

> From: Stefan Håkansson LK [mailto:[email-masked]] 
> Sent: 19 August 2011 10:50
> To: Elwell, John; [email-masked]; [email-masked]
> Subject: RE: Use cases - recording and voicemail
> 
> John,
> 
> I my view we should definitely support local recording of 
> streams (regardless of if they are generated by local devices 
> or received via RTP), and this could be done in parallel to 
> rendering them or not (up to the app).
> 
> I'm less sure about that the recorded media should also be an 
> RTP source - couldn't you just as well send the file over and 
> then play it at the remote end?
[JRE] Yes, the file could be streamed across, but there are folks who want it to get across more or less in real time, e.g., where the recorder is performing real-time analytics, perhaps in a contact centre. This is the basis for the work done in the IETF SIPREC WG. The same considerations that require RTP browser-to-browser for RTC-Web also dictate RTP as the transport for sending media to a recorder.

In terms of what impact this would have on the solution, perhaps it would have very little impact. Perhaps it could be treated as two separate sessions:
- one that exchanges audio/video between capture/rendering devices and the remote entity and copies to a local file; and
- one that takes audio/video from the local file and sends it to a remote recorder.
The main requirement then is to be able to read from the local file at the same time as writing to the local file.

John

> From: [email-masked] [[email-masked]] On 
> Behalf Of Elwell, John [[email-masked]]
> I did indeed propose recording use cases. I see that Randell 
> has applied that to mail (voicemail / videomail) use cases. I 
> would like to try to provide some separation here.
> 
> The recording use cases I had in mind were cases where 
> audio/video are captured/rendered locally (through 
> camera/mic/display/speaker) and IN ADDITION are recorded, 
> either locally or remotely. I tend to think of 
> voicemail/videomail as being a replacement for local 
> capture/rendering, or to put it another way, the storage 
> device acts as the capture/rendering device. So in the 
> voicemail/videomail case, recording is INSTEAD OF 
> conventional capture/rendering.
> 
> Voicemail/videomail can be local or remote. In the remote 
> case, the inbound call is diverted to a remote mailbox 
> device. This is a signalling plane matter, and outside the 
> scope of RTC-Web, I believe.
> 
> So the interesting case is local voicemail/videomail. It 
> simply means that a mailboxes or files replace the 
> conventional capture rendering devices. So received audio and 
> video can be sent to files, and files can be used as the 
> source of prompts or other information to be fed back to a 
> caller. The basic requirement seems to be to use files as 
> sources and sinks of media sent/received over RTP. One 
> additional requirement is some means by which the caller can 
> control the mailbox - this is conventionally done by DTMF or 
> voice recognition - in fact these are the only methods 
> available when a call comes from PSTN. So this could also 
> lead to a requirement for receiving DTMF.
> 
> Local recording, on the other hand, means that media captured 
> from local devices and sent via RTP and media received via 
> RTP and rendered at local devices are also copied to local files.
> 
> Remote recording means that media captured from local devices 
> and sent via RTP and media received via RTP and rendered at 
> local devices are also sent via RTP to a remote recording device.
> 
> Basically, all these use cases, if we choose to progress 
> them, have a general impact on requirements - some sort of 
> flexibility in terms of where media is sourced and sunk, 
> including the ability to use a file as a source/sink and the 
> ability to have multiple sinks. It seems that a well chosen 
> API architecture could accommodate these, but leaving these 
> requirements until later might mean that the chosen API 
> architecture is not sufficiently flexible to accommodate 
> these requirements later. DTMF reception, if we decide we 
> need it, would be a separate requirement.
> 
> So I would welcome feedback on which of these use cases are 
> interesting (I have already heard some support for recording 
> use cases), and I could propose text if necessary.
> 
> John
> 
> John Elwell

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 19 Aug 2011 14:33:35 +0200

John,
>[JRE] Yes, the file could be streamed across, but there are folks who want it to get across more or less in real time, e.g., where the recorder is performing real-time analytics, perhaps in a >contact centre. This is the basis for the work done in the IETF SIPREC WG. The same considerations that require RTP browser-to-browser for RTC-Web also dictate RTP as the transport for >sending media to a recorder.
[Stefan] If you want to do stuff remotely in real time, could you not just avoid recording locally? Regardless of if the analysis is to happen on a locally generated stream, or on a stream received via RTP, that stream can be streamed to the analysing server (via a p2p-connection), right? (Probably I am missing something).

Stefan

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 19 Aug 2011 09:07:49 -0400

> I'm less sure about that the recorded media should also be an RTP source - couldn't you just as well send the file over and then play it at the remote end?

I think discussing this in terms of a "file" is an over simplification.
While it could be a file, it could also be synthesized, or assembled 
from pieces, interactively as a result of received media or signaling. 
In such cases you can't just "send the file".

Also, sending the file only works if the recipient can always deal with 
a file equally to receiving an RTP stream. That seems like a burden to 
levy on all possible recipients. For instance, it would mean that 
rtcweb:pstn gateways would all require the capability to buffer and 
playback audio/video files.

So perhaps what is needed is the concept of a software agent that can 
take on the role of a local source/sink of media streams.

	Thanks,
	Paul

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 19 Aug 2011 15:21:30 +0200

>In such cases you can't just "send the file".
Maybe it was an over simplification, but synthesizing/assembling etc. sound a bit overly complex to me, at least in a first version. Just my personal opinion. In any case we would have to specify more precisely what should be possible.

---

From: "Elwell, John" <[email-masked]>
Date: Fri, 19 Aug 2011 15:25:35 +0200

Stefan,

No, I am not suggesting both local recording and remote recording at the same time. Sometimes local recording will be required, sometimes remote recording.

However, I did suggest (in other text in my previous message) that one possible solution might be to record locally and use a second RTC-Web session to transmit from the local file to the remote recorder. What I failed to say was that in this case the local file would be a temporary repository - just a buffer between the two sessions.

John

> Sent: 19 August 2011 13:34
> >[JRE] Yes, the file could be streamed across, but there are 
> folks who want it to get across more or less in real time, 
> e.g., where the recorder is performing real-time analytics, 
> perhaps in a >contact centre. This is the basis for the work 
> done in the IETF SIPREC WG. The same considerations that 
> require RTP browser-to-browser for RTC-Web also dictate RTP 
> as the transport for >sending media to a recorder.
> [Stefan] If you want to do stuff remotely in real time, could 
> you not just avoid recording locally? Regardless of if the 
> analysis is to happen on a locally generated stream, or on a 
> stream received via RTP, that stream can be streamed to the 
> analysing server (via a p2p-connection), right? (Probably I 
> am missing something).
> 
> Stefan

---

From: "Elwell, John" <[email-masked]>
Date: Fri, 19 Aug 2011 15:30:50 +0200

Paul,

I agree file might not be the right term. What we need is a way of passing the stream, within the browser, between one session and a second session (in SIPREC terms, between the communication session and the recording session).

John

> [mailto:[email-masked]] On Behalf Of Paul Kyzivat
> Sent: 19 August 2011 14:08
> To: [email-masked]
> Subject: Re: [rtcweb] Use cases - recording and voicemail
> 
> > The recorded media should also be possible to render 
> locally (be the source for a video element).
> >
> > I'm less sure about that the recorded media should also be 
> an RTP source - couldn't you just as well send the file over 
> and then play it at the remote end?
> 
> > I did indeed propose recording use cases. I see that 
> Randell has applied that to mail (voicemail / videomail) use 
> cases. I would like to try to provide some separation here.
> >
> > Local recording, on the other hand, means that media 
> captured from local devices and sent via RTP and media 
> received via RTP and rendered at local devices are also 
> copied to local files.
> >
> > Remote recording means that media captured from local 
> devices and sent via RTP and media received via RTP and 
> rendered at local devices are also sent via RTP to a remote 
> recording device.
> >

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 19 Aug 2011 16:55:40 +0200

>However, I did suggest (in other text in my previous message) that one possible solution might be to record locally and use a second RTC-Web session to transmit from the local file to the >remote recorder. What I failed to say was that in this case the local file would be a temporary repository - just a buffer between the two sessions.
This makes sense. Also, if you look at the API proposals available, it would be quite easy to forward (in real time) a stream being received to another entity. There is no explicit recording, a stream being received (via RTP) is just streamed to another entity (via a separate RTC-Web session). I think this would solve this case.

Stefan

---

From: "Ravindran Parthasarathi" <[email-masked]>
Date: Fri, 19 Aug 2011 22:48:36 +0530

Stefan,

In case recording similar to SIPREC, it is little bit more than spanning two media (RTP stream) alone because recording has to include some context data about recording apart from the media stream.

Thanks
Partha

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Stefan Håkansson LK
> Sent: Friday, August 19, 2011 8:26 PM
> >However, I did suggest (in other text in my previous message) that one
> possible solution might be to record locally and use a second RTC-Web
> session to transmit from the local file to the >remote recorder. What I
> failed to say was that in this case the local file would be a temporary
> repository - just a buffer between the two sessions.
> This makes sense. Also, if you look at the API proposals available, it
> would be quite easy to forward (in real time) a stream being received
> to another entity. There is no explicit recording, a stream being
> received (via RTP) is just streamed to another entity (via a separate
> RTC-Web session). I think this would solve this case.
> 
> Stefan

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 19 Aug 2011 19:46:52 +0200

Partha,

thanks for updating me.

Stefan
________________________________________
From: Ravindran Parthasarathi [[email-masked]]
Sent: Friday, August 19, 2011 7:18 PM
To: Stefan Håkansson LK; Elwell, John; [email-masked]; [email-masked]
Subject: RE: [rtcweb] Use cases - recording and voicemail

Stefan,

Thanks
Partha

---

From: Randell Jesup <[email-masked]>
Date: Fri, 19 Aug 2011 17:51:38 -0400

> I my view we should definitely support local recording of streams (regardless of if they are generated by local devices or received via RTP), and this could be done in parallel to rendering them or…

Agreed.   Note that there are legal requirements in various locations around recording
conversations; that's up to the application IMHO -- however we'll want to make sure it's
reasonably easy for the application to do.  While I'm not an expert, recording someone
in many jurisdictions requires periodic beeps, etc.  They'd have to mix it into the outgoing
stream, but it would have to remain there even if the user "muted".  I want to make sure
we're providing something that won't be a hassle for the application developers.

> The recorded media should also be possible to render locally (be the source for a video element).
Yes.

> I'm less sure about that the recorded media should also be an RTP source - couldn't you just as well send the file over and then play it at the remote end?

That might not work for cases where the two users are talking through different
providers/apps, and it also would imply a much longer delay in many cases, plus
local storage requirements, etc.   Think a recorded greeting played to callers if
no one answers, for example.

Assuming that recording and playback are of encoded media:

There are issues with recording and playback having to do with error recovery.
For recording an incoming stream, it's less of an issue - you do normal error
recovery, and on playback it would look the same as it would have if the call had
been live.

For sending a pre-recorded stream (greeting, etc): I'd assume it was recorded
without loss.  However, the other side may experience loss in receiving it.
To deal with this, we can a) decode and re-encode the media, allowing us to react
to incoming loss reports, or b) include periodic IDRs or equivalent.  I would
lean to a) (decode&  re-encode), that also handles issues with codec parameters,
codec choice, etc.  So an input would be from a decoded stream.  This may use more
resources than b; the application could (at its discretion) not render incoming media
while playing back.

-- 
Randell Jesup
[email-masked]

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 19 Aug 2011 19:52:03 -0400

Randell,

Many of the issues you are bringing up are things we have been 
addressing in the siprec WG. RTCWEB can either reinvent all that, or we 
can find some way to reuse that work. At the moment, siprec assumes that 
the agent that is initiating the recording is in the signaling path of a 
sip call, and that it is capable of establishing another sip session to 
the recorder. Those assumptions would seem to be wrong for RTCWEB.

	Thanks,
	Paul

On 8/19/11 5:51 PM, Randell Jesup wrote:
>> I my view we should definitely support local recording of streams
>> (regardless of if they are generated by local devices or received via
>> RTP), and this could be done in parallel to rendering them or not (up
>> to the app).
>
>> I'm less sure about that the recorded media should also be an RTP
>> source - couldn't you just as well send the file over and then play it
>> at the remote end?
>

---

From: Harald Alvestrand <[email-masked]>
Date: Sat, 20 Aug 2011 17:49:32 +0200

On 08/20/11 01:52, Paul Kyzivat wrote:
> addressing in the siprec WG. RTCWEB can either reinvent all that, or 
> we can find some way to reuse that work. At the moment, siprec assumes 
> that the agent that is initiating the recording is in the signaling 
> path of a sip call, and that it is capable of establishing another sip 
> session to the recorder. Those assumptions would seem to be wrong for 
> RTCWEB.
Hm.

RTCWEB doesn't seem to going in the direction of mandating SIP, but 
still, I would think that it is reasonable to say something along the 
lines of "the remote recording case is handled by connecting to a 
SIPREC-capable recorder" (with the usual degree of gatewaying help from 
our signaling proxies).

That will, of course, require that the SIPREC recorder is capable of 
participiating in an RTCWEB session (that is, support ICE and the 
mandatory codecs), that the RTCWEB implementation be capable of copying 
incoming media streams to an outgoing interface, and that negotiation 
can down-negotiate to something that is supported by both call 
participants and the recording device. Does SIPREC envision establishing 
minimum requirements for codecs and profiles?

Once we can satisfy ourselves that we have all the pieces required to 
send media off to a remote API, we should see if we can do something 
very similar for sending media off to some kind of local recorder; it 
seems less likely that we'll get into trouble with locking ourselves 
into a wrong model if we do things in that order.

My $0.02.

               Harald

---

From: "Elwell, John" <[email-masked]>
Date: Mon, 22 Aug 2011 08:28:31 +0200

Partha,

You are talking here about the metadata, I think. I assume the web page / JavaScript has to deal with that - not the browser.

John

> From: Ravindran Parthasarathi [mailto:[email-masked]] 
> Sent: 19 August 2011 18:19
> In case recording similar to SIPREC, it is little bit more 
> than spanning two media (RTP stream) alone because recording 
> has to include some context data about recording apart from 
> the media stream.
> 
> Thanks
> Partha
>

---

From: "Ravindran Parthasarathi" <[email-masked]>
Date: Mon, 22 Aug 2011 12:12:25 +0530

John,

I agree with you. JavaScript API should have the provision to pass the metadata.

Thanks
Partha

>From: Elwell, John [mailto:[email-masked]]
>Sent: Monday, August 22, 2011 11:59 AM
>To: Ravindran Parthasarathi; Stefan Håkansson LK; [email-masked];
>[email-masked]
>You are talking here about the metadata, I think. I assume the web page
>/ JavaScript has to deal with that - not the browser.
>
>John
>
>

---

From: "Avasarala, Ranjit" <[email-masked]>
Date: Mon, 22 Aug 2011 14:52:09 +0800

Hi

We could use websockets protocol to pass metadata information. 

Regards
Ranjit

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Ravindran Parthasarathi
Sent: Monday, August 22, 2011 12:12 PM
To: Elwell, John; Stefan Håkansson LK; [email-masked]; [email-masked]
Subject: Re: [rtcweb] Use cases - recording and voicemail

John,

Thanks
Partha

---

From: "Ravindran Parthasarathi" <[email-masked]>
Date: Mon, 22 Aug 2011 13:46:27 +0530

Hi Ranjit,

I have mentioned as JavaScript API because there is no specific dependency on browser. Let us discuss about the exact mechanism later. Currently, let us have common understanding whether recording usecase has to be added in RTCWeb or not.

Thanks
Partha 

>From: Avasarala, Ranjit [mailto:[email-masked]]
>Sent: Monday, August 22, 2011 12:22 PM
>To: Ravindran Parthasarathi; Elwell, John; Stefan Håkansson LK;
>[email-masked]; [email-masked]

---

From: "Hutton, Andrew" <[email-masked]>
Date: Mon, 22 Aug 2011 10:32:04 +0200

> Sent: 19 August 2011 15:56

[AndyH] It might do but I assume this would limit the RTP model that could be used to the endpoint model. We are currently reviewing how the RTP model for a session recording client should look like with the AVT experts (See http://tools.ietf.org/html/draft-eckel-siprec-rtp-rec-01). This might result in some requirements on how the client handles RTCP for example.

---

From: "Hutton, Andrew" <[email-masked]>
Date: Mon, 22 Aug 2011 11:25:40 +0200

> SIPREC-capable recorder" (with the usual degree of gatewaying help from
[AndyH] - I agree this would just seem to be another example of how the Session Recording Client can be decomposed and is similar to Figure 4 in the SIPREC architecture draft (http://tools.ietf.org/html/draft-ietf-siprec-architecture-02#section-3.1.2) but in the case of RTCWEB the media server is the browser and the application is the web server and of course RTCWEB will not use mediactrl. I have been resisting attempts to add more diagrams to the SIPREC architecture draft to show the multitude of ways the SRC and SRS could be decomposed but maybe RTCWEB is a good reason to do this.

> minimum requirements for codec's and profiles?

[AndyH] - No the SIPREC requirements (http://tools.ietf.org/html/rfc6341) don't include minimum codec requirements.

---

From: "Avasarala, Ranjit" <[email-masked]>
Date: Mon, 22 Aug 2011 18:17:22 +0800

Hi Partha

I agree with Paul that we could use the recording mechanism discussed in siprec mailing list could be added here as webbrowser would essentially be a SIP UA which could initiate and terminate SIP sessions. 

Regards
Ranjit

-----Original Message-----
From: Ravindran Parthasarathi [mailto:[email-masked]] 
Sent: Monday, August 22, 2011 1:46 PM
To: Avasarala, Ranjit; Elwell, John; Stefan Håkansson LK; [email-masked]; [email-masked]
Subject: RE: [rtcweb] Use cases - recording and voicemail

Hi Ranjit,

Thanks
Partha

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 22 Aug 2011 13:10:19 +0200

> I agree with Paul that we could use the recording mechanism discussed in siprec mailing list could be added here as webbrowser would essentially be a SIP UA which could initiate and terminate SIP s…
mandatory standard clarification (?):

.. as somewhere in the combination of the Web browser, the Javascript 
and the Web server it connects to, there would be something that 
performs the functions of a SIP UA which could initiate and terminate 
SIP sessions ...
>>>>> session to transmit from the local file to the>remote
>>>> recorder. What I

---

From: Stefan Håkansson LK <[email-masked]>
Date: Mon, 22 Aug 2011 13:19:39 +0200

On 2011-08-22 10:16, Ravindran Parthasarathi wrote:

> ... Currently, let us have common understanding whether recording usecase has to be added in RTCWeb or not.
Agree. And also _which_ recording use case(s) in that case. I think this 
is what John was looking for when starting the thread.

My $0.02 says that we need to take some care before adding more and more 
usages and reqs - after all the schedules for the WGs are quite aggressive.

---

From: "Avasarala, Ranjit" <[email-masked]>
Date: Mon, 22 Aug 2011 19:24:11 +0800

True. One possible model could be integrating SIP stack with webbrowser and send / receive SIP messages over websockets. 

Regards
Ranjit

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 22 Aug 2011 09:12:48 -0400

On 8/22/11 7:10 AM, Harald Alvestrand wrote:
>> I agree with Paul that we could use the recording mechanism discussed
>> in siprec mailing list could be added here as webbrowser would
>> essentially be a SIP UA which could initiate and terminate SIP sessions.

Yes, *something* of this sort - the devil is in the details.

IIUC, its the intent of RTCWEB that the media flow directly between 
source and recipient, rather than following the path of the "signaling".

(That of course was also the intent of SIP with proxies, but the almost 
pervasive use of SBCs has usually forced media to follow the signaling 
path.)

In siprec, it is assumed that an entity called the SRC is in the 
signaling and media path of the call being recorded, as well as in the 
signaling and media path of a session with the recorder.

If we assume that the RTCWEB "client" is in the media path, but not in a 
sip signaling path, and that the RTCWEB "server" is sip-capable but not 
in the media path, then we have a structure that doesn't quite fit the 
siprec model.

However this could perhaps be made to work with siprec via some 
additional functionality in the RTCWEB client and server, and maybe some 
tweaks to what is being specified for siprec. The details will depend on 
which end is making the decision to do recording. If its the client, 
then that client will need a way to ask "the" server to establish a 
siprec session and to pass metadata. If its the server that makes the 
decision, then there needs to be a way for the server to ask the client 
to fork media and send it to multiple destinations.

There are of course privacy concerns in this. Whether they are different 
in RTCWEB than they are for siprec in general remains to be seen.

	Thanks,
	Paul

>> I have mentioned as JavaScript API because there is no specific
>> dependency on browser. Let us discuss about the exact mechanism later.
>> Currently, let us have common understanding whether recording usecase
>> has to be added in RTCWeb or not.
>>
>> Thanks
>> Partha
>>

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 22 Aug 2011 09:18:44 -0400

> You are talking here about the metadata, I think. I assume the web page / JavaScript has to deal with that - not the browser.

I don't understand your point John. The javascript is executed by the 
browser.

The actual awareness of information that is to become siprec metadata 
may well be distributed - some of it in the client/browser, and some of 
it in the server. That will somewhat complicate getting it all together.

	Thanks,
	Paul

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 22 Aug 2011 16:09:43 +0200

On 08/22/11 13:19, Stefan Håkansson LK wrote:
>> ... Currently, let us have common understanding whether recording 
>> usecase has to be added in RTCWeb or not.
> Agree. And also _which_ recording use case(s) in that case. I think 
> this is what John was looking for when starting the thread.
>
> My $0.02 says that we need to take some care before adding more and 
> more usages and reqs - after all the schedules for the WGs are quite 
> aggressive.
Indeed. When thinking about the recording use cases, I could come up 
with a few interpretations:

- Record one media stream, as it arrives (the "voice mailbox" concept)
- Record a conversation: Outgoing and incoming audio and video in an 
1-on-1 or 1-on-N context (typically as permanent records of a meeting or 
conversation)
- Record the application screen as presented to the user, together with 
the audio tracks as presented to the user's audio device, but don't 
bother with incoming audio from the user at all (game recording, for 
instance - the "WoW movie" kind)

In each instance, recording technology might include recording:

   - As one track, somehow mixed, ready for playback
   - As multiple tracks recorded separately within a container that 
requires special playback devices
   - As multiple tracks, recorded separately to separate objects
   - As streams sent to some remote recording entity, that may choose 
one of the options above

I'd be happier about discussing which ones of these people find it 
critical to support before we dive into figuring out how to support them.

Send text!

                       Harald

---

From: "Elwell, John" <[email-masked]>
Date: Mon, 22 Aug 2011 16:41:20 +0200

> From: Harald Alvestrand [mailto:[email-masked]] 
> Sent: 22 August 2011 15:10
> To: Stefan Håkansson LK
> Cc: Ravindran Parthasarathi; Avasarala, Ranjit; Elwell, John; 
[JRE] This was not what I originally proposed, but emerged during discussion. From my point of view a mailbox is more likely to be centralized, not on a user's device, so the web application can divert calls to a mailbox using signalling means, without any impact on the browser. Some people might want to support a local mailbox, in which case browser support would be needed, but that is second priority, from my perspective.

> 1-on-1 or 1-on-N context (typically as permanent records of a 
[JRE] This was what I was originally asking about. Seems to be important to have this capability from day 1, or at least an architecture that can be extended to do this at day 2.

> instance - the "WoW movie" kind)
[JRE] Not high priority from my perspective.

> requires special playback devices
[JRE] I think the container here is what is known as metadata, from a session recording point of view. That container can be a matter for the web application.

>    - As multiple tracks, recorded separately to separate objects
[JRE] Yes, although the possibility of mixing (e.g., the two directions of an audio stream) would be good to have.

>    - As streams sent to some remote recording entity, that may choose 
[JRE] Yes, this is my primary concern, coupled with the "record a conversation" above.

> critical to support before we dive into figuring out how to 
[JRE] I was just about to - look out for separate postings for local and remote recording (I will leave others to proposed text for mailbox, if they require it).

John

John Elwell

---

From: "Elwell, John" <[email-masked]>
Date: Mon, 22 Aug 2011 17:15:19 +0200

> Sent: 22 August 2011 14:19
> To: [email-masked]
> > You are talking here about the metadata, I think. I assume 
> the web page / JavaScript has to deal with that - not the browser.
> 
[JRE] My point is that in the same way as SIP signalling (at least in some people's minds) is a matter for JS, the inclusion of a metadata body part in that SIP signalling is also a matter for the JS. Of course, if SIP is included in the browser, this is a different matter.

John

John Elwell

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 22 Aug 2011 11:45:41 -0400

> [JRE] My point is that in the same way as SIP signalling (at least in some people's minds) is a matter for JS, the inclusion of a metadata body part in that SIP signalling is also a matter for the…

OK. Now I get it.

Well, then again this takes on a different cast based on whether, or 
not, you think the browser should have a sip stack that is "driven" by JS.

If so, then probably the JS will also need to establish the SIP 
Recording Session, and send the metadata in the sip signaling. 
Presumably that would have to be part of the JS API for dealing with the 
sip stack.

OTOH, if you don't think the browser and JS should be doing sip 
signaling, then the web server is probably establishing the Recording 
Session, and sending the metadata. But the media streams may not flow 
through the web server. So in that case, the web server, via the JS that 
it provides, will require the capability to provide a forked media 
stream that the server can then reference in its SDP for the RS. And 
also, the JS will need *some* way to provide to the server the info that 
should go into the metadata.

There had better be *some* standards for how to do it or else it will be 
a high bar for supporting recording. Perhaps the metadata could be 
provided by the JS to the web server using the same XML schema being 
defined by siprec, but conveyed differently, using http. But I expect it 
will be at least a little more complicated than that.

	Thanks,
	Paul

---

From: "Ravindran Parthasarathi" <[email-masked]>
Date: Tue, 23 Aug 2011 00:25:58 +0530

Please read inline

Thanks
Partha

>From: [email-masked] [mailto:[email-masked]] On Behalf
>Of Paul Kyzivat
>Sent: Monday, August 22, 2011 9:16 PM
>To: Elwell, John
>Cc: [email-masked]
>> [JRE] My point is that in the same way as SIP signalling (at least in
>some people's minds) is a matter for JS, the inclusion of a metadata
>body part in that SIP signalling is also a matter for the JS. Of course,
>if SIP is included in the browser, this is a different matter.
>>
>> John
>
>OK. Now I get it.
>
[Partha] This is my understanding. Here, it is possible to complete reuse SIPREC work with minor extensions specific to RTCWeb like codec profile.

>will be at least a little more complicated than that.
[Partha] I agree with you that two way for passing metadata & media is more complicated which is avoided in SIPREC protocol design.

---

From: Randell Jesup <[email-masked]>
Date: Mon, 22 Aug 2011 16:11:21 -0400

> in RTCWEB than they are for siprec in general remains to be seen.

There are also other practical issues.  In particular with bandwidth: you've
now roughly tripled the bandwidth requirement of the endpoint: it must send
to the other end, AND mirror both outgoing (which it controls) and incoming
(which it doesn't, or only controls very indirectly) to the recording server.

For cases where you have bandwidth to burn (audio-only calls on broadband)
you might get away with it.  For a video call under any sort of bandwidth
restraint, it would be a huge problem and would seriously degrade quality.

-- 
Randell Jesup
[email-masked]

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 22 Aug 2011 16:46:34 -0400

> restraint, it would be a huge problem and would seriously degrade quality.

Re recognize this in siprec. The SRC can be inserted into the signaling 
and media path in a variety of ways. Typical sip deployments today have 
servers in the path that are often not bandwidth constrained.

RTCWEB seems to be revisiting the deployment topologies. Topologies that 
work for recording may also need to be revisited.

	Thanks,
	Paul