Subject: [dispatch] RTC-Web Use cases

From: Stefan Håkansson LK <[email-masked]>
Date: Thu, 20 Jan 2011 11:42:11 +0100

Just to start the discussion, I've put together the simplest use case you
can imagine (with an additional error case). Some requirements are drawn
from it. The intention is to add more use cases, but before spending any
more energy I though it would be good to display this to the community to
start the discussion and to get feedback on whether this is a workable
model or not.
 
The requirements are split in three categories: 
1. User/privacy: the intention is to list what an end user can expect in
   terms of what an web application can do without user consent and so on
2. API dimension: lists what must be accessible by JS and signalled to JS
   via APIs
3. Functional dimension: lists functionality that must be supported by the
   browser (in combination with the underlying operating system, drivers 
   and HW) even though it is not visible from the API. This list is rather
   short since most of the functionality is visible in the API
 
I can't say that I am extremely satisfied with this division. Perhaps we
could come up with something better. Also, some of the requirements in the
User/privacy and Functional dimensions are not drawn (e.g. ask for user
consent) from this use case per se, but are instead generic requirements.
Perhaps they should be handled separately?
 
The term "stream" is used. The thinking (inspired by the WhatWG HTML5 draft
<http://www.whatwg.org/specs/web-apps/current-work/multipage/commands.html#devices>)
is that some kind of Stream API should represent a stream. A stream in turn
is what is generated by a mic or a cam, but also what would be sent between
browsers. Obviously these streams would have different formats (the first
being raw audio/video samples, the second encoded audio/video in RTP 
packets). But the Steam API is what the JS code uses to manage the stream.
 
OK, to the use case:

====================
1. Simple Video Chat
====================
A simple video chat service has been developed. In the service the users are
logged on to the same chat web server. The web server publishes information
about user login status, pushing updates to the web apps in the browsers. By
clicking on an online peer user name, a 1-1 video chat session between the
two browsers is initiated. The invited peer is presented with a choice of
joining or rejecting the session.
 
The web author developing the application has decided to display a self-view
as well as the video from the remote side in rather small windows, but the
user can change the display size during the session. The application also
supports if a participant (for a longer or shorter time) would like to stop
sending audio (but keep video) or video (keep audio) to the other peer
("mute").
 
Any of the two participants can at any time end the chat by clicking a
button.
 
In this specific case two users are using lap-tops in their respective
homes. They are connected to the public Internet with a desktop browser
using WiFi behind NATs. One of the users has an ADSL connection to the home,
and the other fiber access. Most of the time headsets are used, but not always.

1.1 Requirements
================

1.1.1 User/privacy dimension
----------------------------
--------------------------------------------------------------------------------
!Requirement. The user must:            !Comment                               !
--------------------------------------------------------------------------------
!give explicit consent before a device  !                                      !
!can be used to capture audio or video  !                                      !
--------------------------------------------------------------------------------
!be able to in an intuitive way revoke  !                                      !
!and change capturing permissions       !                                      !
--------------------------------------------------------------------------------
!be able to easily understand that audio!                                      !
!or video is being captured             !                                      !
--------------------------------------------------------------------------------
!be informed about that an invitation to!                                      !
!a peer video chat session has been     !                                      !
!received                               !                                      !
--------------------------------------------------------------------------------
!be able to accept or reject an         !                                      !
!invitation to a peer video chat session!                                      !
--------------------------------------------------------------------------------
!be able to stop a media stream from    !                                      !
!being transmitted                      !                                      !
--------------------------------------------------------------------------------

1.1.2 API dimension
-------------------
--------------------------------------------------------------------------------
!Requirement.                           !Comment                               !
--------------------------------------------------------------------------------
!It must be possible to update presence !Event. Out of scope for RTC-Web?      !
!info from web server and make web      !                                      !
!application aware.                     !                                      !
--------------------------------------------------------------------------------
!It must be possible to propagate       !Out of scope for RTC-Web?             !
!intention to start a chat session from !                                      !
!one web app (via server), and make     !                                      !
!receiving web application aware.       !                                      !
!Likewise, the receiving web application!                                      !
!must be able to propagate its accept/  !                                      !
!reject to the initiating web app.      !                                      !
--------------------------------------------------------------------------------
!The web application must be able to use!Provided the user has given consent.  !
!cams and mics as input devices.        !                                      !
--------------------------------------------------------------------------------
!The web application must be able to    !I.e. how they are routed. To e.g. both!
!control how streams generated by input !a self-view and a peer                !
!devices are used                       !                                      !
--------------------------------------------------------------------------------
!The web application must be able to    !Use the audio and video elements?     !
!control how streams are rendered and   !                                      !
!displayed                              !                                      !
--------------------------------------------------------------------------------
!The web application must be able to    !                                      !
!initiate sending of streams to a peer  !                                      !
--------------------------------------------------------------------------------
!The web application must be able to    !If the video is going to be displayed !
!define the media format to be used for ! in a large window, use higher bit-   !
!the streams sent to a peer.            !rate/resolution. Should media settings!
!                                       !be allowed to be changed during a     !
!                                       !session (at e.g. window resize)?      !
--------------------------------------------------------------------------------
!The web application must be made aware !Event.                                !
!of whether set up of stream sending was!                                      !
!successful or not                      !                                      !
--------------------------------------------------------------------------------
!The web application must be made aware !Event. To be able to (with or without !
!when a stream from a peer is received  !user involvement) accept or reject,   !
!                                       !and to connect the stream to the right!
!                                       !display/rendering element.            !
--------------------------------------------------------------------------------
!The web application must be made aware !Event.                                !
!of when a stream from a peer is not    !                                      !
!received any more                      !                                      !
--------------------------------------------------------------------------------
!The web application in a session must  !                                      !
!be able to terminate all incoming and  !                                      !
!outgoing streams                       !                                      !
--------------------------------------------------------------------------------
	 
1.1.3 Functional dimension
--------------------------
--------------------------------------------------------------------------------
!Requirement.                           !Comment                               !
--------------------------------------------------------------------------------
!The browser must be able to have an    !Out of scope for RTC-Web? Use WS or   !
!always on connection with the web      !S-SE?                                 !
!server to be able to receive presence  !                                      !
!updates and chat initiations           !                                      !
--------------------------------------------------------------------------------
!The browser must be able to use mics   !                                      !
!and cams as input devices              !                                      !
--------------------------------------------------------------------------------
!The browser must be able to send       !                                      !
!streams (includes the associated       !                                      !
!processing like coding, framing, etc.) !                                      !
!to a peer in presence of NATs.         !                                      !
--------------------------------------------------------------------------------
!The browser must be able to receive    !                                      !
!streams (associated processing) from   !                                      !
!peers and render them                  !                                      !
--------------------------------------------------------------------------------
!Streams being transmitted must be      !Do not starve other traffic (e.g. on  !
!subject to rate control                !ADSL link)                            !
--------------------------------------------------------------------------------
!When there is both incoming and        !Headsets not always used              !
!outgoing audio streams, echo           !                                      !
!cancellation must be provided to avoid !                                      !
!disturbing echo during conversation    !                                      !
--------------------------------------------------------------------------------
!Synchronization between audio and video!                                      !
!must be supported                      !                                      !
--------------------------------------------------------------------------------

2 Simple video chat with link loss
==================================
The use case is identical to the previous one, but in this case the ADSL link 
is unreliable and goes down from time to time.

2.1 Requirements
----------------
Only the delta is documented (the other reqs remain).

2.1.1 User/privacy dimension
----------------------------
--------------------------------------------------------------------------------
!Requirement.                           !Comment                               !
-------------------------------------------------------------------------------- 
!The user must be informed that the     !                                      !
!communication has ceased               !                                      !
--------------------------------------------------------------------------------
	 
2.1.2 API dimension
-------------------
--------------------------------------------------------------------------------
!Requirement.                           !Comment                               !
--------------------------------------------------------------------------------  
!The web application must be made aware !To be able to inform user and take    !
!of that the connection with the server !action. Out of scope for RTC-Web?     !
!has been dropped                       !                                      !
--------------------------------------------------------------------------------
!The web application must be made aware !To be able to inform user and take    !
!of when streams from a peer are no     !action (one of the peers still has    !
!longer received                        !connection with the server)           !
--------------------------------------------------------------------------------

2.1.3 Functional dimension
--------------------------
--------------------------------------------------------------------------------
!Requirement.                           !Comment                               !
-------------------------------------------------------------------------------- 
!The browser must detect when no streams!                                      !
!are received from a peer               !                                      !
--------------------------------------------------------------------------------

//Stefan

---

From: Henry Sinnreich <[email-masked]>
Date: Thu, 20 Jan 2011 09:48:52 -0600

Hi Stefan,

I believe there is a critical difference between a stream for watching TV
for example, versus A/V streams in an interactive conversation.

> browsers.

For interactive voice, the holy grail is 150 ms, though often this is
somewhat exceeded and there are several limits defined by standards, such as
max 500ms allowed.
For lip synchronization, interactive video must match voice.

For this reason, the term "Interactive A/V Streams" may be a more accurate
definition, or something similar, but distinct from present A/V streaming.

What is the sentiment on list?

Thanks, Henry 

On 1/20/11 4:42 AM, "Stefan Håkansson LK" <[email-masked]>
wrote:

> !received                               !
> !
> !being transmitted                      !
> !
> !Requirement.                           !Comment
> !
> !application aware.                     !
> !
> !devices are used                       !
> !
> !displayed                              !
> !
> !successful or not                      !
> !
> !received any more                      !
> !
> !outgoing streams                       !
> !
> !Requirement.                           !Comment
> !
> !must be supported                      !
> !
> !Requirement.                           !Comment
> !
> !Requirement.                           !Comment
> !
> !has been dropped                       !
> !
> !Requirement.                           !Comment
> !
> _______________________________________________
> dispatch mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dispatch