Subject: [rtcweb] Screen sharing

From: Stefan Håkansson LK <[email-masked]>
Date: Thu, 1 Dec 2011 16:19:52 +0100

Related to use-case 4.2.7 "Simple Video Communication Service with 
sharing" 
(<http://datatracker.ietf.org/doc/draft-ietf-rtcweb-use-cases-and-requirements/?include_text=1>) 
we had a student experimenting a bit.

Essentially, the screen was treated as another video source (just as a 
camera), and was transmitted to the other end as a MediaStream over a 
PeerConnection.

The implementation of getUserMedia allowed the screen to be selected as 
a video source.

Perhaps this could be a way forward for meeting this use-case? And 
perhaps "screen" should be defined as a valid options argument to 
getUserMedia?

(there is some more info about what the student did at 
<https://labs.ericsson.com/developer-community/blog/screen-sharing>)

Stefan

---

From: Henry Sinnreich <[email-masked]>
Date: Thu, 01 Dec 2011 13:18:26 -0600

Hi Stefan,

Video sucks for screen sharing, displaying documents or even plain bullet
slides.
Displaying the screen and documents is in a completely other league IMO.

Thanks, Henry

On 12/1/11 9:19 AM, "Stefan Håkansson LK" <[email-masked]>
wrote:

> _______________________________________________
> rtcweb mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtcweb

---

From: Justin Uberti <[email-masked]>
Date: Thu, 1 Dec 2011 14:47:26 -0500

It works pretty well for us in Hangouts, and we'd like to continue to
support this functionality in WebRTC, so I think this area is definitely
something this group should explore.

getUserMedia("screen") seems like a good API to start with; it already
needs to pop up a chooser dialog, and it has a mechanism to handling
success or failure.

As the demo indicates, there are many types of "screens" that the user
could be prompted to share:
- The entire desktop
- Individual windows on the desktop
- Windows owned by the browser
- Specific tabs within the browser
- A user-selectable onscreen region

The exact details can be browser-specific, but we'll need to give some
guidance here so that applications can have a consistent experience across
browsers.

On Thu, Dec 1, 2011 at 2:18 PM, Henry Sinnreich
<[email-masked]>wrote:

> > (<
> http://datatracker.ietf.org/doc/draft-ietf-rtcweb-use-cases-and-requirements
> > /?include_text=1>)

---

From: Randell Jesup <[email-masked]>
Date: Thu, 01 Dec 2011 14:55:49 -0500

> Displaying the screen and documents is in a completely other league IMO.

It can suck, for sure, depending on resolution.  If you change the 
encoding characteristics to be appropriate for a screen, it can be much 
better.

We used to deliver Internet-over-Cable-Settops (at WorldGate), using 
MPEG-2 p-frames and Xvfb with damage regions.  The trick it to not run 
steady video-like frame rates in CBR or near-CBR, but to instead adapt 
the codec to the source material - run highly variable framerates in 
partial VBR, and allow some initial fuzziness on a large changes to 
sharpen in subsequent frames (and maybe delay the first frame of a large 
change and use more bits for it), then stop updates if the source is 
static.  Also you'd probably want to use a heuristic or other mechanism 
to detect sharing a window with video or animation in it and shift 
encoding modes to a video-centric mode.  And obviously use as large a 
resolution as is useful at the other end, if the bandwidth supports it. 
  And never, ever send an iframe unless needed. :-)

(Note: I wasn't the video expert on this; I handled the browser side.)

-- 
Randell Jesup
[email-masked]

---

From: Aron Rosenberg <[email-masked]>
Date: Thu, 1 Dec 2011 15:05:02 -0800

Screen sharing works great over H.264 or VP8 for us. You have to tweak the
codec encoding parameters and FPS a bit, but you get great quality even on
text.

Aron Rosenberg
LifeSize, a division of Logitech

---

From: Henry Sinnreich <[email-masked]>
Date: Thu, 01 Dec 2011 18:31:34 -0600

Can remote meeting attendees read the print on say a legal document under
discussion?

If yes for commercial grade quality (I still have doubts), how is the codec
tweaking enabled via the API?

Thanks, Henry

On 12/1/11 5:05 PM, "Aron Rosenberg" <[email-masked]> wrote:

> On Thu, Dec 1, 2011 at 11:18 AM, Henry Sinnreich <[email-masked]>
> wrote:
>> Hi Stefan,
>> 
(<http://datatracker.ietf.org/doc/draft-ietf-rtcweb-use-cases-and-requiremen>>>
ts

---

From: Justin Uberti <[email-masked]>
Date: Thu, 1 Dec 2011 19:57:48 -0500

On Thu, Dec 1, 2011 at 7:31 PM, Henry Sinnreich
<[email-masked]>wrote:

>  Can remote meeting attendees read the print on say a legal document
> under discussion?
>

In the implementations we have been referring to, yes.

>
> If yes for commercial grade quality (I still have doubts), how is the
> codec tweaking enabled via the API?
>

It's automatically enabled when attaching a source of type "screen" to a
PeerConnection.

> On Thu, Dec 1, 2011 at 11:18 AM, Henry Sinnreich <
> [email-masked]> wrote:
>
> Hi Stefan,
>
> ------------------------------

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 1 Dec 2011 20:02:08 -0500

To be clear, we're talking about H.264/SVC, right?  

On Dec 1, 2011, at 18:05, "Aron Rosenberg" <[email-masked]> wrote:

> Screen sharing works great over H.264 or VP8 for us. You have to tweak the codec encoding parameters and FPS a bit, but you get great quality even on text.
> 
> Aron Rosenberg
> On Thu, Dec 1, 2011 at 11:18 AM, Henry Sinnreich <[email-masked]> wrote:
> Hi Stefan,
>

---

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 2 Dec 2011 14:49:46 +0100

> Can remote meeting attendees read the print on say a legal document
All I can say is that in the little testing we did it looks quite OK. If 
you want to you can test our using our lib; depending on what GStreamer 
plug-ins you add Theora (no plug-in added), VP8 or H.264 would be used.

But probably Aron's/Justin's experiences are more valid as those are 
based on live services if I understand correctly.

My conclusion is that using video for screen sharing can be a quick way 
forward.
> codec tweaking enabled via the API?
I proposed that a "screen" option could be added to the API (now "audio" 
and "video" is there). If "screen" is selected, the codec parameters 
could be tweaked accordingly (we have no tweaking in our current lib though)
>     Screen sharing works great over H.264 or VP8 for us. You have to
>     tweak the codec encoding parameters and FPS a bit, but you get great
>     quality even on text.
>
>     Aron Rosenberg
>     <[email-masked]> wrote:
>
>         Hi Stefan,
>
>         <https://labs.ericsson.com/developer-community/blog/screen-sharing>
>         <https://labs.ericsson.com/developer-community/blog/screen-sharing>>)
>         >
>         >  Stefan
>     ------------------------------------------------------------------------

---

From: Emil Ivov <[email-masked]>
Date: Fri, 02 Dec 2011 15:18:18 +0100

FWIW, that's how we've been doing desktop sharing in Jitsi for quite a
while now and the feedback is quite positive. Quality more than
satisfactory and bandwidth is way better than what you get with things
like VNC.

-- 
http://jitsi.org