You are here›Research Topics›Communication›Blog›Beyond HTML5 - Implementing <device> and stream management in WebKit
Beyond HTML5 - Implementing <device> and stream management in WebKit
As described in the previous post Beyond HTML5 - Full-duplex Conversational Voice and Video in Browsers and Web Runtimes we have, as part of a research activity, the ambition to make a test implementation of support for full-duplex audio and video conversations in the web browser/runtime using
<video>, Stream API and Web Sockets.
This post deals with our implementation of support for stream management and
WebKit does not currently support stream management. We have added this in form of a separate WebKit module called MediaSteamManager. The streams must be addressable in order to enable the web developer to access them. The current HTML5 Editors draft of WhatWG stipulates in the section about the Stream API a File URN to be used, referencing the File API W3C draft. However, the File API draft has moved away from
urn: url's, in the latest version
blob: is used instead. Pending a final decision we are currently using
streamdata: url's for addressing streams in our implementation. An example of a stream url:
Implementation of the
As described in an earlier post, the
<device> element represents a device selector which allows the user to grant a web page/app access to input devices such as a microphone or web camera. We've implemented support for
<device> in WebKit, but since the
<device> to be able to query for available devices and to select a device. These extensions must of course be removed at a later stage since they circumvent the security part of the
<device> element. The platform device query can return the default GStreamer V4L2 video source and ALSA audio source. To aid debugging the GStreamer video and audio test sources are also currently returned, but should not be in a final version.
OK, enough about implementation. We have set up this simple demo of a self-view (no audio!):
(in later HTML5 Editor drafts "video_capture" is not a valid type any more and is replaced by "media" - we will update)
Note the inclusion of the file "device_dialog.js" that renders a device selector button and the actual selector as mentioned before. At window.onload the videoDevice is set up. At onchange of the videoDevice, i.e. the user has selected a device, the stream generated by that device is played out in the video element.
The demo displays the html file, starts GtkLauncher, and then demonstrates the selection of the camera and the GStreamer video test source as sources. It can be viewed in this video clip (view in 480p or 720p):
Currently we are looking into sending and receiving streams on Web Sockets. The GTK+ port of WebKit has no WebSocket support, so that is something we are adding. As the summer vacations are starting, there may be some time before the next post!
--Nicklas Sandgren, Adam Bergkvist, Stefan Håkansson