Communication

In this blog post, I will present a new approach to handling local device access from web pages or web apps. The purpose is to get rid of the dreaded modal prompting as well as providing the user with full control at any time over what information leaves her computer or handheld in response to HTML5 device access APIs.

HTML5 standards are based on a security/privacy concept where the user is prompted with a modal dialog each time a web app or web site requests access to devices like the camera, microphone, location provider, etc.

For example, access to a camera is given to a web app in response to the JavaScript call getUserMedia(“video”, callback). First, the user will be prompted for device access and may choose one of the available devices in a modal dialog. Then the video stream is returned to the web app in the callback dispatched when the modal dialog is dismissed.

The problem with a modal dialog is two-fold:

1. Modal dialogs tend to annoy the user since all execution is stopped until the user has acknowledged by either granting or denying access. Many users, therefore, get in the habit of just “OK”-ing all modal dialogs immediately on sight without reflecting on the consequences.
2. Once access is granted or denied, there is no way for the user to get an overview of the devices a web app or web page has been granted or denied access to, nor is there a way to change an earlier decision.

This post describes a means of giving the user full control, at any time, over external access to local devices such as a camera, microphone, and location without the need for modal dialogs. An added benefit is that the user may always have an overview of the outgoing information. For example, the browser vendor (as opposed to the web app developer) can provide a self-view on outgoing video streams from the camera that cannot be disabled by the app or a third party. We call this mechanism a Device Access Proxy (DAP), and it has been implemented by Ericsson Research in a prototype browser.

Figure: Overview

On browser start-up, the DAP manager aquires access to all the available devices in the platform. Furthermore, the DAP manager also provides a default stream for each type of device. A default stream is a benign data stream such as a test image or a pre-recorded video that serves as a placeholder. What's important is that it does not come from a real device.

A new DAP instance is created in response to each request for device access. Access is immediately granted without user interaction, which is OK, since the default content doesn't originate from any real device. Each proxy instance has a corresponding UI component that allows the user to change the source of the content at any time.

To indicate to the user that a web app/site has requested access, part of the browser chrome (i.e. outside of the HTML view) changes state and its visual appearance reflects the streaming state of the current page. (See figure below.)

Figure: DAP State Top: No requests for devices made. Middle: Outgoing streams are default streams. Bottom: At least one stream is from a real device.

Additionally, the browser chrome provides a means to bring up a status view. As shown in the overview figure above, each call to getUserMedia() will instantiate a proxy, and each proxy has a UI component that is displayed in the status view for the corresponding page.

The UI component shows the available input sources for the stream leaving its proxy (including the fake default stream) as well as the stream from the currently selected source (i.e. a trusted self-view).
The user may freely change the source for any proxy, and the trusted self-view is updated accordingly. The status view may be revisited at any time to change the sources of outgoing streams. The figure below shows our implementation of the Proxy UI component in the prototype.

Figure: Proxy UI Component Reading from top-left to lower-right, this picture shows: i) a default stream, ii) the status view with the proxy UI component for the stream, iii) selecting a new source, and iv) a real stream.

If tabbed browsing is in effect, the streaming state of the background tabs can be communicated to the user by, for example, the color of the label text as shown below.

Figure: Streaming State An example showing how the streaming state of background tabs can be communicated through tab label color. Blue means only default streams; red means at least one outgoing stream from a real device.

The following shows two screen recordings of camera access augmented by the DAP implementation. The first one is a walk-through of the UI elements used in our prototype, and the second one shows user interaction in a WebRTC session where the benefits of the DAP are highlighted. Note that the caller (overlay video in top right corner) is not using a DAP-enabled browser.