Vonage Video Express developer guide

This developer guide provides detailed information on using Video Express.

Basics
UI and layout
Screen sharing
Events
Advanced features
Using Video Express with the Enterprise environment
Using Video Express in mobile apps
Using Video Express with React
Known issues and limitations

Basics

For an simple example of implementing Video Express, see this Quick start code.

About Video Express

Vonage Video Express is a JavaScript library to quickly create a multiparty video conference web application. It works on top of OpenTok.js, the Vonage Video API for web applications.

Set up a Video Express room, based on a Video API session, to enable multiple web-based clients to meet.

Video Express includes the following:

A layout manager, which handles arranging video elements in the HTML page
A quality manager that continuously optimizes stream resolution and frame rate, reducing client bandwidth usage.
An experience manager that dynamically sets speaker priority and auto-mutes joiners for larger meetings (see Active speaker detection).
A Room Manager, which handles low-level interaction with WebRTC and delivers higher-level primitives which are easier to build with.

Important: Before using Video Express, be sure to activate the Video Express add-on for your account.

Go to your Video API Account and click Account Settings in the left-hand menu.
In the list of Account add-ons, find Video Express and click Add to account. Then follow the remaining instructions to enable the add-on.

Loading the Video Express library

Video Express is available as an Node module at npmjs.com:

$ npm i --save @vonage/video-express

You can also load the Video Express library using a script tag an HTML page:

<script src="https://static.opentok.com/v1/js/video-express.js"></script>

Currently, Vonage Video Express doesn't include a default UI. So you will need to add CSS to style the room components in your app:

For a quick start, copy the video-express-styles.css file and include it in the head section of the page using Video Express:

<link rel="stylesheet" href="/path/to/video-express-styles.css" media="screen" charset="utf-8">

Joining a room and publishing a camera stream

Call the Room constructor to join a Video Express Room

const room = new VideoExpress.Room({
  apiKey: '44444444',
  sessionId: '1_this1is2my3new4session5id6',
  token: 'T1==this1is2a3token5nekot6a7si8siht9...',
});

The apiKey, sessionId, and token are the API key for your Video API project, a session ID for the room, and a token. The token must include a "publisher" role. You obtain the API key at your Video API account page. Create session IDs and tokens using the OpenTok server SDKs. See

The following instantiates a Room instance (with all parameters)

const room = new VideoExpress.Room({
  apiKey: "44444444",
  sessionId: "1_this1is2my3new4session5id6",
  token: "T1==this1is2a3token5nekot6a7si8siht9",
  roomContainer: "roomcontainer",
  managedLayoutOptions: {
    layoutMode: "active-speaker",
    cameraPublisherContainer: "cameraPublisherContainerDiv",
    screenPublisherContainer: "screenPublisherContainerDiv",
  }
});

The following instantiates a Room instance but excludes the camera and screen-sharing publisher containers:

const room = new VideoExpress.Room({
  apiKey: "44444444",
  sessionId: "1_this1is2my3new4session5id6",
  token: "T1==this1is2a3token5nekot6a7si8siht9",
  roomContainer: "roomcontainer",
  managedLayoutOptions: {
    layoutMode: "active-speaker",
  }
});

When the client joins the room, the Room object dispatches a connected event:

room.on('connected', () => {
  console.log('Connected');
  // Can use this event to update visual indicator for connection status
});

See Events for more information on the Video Express event model.

Upon joining a room, Video Express creates a camera publisher for the client. This sends the client's video to the other participants in the room. The user is prompted to grant access to the camera and microphone. If the user denies access to the camera and microphone, the client is disconnected from the room, since you are not allowed to join a room without publishing a camera stream.

Important: Vonage Video Express 1.0 currently supports 25 simultaneous participants in a room. If you add more than 25 people in a session, you will need to ensure that participants’ network and hardware performance is sufficient.

Leaving a room

To leave the room, close the browser window or call the leave() method of the Room object:

room.leave();

Detecting when the local client leaves the Room

The Room object dispatches a disconnected event when the client leaves the room:

room.on('disconnected', (reason) => {
  console.log('disconnected reason: ', reason);
  // Can use this event to update visual indicator for connection status
});

The event emits reason string that identifies the reason the client disconnected. See the reference documentation.

The Room object dispatches reconnecting and reconnected events if the client loses its connection to the room and reconnects:

// You can use these event to update a UI indicator for connection status
room.on('reconnecting', () => {
  console.log('Temporarily disconnected.');
});

room.on('reconnected', () => {
  console.log('Reconnected.');
});

Detecting when other clients join and leave the Room

The Room object dispatches a participantJoined event when another participant (not the local user) joins the room:

room.on('participantJoined', (participant) => {
  console.log('participant joined: ', participant.name);
});

The event emits a Participant object identifying the participant that joined. See the reference documentation.

The Room object dispatches a participantLeft event when a participant (other than the local user) leaves the room:

room.on('participantLeft', (participant, reason) => {
  console.log('participant left: ', participant.name);
  console.log('reason: ', reason);
});

The event emits a Participant object identifying the participant that left and and a reason string indicating the reason for the client leaving. See the reference documentation.

Detecting when other clients publish camera streams

When other users publish camera streams, the Video Express layout manager automatically adds the video to the HTML page.

A Participant object dispatches a cameraCreated event when a participant (not the local user) publishes a camera stream:

participant.on('cameraCreated', (cameraSubscriber) => {
  console.log('new camera stream for ', participant.name);
});

The event emits a CameraSubscriber object. The CameraSubscriber object includes methods to enable and disable audio and video playback in the local client (see Enabling and disabling a camera subscriber's audio and video).

A Participant object dispatches a cameraDestroyed event when a participant (not the local user) stops publishing a camera stream:

participant.on('cameraDestroyed', () => {
  console.log('camera destroyed for ', participant.name);
});

UI and layout

Video Express includes a layout manager that automatically arranges video elements in the HTML DOM and adjusts the layout on mobile screens. The layout manager automatically highlights active speakers in a Room. The layout automatically adjusts to make active speakers larger. The layout manager automatically enlarges a screen-sharing video, if there is one.

The layout manager will put a screen-share video in the largest tile of the layout, even if active speaker is applied.

You can set the screenPublisherContainer and cameraPublisherContainer options of the Room() construction to have the video for the local client's published camera and screen-sharing videos appear outside of the layout manager's UI.

On a mobile device, the layout manager shows only 4 camera videos. Also, a screen-sharing and active speaker videos take up the entire screen on a mobile device. You can use the participantJoined and participantLeft events dispatched by the Room object to have a custom UI indicator showing the number of total participants in a room.

By default, the layout manager automatically determines whether a client is running on a mobile device, based on the client's user agent. The managedLayoutOptions parameter of the Room() constructor includes an optional deviceLayoutMode property. Set this to "desktop" or "mobile" to explicitly set the device layout.

Layout options

There are two layouts:

grid — Videos are arranged in a grid layout.
active-speaker — An active speakers (the participant currently speaking) is automatically highlighted in a larger DOM element.

The Room() constructor includes a managedLayoutOptions parameter that lets you set the layout when you join a Room:

const room = new VideoExpress.Room({
  apiKey: '44444444',
  sessionId: '1_this1is2my3new4session5id6',
  token: 'T1==this1is2a3token5nekot6a7si8siht9...',
  roomContainer: 'roomContainer',
  managedLayoutOptions: {
    layoutMode: 'active-speaker'
  }
});

You can also change the layout mode by calling the Room.setLayoutMode() method:

room.setLayoutMode('grid'); // Set layout mode to grid
room.setLayoutMode('active-speaker'); // Set layout mode to active-speaker

The managedLayoutOptions parameter of the Room() constructor also includes an optional deviceLayoutMode property. Set this to "desktop" or "mobile" to explicitly set the device layout. By default, the layout manager automatically determines whether a client is running on a mobile device, based on the client's user agent.

The optional mediaShutoffThreshold parameter of the Room() constructor sets a threshold for the current number of participants in the Room (upon joining) that will prevent the local client’s CameraPublisher from publishing audio and video.

The optional maxVideoParticipantsOnScreen parameter of the Room() constructor sets the maximum number of CameraSubscriber videos shown in a room simultaneously. This affects the display of videos in the local client only.

Changing the camera and microphone used

Setting camera publisher's audio/video devices

// Changes the audio input device
room.camera.setAudioDevice(audioDeviceId);

// Changes the video input device
room.camera.setVideoDevice(videoDeviceId);

Getting camera publisher's audio/video devices

// Returns the current audio input device
const currentAudioDevice = await room.camera.getAudioDevice();

// Returns the current video input device
const currentVideoDevice = room.camera.getVideoDevice();

Setting the audio output device

To get an array of available audio output devices, call the VideoExpress.getAudioOutputDevices() method:

const devices = await VideoExpress.getAudioOutputDevices();

To get the device ID and label for the current audio output device, call the VideoExpress.getActiveAudioOutputDevice() method:

const device = await VideoExpress.getActiveAudioOutputDevice();
console.log(device.deviceId, device.label);

To set the audio output device, call the VideoExpress.setAudioOutputDevice() method, passing in a device ID:

const devices = await VideoExpress.setAudioOutputDevice(deviceId);

The following code shows how to implement methods to cycle through available audio output devices:

const currentAudioIndex;

const devices = await VideoExpress.getAudioOutputDevices;
const currentDevice = await devices.getActiveAudioOutputDevice();
  devices.forEach((device, index) => {
    if (device.label === currentDevice.label) {
      currentAudioIndex = index;
    }
  });

const cycleAudioOutput = () => {
  currentAudioIndex += 1;
  let deviceId = devices[currentAudioIndex % devices.length].deviceId;
  VideoExpress.setAudioOutputDevice(deviceId);
};

Creating a preview publisher

You can optionally set up a preview publisher element before joining the room:

const previewPublisher = new VideoExpress.PreviewPublisher('previewContainer');
await previewPublisher.previewMedia(
  publisherOptions: {
    targetElement: 'previewContainer',
    publisherProperties: {
      resolution: '1280x720'
    },
  },
);

The user is prompted to grant access to the camera and microphone. If the user grants access, the promise returned by the previewPublisher.previewMedia() method is resolved. Otherwise it is rejected.

You can use the preview publisher to have the user select the camera and microphone before joining the room:

// Get the current audio input device
const currentAudioDevice = await previewPublisher.getAudioDevice();

// Change the audio input device
previewPublisher.setAudioDevice(audioDeviceId);

// Get the current video input device
const currentVideoDevice = previewPublisher.getVideoDevice();

// Change the video input device
previewPublisher.setVideoDevice(videoDeviceId);

To get an array of audio and video input devices available, call the VideoExpress.getDevices() method:

const devices = await VideoExpress.getDevices();

The following code shows how to implement methods to cycle through available audio and video input devices:

let currentAudioIndex;
let currentVideoIndex;

VideoExpress.getDevices((err, devices) => {
  audioInputs = devices.filter((device) => device.kind === 'audioInput');
  // Find the right starting index for cycleMicrophone
  audioInputs.forEach((device, index) => {
    if (device.label === previewPublisher.getAudioSource().label) {
      currentAudioIndex = index;
    }
  });

  currentAudioIndex = devices
    // Get all video inputs
    .filter((device) => device.kind === 'audioInput')
    // Find the right starting index for cycleMicrophone
    .findIndex((device) => device.label === previewPublisher.getAudioSource().label);

  currentVideoIndex = devices
    // Get all video inputs
    .filter((device) => device.kind === 'videoInput')
    // Find the right starting index for cycleCamera
    .findIndex((device) => device.label === previewPublisher.getVideoDevice().label);
});

const cycleMicrophone = () => {
  currentAudioIndex += 1;
  let deviceId = audioInputs[currentAudioIndex % audioInputs.length].deviceId;
  previewPublisher.setAudioSource(deviceId);
};

const cycleCamera = () => {
  currentVideoIndex += 1;
  let deviceId = videoInputs[currentVideoIndex % videoInputs.length].deviceId;
  previewPublisher.setVideoSource(deviceId);
};

Before joining the room, you can destroy the preview publisher (removing it from the page):

previewPublisher.destroy();

Enabling and disabling published video

Accessing camera publisher's audio/video

// Check whether a camera publisher video is enabled or not:
room.camera.isVideoEnabled();

// Check whether a camera publisher audio is enabled or not:
room.camera.isAudioEnabled();

// Enable the camera publisher's video:
room.camera.enableVideo();

// Disable the camera publisher's video:
room.camera.disableVideo();

// Enable the camera publisher's audio:
room.camera.enableAudio();

// Disable the camera publisher's audio:
room.camera.disableAudio();

Enabling and disabling a camera subscriber's audio and video

These methods affect the audio and video in the local client's subscriber only.

// Check whether a camera subscriber's video is enabled or not:
participant.camera.isVideoEnabled();

// Check whether a camera subscriber's audio is enabled or not:
participant.camera.isAudioEnabled();

// Enable a camera subscriber's video:
participant.camera.enableVideo();

// Disable a camera subscriber's video:
participant.camera.disableVideo();

// Enable a camera subscriber's audio:
participant.camera.enableAudio();

// Disable a camera subscriber's audio:
participant.camera.disableAudio();

Setting video filters

You can set background blur and background replacement filters to a CameraPublisher video stream with the CameraPublisher.setVideoFilter() method. You can remove filters with the CameraPublisher.clearVideoFilter() method.

// Applying a background blur filter
room.camera.setVideoFilter({
  type: 'backgroundBlur',
  blurStrength: 'high'
});

// Removing the filter
room.camera.clearVideoFilter();

Setting images for when a video is disabled

You can set a video's background image with the setDisabledImageURI method available for the CameraSubscriber, CameraPublisher, ScreenSubscriber, ScreenPublisher, and PreviewPublisher classes. This image will appear when the video is muted.

const disabledImageURI = 'https://path-to.file/dog-dance.jpg';
room.camera.setDisabledImageURI(disabledImageURI);

Setting initials for when a video is disabled

You can set a video's background to be one to two characters with the participantInitials option of the Room() constructor. The initials will appear when the video is muted and the disabledImageURI has not been set.

const room = new VideoExpress.Room({
  apiKey: '4815162342',
  sessionId: '1_this1is2my3new4session5id6',
  token: 'T1==this1is2a3token5nekot6a7si8siht9...',
  participantInitials: 'HI',
});

Active speaker detection

The Video Express layout manager automatically arranges video elements in the HTML DOM and adjusts the layout on mobile screens. The layout manager automatically highlights active speakers in a Room. The layout automatically adjusts to make the active speaker larger. The layout manager automatically enlarges a screen-sharing video, if there is one. The layout manager automatically displays the active speaker if they are hidden.

To enable the active speaker highlight, set the managedLayoutOptions.speakerHighlightEnabled option of the Room() constructor to true. To customize the highlight color, set the managedLayoutOptions.speakerHighlightColor option of the Room() constructor.

The Room object dispatches an activeSpeakerChanged event when there is a new active speaker in the room. When this event is dispatched, you can add UI effects based on which participant is actively speaking.

room.on('activeSpeakerChanged', (participant) => {
  console.log('Active speaker: ', participant.name);
});

The event emits a Participant object, identifying the active speaker.

To start screen sharing, call the Room.startScreensharing() method:

room.startScreensharing('screenContainer')
  .then(() => console.log('Started screen sharing'))
  .catch((err) => console.err(err));

The method takes one optional parameter: the target HTML element to be the container of the screen-sharing video. This can be an HTMLElement or a string (the id of the HTML element). If no parameter is passed in, the container is the screenPublisherContainer property of the managedLayoutOptions parameter passed into the Room() constructor, or to the roomContainer property passed into the Room() constructor. If none of these are specified, the screen-sharing publisher video is added as a child of the body element of the HTML page.

The user is prompted to grant access to the screen. If the user grants access to the screen, the Promise returned by the Room.startScreensharing() method is resolved when screen sharing starts. Otherwise the Promise is rejected.

To stop screen sharing, call the Room.stopScreenSharing() method:

room.stopScreenSharing();

When the screen-sharing publisher starts, a corresponding screen-sharing subscriber is automatically added to the layout manager in each other participant's page.

The following code checks whether a screen-sharing publisher video is enabled or not:

room.screen.isVideoEnabled();

The following code checks whether a screen-sharing publisher audio is enabled or not:

room.screen.isAudioEnabled();

To enable the screen-sharing publisher's video:

room.screen.enableVideo();

To disable the screen-sharing publisher's video:

room.screen.disableVideo();

To enable the screen-sharing publisher's audio:

room.screen.enableAudio();

To disabled the screen-sharing publisher's audio:

room.screen.disableAudio()

When other users publish screen-sharing streams, the Video Express layout manager automatically adds the video to the HTML page.

A Participant object dispatches a screenCreated event when the participant (not the local user) publishes a screen-sharing stream:

participant.on('screenCreated', (screenSubscriber) => {
  console.log('new screen-sharing stream for ', participant.name);
});

The event emits a ScreenSubscriber object. The ScreenSubscriber object includes methods to enable and disable audio and video playback for the screen-sharing stream in the local client (see Enabling and disabling a screen-sharing subscriber's audio and video).

A Participant object dispatches a screenDestroyed event when a participant (not the local user) stops publishing a camera stream:

participant.on('screenDestroyed', () => {
  console.log('screen-sharing stream destroyed for ', participant.name);
});

// Check whether a screen-sharing subscriber video is enabled or not:
participant.screen.isVideoEnabled();

// Check whether a screen-sharing subscriber audio is enabled or not:
participant.screen.isAudioEnabled();

// Enable a screen-sharing subscriber's video:
participant.screen.enableVideo();

// Disable a screen-sharing subscriber's video:
participant.screen.disableVideo();

// Enable a screen-sharing subscriber's audio:
participant.screen.enableAudio();

// Disable a screen-sharing subscriber's audio:
participant.screen.disableAudio();

Events

Video Express events use the event emitter pattern.

You add event listeners for an object by calling its on() method. For example, the following code adds a listener for the connected event dispatch by the Room object:

room.on('connected', () => {
  console.log('Connected');
});

The on() method takes two parameters: the name of the event (a string) and a callback function that is called when the event is dispatched.

For some events, arguments are passed into the callback function. The event is said to emit these objects. For example, the Room disconnected event emits a reason string:

room.on('disconnected', (reason) => {
  console.log('Disconnected:', reason);
});

The Room participantLeft event emits two objects — a Participant object and a reason string:

room.on('connected', (participant, reason) => {
  console.log('participant left: ', participant.name);
  console.log('reason: ', reason);
});

Advanced features

End-to-end encryption

To use end-to-end encryption, instantiate the Room object with an encryption secret:

const room = new VideoExpress.Room({
  apiKey: '44444444',
  sessionId: '1_this1is2my3new4session5id6',
  token: 'T1==this1is2a3token5nekot6a7si8siht9...',
  encryptionSecret: 'this-is-a-secret',
});

After a room is created with an encryption secret, it can be updated to use a new secret by calling the Room.setEncryptionSecret(encryptionSecret) method:

room.setEncryptionSecret('this-is-a-secret');

Note: End-To-End encryption needs to be enabled by the server SDKs first. See the Video Express developer guide.

Signaling

Once you have joined a room, you can use the Room.signal() method to send a signal to one or more participants in the room.

The following code sends a signal to all participants in the Room:

room.signal({
  data: 'hello.'
});

The following code sends a signal to a specific participant in the Room:

room.signal({
  to: participant, // this is a Participant object
  data: `hello ${participant.name}.`
});

You can include an optional type property of the signal options to help identify the signal:

room.signal({
  type: 'greeting',
  data: `hello.`
});

When a signal is received, the Room object dispatches a signal event, which emits a SignalEvent object that defines the signal:

room.on('signal', (signalEvent) => {
  console.log(`Signal received from ${signalEvent.from}. Data: ${signalEvent.data}.`);
});

When a signal that includes a type is received, the Room object dispatches a signal:type event, which emits a SignalEvent object that defines the signal:

room.on('signal:greeting', (signalEvent) => {
  console.log(`Signal received from ${signalEvent.from}. Data: ${signalEvent.data}.`);
});

For more information, see the Room.signal() and the signal and signal: events in the Video Express reference documentation. Also see the Signaling overview.

Setting a proxy URL

With the IP proxy feature, clients route all internet traffic (except for media streams) via your proxy server. Non-media traffic includes communication to the Video API servers and logging infrastructure.

The IP proxy feature is available as an add-on feature.

The following code sets the URL of the IP proxy server. Call it before calling the Room() constructor:

VideoExpress.setProxyUrl('https://123.123.123.123:8080');

For more information, see the IP proxy developer guide.

Using custom TURN servers

The configurable TURN server feature is available as an add-on feature. To configure the TURN servers used by the Video Express client, set the options of the iceConfig property of the options parameter of the Room() constructor.

For more information, see the configurable TURN server developer guide.

Using connection data

When you call the joining a Video Express room, you pass in a connection token. A token can have an optional connection data string, containing metadata. You can access this data with the Room.participantConnectionData and Participant.connectionData properties.

Server-side code (Node.js)

This example shows how to add connection data when generating a token using the OpenTok Node SDK:

// Generating token from your server
const token = opentok.generateToken(sessionId, {
  role: 'moderator',
  data: '{"disabledImageURI":"https://path-to.file/jason.jpg"}',
});

This example shows how to enable end-to-end-encryption when creating a new session using the OpenTok Node SDK:

opentok.createSession({
  mediaMode: 'routed',
  e2ee: true, 
});

Client-side code (Video Express)

// Setting a participant's disabledImageURI with data from token
participant.on('cameraCreated', (cameraSubscriber) => {
  // connectionData is a JSON string from some your server
  const { connectionData } = participant;
  const { disabledImageURI } = JSON.parse(connectionData);
  cameraSubscriber.setDisabledImageURI(disabledImageURI);
});

For more information, see Token Creation Overview.

Using Video Express with the Enterprise environment

Vonage Video Express currently only offers a Standard environment. However, you can use Enterprise projects with Video Express. This ensures that Video Express utilizes OpenTok signaling and media servers that are dedicated to partner applications using the Enterprise environment. This offers greater stability from changes and more resilience against platform load spikes.

Using Video Express in mobile apps

See the Vonage Video Express iOS and Android demo app on GitHub. This app uses Video Express in web views (such as WKWebView in iOS and WebView in Android) for mobile apps.

Using Video Express with React

See the Vonage Video Express React demo app on GitHub. This app uses Video Express and React for a multi-party video app. In addition to features provided by Video Express, the app includes room and participant name selection, archiving (recording), and more.

Known issues and limitations

Vonage Video Express works on top of OpenTok.js, the Vonage Video API for JavaScript. All clients connecting to a Vonage Video Express room should use Vonage Video Express. Using other Vonage Video API client SDKs (including web clients built directly using OpenTok.js APIs) can disable some features in Vonage Video Express.

There are some limitations of OpenTok.js and Video API platform features that Vonage Video Express does not support:

Video Express does not support large interactive broadcasts because Video Express currently requires every client that joins a room to publish a camera stream.
Video Express may take slightly longer to connect when used with the
Allowed IP list feature. Video Express focuses on simplifying the developer experience. For this reason, Video Express will try downloading configuration files via CDN (which may be blocked by your firewall) before it uses the addresses on the allowed IP list. You cannot specify that a client should use the allowed IP addresses only (as you would with the ipWhitelist option in OpenTok.js OT.initSession() method). A Video Express client may take slightly longer to connect than a client using OpenTok.js configured with the ipWhitelist option set to true when using Allowed IP list with firewalls which block content delivery networks.

Participants in Video Express correspond to OpenTok connections. Camera and screen-sharing streams in Video Express correspond to OpenTok streams. Data and events for connections and streams show up in developer tools such as Inspector, Insights, and others. Video Express provides connection IDs, through Participant.id, Room.participantId, and the keys of Room.participants, which can be used to identify specific users for connection and stream events and data. Some features that require connection and stream IDs are not supported with Video Express. These include:

Most Moderation features. However, you can use the REST API method to force all streams in a Video Express room to mute audio (though you cannot include a list of stream IDs to exclude). More Video Express Moderation features are coming soon.
Selecting streams to be included in an archive or live streaming broadcast.
Changing the layout class for an archive or live streaming broadcast that uses a custom layout.

Vonage Video Express does not provide access to the underlying OpenTok.js Publisher and Subscriber objects. So, you cannot use the getStats() and getRtcStatsReport() methods of these objects. Video Express automatically optimizes the frame rate and resolution of all video streams so there is no need to call the setPreferredFrameRate() and setPreferredResolution() methods for a subscriber or publisher.

For details on known issues and fixed issues, see the Vonage Video Express release notes.

For details on known issues regarding OpenTok.js, see the OpenTok.js Known Issues.

Could we improve this page? Let us know what's missing.

Feedback

Suggestions