Video Stream Layout Management

Problem

A common challenge developers face when creating video chat applications is how to lay out the video streams on the page in an elegant manner. Many video chat applications involve a variable number of streams, users are constantly joining and leaving the chat session—dealing with these changes so that the streams do not affect other elements on the page can be difficult.

Solution

One way of dealing with this challenge is to define a fixed-size container where you want all your streams to live and then dynamically resize the streams within the container so that all the streams can fit. This is the approach the TokBox team took when developing the OpenTok HTML Embed Widget. You can see this approach in action in the sample app we are going to build here.

Tutorial Overview

In this tutorial I will cover the LayoutContainer class, a javascript class created for the HTML Embed Widget which makes it simple for OpenTok developers to lay out a variable number of streams on the page in a way that minimizes empty space and does not affect other elements on the page. This tutorial assumes that you have a basic knowledge of how an OpenTok application works. If you do not, please first read the basic tutorial on how to get started.

The first part of this tutorial will go over the LayoutContainer class and explain its methods. The second part will go over usage of the LayoutContainer class within a simple OpenTok application. I suggest that you first try the LayoutContainer Sample app we are going to build to understand the problem and how this approach solves it before continuing.

Before we start, if you would like to download the LayoutContainer.js class and the sample code for what we are going to build, you can download both from the Github repository here.

The LayoutContainer Class

The LayoutContainer class is made up of four methods. I will go through each of these four methods and explain when and how to use them.

LayoutContainer.init(divId, width, height)

The LayoutContainer.init() method initializes the div where you want your streams to be displayed and should be called once at the beginning of your application. This method takes as parameters the div id of the div to be used to manage the streams, and the height and width that you would like that div to be (Note: you do not need to set this height and width in the markup or CSS, the LayoutContainer class will do it for you).

LayoutContainer.addStream(divId, publisher)

The LayoutContainer.addStream() method adds a stream to be managed by the LayoutContainer. This method will usually be called right before you subscribe or publish a stream. This method takes as parameters the id that you pass in to the Session.publish() or Session.subscribe() methods and also a boolean variable publisher that is true if the id you are passing in is from a publisher or false if the id is from a subscriber.

LayoutContainer.removeStream(subscriberId)

The LayoutContainer.removeStream() method removes a stream from being managed by the LayoutContainer. This method takes one parameter, the subscriber id of the subscriber from the stream to be removed.

LayoutContainer.layout()

The LayoutContainer.layout() method lays out the LayoutContainer to display changes from any streams that were added or removed. This method calculates the optimal row and column configuration to use to arrange the videos to minimize the amount of wasted space within the container (i.e. makes each stream as large as possible without going beyond the edge of the LayoutContainer). This method should usually be called after you have added or removed streams from the LayoutContainer to update the display.

Using the LayoutContainer Class

The first thing we must do is include the appropriate JavaScript files. We are going to include the LayoutContainer.js for the LayoutContainer and TB.min.js which is the OpenTok JavaScript API.

[javascript]
<script src="OT_LayoutContainer.js" type="text/javascript" charset="utf-8"></script>
<script src="http://staging.tokbox.com/v0.91/js/TB.min.js" type="text/javascript" charset="utf-8"></script>
[/javascript]

Next we will initialize the session object and set up the event handlers we need.
[javascript]
var apiKey = 1127; // OpenTok sample API key. Replace with your own API key.
var sessionId = ‘287a9e4ad0aa8e501309df11fe53831452fa1167’; // Replace with your session ID.
var token = ‘devtoken’; // Should not be hard-coded.

session = TB.initSession(sessionId);

session.addEventListener(‘sessionConnected’, sessionConnectedHandler);
session.addEventListener(‘streamCreated’, streamCreatedHandler);
session.addEventListener(‘streamDestroyed’, streamDestroyedHandler);
[/javascript]

We initialize our layout container (and we have a div with id “streamContainer” in the HTML). In this example I picked a resolution of 500px by 340px—you can adjust this to whatever size you would like your container to be.
[javascript]
OT_LayoutContainer.init("streamContainer", 500, 340);
[/javascript]

Now before we define our sessionConnected, streamCreated, and streamDestroyed event handlers, we are going to create two helper methods for publishing and subscribing to streams.
[javascript]
function publishStream() {
// Make up an id for our publisher
var divId = ‘opentok_publisher’;

// Pass in TRUE since this is a publisher
OT_LayoutContainer.addStream(divId, true);

session.publish(divId);
}
[/javascript]
In the previous method we define an id, pass it to the LayoutContainer which will create the div for us and manage it for us, and then publish with it.

[javascript]
function subscribeToStreams(streams) {
// For each stream
for (var i = 0; i < streams.length; i++) {
// Check if this is the stream that I am publishing, and if so do not subscribe.
if (streams[i].connection.connectionId != session.connection.connectionId) {
// Make a unique div id for this stream
var divId = ‘stream_’ + streams[i].streamId;

// Pass in FALSE since this is a subscriber
OT_LayoutContainer.addStream(divId, false);

session.subscribe(streams[i], divId);
}
}
}
[/javascript]
In the previous method we take an array of streams, define unique ids for each one, and then again pass those ids to the LayoutContainer which takes care of creating and managing the divs for us. Finally, we subscribe to each stream.

Now that we have our helper methods set up and the session and LayoutContainer initialized, all that is left to do is set up our sessionConnected, streamCreated, and streamDestroyed event handlers.

[javascript]
function sessionConnectedHandler(event) {
// Publish my stream to the session
publishStream();

// Subscribe to all streams currently in the Session
subscribeToStreams(event.streams);

// Re-layout the container with the new streams
OT_LayoutContainer.layout();
}

function streamCreatedHandler(event) {
// Subscribe to the newly created streams
subscribeToStreams(event.streams);

// Re-layout the container with the new streams
OT_LayoutContainer.layout();
}

function streamDestroyedHandler(event) {
// Get all destroyed streams
for (var i = 0; i < event.streams.length; i++) {
// For each stream get the subscriber to that stream
var subscribers = session.getSubscribersForStream(event.streams[i]);
for (var j = 0; j < subscribers.length; j++) {
// Then remove each stream
OT_LayoutContainer.removeStream(subscribers[j].id);
}
}

// Re-layout the container without the removed streams
OT_LayoutContainer.layout();

}
[/javascript]
These event handlers are pretty straight-forward. Whenever somebody connects they first publish and then subscribe to all existing streams. We subscribe to new streams when they are created and remove streams whenever they are destroyed. Notice that we call OT_LayoutContainer.layout() at the end of each event handler to update the display of the LayoutContainer with changes.

Now when we connect to the session we will have a div that contains all our streams. As people join and leave the session, the streams within the container will re-arrange themselves to make the most efficient use of the available space. Since all streams are contained within a fixed-size div, we do not need to worry about how the the varying number of streams might affect the rest of the page.

Conclusion

The LayoutContainer class is a simple solution for solving a common problem when it comes to video chat applications. Using this class is one solution, but it is certainly not the only one. If you have any ideas or are interested in hacking together your own classes and modules for other OpenTok developers to use, I would love you to post your thoughts here or tweet them @tokboxdev.

The LayoutContainer class, sample application, and documentation is available for download here.