Building the HTML client for thev v OpenTok TokShow app

Building the TokShow app gave us another great opportunity to test our API in a real-world app. And, sure enough, we discovered some additions for the OpenTok API.

Not all clients using the app and participating in the session have the same audio-video capabilities. Some clients use older versions of Flash Player or hardware that does not support acoustic echo cancellation. The acoustic echo cancellation feature was added in OpenTok v0.91.18 (in June), and it works great. It pretty much eliminates acoustic audio feedback. We wanted to build an app that allowed the administrator to see if a potential participant (fan) had acoustic echo cancellation supported on their machine. However, the OpenTok API did not provide that information … so we added it. And the new API includes other information on the quality of each stream in the session. In addition to acoustic echo cancellation support, the API provides other information about the publisher of a stream, including the upstream bandwidth, whether the microphone and camera are enabled, and whether H.264 video is supported. These enhancements were added in the November 10 release of the OpenTok API (v0.91.35).

The app consists of three separate views: one for the artist, one for fans, and one for the administrator. We managed each of these in corresponding JavaScript files. Adding OpenTok audio-video streams to the page was simple, as you simply need to specify the HTML element that a stream replaces. This means that you can easily integrate OpenTok with jQuery or other frameworks. We used Backbone.js to manage views and associated data models.

The administrator page is used by the director or producer of the live event. In addition to the audio-video streams of the artist and the current fan talking to the artist, the administrator page includes a list of fans waiting to chat with the artist. The administrator can then queue up the next fan to chat with the artist, and then put that fan “on-stage” (while removing the current fan). This administration page included information on each fan’s capabilities, including their acoustic echo cancellation capabilities (as mentioned above), providing the administrator with information on which fans had the requirements needed for a good connection.

As we were in the process of development, we became aware of the need to maintain states for each view. For example, a fan’s page could be in one of the following states: (a) just watching the artist chat with other fans, (b) in line waiting to chat with the artist, and (c) “onstage,” chatting with the artist live. As states change, we published or subscribed to OpenTok streams, depending on the needs of the current state.

We needed to build an app that would support thousands of simultaneous connections to the OpenTok session, without burdening any client with unneeded events or audio-video data. OpenTok supports the numbers our app requires, but we made sure that each view was only getting the data it needs. For example, the fan and artist pages would only subscribe to two audio-video streams: that of the artist and that of the fan talking to the artist. On the administrator page, we included controls for turning on and off audio and video for fans that were in line. (This is easy to do. In the OpenTok API, each Subscriber object has a subscribeToAudio() method and a subscribeToVideo() method, which you toggle audio and video on and off.) Also, we add “artist” as connection data when generating the artist’s token; this way pages can identify the artist’s stream by checking the connection.data property of the Stream object.

We used the OpenTok signaling API to have the administrator send information to other participants in the session. Basically, the administrator page would make an Ajax call to store information on the server. Then it would call the OpenTok Session.signal() method. Fan pages connected to the session (as well as the artist page) would then get a signalReceived event (defined by the OpenTok API) and use an Ajax call to look up the data. (See the blog post about the server-side code for the app.) For example, as a new fan went “on-stage”, the administrator page would relay the ID for the fan’s stream, using this signaling method, and other pages connected to the session (fans and the artist) then subscribed to the stream (using the Session.subscribe() method of the OpenTok API). Also, we add “administrator” as connection data when generating the administrator’s token; pages check the fromConnection.data property of the signalReceived event to make sure that the signal event is from the administrator (and worth processing).

In an upcoming build of OpenTok, we will be adding an API to save and transmit state information about sessions. This will make it easier to relay information among participants without relying on storing data on the server.