Demo application: OpenTok 3D


The OpenTok.js SDK integrates beautifully into current HTML elements, providing a great variety of layouts and styles. But why should we stick to the traditional 2D design? Modern browsers offer us the power of 3D visualization with WebGL, a technology that has already opened up a new world of interaction and presentation of data within the browser domain.

With an objective to take advantage of the possibilities of 3D within the browser,  we created the  OpenTok 3D demo. The OpenTok 3D demo is a multi-party video application which shows how we can integrate the OpenTok.js API with WebGL technology using the three.js library. One of the objectives of this demo application is to inspire people building on top of the OpenTok.js SDK, showing them the beginning of endless possibilities on how we can present the video screens in a true 3D world.  Cameras, lights, textures, rendering effects, and more, can be leveraged to enrich the final experience.

So, how do we use the video from the publisher and subscribers in the 3D world that we have built? Through textures created after the HTML video element.

Here’s how:

1. WebGL can take several types of input as sources for textures. Specifically, a HTML video element can be used directly as a texture source:


2.  or, if we are using three.js,


3. The only thing left is to get the video element from the OpenTok.js publishers and subscribers. Since the SDK creates them for us, we only need to retrieve it from the “div” using “querySelector”:


Et voilà!

The source code of the project lives in GitHub.

This project only works in Chrome and Firefox. Even though WebGL and three.js work in all browsers, a native webrtc implementation is required, only supported in the mentions browsers.