Last week, we released OpenTok v2.14, the latest version of our Client SDKs. We wanted to update you on some of the great new features included and how you can use them.
With the Frame Metadata API, cycleVideo and facingMode APIs and other features, there is plenty to make it even easier for you to build great live video apps on OpenTok.
Frame Metadata API
The new Frame Metadata API provides developers with a simple way to add metadata to each frame as it is captured. The metadata is embedded with the video stream so that it arrives at the receiver at the same time. The subscriber to the stream can extract this metadata and use it to enhance the experience of the end user.
The metadata can be used for any purpose the developer deems fit. It could be used to simply carry a timestamp or some other test data. It can also be used in a more sophisticated AR application to capture the location of the camera in 3D space.
Another use is with a frame level graphics transform on a region of interest to enhance the video quality of that region. Our CTO, Badri Rajasekar, gave a presentation at last year’s Kranky Geek Event that demonstrated using this technique to perform “content-aware video encoding.” An app can detect areas where faces are in a video stream and provide more pixels to those areas compared to the other background areas. The app can add metadata to each video frame to indicate the areas that contain faces. It can then use this metadata to properly render the video in subscribing clients.
The Frame Metadata API is available in the OpenTok iOS, Android, and Windows SDKs.
You can find sample applications using the Frame Metadata API here:
cycleVideo and facingMode APIs
Codec not supported Event
With Safari’s lack of VP8 support and with Chrome on Android not always supporting H.264, there are times when you will not be able to see the video of the other participant. When this happens, OpenTok.js displays the “Video format not supported” message (shown above) to end users. We now also surface an Event to our developers in case they want to supply their own UI or provide more details to the end user. The videoDisabled Event in the Subscriber now has a new reason (“codecNotSupported”) to account for this.
Other bug fixes and performance improvements in OpenTok v2.14
- The OpenTok plugin for Internet Explorer now supports the setAudioVolume() and getAudioVolume() methods of the Subscriber.
- The audioLevelUpdated Event now works in Safari on iOS without a user gesture. This is actually not something we fixed but another example of us working closely with a browser vendor, in this case Apple, to get an issue fixed and shipped in the browser.
- We found and fixed some memory leaks in OpenTok.js that will improve performance and prevent crashes in long running sessions.
- We did some refactoring of OpenTok.js to remove some race conditions from the code. This has resulted in a reduction in the number of errors we’re seeing!
- We also made some changes in our native SDKs (iOS, Android, and Windows). These will result in performance improvements, including lower CPU and memory usage.
If you’d like to start building with some of these features, sign up for your free OpenTok account here.