Suggestions

Searching ...

The OpenTok Android SDK includes methods for Android clients to publish and receive captions in an OpenTok session.

Live captioning must be enabled at the session level via the REST API.

Live captioning is only supported in routed sessions.

This topic includes the following sections:

• Publishing live captions
• Subscribing to live captions

Publishing live captions

An OpenTok publisher can start or stop publishing real-time live captions by calling the setPublishCaptions() method of the PublisherKit object:

publisher.setPublishCaptions(true);

If the publisher does not include an audio track, the PublisherKit.PublisherListener.onError() method is called with an error (with the code property of the error set to PublisherMissingAudioTrack).

The Vonage Android SDK does not support a publisher receiving events for its own captions. To render the speaker’s own captions, create a hidden subscriber (to the publisher’s stream) to listen for the caption events. (See the next section.)

Subscribing to live captions

A subscriber may start or stop receiving captions by calling the subscribeToCaptions() method of the SubscriberKit object:

subscriber.setSubscribeToCaptions(true);

You can call this method regardless of whether the publisher of the stream is currently publishing live captions. The subscriber will start receiving captions data once the publisher begins publishing captions.

To stop receiving captions, pass false into the method:

subscriber.setSubscribeToCaptions(false);

Subscribers can verify whether they are actively subscribed to a stream's live captions using the isSubscribedToCaptions() method:

boolean isSubscribed = subscriber.getSubscribeToCaptions();

Subscribers receive captions via events. The OpenTok SDK does not display the text of the captions in the UI. Use the SubscriberKit.CaptionsListener interface to set up a listener for captions events:

SubscriberKit.CaptionsListener captionsListener = new SubscriberKit.CaptionsListener() {
    @Override
    public void onCaptionText(SubscriberKit subscriber, String text, boolean isFinal) {
        // Display the text in the UI.
    }
}

The hasCaptions() method of a Stream object reports whether the stream has captions:

boolean hasCaptions = stream.hasCaptions();

Implement the onStreamHasCaptionsChanged() method of the Session.StreamPropertiesListener interface to monitor when a stream has captions enabled and disabled:

Session.StreamPropertiesListener captioningListener = new Session.StreamPropertiesListener() {
    @Override
    public void onStreamHasCaptionsChanged(Session session, Stream stream, boolean hasCaptions) {
        // Adjust UI to indicate that captions are or are not available.
    }
}

Receiving your own live captions

The Vonage Video API does not support a publisher receiving events for its own captions. To render the speaker's own captions, create a hidden subscriber (to the local publisher's stream) to listen for caption events. Do not display this subscriber's view in the UI, and do not subscribe to its audio (to avoid echo). You can then add the captions text to the UI.

The Publisher by default does not publish captions, so call setPublishCaptions() passing in true:

    publisher = new Publisher.Builder(this).name(mUsername).build();
    publisher.setPublishCaptions(true);

Subscribe to the stream in the publisher's onStreamCreated() callback:

@Override
public void onStreamCreated(PublisherKit publisher, Stream stream) {
    Log.d(LOG_TAG, "onStreamCreated: Publisher Stream Created. Own stream " + stream.getStreamId());
    subscriber = new Subscriber.Builder(MainActivity.this, stream).build();
    subscriber.setSubscribeToAudio(false);
    subscriber.setSubscribeToVideo(true); // This is a workaround for an OpenTok bug.
    subscriber.setSubscribeToCaptions(true);
    subscriber.setCaptionsListener(this);

    selfPublisherStreamId = stream.getStreamId();
}

As noted above, you must subscribe to video to receive captions. (This will be fixed in a future version.)

This example stores the publisher's stream ID in a selfPublisherStreamId variable, used here to prevent its view from being added to the UI:

@Override
public void onStreamReceived(Session session, Stream stream) {
    Log.d(TAG, "onStreamReceived: New Stream Received " + stream.getStreamId() + " in session: " + session.getSessionId());

    if(stream.getStreamId() == selfPublisherStreamId) {
        return;
    }

    // Show subscriber in the UI
    if (subscriber == null) {
        subscriber = new Subscriber.Builder(MainActivity.this, stream).build();
        subscriber.getRenderer().setStyle(BaseVideoRenderer.STYLE_VIDEO_SCALE, BaseVideoRenderer.STYLE_VIDEO_FILL);
        subscriber.setSubscriberListener(subscriberListener);
        session.subscribe(subscriber);
        subscriberViewContainer.addView(subscriber.getView());
    }
}

The subscriber has the transcribed text (for your published stream) in the onCaptionText() listener:

@Override
public void onCaptionText(SubscriberKit subscriber, String text, boolean isFinal) {
    // Adjust UI to show captions.
}