Suggestions

close search

Searching ...

The OpenTok Windows SDK includes methods for Windows clients to publish and receive captions in an OpenTok session.

Live captioning must be enabled at the session level via the REST API.

Live captioning is only supported in routed sessions.

This topic includes the following sections:

• Publishing live captions
• Subscribing to live captions

Publishing live captions

An OpenTok publisher can start or stop publishing real-time live captions by setting the PublishCaptions property of the Publisher object:

publisher.PublishCaptions = true;

If the publisher does not include an audio track, the PublisherKit.Error event is dispatched, with the ErrorCode property of the ErrorEventArgs object set to PublisherMissingAudioTrack.

The Vonage Windows SDK does not support a publisher receiving events for its own captions. To render the speaker’s own captions, create a hidden subscriber (to the publisher’s stream) to listen for the caption events. (See the next section.)

Subscribing to live captions

A subscriber may start or stop receiving captions by setting the SubscribeToCaptions property of the Subscriber object:

subscriber.SubscribeToCaptions = true;

You can set this property regardless of whether the publisher of the stream is currently publishing live captions. The subscriber will start receiving captions data once the publisher begins publishing captions.

To stop receiving captions, set the property to false:

subscriber.SubscribeToCaptions = false;

Subscribers receive captions via events. The OpenTok SDK does not display the text of the captions in the UI. Add an event listener for the Subscriber.CaptionText event:

subscriber.CaptionText += (object sender, Subscriber.CaptionTextArgs e) =>
{
    // Display the text in the UI.
};

The HasCaptions property of a Stream object reports whether the stream has captions:

stream.HasCaptions;

The Session.StreamHasCaptionsChanged event is dispatched when a stream has captions enabled and disabled:

session.StreamHasCaptionsChanged += (object sender, StreamEventArgs e) =>
{
    // Adjust UI to indicate that captions are or are not available.
};

Receiving your own live captions

The Vonage Video API does not support a publisher receiving events for its own captions. To render the speaker's own captions, create a hidden subscriber (to the local publisher's stream) to listen for caption events. Do not display this subscriber's view in the UI, and do not subscribe to its audio (to avoid echo). You can then add the captions text to the UI.

The Publisher by default does not publish captions, so set the PublishCaptions property of the Publisher object to true:

    Publisher = new Publisher.Builder(context)
    {
        Capturer = capturerDevices[0].CreateVideoCapturer(VideoCapturer.Resolution.High, VideoCapturer.FrameRate.Fps30),
        Renderer = PublisherVideo
    }.Build();
    Publisher.PublishCaptions = true;

Subscribe to the stream in the publisher's Publisher.StreamCreated event handler:

private void Publisher_StreamCreated(object sender, Publisher.StreamEventArgs e)
{
    Console.WriteLine("Publisher_StreamCreated");
    Subscriber subscriber = new Subscriber.Builder(context, e.Stream)
    {
        Renderer = SubscriberVideo
    }.Build();
    subscriber.SubscribeToAudio = false;
    subscriber.SubscribeToVideo = true; // This is a workaround for an OpenTok bug.
    subscriber.SubscribeToCaptions = true;
    subscriberCallbacks.on_caption_text += Subscriber_OnCaptionText;

    selfPublisherStreamId = e.Stream;
}

As noted above, you must subscribe to video to receive captions. (This will be fixed in a future version.)

This example stores the publisher's stream ID in a selfPublisherStreamId variable, used here to prevent its view from being added to the UI:

private void Subscriber_Connected(object sender, EventArgs e)
{
    if (e.Stream == selfPublisherStreamId) {
        return;
    }

    // Show other subscribers in the UI
}

The subscriber has the transcribed text (for your published stream) in the following callback:

public void Subscriber_CaptionText(object sender, Subscriber.CaptionTextArgs e) {
    // Adjust UI to show captions.
}