Suggestions

close search

Searching ...

The OpenTok iOS SDK includes methods for iOS clients to publish and receive captions in an OpenTok session.

Live captioning must be enabled at the session level via the REST API.

Live captioning is only supported in routed sessions.

This topic includes the following sections:

• Publishing live captions
• Subscribing to live captions

Publishing live captions

An OpenTok publisher can start or stop publishing real-time live captions by setting the publishCaptions property of the OTPublisherKit object:

publisher.publishCaptions = YES;

If the publisher does not include an audio track, the [PublisherKit publisher:didFailWithError:] message is sent, with the code property of the error set to OTPublisherMissingAudioTrack.

The Vonage iOS SDK does not support a publisher receiving events for its own captions. To render the speaker’s own captions, create a hidden subscriber (to the publisher’s stream) to listen for the caption events. (See the next section.)

Subscribing to live captions

A subscriber may start or stop receiving captions by setting the subscribeToCaptions property of the OTSubscriberKit object:

subscriber.subscribeToCaptions = YES;

You can call this method regardless of whether the publisher of the stream is currently publishing live captions. The subscriber will start receiving captions data once the publisher begins publishing captions.

To stop receiving captions, set the property to NO:

subscriber.setSubscribeToCaptions = NO;

The [OTSubscriberKitCaptionsDelegate subscriber:caption:isFinal:] message is sent when a subscriber to a stream receives captions:

// OTSubscriber delegate callbacks:
-(void) subscriber:(OTSubscriberKit *)subscriber caption:(NSString *)text isFinal:(BOOL)final {
    // Display caption text in UI.
}

You can set up a key-value observer for the hasCaptions property of an OTStream to see when the stream has captions enabled and disabled:

//add observer on a stream of interest
[_subscriber.stream addObserver:self forKeyPath:@"hasCaptions" options:NSKeyValueObservingOptionNew context:NULL];
    
- (void)observeValueForKeyPath:(NSString *)keyPath
                      ofObject:(id)object
                        change:(NSDictionary *)change
                       context:(void *)context {
    NSLog(@"KVO change for %@ = %@",keyPath, change); 
    //manipulate GUI element
}

Receiving your own live captions

The Vonage Video API does not support a publisher receiving events for its own captions. To render the speaker's own captions, create a hidden subscriber (to the local publisher's stream) to listen for caption events. Do not display this subscriber's view in the UI, and do not subscribe to its audio (to avoid echo). You can then add the captions text to the UI.

The Publisher by default does not publish captions, so set the publishCaptions property of the publisher to TRUE:

    _publisher =
    [[OTPublisher alloc] initWithDelegate:self
                                 settings:settings];
    _publisher.publishCaptions = TRUE;
    OTError *error = nil;
    [_session publish:_publisher error:&error];

Subscribe to the stream in the [OTPublisherKitDelegate publisher:streamCreated:] method:

- (void)publisher:(OTPublisherKit *)publisher
    streamCreated:(OTStream *)stream
{
    // Self subscribe to our own publisher stream
    _subscriber = [[OTSubscriber alloc] initWithStream:stream delegate:self];
    _subscriber.subscribeToAudio = false;
    _subscriber.subscribeToVideo = true;  // This is a workaround for an OpenTok bug.
    _subscriber.subscribeToCaptions = true;
    _subscriber.captionsDelegate = self;
    OTError *error = nil;
    [_session subscribe:_subscriber error:&error];
    if (error)
    {
        [self showAlert:[error localizedDescription]];
    }
    _selfPublisherStreamId = stream.streamId;
}

As noted above, you must subscribe to video to receive captions. (This will be fixed in a future version.)

This example stores the publisher's stream ID in a _selfPublisherStreamId variable, used here to prevent its view from being added to the UI:

- (void)subscriberDidConnectToStream:(OTSubscriberKit*)subscriber
{
    if ([subscriber.stream.streamId isEqualToString:_selfPublisherStreamId]) {
        return; // Don't show the self subscriber view.
    }
    
    [_subscriber.view setFrame:CGRectMake(0, widgetHeight, widgetWidth,
                                          widgetHeight)];
    [self.view addSubview:_subscriber.view];
}

The subscriber has the transcribed text (for your published stream) in the following delegate:

- (void)subscriber:(nonnull OTSubscriberKit*)subscriber
           caption:(NSString*_Nonnull)text
           isFinal:(BOOL)isFinal {
    // Adjust UI to show captions.
    if ([subscriber.stream.streamId isEqualToString:_selfPublisherStreamId])
    {
        // text is your own audio as transcribed text or captions
        NSLog(@"My own caption %@", text);
    } else {
        // other subscribers captions
    }
}