Suggestions

close search

Back to Screen Sharing Overview

Screen Sharing Step 1: Adding screen sharing

To see the code for this sample, switch to the screen-sharing branch. The code for this section is in the signaling branch of the learning-opentok-ios repo:

git checkout screen-sharing

This page shows the difference between this branch and the video-capturer-basic branch which it was built upon.

This branch shows you how to capture the screen (a UIView) using a custom video capturer.

Before studying this sample, see the video-capturer-basic sample.

This sample code demonstrates how to use the OpenTok iOS SDK to publish a screen-sharing video, using the device screen as the source for the stream's video. The sample uses the initCapture, releaseCapture, startCapture, stopCapture, and isCaptureStarted methods of the OTVideoKit class to manage capture functions of the application. The ViewController class creates a session, instantiates subscribers and sets up the publisher. The OTKBasicVideoCapturer class creates a frame, captures a screenshot, tags the frame with a timestamp and saves it in an instance of consumer. The publisher accesses the consumer to obtain the frame.

The initCapture method is used to initialize the capture and sets value for the pixel format of an OTVideoFrame object. In this example, it is set to ARGB.

- (void)initCapture
{
    self.format = [[OTVideoFormat alloc] init];
    self.format.pixelFormat = OTPixelFormatARGB;
}

The releaseCapture method clears the memory buffer:

- (void)releaseCapture
{
    self.format = nil;
}

The startCapture method creates a separate thread and calls the produceFrame method to start screen captures:

- (int32_t)startCapture
{
    self.captureStarted = YES;
    dispatch_after(kTimerInterval,
                   dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0),
                   ^{
                       @autoreleasepool {
                           [self produceFrame];
                       }
                   });

    return 0;
}

The produceFrame method:

The frame for the captured images is set as an object of OTVideoFrame. Properties of OTVideoFrame define the planes, timestamp, orientation and format of a frame.

OTVideoFrame *frame = [[OTVideoFrame alloc] initWithFormat:self.format];

A timestamp is created to tag the image. Every image is tagged with a timestamp so both publisher and subscriber are able to create the same timeline and reference the frames in the same order.

static mach_timebase_info_data_t time_info;
uint64_t time_stamp = 0;

time_stamp = mach_absolute_time();
time_stamp *= time_info.numer;
time_stamp /= time_info.denom;

The screenshot method is called to obtain an image of the screen.

CGImageRef screenshot = [[self screenshot] CGImage];

The fillPixelBufferFromCGImage method converts the image data of a CGImage into a CVPixelBuffer.

[self fillPixelBufferFromCGImage:screenshot];

The frame is tagged with a timestamp and capture rate in frames per second and delay between captures are set.

CMTime time = CMTimeMake(time_stamp, 1000);   
frame.timestamp = time;
frame.format.estimatedFramesPerSecond = kFramesPerSecond;
frame.format.estimatedCaptureDelay = 100;

The number of bytes in a single row is multiplied with the height of the image to obtain the size of the image. Note, the single element array and bytes per row are based on a 4-byte, single plane specification of an RGB image.

frame.format.imageWidth = CVPixelBufferGetWidth(pixelBuffer);
frame.format.imageHeight = CVPixelBufferGetHeight(pixelBuffer);
frame.format.bytesPerRow = [@[@(frame.format.imageWidth * 4)] mutableCopy];
frame.orientation = OTVideoOrientationUp;

CVPixelBufferLockBaseAddress(pixelBuffer, 0);
uint8_t *planes[1];

planes[0] = CVPixelBufferGetBaseAddress(pixelBuffer);
[frame setPlanesWithPointers:planes numPlanes:1];

The frame is saved in an instance of consumer. The publisher accesses captured images through the consumer instance.

[self.consumer consumeFrame:frame];

The pixel buffer is cleared and a background-priority queue (separate from the queue used by the UI) is used to capture images. If image capture is in progress, the produceFrame method calls itself 15 times per second.

 CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);   
 if (self.captureStarted) {
        dispatch_after(kTimerInterval,
                       dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0),
                       ^{
                           @autoreleasepool {
                               [self produceFrame];
                           }
                       });
    }

The screenshot method takes a screenshot and returns an image. This method is called by the produceFrame method.

- (UIImage *)screenshot
{
    CGSize imageSize = CGSizeZero;

    imageSize = [UIScreen mainScreen].bounds.size;

    UIGraphicsBeginImageContextWithOptions(imageSize, NO, 0);
    UIWindow *window = [UIApplication sharedApplication].keyWindow;

    if ([window respondsToSelector:
         @selector(drawViewHierarchyInRect:afterScreenUpdates:)])
    {
        [window drawViewHierarchyInRect:window.bounds afterScreenUpdates:NO];
    }
    else {
        [window.layer renderInContext:UIGraphicsGetCurrentContext()];
    }

    UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();
    return image;
}