Suggestions

close search

Back to Tutorials

Custom Video Capturing (Android)

Overview

This tutorial walks through the steps required to make modifications to the video capturer in your OpenTok Android application.

Step 1: Implementation

The code for this section is in the basic-capturer branch of the learning-opentok-android repo, so if you haven't already, you'll need to clone the repo into a local directory — this can be done using the command line:

git clone https://github.com/opentok/learning-opentok-android.git

Then check out the branch:

git checkout basic-capturer

Open the project in Android Studio to follow along. This branch shows you how to make minor modifications to the video capturer used by the Publisher class.

In this example, the app uses a custom video capturer to publish random pixels (white noise). This is done simply to illustrate the basic principals of setting up a custom video capturer. (For a more practical example, see the Camera Video Capturer and Screen Video Capturer examples, described in the sections that follow.)

In the initializePublisher() method ChatActivity class, after creating a Publisher object, the code calls the setCapturer(capturer) method of the Publisher object, passing in a NoiseVideoCapturer object as a parameter:

mPublisher.setCapturer(new NoiseVideoCapturer(320, 240));

NoiseVideoCapturer is a custom class that extends the BaseVideoCapturer class (defined in the OpenTok iOS SDK). This class lets you define a custom video capturer to be used by an OpenTok publisher.

The BaseVideoCapturer.init() method initializes capture settings to be used by the custom video capturer. In this sample's custom implementation of BaseVideoCapturer (NoiseVideoCapturer), the initCapture() method sets properties of a mCapturerSettings property:

@Override
public void init() {
    mCapturerHasStarted = false;
    mCapturerIsPaused = false;

    mCapturerSettings = new CaptureSettings();
    mCapturerSettings.height = mHeight;
    mCapturerSettings.width = mWidth;
    mCapturerSettings.format = BaseVideoCapturer.ARGB;
    mCapturerSettings.fps = FPS;
    mCapturerSettings.expectedDelay = 0;
}

The BaseVideoCapturer.CaptureSettings class (which defines the mCapturerSettings property) is defined by the OpenTok Android SDK. In this sample code, the format of the video capturer is set to use ARGB as the pixel format, with a specific number of frames per second, a specific height, and a specific width.

The BaseVideoCapturer startCapture() method is called when a publisher starts capturing video to be sent as a stream to the OpenTok session. This will occur after the Session.publish(publisher) method is called. In the NoiseVideoCapturer implementation of this method, the run() mFrameProducer thread is started:

@Override
public int startCapture() {
    mCapturerHasStarted = true;
    mFrameProducer.run();
    return 0;
}

The [self produceFrame] method creates a buffer of bytes and fills it with random noise. (Note that each frame is four bytes in the buffer, defining the constituent ARGB values for the frame.) It then passes the buffer into the provideByteArrayFrame(data, format, width, height, rotation, mirrorX) method, defined by the BaseVideoCapturer class (in the OpenTok Android SDK):

Runnable mFrameProducer = new Runnable() {
    @Override
    public void run() {
        Random random = new Random();
        byte[] buffer = new byte[mWidth * mHeight * 4];
        byte[] randoms = new byte[4];
        for (int i = 0; i < mWidth * mHeight * 4; i += 4) {
            random.nextBytes(randoms);
            buffer[i] = randoms[0];
            buffer[i + 1] = randoms[1];
            buffer[i + 2] = randoms[2];
            buffer[i + 3] = randoms[3];
        }

        provideByteArrayFrame(buffer, BaseVideoCapturer.ARGB,
                mWidth, mHeight, Surface.ROTATION_0, false);

        if (mCapturerHasStarted && !mCapturerIsPaused) {
            mFrameProducerHandler.postDelayed(mFrameProducer, mFrameProducerIntervalMillis);
        }
    }
};

This causes the publisher to send the frame of data to the video stream in the session. If the session is still publishing data, the mFrameProducer thread is run again after a specified delay (mFrameProducerIntervalMillis), causing another frame of video to be captured and published.

Step 2: Configuration

To see the code for this sample, switch to the camera-capturer branch of the learning-opentok-android repo:

git checkout camera-capturer

This branch shows you how to use a custom video capturer using the device camera as the video source.

Before studying this sample, see the basic-capturer sample.

In the initializePublisher() method of the ChatActivity class, after creating a Publisher object, the code calls the setCapturer(capturer) method of the Publisher object, passing in a CameraVideoCapturer object as a parameter:

mPublisher.setCapturer(new CameraVideoCapturer(this, 640, 480, 30));

CameraVideoCapturer is a custom class that extends the BaseVideoCapturer class (defined in the OpenTok iOS SDK). This class lets you define a custom video capturer to be used by an OpenTok publisher. The CameraVideoCapturer class uses a FrontCameraFrameProvider object to capture frames from the Android camera:

public CameraVideoCapturer(Context context, int width, int height, int fps) {
    mWidth = width;
    mHeight = height;
    mDesiredFps = fps;

    mHelper = new FrontCameraFrameProvider(context, mWidth, mHeight, mDesiredFps);
    mHelper.setHelperListener(this);
}

The FrontCameraFrameProvider class uses a FrontCameraFrameProvider.ProviderListener interface to send events when frames are available for capture. The CameraVideoCapturer class implements this interface and its cameraFrameReady() method.

The BaseVideoCapturer.init() method initializes capture settings to be used by the custom video capturer. In this sample's custom implementation of BaseVideoCapturer (CameraVideoCapturer) the initCapture() it also calls the init() method of the FrontCameraFrameProvider class:

@Override
public void init() {
    mCapturerHasStarted = false;
    mCapturerIsPaused = false;

    mCapturerSettings = new CaptureSettings();
    mCapturerSettings.height = mHeight;
    mCapturerSettings.width = mWidth;
    mCapturerSettings.expectedDelay = 0;
    mCapturerSettings.format = BaseVideoCapturer.NV21;

    mHelper.init();
}

The FrontCameraFrameProvider.init() method creates a Camera object.

The BaseVideoCapturer.startCapture() method of is called when a Publisher using the video capturer starts capturing video to send as a stream to the OpenTok session. This will occur after the Session.publish(publisher) method is called. The CameraVideoCapturer of this method calls the startCapture() method of the FrontCameraFrameProvider object:

@Override
public int startCapture() {
    mCapturerHasStarted = true;
    mHelper.startCapture();
    return 0;
}

The FrontCameraFrameProvider.startCapture() method sets some parameters of the Android Camera, based on parameters set at initialization. It creates a callback buffer for the camera (calling the Camera.addCallbackBuffer(buffer) method). It also sets itself as the implementer of the Camera.PreviewCallback interface, and it calls the startPreview() method of the Camera object:

mCamera.setPreviewCallbackWithBuffer(this);
mCamera.startPreview();

The FrontCameraFrameProvider implementation of the FrontCameraFrameProvider.onPreviewFrame(data, camera) method. This method is called when preview frames are displayed by the camera. The FrontCameraFrameProvider calls the cameraFrameReady() method of the ProviderListener object, which is the CameraVideoCapturer object in this app:

@Override
public void onPreviewFrame(byte[] data, Camera camera) {
    mPreviewBufferLock.lock();
    if (data.length == mExpectedFrameSize) {
        // Get the rotation of the camera
        int currentRotation = compensateCameraRotation(mCurrentDisplay
                .getRotation());

        if (mListener != null) {
            mListener.cameraFrameReady(data, mCaptureActualWidth, mCaptureActualHeight,
                    currentRotation, isFrontCamera());
        }

        // Reuse the video buffer
        camera.addCallbackBuffer(data);
    }
    mPreviewBufferLock.unlock();
}

The CameraVideoCapturer implementation of the cameraFrameReady() method (defined by the Camera.PreviewCallback interface) calls the provideByteArrayFrame() method, defined by the BaseVideoCapturer class (which the CameraVideoCapturer class extends):

@Override
public void cameraFrameReady(byte[] data, int width, int height, int rotation, boolean mirror) {
    provideByteArrayFrame(data, NV21, width,
            height, rotation, mirror);
}

This sends a byte array of data to the publisher, to be used for the next video frame published.

Note that the CameraVideoCapturer also includes code for determining which camera is being used, working with the camera orientation and rotation, and determining the image size.

Congratulations! You've finished the Custom Video Capturer Tutorial for Android.
You can continue to play with and adjust the code you've developed here, or check out the Next Steps below.