BlueJeans is being sunset. Please refer to more details

Custom Video Source

The Custom Video Source feature allows developers to integrate a variety of video frames, including pre-recorded clips, 3D renders, or a blend of both, as a video input, offering endless possibilities for video customization. For example, the Custom Video Source feature provides the capability to alter the plain/vanilla (default/raw) video stream from the device's front or rear camera to include any effects on top of the video stream and send it back to the BlueJeans platform. The effects may include visual enhancements, such as placing stickers on top of someone's face, introducing AR objects into the video frame, or displaying images or GIFs of products during a live video call.

Using the BlueJeans Custom Video Source feature, a retail store can enhance its virtual customer experience by showcasing its latest Android mobiles as Augmented Reality (AR) objects in a live video call. Instead of the default camera feed, a sales representative can replace it with a customized video feed that allows for the use of virtual objects. This feature enables participants to engage in an immersive and interactive meeting experience, where they can manipulate virtual objects and interact with the sales representative to learn about the features of the Android mobiles.

Pre-requisites

The supported aspect ratios are 4:3, 3:4, 16:9 and 9:16

Steps for Building Custom Video Source

Here are the steps for creating a custom video source using the Android Client SDK:

Step 1

Set your video source to "Custom" for initiating the custom video source flow.

setVideoSource(videoSource: VideoSource)

When you call video source with "Custom" that means you are shutting off the camera.

Note
By default, the video source is set as "Camera", which means you will see a video feed from your default front or back camera.

Step 2

The pushCustomVideoFrame()  API is a function that allows you to send your custom video feed one frame at a time. In order to use the pushCustomVideoFrame() API, you must first set the video source to Custom by calling the setVideoSource API. The pushCustomVideoFrame() API takes the BJNVideoFrame object as an argument.

class BJNVideoFrame {
    frameData: CustomFrameData
    frameOrientation: FrameOrientation
    frameFormat: FrameFormat
    isMirrored: Boolean
}
  • CustomFrameData is an object that contains raw pixels as well as some other frame-related metadata such as frame width and height.  
    class CustomFrameData {
                        buffer: ByteBuffer, yStride: Int, uBuffer: ByteBuffer? = null, 
                        val vBuffer: ByteBuffer? = null, uvRowStride: Int = 0, 
                        uvPixelStride: Int = 0, width: Int, height: Int 
                       } 

    Irrespective of the frame format chosen, a CustomFrameData object should always contain the raw pixels in buffer field and should also specify the width and height of the frame in width and height fields of CustomFrameData object. 

    Note

    If the frame format is set to NV21, you must provide "U and V buffers" along with their corresponding row and pixel strides.

  • To display the custom video frame in your desired orientation, you must specify its orientation. The custom video frame can be displayed in any of the following orientations. 
    ORIENTATION_0,
ORIENTATION_90,
ORIENTATION_180  
ORIENTATION_270
    • ORIENTATION_0 sends a custom video frame as it is without applying any rotations.
    • ORIENTATION_90 rotates the custom video frame by 90 degrees.
    • ORIENTATION_180 rotates the custom video frame by 180 degrees.
    • ORIENTATION_270 rotates the custom video frame by 270 degrees.
  • The following is a list of the frame formats that are supported for custom video
    • ARGB,
    • I420,
    • RGB565
    • NV21

Step 3

You are now ready to send your custom video frame-by-frame using the following API with 'BJNVideoFrame' as its argument. As this API can only take one frame at a time, it must be called every time a custom video frame is available for transmission.

pushCustomVideoFrame(frame: BJNVideoFrame) : PushVideoFrameResult

The results of the pushCustomVideoFrame() API are below:

Response Codes

Description
Success This means that the frame has been sent to all remote participants successfully.
IncorrectFrameFormat The error message signifies that the format of the attached frame does not match the supported frame formats.
NotUsingCustomVideoSource You will get this error when you call pushCustomVideoFrame() directly without setting setVideoSource to "Custom".
VideoMuted You are sending custom video frames while your video is on mute.
UnsupportedAspectRatio This error will be triggered when the aspect ratio of the custom video frame fall outside the aspect ratios of 4:3, 3:4, 16:9, and 9:16.

Congratulation! You are now ready to display your custom video in the meeting.

Use Cases of Custom Video Source

Retail Store

The custom video source feature enables retailers to offer an immersive and interactive shopping experience for their customers, which can increase sales and differentiate them from competitors. An example of this is selling headsets through a virtual store where customers can see and experience the product in 3D.

Remote Learning

You can use the Custom Video Source feature to enhance the learning experience by bringing in visual aids such as diagrams, charts, and animations that can be manipulated by all participants/students. This feature can help students better understand complex concepts, leading to more engagement and participation in the virtual class.

For example, a teacher is using a 3D representation of the sun and earth to show students how the earth revolves around the sun, and how its own rotations are related. By using this feature, teachers can create a more immersive and engaging learning experience for their students in a virtual class. Other potential use cases for the feature in remote learning include virtual field trips and interactive history lessons.

AR object

Custom video source enables you to utilize AR objects and visual improvements in the live video stream. For instance, you can enhance the video's appeal by placing glasses or applying various AR filters near your face, as depicted in the image below. Additionally, you have the option to insert AR objects, like virtual products or brand logos, into the video frame to advertise your sponsorships or merchandise.

Sample App

Within our sample application, we have incorporated AR objects as examples, which feature virtual stickers such as fox ears and a nose, adding a playful element to your videos. For more information on how this implementation works, refer to our sample app.

Additionally, you could use the custom video source to display images, or GIFs during the live video call, such as behind-the-scenes footage or funny memes, to keep your audience engaged and entertained.