Video streaming is predicted to account for a staggering 82 percent of the total internet traffic by 2022. From basic android mobile to 4K premium tv – viewers are watching content on devices with varied resolutions, compute & capability. The massive amount of data streaming presents challenges to broadcasters, content distribution networks, such as buffering issues, low resolution, poor quality, high operating cost, monetization, etc. AI/ML helps to overcome these challenges by providing the highest possible quality of experience (QoE) and quality of service (QoS).
AI/ML models can enable broadcasters and content providers to unearth opportunities to reduce streaming costs, enhance the viewer quality of experience, and improve viewer engagement using smart advertisement & video analytics by using AI/ML models throughout the media workflow.
This blog discusses the use of different AI/ML models in video streaming to enhance QoE and QoS.
Streaming Cost Optimization
YouTube and Netflix employ machine learning to optimize the encoding parameters dynamically. This not only increases user QoE and QoS but also reduces the number of bits required for the required quality. Encoding optimization using machine learning can also help optimize costs in the form of less bandwidth usage, storage costs, etc. Neural networks (NN) dynamically predict the video encode quantization levels (QL) that can produce target bitrate and achieve the performance of dual-pass encoding in a single pass. As a result, it will also reduce overall video latency and encoding costs.
An ML Engine can take in parameters for computing, network, storage costs, and user parameters (screen size, etc.), and provides optimal transcoding parameters. These optimal settings will help to reduce CDN operating costs.
The place for AI in video advertisements lies in its potential for delivering dynamic ads based on the geography, language, demographics, content being played, and the viewer’s interests. Video content producers can structure the creation of their videos in such a way that specific locations can be placeholders for advertisements. The AI technology can dynamically place advertisements into these placeholders based on a range of factors.
The use of AI ensures that content producers are no longer limited to the manual tags (metadata) provided. It can dynamically place advertisements with an even more personalized and localized approach. It also helps in ad breaks and avoids abrupt placement of ads between the scenes by locating a suitable spot.
Automatic Video Synthesis for Smart Advertisement
Style transfer of one image to another image is well understood in the AI space. Also, the same can extend to videos. In addition, Synthetic data can be created and embedded along with the broadcast content seamlessly by AI engines.
The synthesized video can for example generate custom textures, messages, logos, with an active scene without disrupting the flow. A new advertisement can replace the existing banner ad within the scene without any human involvement (Of course, this is doable where the content owner has rights to modify). Certainly, this can enable different in-scene ads for different regions.
Compliance to Regulatory Warnings
Compliance often requires the content provider to identify events/scenes in a video that may be restricted in specific territories. The restriction can be to remove the scene or to add statutory warnings.
AI engines identify such scenes and present “time-in” and “time-out” points to an editing system to perform edits. For example, the appearance of alcohol or tobacco in the UAE—or a show that uses a denigrating ethnic term in India—is a serious violation in those and other countries. Moreover, AI can quickly flag objectionable content and helps comply with regulatory requirements and prevent snafus.
The commercial compliance requirements (logo colors/dimensions) take quite a lot of manual checks and clean up. AI/ML engines can perform these tasks very effectively. All social media platforms are trying to employ AI to remove and prevent harmful content before it goes live on their platforms. Moreover, Image analysis techniques such as image classification and object detection are in use to categorize and detect specific images. Certainly Image segmentation puts those images into context, which is applicable to broadcast content as well to some degree.
We are just at the beginning of the applications of AI to video streaming. Identifying objects, recognizing faces, inserting customized advertisements, complying with regulatory warnings, generating subtitles at high speeds, transmitting the video intelligently at low bandwidth and lower costs are a few of the tasks that can be managed by AI engines very effectively. AI and deep neural network-based enhancements will dramatically improve user QoE and alter video consumption forever. AI solutions will soon become a widespread standard and will continue to redefine the streaming media ecosystem with emerging innovations.