Content Based Video Encoding Based on Spatial and TemporalInformation

Abstract: A significant amount of video content is produced today that needs to be stored, distributed, andstreamed globally in a cost-effective way. This relies on video compression, a process performedby encoding software that remove spatial redundancy, the similarities within a frame, andtemporal redundancy, the similarities between temporarily adjacent frames. In video encoding,there's always a trade-off between preserving video quality high while reducing file sizes throughcompression. This thesis focuses on Content-Based Encoding (CBE), an approach aiming tooptimize this trade-off by adjusting the encoder's parameters based on the content complexity ofthe video. Content complexity here refers to the amount of data and detail present in a videosequence that needs to be preserved during compression. This work explores the potential of twocritical features in video encoding namely the spatial and temporal information (SI-TI) fordescribing content complexity. The spatial perceptual information (SI) is based on the Sobel edgedetection algorithm and indicates the level of detail within a frame. On the other hand, temporalinformation (TI) is based on motion difference between consecutive frames and captures the rateof change within a video. Videos from three diverse datasets are encoded in various resolutionsand bitrate con- straints. Then, the video compressibility of each video is computed based on thequality degradation after encoding. The standard SI-TI features show low correlation withcompressibility. Statistical variations of SI-TI with stronger correlation are identified. Based onthese, a binary decision tree classifier is developed that accurately distinguishes simple andcomplex videos in terms of encoding complexity. The suggested approach can potentially reducevideo storage and distribution costs by encoding low complexity videos with higher compressionwithout sacrificing user experience.