Abstract

In recent years, with the rapid development of digital technology, the public can capture and produce high-quality visual media conten. However, the current visual media processing techniques cannot satisfy the increasing demands of automatically organize, understand and enhance the media data according to the human intentions and potentially billions of personalized editing. So we focus on how to analysis the images and videos in a perception-oriented way. Personal videos often contain visual distractors, i.e. objects that are accidentally captured that can distract viewers from focusing on the main subjects. We propose methods to automatically detect and localize these distractors. A rule-based method is firstly proposed to detect common distractors. Then a learning based framework is proposed with a manually-labeled dataset. To achieve spatially- and temporally-coherent detection, we propose to extract features at the Temporal-Superpixel (TSP) level in a traditional SVM-based learning framework. We have also experimented with end-to-end learning with Convolutional Neural Networks (CNNs), which achieve slightly higher performance. The classification result is further refined in a post-processing step based on graph-cut optimization. Experimental results show that our method achieves an accuracy of 81% and a recall of 86%. We demonstrate several ways to remove the detected distractors to improve the video quality, including video hole filling; video frame replacement; and camera path re-planning.