Accurate Dynamic Sketching of Faces from Video

Authors:
Zijian, Xu
Jiebo, Luo

Abstract:
A sketch captures the most informative part of an object, in a much more concise and potentially robust representation (e.g., for face recognition or new capabilities of manipulating faces). We have previously developed a framework for generating face sketches from still images. A more interesting question is can we generate an animated sketch from video? We adopt the same hierarchical compositional graph model originally developed for still images for face representation, where each graph node corresponds to a multimodal model of a certain facial feature (e.g., close mouth, open mouth, and wide-open mouth). To enforce temporal-spatial consistency and improve tracking efficiency, we constrain the transition of a graph node to be only between immediate neighboring modes (e.g. from closed mouth to open mouth but not to wide-open mouth), as well as by its corresponding parts in the neighboring frames. To improve the matching accuracy, we model the local structure of a given mode as a shape-constrained Markov network (SCMN) of image patches. The preliminary results show accurate sketching results from video clips.

Summary:
The method similar to the face to sketch recognition, it uses 3 layers - face --> parts --> sketch layers. AS it moves from one layer to the other, the algorithm identifies the miniscule details. First, moving to the parts layer, the face is fragmented into its parts (eyes, mouth,...) and the orientation, length, position and other features are found. Then moving on to the next layer, the line segments, interesections,... are identified and the information from the parts layer are then embedded on them.