Video Motion Stylization by 2D Rigidification

Our method takes as input a video and its optical flow (a). We segment the video and optimize its pixel trajectories to produce a new video that exhibits piecewise-rigid motion (b). The resulting rigidified video can be stylized with existing algorithms (c) to produce animations where the style elements (brush strokes, paper texture) produce a strong sense of 2D motion.

Abstract

This paper introduces a video stylization method that increases the apparent rigidity of motion. Existing stylization methods often retain the 3D motion of the original video, making the result look like a 3D scene covered in paint rather than a 2D painting of a scene. In contrast, traditional hand-drawn animations often exhibit simplified in-plane motion, such as in the case of cutout animations where the animator moves pieces of paper from frame to frame. Inspired by this technique, we propose to modify a video such that its content undergoes 2D rigid transforms. To achieve this goal, our approach applies motion segmentation and optimization to best approximate the input optical flow with piecewise-rigid transforms, and re-renders the video such that its content follows the simplified motion. The output of our method is a new video and its optical flow, which can be fed to any existing video stylization algorithm.

Video

BibTex

@inproceedings{delanoy2019video,
	title={Video motion stylization by 2D rigidification},
	author={Delanoy, Johanna and Bousseau, Adrien and Hertzmann, Aaron},
	booktitle = {{Expressive 2019 - 8th ACM/ Eurographics Proceedings of the Symposium}},
	year={2019},
	publisher = {The Eurographics Association},
	doi = {https://doi.org/10.2312/exp.20191072},
	url = {https://diglib.eg.org/handle/10.2312/exp20191072}
}