Abstract:
Abstract—Artists use different means of stylization to control the
focus on different objects in the scene. This allows them to portray complex meaning and achieve certain artistic effects. Most
prior work on painterly rendering of videos, however, uses only
a single painting style, with fixed global parameters, irrespective of objects and their layout in the images. This often leads to inadequate artistic control. Moreover, brush stroke orientation is typically assumed to follow an everywhere continuous directional field. In this paper, we propose a video painting system that accounts for the spatial support of objects in the images or video, and uses this information to specify style parameters and stroke orientation for painterly rendering. Since objects occupy distinct image locations and move relatively smoothly from one video frame to another, our object-based painterly rendering approach is characterized by style parameters that coherently vary in space and time. Spatiotemporal coherence of varying style parameters enables more artistic freedom, such as emphasis/deemphasis, increase or decrease of contrast, exaggeration or trivialization of different objects in the scene in a temporally coherent fashion.
Given a video that has been segmented into temporally moving
or deforming objects as well as the background, the user can
specify style parameters, such as stroke color, size, and opacity
for each target object in some keyframes. Due to spatiotemporal
coherence of the video object segmentation, this information can
be propagated to the target object in all other frames. The user
may easily reselect new objects and their style parameters in any
frame, which will be then automatically propagated to the entire
video.
Similarly, brush stroke orientations can be specified per object,
and thus can be spatially discontinuous. In addition, brush
stroke orientations can be also be specified for a target object
in some keyframes and propagated to other frames. Unlike
style parameters, brush stroke orientations are transported by
taking into account the underlying object’s movement such as
translation and rotation.
To generate the painterly processed video, we have developed
a novel image-based renderer that supports object-based style parameters and orientation fields. Our renderer is inspired by flow visualization techniques. Given a frame, it first places the
seeds of brush strokes such as solid disks on the canvas. The
canvas is then advected according to the brush stroke orientation
field. By iteratively blending the advected image with the original canvas of stroke seeds, we can interactively obtain a painted image in which contrast between neighboring strokes is less obvious than traditional renderers in which each curved strokes are constructed explicitly. This helps alleviate the flickering effect often associated with video painting.
The utility of our approach is demonstrated through a number
of artistic operations on images and videos resulting in their
high-quality, multi-style rendering.
