| Abstract |
The vision-based tracking and the segmentation of an object of interest in an image sequence are two challenging, tightly coupled computer vision problems. By solving the segmentation problem, a solution to the tracking problem can be obtained, while tracking may provide important input to segmentation. The coupling between these two problems is an active research topic because, besides its theoretic interest, it may lead to robust solutions in a number of important applications including object localization and recognition, vision-based automated surveillance, activity recognition, human-computer/robot interaction, etc.
In this work we propose a new method for integrated tracking and segmentation of a single non-rigid object in a monocular video, captured by a possibly moving camera. It is assumed that a binary mask is available for the initial frame of an image sequence, fully or partially indicating the previously unseen object of interest that is to be segmented and tracked throughout that image sequence. A closed-loop interaction between Expectation Maximization (EM) color-based tracking and Random Walker-based image segmentation is proposed. The tracking algorithm represents the position and the area of the object in the form of an ellipse in each frame of the image sequence. At each frame, a finely segmented object mask is available from the segmentation performed at the previous frame. The spatial position and variance of the object mask are utilized to initialize the ellipse of the tracking algorithm for the current frame. Through EM iterations performed by the tracking method, a new ellipse is computed, estimating the new position and variance of the object in the current frame. The initial and the evolved ellipses are used to estimate a 2D affine transformation that propagates the segmented object shape of the previous frame to the current frame. A shape band is then defined indicating a region of uncertainty where the true object boundaries lie. In the following, pixel-wise spatial and color image cues are fused using Bayesian inference to guide object segmentation. A finely segmented object mask of the target object is finally computed in the current frame using the Random Walker-based segmentation methodology, closing the loop between tracking and segmentation.
The proposed method efficiently tracks and segments previously unseen objects requiring no off-line training or prior knowledge regarding the object of interest and its scene context. As confirmed by both the qualitative and quantitative experimental evaluation carried out on a variety of image sequences, the proposed methodology results in reduced tracking drifts and in fine object segmentation. Additionally, it operates effectively for previously unseen objects of varying appearance and shape that perform complex motions under varying illumination conditions.
|
Visual object tracking and segmentation in a closed loop
