Figure 1. A sample hand image captured by projector-camera system.

In this work, we introduce a coarse-to-fine approach to solve the aforementioned problems. The main idea of the method is to combine contrast saliency map with mean-shift based smoothing and segmentation by a confidence function. Low-level contrast saliency detection enables the hand region to be highlighted roughly, and mean-shift based smoothing method removes the noises induced by projection contents without demolishing discontinuity information. Moreover, without any pre-training and pre-calibration procedures, the robust, precise and also rapid hand segmentation can be derived.

The approaches for hand segmentation have been studied extensively in computer vision society. Among them, skin color detection is very common for its simpleness and easy implementation, and is very efficient against simple background or in the scene of hand being the only skin-colored object. However, diverse video contents are projected continuously in projector-camera scenario, when some skin-colored objects are projected on the background (Region A in Fig. 1) or non-skin-colored objects are projected on the hand (Region B in Fig. 1), the skin color based methods will be influenced severely. Since the geometrically and radiometrically calibrated projector-camera system can predict where the video contents are projected and how they should appear in the image data, background subtraction is adopted to segment the hand as the set of pixels that are out of expectation on the projection surface, but suffers from separating hand region from the hand-cast shadows (Region C in Fig. 1), let alone calibration procedures and constraints of constant ambient illuminations and fixed projection surface.

Visual comparison: (a) original image; (b) ground-truth; (c) our method; (d) SCM; (e) BkSub; (f) GB. The yellow (top-left) and green (top-right) numbers in each result image are the corresponding precision p and recall r values, respectively.

(To view the high-resolution version, please click this image)

[1] J. Dai and R. Chung. Combining Contrast Saliency and Region Discontinuity for Precise Hand Segmentation in Projector-Camera System. In Proc. of The 21st International Conference on Pattern Recognition, pages 2161-2164, 2012.