Touchscreen Everywhere
On Transferring a Normal Planar Surface to a Touch-Sensitive Display


(Note: To view HD quality, please choose 720p)

Abstract

We address how an HCl (Human-Computer Interface) with small device size, large display, and touch input facility can be made possible by a mere projector and camera. The realization is through the use of a properly embedded structured light sensing scheme that enables a regular light-colored table surface to serve the dual roles of both a projection screen and a touch-sensitive display surface. A random binary pattern is employed to code structured light in pixel accuracy, which is embedded into the regular projection display in a way that the user perceives only regular display but not the structured pattern hidden in the display. With the projection display on the table surface being imaged by a camera, the observed image data, plus the known projection content, can work together to probe the 3D world immediately above the table surface, like deciding if there is a finger present and if the finger touches the table surface, and if so at what position on the table surface the finger tip makes the contact. All the decisions hinge upon a careful calibration of the projector-camera-table surface system, intelligent segmentation of the hand in the image data, and exploitation of the homography mapping existing between the projector’s display panel and the camera’s image plane.

More HD Videos

Finger Touch Detection (Gray Surface + Normal Ambient Light)
Finger Touch Detection (Gray Surface + Dark Ambient Light)
Finger Touch Detection (Yellow Surface + Normal Ambient Light)
Finger Touch Detection (Yellow Surface + Dark Ambient Light)
Finger Touch Detection (Artifact Surface + Normal Ambient Light)
Finger Touch Detection (Artifact Surface + Dark Ambient Light)

Finger Dragging Tracking
Multiple Touching

Reference

[1] J. Dai and R. Chung. Touchscreen Everywhere: On Transferring a Normal Planar Surface to a Touch-Sensitive Display. IEEE Transactions on System, Man and Cybernetics, Part B, In Press, 2013..
[2] J. Dai and R. Chung. Making Any Planar Surface into a Touch-sensitive Display by a Mere Projector and Camera. In Proc. of 25th IEEE Conference on Computer Vision and Pattern Recognition (CVPR'12) - Workshop (PROCAMS'12), pages 35-42, 2012.

Back to Projects Page