HOP: Hands-On Projection

Design Team Members: Al Amir-Khalili, Kyle Morrison, Christopher Best
Supervisor: Alexander Wong

Download Detailed Report

Background

Many of us have been frustrated when try to perform some visual task on the computer, whether it be resizing some data table, moving an object in a CAD program, panning across a document, or selecting some text that you wish to copy. There are times when you wish you could reach into the computer and physically grab that object, and perform the manipulation manually. The aim of this project is to allow such physical manipulation to be integrated with computerized design.

Project Description

The proposed design draws upon image processing, physical markers and physical image projection to create an integrated physical computing system for the user. The markers will be detected by the camera and processed by the computing system. Based on an analysis of these markers, their meanings will be superimposed on top of them by the projector. The end result is that the user will have real-time physical input to the computerized system, allowing for fast and intuitive manipulation of objects.

It should be noted that the current system is application agnostic. It could be used for applications like board games, urban planning, military strategizing, interior design, or any of the host of applications that can benefit from the ability to physically manipulate a two-dimensional space.

Design Methodology

Research

The two areas of research were for determining the appropriate hardware for the project, and developing proper software systems to utilize this hardware. The first area dealt mostly with determining a suitable capture and projection system, and we eventually decided on the PlayStation Eye for the capture device and back projection to display the virtual design.

For the software, we delved into state of the art circle detection and bit extraction methodologies, but eventually decided that homespun algorithms developed specifically for this application would yield the best results. This is because the design of the marker and projection environment is also determined by us, and thus can be designed in such a way to make detection and recognition tasks easier.

Design and Analysis

A set of markers will be designed to act as machine-recognizable components of a physical scene. An image detection algorithm will be implemented in C++ using the OpenCV computer vision library to recognize these markers in real time. A physical prototype will be created using consumer electronics which allows input into a virtual application through manipulation of the physical markers. The system will project information into the physical scene, making the application fully interactive.

Testing

Different tests are to be done at each of the design stages. Such tests include testing the Worst Case Execution Time (WCET), maximum number of unique markers detected, maximum number of repeated markers detected, response to obstructions, etc. However, for the purpose of a holistic test, the design symposium serves as an ideal medium for obtaining user feedback on the responsiveness and intuitiveness of the finished integrated system.