SixthSense Stall By S6 IT Students…. In SAAGA ‘10

SixthSense is a wearable gestural interface device that augments the physical world with digital information and lets people use natural hand gestures to interact with that information. It was developed by Pranav Mistry, a PhD student in the Fluid Interfaces Group at the MIT Media Lab.

Components and principles:

The SixthSense prototype is composed of a pocket projector, a mirror, and a camera. The hardware components are coupled in a pendant-like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket, working such as :

      The projector: projects visual information, enabling surfaces, walls and physical objects around the wearer to be used as interfaces;

      The camera and hands: recognizes and tracks the user's hand gestures and physical objects using computer-vision based techniques.

      The software program: processes the video stream data took by the camera and tracks the locations of the colored markers (visual tracking of colored fingers) at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces.

The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction.

Interest:

The SixthSense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by multi-touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements. The drawing application lets the user draw on any surface by tracking the fingertip movements of the user’s index finger. SixthSense also recognizes user’s freehand gestures (postures). For example, it implements a gestural camera that takes photos of the scene the user is looking at by detecting the ‘framing’ gesture. The user can stop by any surface or wall and flick through the photos he/she has taken. SixthSense also lets the user draw icons or symbols in the air using the movement of the index finger and recognizes those symbols as interaction instructions. For example, drawing a magnifying glass symbol takes the user to the map application or drawing an ‘@’ symbol lets the user check his mail. The SixthSense system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. The gesture of drawing a circle on the user’s wrist projects an analog watch.

Cost and license:

The current prototype system costs approximately $350 to build. The software may be available for free on the model of open and editable freeware.

Implementation in our stall:

As a demo and to make people aware about the latest technology, we made an attempt in creating the device which stands nearest to the actual implementation by Pranav Mistry. Here we used a (i) projector, owned by our college itself, (ii) WII MOTE, a device used for playing computer game (iii) a Laptop and (iv) an IR emitting pen.

Working of our device:

As the real SixthSense device works based on sensing the hand gestures with the help of camera, our device works based on each and every pulse generated by the IR pen. The working of our device is as follows: As we connected the lap with the projector we can project the whole screen to the wall or to a screen. After projecting the screen, we have to connect the WII MOTE device to lap through Bluetooth. After that we must set ‘calibration’, the process of introducing the position of screen to WII MOTE.

After calibration, we can use the IR pen to touch the screen on wall to control the computer. Each pulse in the IR pen acts as each mouse click in computer and thus we can use the system using that pen only.

Each and every pulse in IR pen is detected by the WII MOTE device, and is sent to the computer through Bluetooth. Based on the program we set in C#, the current position of pen click is calculated and corresponding change takes place in the computer. Thus the whole device (or we can call it as a system) works.

We know it actually cannot compete with real SixthSense technology. But with our limitation, time and knowledge we tried our level best. We know, we are now explaining this to those who are having much more potential than us. But we know you all will encourage us. Thanking you all.

Me, Nisthul & Maneesha are thankful to all our friends who are with us in two days. We are thankful to those who help us in providing each and every useful part of this device. We are also thankful to our teachers for their encouragement and helpful mentality; also we are thankful to those who visited our stall to make the program a success….