SixthSense: A Wearable, Gestural Interface to Augment Our World
Turning any surface into a touch-screen display
SixthSense is a wearable, gestural interface that augments our physical world with digital information, and lets us use natural hand gestures to interact with that information.
SixthSense brings intangible, digital information into the tangible world, and allows us to interact with this information via natural hand gestures. SixthSense frees information from its confines, seamlessly integrating it with reality, thus making the entire world your computer.
The SixthSense prototype comprises a pocket projector, mirror, and camera worn in a pendant-like mobile device. Both the projector and the camera are connected to a mobile computing device in the user’s pocket. The system projects information onto the surfaces and physical objects around us, making any surface into a digital interface; the camera recognizes and tracks both the user's hand gestures and physical objects using computer-vision-based techniques. SixthSense uses simple computer-vision techniques to process the video-stream data captured by the camera and follows the locations of colored markers on the user’s fingertips (which are used for visual tracking). In addition, the software interprets the data into gestures to use for interacting with the projected application interfaces.
The current SixthSense prototype supports several types of gesture-based interactions, demonstrating the usefulness, viability, and flexibility of the system. The current prototype system costs approximately $350 to build.
TOFU is a project to explore new ways of robotic social expression by leveraging techniques that have been used in 2d animation for decades. Disney Animation Studios pioneered animation tools such as "squash and stretch" and "secondary motion" in the 50's. Such techniques have since been used widely by animators, but are not commonly used to design robots. TOFU, who is named after the squashing and stretching food product, can also squash and stretch. Clever use of compliant materials and elastic coupling, provide an actuation method that is vibrant yet robust. Instead of using eyes actuated by motors, TOFU uses inexpensive OLED displays, which offer highly dynamic and lifelike motion.
Can traditional values be embedded into a digital object?
In this research we implement a special guitar that combines physical acoustic properties with virtual capabilities. A wooden resonator - a unique, replaceable piece of wood that gives the guitar a unique acoustic sound, will embody the acoustical values. The acoustic signal created by this wooden heart will be digitally processed in a virtual sound box in order to create flexible sound design.
Today’s tools and instruments, whether musical or graphical, fall into two very distinct classes, each with its own benefits and drawbacks. Traditional physical instruments offer a richness and uniqueness of qualities that result from the unique properties of the physical materials used to make them. The hand crafted, construction qualities are also very important for those tools. In contrast electronic and computer based instruments lack this richness and uniqueness; they produce very predictable and generic results, but offer the advantage of flexibility: they can be many instruments in one. We propose a new approach to designing and building instruments, which attempts to combine the best of both. The approach is characterized by a sampling of the instrument's physical matter and its properties and complemented by a physically simulated, virtual shape. This approach to building digital objects maintains some of the rich qualities and variation found in real instruments (the result of natural materials combined with craft) with the flexibility and open-endedness of digital ones.
