Responsive Environments
The Responsive Environments group explores how sensor networks augment and mediate human experience, interaction and perception, while developing new sensing modalities and enabling technologies that create new forms of interactive experience and expression. Our current research encompasses the development and application of various types of sensor networks, energy harvesting and power management, and the technical foundation of ubiquitous computing. Our work is highlighted in diverse application areas, which have included automotive systems, smart highways, medical instrumentation, RFID, wearable computing, and interactive media.
Research Projects
Configurable Dynamic Privacy for Pervasive Sensor Networks
Joe Paradiso and Nan-Wei GongWe have built a configurable infrastructure to protect users' dynamic levels of privacy in a pervasive sensor network. Our system is based around a badge that can alert the user to the presence of participating sensor networks, plus emit an RF beacon with which the network can gauge the level of privacy desired. Badges either periodically emit an "opt out" signal, blocking sensing within their RF (and sensor perceptual) range, or allow the users' desired level of privacy to be preconfigured online. This privacy level depends on user location, and can eliminate or "blur" the data and calculated features available from various sensors and sensor nodes. The privacy level can also be dependent on the status of the client browsing the sensor network—the badge user can assign different levels of privacy to different groups of people.
DoppelLab
Joe Paradiso, Gershon Dublon, Nan-Wei Gong, Mathew Laibowitz, Alexander Reben, David Small and Laurel Smith PardueWe are populating a detailed 3-D graphic model of the Media Lab complex (buildings E15 and E14) created by the Design Ecology group with graphics and audio driven by sensor information derived from our Ubiquitous Sensor Portals system and other embedded sensor/actuator infrastructures around our Lab facilities. The goal of this work is to explore fluid ways of visualizing information derived from a dense sensor network in a cross-reality implementation that will always be running, so remote participants can browse and interact with our physical facility from anywhere in the world.
Funk2: Causal Reflective Programming
Marvin Minsky, Joe Paradiso and Bo MorganFunk2 is a novel process-description language that keeps track of everything that it does. Remembering these causal execution traces allows parallel threads to reflect, recognize, and react to the history and status of other threads. Novel forms of complex, adaptive, nonlinear control algorithms can be written in the Funk2 programming language. Currently, Funk2 is implemented to take advantage of distributed grid processors consisting of a heterogeneous network of computers, so that hundreds of thousands of parallel threads can be run concurrently, each using many gigabytes of memory. Funk2 is inspired by Marvin Minsky's Critic-Selector theory of human cognitive reflection.
Interaction with Ubiquitous Dynamically Responsive Media
Joe Paradiso, Nan-Wei Gong, Mathew Laibowitz and Alexander RebenThis project takes advantage of our group's Ubiquitous Media Portals platform, which enables a large suite of research around the broad theme of what we call Dynamic Ubiquitous Media. This will include relevant, personalized information delivered ambiently to Lab visitors, with intuitive non-contact gestural input for interacting with this information. This project will build a framework for implementing dynamic media, and demonstrate it running throughout our building through a variety of applications.
Lab-Wide and Wearable Sensor and Video Network
Joe Paradiso and Mathew LaibowitzThis is a suite of devices and protocols to support applications in wearable human/social sensing linked to a distributed camera and vision system. The current system includes a sensate wristwatch with biological and gestural sensors, and a lapel-pin device with motion and audio-affect sensing. These all communicate with wall-mounted devices (Portals), each of which has a high-resolution camera, environmental sensors, and a localization system for all devices in the network. All devices record data and audio in sync with the recorded video. A full-spec Zigbee network supports device synchronization and mesh networking. All devices have enough on-board power to extract features from the data.
Moral Compass: A Model of Self-Conscious Learning
Ed Boyden, Henry Lieberman, Marvin Minsky, Joe Paradiso and Bo MorganMoral Compass is a model of how children learn in a problem-solving environment where the child is learning to accomplish goals in the context of parents, strangers, and cultural knowledge. The child learns in multiple ways: playing alone, being told stories, and being rewarded or punished. Our model aims to provide an explanation for relatively complex reflective states of mind, such as desire, avoidance, focus, Ignorance, and personality traits. Our model also emphasizes different types of failure in its reflective approach to learning, including: surprise, disappointment, and guilt. Possible applications include better understanding of the mental health of cognition in social domains.
Sensor-Enabled Active Buildings
Joe Paradiso and Mark FeldmeierThis project explores the wide-scale distribution of low-power, low-cost sensor nodes that can measure temperature, humidity, light levels, and human presence. These sensor nodes will enable buildings to react quickly and effectively to the changing needs of their inhabitants, automatically controlling, for example, heating/air conditioning, windows (opening and shades), and lighting. Total building power consumption can be reduced, and repair requests can be made automatically.
Spinner
Joe Paradiso and Mathew LaibowitzSpinner is a Lab-wide sensor network platform designed to detect and capture fragmented events of human behavior that can be collected and sequenced into a cohesive narrative conveying a larger overall meaning. This project also looks at the development of parametric models of narrative that can be mapped on to sensor-detectable elements of human activity.
Ubiquitous Media Portals
Joe Paradiso, Nan-Wei Gong and Mathew LaibowitzWe have turned our building into a living laboratory by distributing 45 Ubiquitous Media Portals throughout our facility. These sensor-rich platforms capture video and stereo audio in addition to measuring nearby motion, temperature/humidity, light levels, and IR detection of active badges. The portals also work as Zigbee base stations, enabling communication and localization of various low-power, wireless, wearable devices that our group and others have developed for on-body sensing and identification/tracking of people. They also feature a small touch-screen display and speaker, allowing them to render and present dynamic graphics and information.
Wearable, Wireless Sensor System for Sports Medicine and Interactive Media
Joe Paradiso, Michael Thomas Lapinski, Dr. Eric Berkson and MGH Sports MedicineThis project is a system of compact, wearable, wireless sensor nodes, equipped with full six-degree-of-freedom inertial measurement units and node-to-node capacitive proximity sensing. A high-bandwidth, channel-shared RF protocol has been developed to acquire data from many (e.g., 25) of these sensors at 100 Hz full-state update rates, and software is being developed to fuse this data into a compact set of descriptive parameters in real time. A base station and central computer clock the network and process received data. We aim to capture and analyze the physical movements of multiple people in real time, using unobtrusive sensors worn on the body. Applications abound in biomotion analysis, sports medicine, health monitoring, interactive exercise, immersive gaming, and interactive dance ensemble performance.