MIThril project overview

The MIThril hardware platform combines body-worn computation, sensing, and networking in a clothing-integrated design. The MIThril software platform is a combination of user interface elements and machine learning tools built on the Linux operating system.

This document provides a brief overview of some of the major components of the MIThril system; It is necessarily incomplete and somewhat out-of-date. If you have questions that are not answered by this document or the MIThril FAQ, contact the MIThril team at MIThril@media.mit.edu.

MIThril hardware may be broken down into three classes of components: computing cores, networking and body bus, and sensors and peripherals. MIThril software may be broken down four categories: OS, HCI components, machine learning/classification tools, and prototype applications.

The Linux cores provide substantial on-body computation, signal processing/classification, and networking resources to the user. They communicate with each other and the outside world via on-body 10 Mbit Ethernet. The wireless bridge provides a high-bandwidth connection to off-body resources. The SAK board provides a self-contained, inexpensive light-weight data acquisition system that is compatible with the MIThril body bus.

• MIThril Body Network
  The MIThril Body Network is the Ethernet/power network which connects the MIThril cores, providing peer-to-peer TCP/IP networking and power. The current design employs standard 4-twisted pair Category 5 Ethernet cable and a modified Ethernet hub to distribute both power and data.

• MIThril Body Bus
  The MIThril body bus is a branching single-cable power/data network design that provides regulated 5V and unregulated 12V power, USB and I²C through a 10 wire cable. The USB and I²C protocols were selected to provide maximal compatibility with current COTS (cheap, off-the-shelf) USB devices like cameras and microphones, and hackability, since it is relatively easy to build simple microcontroller-based I²C sensors and peripherals. The connectors and cables were chosen to be mechanically and electrically stable, robust, and locking. All components are flat and small, making it easy to integrate the MIThril body bus with existing clothing or new designs.

  MIThril body bus junctions are small, flat hubs that act as 3-way branches for the bus and tie the bus to the physical garment substrate. At present the hubs are passive and do not support USB directly, although USB devices may be plugged into the BSEV core itself. We have a nearly complete design for a USB-enabled body-bus junction, but it is not yet tested.

Working MIThril body-bus sensors include:

• High-precision three-axis accelerometer (custom I²C device).
• IR Active tag reader (custom I²C device).
• Integrated accelerometer/IR active tag reader/microphone/headphone audio board for SAK core.
• USB microphone and headphone driver.
• USB CCD camera
.

The USB and I²C protocols span a wide range of useful sensors and peripherals, but the 12Mbit bandwidth constraint of USB and 400Kbit constraint of I²C do impose some limitations on the type and number of devices which may share this bus. Likewise, debugging console connections should not be routed over the body bus for practical reasons. Thus, it makes sense to connect some devices directly to a MIThril core. These devices include:

• A re-cabled Twiddler One chording keyboard and modified Palm folding keyboard, which are hot-swappable through an exterior plug and connect to the same serial port on the Brightstar.
• The debugging console serial ports on the Brightstar and CerfBoard.
• Earthmate GPS receiver (serial)
• the MicroOptical Clip-on, which is based on the Kopin CyberDisplay 320 Color chip, is one such peripheral. The Kopin display signal is produced through a combination of the MPC823's on-board LCD driver, timing code for the Brightstar's FPGA, and some additional D to A and analog circuitry on a companion board which plugs into the Brightstar IPEngine.

The current FPGA core for the BSEV core generates timing signals for the HMD and provides great flexibility and control over the display. For instance, a combination of the FPGA core and Linux driver allows us to switch from "standard" field-sequential color to high-brightness 180Hz gray scale to low-power monochrome (red "gray scale") display modes on the fly.

We have functioning Linux distributions for both of our current Linux computing cores (Debian on the StrongARM, a Hard Hat Linux derivative on the Brightstar).

We have functioning Linux I²C and USB master drivers for the MPC823, which we will publish in our CVS repository shortly. We also have a field-sequential-color frame-buffer driver for the BSEV core, allowing us to use 24-bit color and X-windows.

We have made substantial progress in implementing the Enchantment user interface, as described in the Enchantment Whitepaper. This includes the Anduin window manager, which provides a mouseless interface to X applications (Anduin is in our MIThril Code CVS repository but not documented elsewhere yet) and the Enchantment IPC whiteboard context server (also in the CVS repository, but not yet documented elsewhere).

The MIThril Inference Engine provides a set of tools for applying statistical machine learning techniques to classifying sensor data in real-time. The inference engine works with the Enchantment IPC whiteboard to provide real-time classification and regression to context-aware applications. (This code can also be found in the CVS repository). We have successfully used the MIThril Inference Engine to implement a sophisticated activity classification system based on accelerometer data; see the Motion Classification Project Paper (gzip compressed postscript) for more details.

At present we are working on three major applications; the Memory Glasses project, the Context-Aware Cell Phone Project and a biomedical patient functional assessment project utilizing the SAK platform (a collaboration with researchers at the Rochester Center for Future Health).