Eye exams via a slit lamp are critical in early diagnosis of diseases such as cataracts, corneal injury, and pterygia, in order to avert vision loss. The slit lamp is one of the most versatile tools in an ophthalmologist's clinic, but is big, expensive, and is designed with specialized ophthalmic clinics in mind. AnEye is a suite of portable, computationally driven solutions that leverage modern optics and commercially available consumer electronics to extend the reach of examinations of the anterior segment of the eye well beyond large hospitals and clinics, into resource-constrained settings such as rural mass-screening camps, mobile ophthalmology clinics, and even primary care.

Fluorescence lifetime imaging is a significant bio-imaging tool that finds important applications in life-sciences. Widely known applications include cancer detection and DNA sequencing. To that end, fluorescence microscopy which is at the heart of bio-imaging is an electronically and optically sophisticated device which is prohibitively expensive. Our work is demonstrates the fluorescence microscopy like functionality can be achieved by a simple, consumer sensor such as the Microsoft Kinect which costs about $100. This is done by trading-off the precision in optics and electronics for sophistication in computational methods. Not only this allows for massive cost reduction but leads to several advances in the area. For example, our method is calibration-free in that we do not assume sample's relative placement with respect to the sensor. Furthermore, our work opens new pathways of interaction between bio-imaging, optics and computer vision communities.

We introduce a novel interactive method to assess cataracts in the human eye by crafting an optical solution that measures the perceptual impact of forward scattering on the foveal region. Current solutions rely on highly trained clinicians to check the back scattering in the crystallin lens and test their predictions on visual acuity tests. Close-range parallax barriers create collimated beams of light to scan through sub-apertures, scattering light as it strikes a cataract. User feedback generates maps for opacity, attenuation, contrast, and local point-spread functions. The goal is to allow a general audience to operate a portable, high-contrast, light-field display to gain a meaningful understanding of their own visual conditions. The compiled maps are used to reconstruct the cataract-affected view of an individual, offering a unique approach for capturing information for screening, diagnostic, and clinical analysis.

We present coded focal stack photography as a computational photography paradigm that combines a focal sweep and a coded sensor readout with novel computational algorithms. We demonstrate various applications of coded focal stacks, including photography with programmable non-planar focal surfaces and multiplexed focal stack acquisition. By leveraging sparse coding techniques, coded focal stacks can also be used to recover a full-resolution depth and all-in-focus (AIF) image from a single photograph. Coded focal stack photography is a significant step towards a computational camera architecture that facilitates high-resolution post-capture refocusing, flexible depth of field, and 3D imaging.

Millions of people worldwide need glasses or contact lenses to see or read properly. We introduce a computational display technology that predistorts the presented content for an observer, so that the target image is perceived without the need for eyewear. We demonstrate a low-cost prototype that can correct myopia, hyperopia, astigmatism, and even higher-order aberrations that are difficult to correct with glasses.

We believe that tough global health problems require an innovation pipeline. We must bring together the people and providers facing health challenges to form what we call an innovation continuum: inventors building new low-cost technologies; developers capable of rapidly iterating on these inventions for use in the real world; clinicians and end users to validate our creations; and entrepreneurs, philanthropists, and development agencies to scale our solutions. We are asking big questions such as: What billion-dollar ideas could impact a billion lives in health, education, transportation through digital interfaces, digital opportunities, and applications for physical systems? Using machine learning, computer vision, Big Data, sensors, mobile technology, diagnostics, and crowdsourcing, we are conducting research at the Media Lab, and also collaborating with innovators in three centers in India and in other centers worldwide. Innovations like this launched the effort to create the Emerging Worlds initiative.

The use of fluorescent probes and the recovery of their lifetimes allow for significant advances in many imaging systems, in particular medical imaging systems. Here, we propose and experimentally demonstrate reconstructing the locations and lifetimes of fluorescent markers hidden behind a turbid layer. This opens the door to various applications for non-invasive diagnosis, analysis, flowmetry, and inspection. The method is based on a time-resolved measurement which captures information about both fluorescence lifetime and spatial position of the probes. To reconstruct the scene, the method relies on a sparse optimization framework to invert time-resolved measurements. This wide-angle technique does not rely on coherence, and does not require the probes to be directly in line of sight of the camera, making it potentially suitable for long-range imaging.

We demonstrate a new method to image through scattering materials like tissue and fog. The demonstration includes imaging an object hidden behind 1.5cm of tissue; it's like imaging through the palm of a hand. Our optical method is based on measuring and using all photons in the signal (as opposed to traditional methods, which use only part of the signal). Specifically, we use a time-resolved method that allows us to distinguish between photons that travel different paths in the tissue. Combining this unique measurement process with novel algorithms allows us to recover the hidden objects. This technique can be used in bio-medical imaging, as well as imaging through fog and clouds.

We are developing tomographic techniques for image synthesis on displays composed of compact volumes of light-attenuating material. Such volumetric attenuators recreate a 4D light field or high-contrast 2D image when illuminated by a uniform backlight. Since arbitrary views may be inconsistent with any single attenuator, iterative tomographic reconstruction minimizes the difference between the emitted and target light fields, subject to physical constraints on attenuation. For 3D displays, spatial resolution, depth of field, and brightness are increased, compared to parallax barriers. We conclude by demonstrating the benefits and limitations of attenuation-based light field displays using an inexpensive fabrication method: separating multiple printed transparencies with acrylic sheets.

Using a femtosecond laser and a camera with a time resolution of about one trillion frames per second, we recover objects hidden out of sight. We measure speed-of-light timing information of light scattered by the hidden objects via diffuse surfaces in the scene. The object data are mixed up and are difficult to decode using traditional cameras. We combine this "time-resolved" information with novel reconstruction algorithms to untangle image information and demonstrate the ability to look around corners.

Can a person look at a portable display, click on a few buttons, and recover his or her refractive condition? Our optometry solution combines inexpensive optical elements and interactive software components to create a new optometry device suitable for developing countries. The technology allows for early, extremely low-cost, mobile, fast, and automated diagnosis of the most common refractive eye disorders: myopia (nearsightedness), hypermetropia (farsightedness), astigmatism, and presbyopia (age-related visual impairment). The patient overlaps lines in up to eight meridians, and the Android app computes the prescription. The average accuracy is comparable to the traditional method -- and in some cases, even better. We propose the use of our technology as a self-evaluation tool for use in homes, schools, and at health centers in developing countries, and in places where an optometrist is not available or is too expensive.

Our deformable camera exploits new, flexible form factors for imaging in turbid media. In this study we enable a brush-like form factor with a time-of-flight camera. This has enabled us to reconstruct images through a set of 1100 optical fibers that are randomly distributed and permuted in a medium.

We introduce polarization field displays as an optically efficient design for dynamic light field display using multi-layered LCDs. Such displays consist of a stacked set of liquid crystal panels with a single pair of crossed linear polarizers. Each layer is modeled as a spatially controllable polarization rotator, as opposed to a conventional spatial light modulator that directly attenuates light. We demonstrate that such displays can be controlled, at interactive refresh rates, by adopting the SART algorithm to tomographically solve for the optimal spatially varying polarization state rotations applied by each layer. We validate our design by constructing a prototype using modified off-the-shelf panels. We demonstrate interactive display using a GPU-based SART implementation supporting both polarization-based and attenuation-based architectures.

We exploit the sub-picosecond time resolution along with spectral resolution provided by terahertz time-domain spectroscopy to extract occluding content from layers whose thicknesses are wavelength comparable. The method uses the statistics of the THz E-field at subwavelength gaps to lock into each layer position and then uses a time-gated spectral kurtosis to tune to highest spectral contrast of the content on that specific layer. To demonstrate, occluding textual content was successfully extracted from a sample similar to a closed book down to nine pages without human supervision. The method provides over an order of magnitude enhancement in the signal contrast and can impact inspection of structural defects in wooden objects, plastic components, composites, drugs, and especially cultural artifacts with subwavelength or wavelength comparable layers.

We have created a 3D motion-tracking system with automatic, real-time vibrotactile feedback and an assembly of photo-sensors, infrared projector pairs, vibration motors, and a wearable suit. This system allows us to enhance and quicken the motor learning process in a variety of fields such as healthcare (physiotherapy), entertainment (dance), and sports (martial arts).

Within the last few years, cellphone subscriptions have spread widely and now cover even the remotest parts of the planet. Adequate access to healthcare, however, is not widely available, especially in developing countries. We propose a new approach to converting cellphones into low-cost scientific devices for microscopy. Cellphone microscopes have the potential to revolutionize health-related screening and analysis for a variety of applications, including blood and water tests. Our optical system is more flexible than previously proposed mobile microscopes, and allows for wide field-of-view panoramic imaging, the acquisition of parallax, and coded background illumination, which optically enhances the contrast of transparent and refractive specimens.

Skin and tissue perfusion measurements are important parameters for diagnosis of wounds and burns, and for monitoring plastic and reconstructive surgeries. In this project, we use a standard camera and a laser source in order to image blood-flow speed in skin tissue. We show results of blood-flow maps of hands, arms, and fingers. We combine the complex scattering of laser light from blood with computational techniques found in computer science.

A smartphone based spectrometer design that is standalone and supported on a wireless platform. The device is low-cost and the power consumption is minimal making it portable to perform a range of studies in the field. Essential components of the device like the light source, spectrometer, filters, microcontroller and wireless circuits have been assembled in a housing that fits into a pocket and the entire device weighs 48 g. The device has a dedicated app on the smartphone to communicate, receive, plot and analyze spectral data. Validations of the device were carried out by demonstrating non-destructive ripeness testing in fruits. Ultra-Violet fluorescence from Chlorophyll present in the skin was measured across various apple varieties during the ripening process and correlated with destructive firmness tests. This demonstration is a step towards possible consumer, bio-sensing and diagnostic applications that can be carried out in a rapid manner.

Surface and object recognition is of significant importance in ubiquitous and wearable computing. While various techniques exist to infer context from material properties and appearance, they are typically neither designed for real-time applications nor for optically complex surfaces that may be specular, textureless, and even transparent. These materials are, however, becoming increasingly relevant in HCI for transparent displays, interactive surfaces, and ubiquitous computing. We present SpecTrans, a new sensing technology for surface classification of exotic materials, such as glass, transparent plastic, and metal. The proposed technique extracts optical features by employing laser and multi-directional, multi-spectral LED illumination that leverages the material's optical properties. The sensor hardware is small in size, and the proposed classification method requires significantly lower computational cost than conventional image-based methods, which use texture features or reflectance analysis, thereby providing real-time performance for ubiquitous computing.

Tensor displays are a family of glasses-free 3D displays comprising all architectures employing (a stack of) time-multiplexed LCDs illuminated by uniform or directional backlighting. We introduce a unified optimization framework that encompasses all tensor display architectures and allows for optimal glasses-free 3D display. We demonstrate the benefits of tensor displays by constructing a reconfigurable prototype using modified LCD panels and a custom integral imaging backlight. Our efficient, GPU-based NTF implementation enables interactive applications. In our experiments we show that tensor displays reveal practical architectures with greater depths of field, wider fields of view, and thinner form factors, compared to prior automultiscopic displays.

This work focuses on bringing powerful concepts from wave optics to the creation of new algorithms and applications for computer vision and graphics. Specifically, ray-based, 4D lightfield representation, based on simple 3D geometric principles, has led to a range of new applications that include digital refocusing, depth estimation, synthetic aperture, and glare reduction within a camera or using an array of cameras. The lightfield representation, however, is inadequate to describe interactions with diffractive or phase-sensitive optical elements. Therefore we use Fourier optics principles to represent wavefronts with additional phase information. We introduce a key modification to the ray-based model to support modeling of wave phenomena. The two key ideas are "negative radiance" and a "virtual light projector." This involves exploiting higher dimensional representation of light transport.

A new method to detect and distinguish between different types of fluorescent materials. The suggested technique has provided a dramatically larger depth range compared to previous methods; thus it enables medical diagnosis of body tissues without removing the tissue from the body, which is the current medical standard. It uses fluorescent probes, which are commonly used in medical diagnosis. One of these parameters is the fluorescence lifetime, that is the average time the fluorescence emission lasts. The new method can distinguish between different fluorescence lifetimes, which allows diagnosis of deep tissues. Locating fluorescence probes in the body using this method can, for example, indicate the location of a tumor in deep tissue, and classify it as malignant or benign according to the fluorescence lifetime, thus eliminating the need for X-ray or biopsy.

Traditional medical ultrasound assumes that we are imaging ideal liquids. We are interested in imaging muscle and bone as well as measuring elastic properties of tissues, all of which are places where this assumption fails quite miserably. Interested in cancer detections, Duchenne muscular dystrophy, and prosthetic fitting, we use tomographic techniques as well as ideas from seismic imaging to deal with these issues.