LaserVision

Inspiration

Lasers are a common trope in Science Fiction, featured in everything from laser guns to laser screwdrivers. They are especially common in weapons, namely automatic sentry turrets and “smart” or targeted weapons. These include laser-sighted rifles, laser following missiles, etc. but in almost all cases, the laser is used as an indicator or control for some other process or device – the laser is an interface.

This project attempts to explore some possible solutions for laser-based interfaces, while also taking the concept of “hands free” direction indication a step further: instead of pointing with your hand or head, you can indicate a vector in space in a natural, intuitive, fast, and precise way by using a part of your body that is often already in perfect alignment with the subject of your attention: your eyes.

Motivation

There are several advantages to eye-based control. When we think of “dexterity” we usually associate nimbleness, precision, and speed with the hands. However, we have other organs, the eyes, which are perhaps even faster and more precise, but are as-yet largely unused for interaction or control. Such an interaction method is, of course, also hands-free, another compelling property, and eye-tracking technology is becoming easy and inexpensive, with hobby-level systems exhibiting performance suitable for most tasks. Finally, when an eye-tracker is hooked up to a laser pointer, there is the undeniable cool-factor of creating a system that essentially allows the user to shoot lasers from (near) their eyes.

Application

The primary exception to this underutilization of eye-tracking and spatial cursors in HCI is for physically disabled users who cannot manipulate or interact with devices in the normal way. Eye-trackers connected or head-mounted lasers have been used to assist individuals with disabilities including ALS and cerebral palsey.

However, there is also class of tasks for-which simply looking is itself a natural interaction method. These tasks usually involve targeting, pointing, selection, etc. via a 3D vector in space – the gaze direction. Although all of these actions are applicable to tasks on a standard computer UI (2D screen), and eye-control of common computer tasks (controlling a mouse, opening email, writing documents, etc.) has been well demonstrated, this is not the best use of our eyes to interact with computers. With the exception of cases involving disabled people, such tasks on a screen are usually accomplished far more efficiently with a standard keyboard and mouse or touchscreen. Where the eyes may represent a natural and powerful interaction method, however, is in spatial interactions with ambient objects in the real world (not directly within reach), where these standard interfaces fail.

System

The system consists of four primary subsystems: a wearable eye-facing camera and computer system for pupil tracking, a head-mounted laser pointer gimbal, and an optionally-wearable laser-tracking turret for interaction demonstration.

• Eye-Tracker The eye tracker is based off the open-source pupil system and consists of a standard usb webcam with the outer case and infrared filter removed. The webcam is mounted on the frames of safety glasses with an adjustable boom. In the first iteration, the webcam is positioned directly in front of the user’s eyeball, a couple centimeters away. Although this allows for highly reliable eye tracking, it is also obtrusive. In the final version, the webcam “sees” the user’s pupil reflected off an infrared mirror placed at an angle in front of the eye. Because this mirror passes visible light, the user can look through it and hardly notice anything is there. Meanwhile the webcam can be situated to look down at the mirror from above the eye, where it is less intrusive. Finally, the webcam status LEDs are replaced with infrared LEDs, which are angled to illuminate the eyeball area for better pupil contrast.

• Laser Pointer : The laserpointer is a simply a small laser module attached to a 2 Degree of Freedom (2-DOF) mini servo gimbal. This assembly is mounted to a GoPro headstrap camera mount (not shown) to be strapped to the user's forehead.

• Tracking Turret:  The turret consists the functional components of a fully automatic NURF gun with a new case mounted on a 2-DOF high-torque servo mechanism. A standard webcam is attached on the base. This assembly can be mounted on a modified GoPro shoulder strap camera mount. (The turret can also be mounted on a standard camera tripod to act as a standalone automated sentry turret, with separate laser targeting system.) Functionally, the turret system operates independently of the other components, and serves only as a demonstration of a separate device making use of the laser-as-interface. The turret automatically tracks laser dots it sees in it's own field of view, and fires when the dots blink.

• Control Electronics The electronics consists of an Arduino Uno micocontroller which controls all servo motors (four in total through PWM), the DC motor of the nerf gun (through a power MOSFET), the laser gimbal laser module (direct), and the illumination LEDs of the turret (through a constant-current LED driver circuit for 1W LEDs) All servos and LEDs are powered by a separate regulated 5V supply, which is derived from a 7.4V 2S 25C LiPo battery pack. The DC motor is powered directly by the by the battery. All electronics are housed in a protective box.
• Computer and Software A macbook air is used to coordinate the system and perform computer vision processing. The two webcams and Arduino are connected to the computer through a usb hub. The open-source Pupil Capture software processes the video stream from the eye-tracking webcam and broadcasts pupil positions on an ftp server. A custom python script subscribes to these messages and commands the laser gimbal accordingly by mapping the pupil position (x,y) screen position to gimbal yaw and pitch angles. Inverse-distance-weighted interpolation is used on a set of 9 calibration points near the corners, edge-centers, and center of the eye’s Field of View (FOV, assumed to be a rectangular frustum.) These corresponding points are recorded during an initial calibration routine in which the laser is pointed at each point and the program records the average pupil position when the user is looking at the laser dot.  After calibration, when the user is targeting the laser with their gaze, a separate python routine is run to control the laser-tracking turret. This routine processes the video stream from the turret webcam to detect laser dots. This process involves color filtering and thresholding to produce a binary mask of likely laser dot pixels. A dilution kernel is applied to eliminate single-pixel outliers. If the number of remaining pixels is sufficient, their centroid of the laser dot is then derived from the first moment of the mask. The (x,y) position of this centroid is linearly mapped to the (yaw, pitch) of the turret based on the webcam’s FOV. The only software connection between the software system for the eye-tracking-laser-gimbal and the laser-tracking-turret is that user winks detected by the eye-tracker activate the firing mechanism on the turret, though this can be done by analyzing the laser dot itself for "blinks".

Discussion

• What would a society with a high percentage of the population wearing systems like this look like? What happens to the individual and their privacy and freedom when the attention of everyone else around them is made glaringly obvious (pun intended), and their own most intimate glances are broadcast for all to see in real time? One could imagine a society with vastly different norms, rules, and customs regulating where one may look, as well as vastly different patterns in where groups look and why.

This project was originally developed for the 2015 SciFab course MAS.S64: http://scifab.media.mit.edu/2015/12/16/saccase-turrets-tracking-lasers-tracking-pupils-tracking-targets/