Video or broadcast news is viewed in a far wider set of circumstances than it ever has been before. It is composed with the assumption of a complete, situated viewing, but in fact it is often grabbed on-the-fly as a momentary experience. As You Need It is a semantic summarizer that deconstructs a multi-part segment for presentation as “chunks of importance.” We are learning if a story can be cut down to a useful update that takes less time than a traffic light, or as much time as a given user has. This project uses and contributes to another group project, SuperGlue.

DbDb (pronounced DubDub) is a collaborative, visually based analysis and simulation platform. We promote open distribution of experimental data by allowing researchers to present a graphical representation of their data and processing techniques that collaborators can build on and augment. This helps test the reproducibility of results and allows others to learn and apply their own techniques. Our intention is for the research community as a whole to benefit from a growing body of open, analytical techniques. DbDb provides an interface for archiving data, executing code, and visualizing a tree of forked analyses. It is part of the Viral initiative on open, author-driven publishing, collaboration, and analysis. It is intended to be linked to PubPub, the main project.

The physical world is increasingly coming online. We have things that measure, sense, and broadcast to the rest of the world. We call this the Internet of Things (IoT). But our cameras are blind to this new layer of metadata on reality. The IoT recorder is a camera that understands what IoT devices it sees and what data they are streaming, thus creating a rich information "caption-track" for the videos it records. Using this meta-data, we intend to explore how this enables new video applications, starting with cooking.

NewsClouds presents a visual exploration of how the news reporting of an event evolves over time. Each “cloud” represents a publication and each competing news organization usually emphasizes different aspects of that same story. Using the time sliders, that evolution becomes evident. In addition, each word or phrase can be expanded to show its links and context. We are building an archive of events associated with ongoing US election developments.

Bitcoin generates net-new value from "mining" in a distributed network. In this work, we explore solar micro-mining rigs that transform excess energy capacity from renewable energy (hard to trade) into money (fungible). Each rig runs a small Bitcoin miner and produces Bitcoin “dust” for micropayments. We envision these micro-miners populating a highly distributed network, across rooftops, billboards, and other outdoor spaces. Where systematic or environmental restrictions limit the ability to freely trade the underlying commodity, micro-mining produces new economic viability. Renewable energy-based, micropayment mining systems can broaden financial inclusion in the Bitcoin network, particularly among populations that need a currency for temporary store of value and must rely on flexible electricity off the grid (e.g., unbanked populations in the developing world). This exploration seeds a longer-term goal to enable open access to digital currency via account-free infrastructure for the public good.

SuperGlue is a core news research initiative that is a "digestion system" and metadata generator for mass media. An evolving set of analysis modules annotate 14 DirecTV live news broadcast channels as well as web pages and tweets. The video is archived and synchronized with the analysis. Currently, the system provides named-entity extraction, audio expression markers, face detectors, scene/edit point locators, excitement trackers, and thumbnail summarization. We use this to organize material for presentation, analysis, and summarization. SuperGlue supports other news-related experiments.

VR Codes are dynamic data invisibly hidden in television and graphic displays. They allow the display to present simultaneously visual information in an unimpeded way, and real-time data to a camera. Our intention is to make social displays that many can use at once; using VR codes, users can draw data from a display and control its use on a mobile device. We think of VR Codes as analogous to QR codes for video, and envision a future where every display in the environment contains latent information embedded in VR codes.