The Audio Notebook
Paper and Pen Interaction with Structured Speech

Lisa J. Stifelman
Speech Interface Group, MIT Media Lab

Illustration Copyright MIT Media Laboratory, 1995

The Audio Notebook allows a user to capture, skim, and browse an audio recording in conjunction with notes written on paper. After attending a lecture or meeting, a user may be missing information from their notes, want more detail for a topic, or to review the original material. The Audio Notebook synchronizes notes written on paper with an audio recording. The user's natural activity (i.e., writing and page turns) serve as implicit indices into the audio. The spatial layout of writing on the page enables a user to rapidly navigate in a time-dependent medium like speech.

• Familiar objects like paper and pen are used for interaction rather than artifacts left over from analog devices (e.g., FF and REW controls).
• An audio scrollbar provides a timeline of the audio associated with each page of notes.
• As a user flips through physical pages of notes, the audio scans to the content for each page.
• Automatic segmentation of the audio based on acoustic cues is combined with the user activity to "structure" the audio.

Publications

• Stifelman, L.J. Augmenting Real-World Objects: A Paper-Based Audio Notebook. Proceedings of CHI '96, ACM SIGCHI. Vancouver, Canada.
  Postscript version
• Stifelman, L.J. The Audio Notebook: Paper and Pen Interaction with Structured Speech. Ph.D. dissertation. Massachusetts Institute of Technology. September, 1997.

Photos & Other Info

• Audio Notebook Prototype 2.0
• Audio Notebook Prototype 1.0