Wearable Wisdom

Wearable Wisdom is a context-aware, audio-based wearable system for mediating advice from mentors to the user. Through audio-based augmented reality glasses, our system offers just-in-time wisdom and advice based on the user’s query and current context by generating a simulated voice of a mentor sharing relevant wisdom and advice with the user. Our system consists of a wearable audio I/O device, a smartphone capable of real-time utterance recognition and context detection, and a backend infrastructure for storing the mentor profiles, processing user inputs and providing responses. 

Based on the literature, we aim to leverage state of the art, context-aware wearables, NLP, and intelligent agent systems to create technology that can provide the user with motivational feedback beyond factual information. Our contributions are: 1) developing and implementing the software architecture for the intelligent wearable audio-based system capable of sensing and offering real-time feedback to the wearer, 2) demonstrating the use of algorithms to effectively deliver relevant, yet abstract wisdom and advice to the user, and 3) exploring the novel area of wisdom computation through user studies. 

We developed Wisdom Paring Algorithm (WPA) based on the Word Mover’s Distance (WMD) method to measure the semantic similarity between two given phrases. At a high level, this algorithm uses a word2vec to output the distance or dissimilarity between two texts. Given two words, i and j, we can compute c(i,j), which is the Euclidean distance or "cost of travel" between them per the word2vec model. The overall distance between two texts is defined as the minimum cumulative cost required to move all words to transform one text to another. Word2vec and WMD work particularly well for our purposes, since we are trying to compare abstract questions and non- factual answers. WMD allows flexibility in deducing the meaning of the text. For the semantic matching scheme in our project, we used the Google News word2vec model. WPA pre-processes the user input by removing stop words and non- alphabetic characters. Given the question from user input, we calculate WMD for all of the questions/answers in the database and can provide an appropriate response from the quotes with the smallest distance. 

Through T-test statistical analysis, our results from 420 ratings (n = 10) show that our algorithm is statistically superior in pairing the wisdom and advice with the questions (P-value = 0.0004), as well as in delivering more inspirational wisdom (P-value = 0.0019) compared to the control. This results demonstrate the efficacy of our algorithm to deliver relevant, yet abstract wisdom and advice to the user.