The Counter Intelligence/Kitchen Sync White Paper is a look at the future of
kitchen and domestic technology. It serves as an introduction to designs, fundamental
technologies and future developments in this field.
Introduction
|
|
 |
This paper presents a vision of a future. In
the last two decades, a great deal of time and research has been spent on
"Living Room of the Future" projects. These usually meant big
screen televisions, quadraphonic sound and strategically placed sofas. The
problem with the living room is that it's a passive environment: the user
sits and absorbs, be it reading the newspaper or watching television. Compare
this to the fundamentally interactive kitchen. |
| |
Let us step back one hundred years into a kitchen
of the 1890s. It was barely recognisable as today's hygenic, (hopefully)
aesthetically pleasing center of the home. Refrigeration was at best performed
by lumps of iced delivered intermitantly; electricity absent, and most heat
provided by coal or wood stoves. It was the domain of the servants, or perhaps
the wife of the household, hidden at the back of house. Think of the progress
of the last one hundred years; then think of the progress of the next one
hundred years. |
| |
The kitchen is an intrinsically dynamic environment. Raw materials
enter, are processed and leave. The user of the kitchen invariably contributes
to that processing, whether it be putting a pre-frozen lasagna in the microwave
or meticulously leafing through The Joy of Cooking for a four-course dinner.
As the two-way conversation of the Internet is to the one-sided oratory
of television, Kitchen Sync is to the Living Room of the Future. |
| |
Kitchen Sync, the vision behind Counter Intelligence, is a
digitally connected, self-aware kitchen which has knowledge and memory of
its activities. It is a multifaceted system, consisting of both intelligent
individual elements and, fundamentally, connectivity between those elements.
In practice, we are building individual components, while maintaining design
philosophies that encourage inter-unit communication. Within the Personal
Information Architecture group, we are also building an infrastructure,
Hive, specifically designed
to enable communication between such elements. |
| |
We present here a basic overview of the three
stages of devices which comprise Kitchen Sync. References to individual
projects will be made clear as we go into more depth on the theories behind
their operation. |
|
The third stage of Kitchen Sync has components that are not
only self aware but have some memory of their use. A Mr. Java coffee cup
is tagged with a simple RFID tag, functioning similarly
to a barcode. A given cup merely knows it is, say,
object #1422722. A single step along is a coffee cup that knows when it
was last used and will release this information when asked - perhaps by
a coffee machine, or a desk that wants to make sure your coffee doesn't
get cold. Perhaps more useful is a fridge door that knows when it was last
opened - and if it's been closed since that point. The simple addition of
memory adds a wealth of possibilities for an item. |
| |
One can also think of this third stage of 'having
memory' as being aware of temporal sequences. Mr. Java, a second stage device,
functions in the instant: when a cup is presented, he makes coffee and plays
the news. Each cup is treated as a separate isolated incident, unrelated
to the one before it. Counter Intelligence, however, is aware of time and
sequence - it's important to add the flour before putting on the icing,
for example. |
| |
We can continue along this trail: one project
for the future is to produce a fridge which is aware of its contents, and
can take action based on that awareness. We envisage a fridge that is not
only aware how much milk it contains, but also orders more should that milk
run out. Once a fridge - or a cupboard, a pantry, a larder - is completed,
Kitchen Sync will have reached a stage where there is a serious possibility
of using the intelligent Kitchen on a day-to-day basis. |
| |
Eventually, we hope to assemble an entire intelligent kitchen
environment and spend time working within the space and using the equipment
on a day to day basis. In the long term, of course, we hope to see Kitchen
Sync projects being used in commercial and residential kitchens. |
|
Mr. Java is an intelligent coffee machine. It's
based on an Acorto 2000s automatic coffee machine, which in its unaltered
state makes a variety of hot coffee and milk based drinks at the touch of
a button. By interfacing with the diagnostic serial port, we were able to
control Mr. Java by means of a tag reader placed under the spout. |
| |
The user places their cup under the spout, as
usual. The reader located under the spout reads the tag on the bottom of
the cup and transmits the result to a computer. The computer would then
issue commands to the Acorto to make the appropriate drink, and play the
associated RealAudio feed through the speakers. |
| |
Mr. Java does not gather information on individual
users' coffee use, although it provided that facility by letting users set
their own URL for their audio feed, which would let users keep track of
their own consumption. However, we did keep track of overall consumption,
including dividing the data by day and by hour over time. For example, we
saw a consistent daily pattern: |
|
Morning coffees peaked at 11am, and we saw another
mid-afternoon peak at 3pm. Barely visible at 9pm is our first espresso peak;
apparently, if you're still drinking coffee at 9pm, then it better be espresso. |
| |
This kind of information was of great interest
to many sponsors: both Kraft Foods, owner of Maxwell House, and P&G, owner
of Folgers, spend a great deal of time and effort tracking usage statistics
such as these. Presently, it's entirely done by hand: someone sits next
to the coffee machine with a clipboard. Mr. Java's type of unobtrusive monitoring
that can actually add value to the product being purchased has possibilities
for a wide variety of applications. |
| |
Mr. Java has been a great success. EDS purchased
an entire system for their Dallas MarketSpace of the Future, and are presently
considering assembling another ten systems for various offices and showrooms.
Kyle Anderson, CEO of Acorto, sees Mr. Java as the missing element between
a regular Acorto automatic espresso machine and the barista: it provides
entertainment. A barista chats about the weather, tells jokes; Mr. Java
adds back that functionality. There are currently plans to exhibit a Mr.
Java in Acorto's main lobby. |
|
The kitchen counter is one of the most used portions
of the kitchen: workspace is invariably prime real estate in food preparation.
A wide variety of tools are used in conjunction with the counterspace in
any food preparation: weighing scales, measuring cups, bowls, spoons, and
- importantly - ingredients. Counter Intelligence tries to integrate itself
into your work habits by serving as an interface between you, the recipe,
and the food being prepared. |
 |
How can a kitchen help you make brownies? You pull
out the cookbook, and start measuring ingredients into a bowl. Out of baking
powder? Well, you remember you can use baking soda, but do you double or
half the quantity? Oops. Just put the eggs in before the milk. Hope it doesn't
matter - wonder why they're listed in that order in the recipe if it doesn't?
If you use semi-sweet chips instead of dark chocolate, how do you adjust
the sugar? Can't see how much butter to add: that dark chocolate from last
time got on the page. |
| |
Counter Intelligence takes away these problems.
It's fully aware of a recipe: the sequence, the ingredients, possible substitutions.
We're in the process of building it as a fully expandable system, enabling
us to modify the user interface as we learn more. The current (extremely)
prototype system uses a barcode scanner, a scales, and a keyboard for input,
and a standard screen for output . It 'knows' a handful of recipes, can
suggest substitutions for one or two products, and has a text-based interface.
|
 |
We envisage a system almost entirely integrated
into a standard workspace area. A scale built into the counter along with
an RFID tag reader lets you identify and tare mixing bowls, whereas other
tag readers, barcode readers or LazyFish could identify ingredients. A LazyFish
would let you select ingredients and finished products by tapping their
picture on the surface, with the entire recipe becoming an interactive experience.
Perhaps instead of a line of text saying "Mix in two cups of flour", Counter
Intelligence will have a pair of elves projected on your counter, apparently
tugging at your real bag of flour. |
| |
The possibilities of Counter Intelligence are practically
endless. We are consciously not predicting an exact path of evolution for
this project, or exact technologies we wish to work with. By letting it
evolve with the technology and change as possibilities arise, we're free
to create and invent entirely new concepts of kitchen interaction without
being locked into an obsolete model. |
|
The concept of an intelligent fridge - Cool I/O
- is one that seems fundamental to the intelligent kitchen. We see a fridge
as performing, a number of functions. |
| |
Primary and most fundamental is the ability to identify objects within
it. This simple sentence encompasses an entirely non-trivial problem:
a later chapter deals with some of the technological possibilities that
could make this occur. What has become clear is that once this hurdle
is overcome, there are a vast number of advantages to everyone who interacts
with a product, from manufacturer to consumer.
|
| |
We see an initial application in the commercial
kitchen, which, by virtue of sheer turnover and capital investment, has
much greater need for Counter Intelligence's information-tracking capabilities.
Specifically, we see a fridge that can keep track of its contents, including,
for any given item, the location in the fridge or fridges, dates that item
entered the fridge or was used, and expiration dates. |
| |
An obvious next step in this endeavor involves
keeping track of desired contents. For example a home might wish to keep
at least one gallon of 2% milk, not more than two days old, at all times.
From this it is a simple matter to have automatic shopping list generation
or, logically, online ordering to replace staple items through a grocery
delivery service. |
| |
An intelligent freezer would perform many of the
same functions, but would perhaps be easier to prototype, as objects that
go in or out of a freezer are generally either in Tupperware-type packaging
or in their original packaging. |
| |
We see the development of CoolIO or a similar
intelligent fridge as a fundamental part of the Counter Intelligence vision.
|
| |
In the summer of 1997, the fledgling Kitchen Sync
team worked on an intelligent fridge that kept track of all contents entering
or leaving the fridge through a doorway-mounted barcode reader. We found
that this user interface was entirely impractical, and massively inadequate
for any consumer use. We expect that a fridge that knows what it contains
will have to rely on an RFID-like tagging scheme. |
|
In an interconnected kitchen, even disposal units
are part of a communications network that keeps track of comings & goings.
We see the sink in such a kitchen at a minimum having a tag reader to read
tags from reusable containers being washed - Tupperware and the like. |
 |
Envision the following scenario. You've had lasagna
for dinner, and there's some left over. You put the leftovers in a Tupperware
container, and put it in the fridge. "Is that lasagna?" asks the fridge
- it remembers you made that for dinner. You confirm. Later on, feeling
peckish, you pull out the leftovers and take half for a snack, putting the
Tupperware back in. CoolIO remembers what was in that tagged container,
and so assumes it still contains lasagna. Hungry again, you pull out the
remainder and eat it for lunch the next day. |
| |
You put the dirty container into Everything Bit,
and wash off the food. As you do that, the RFID tag reader reads the tag
and washes off the data, labeling it as empty. Next time you use that container,
the fridge will ask you what's in the box. |
|
Kitchen!" you announce, bringing Kitchen Sync out of its sleep.
"I'd like to make a chocolate cake for desert tonight."
"I'm afraid we're out of butter: the delivery was delayed. We can
substitute olive oil though - you liked that last time."
"Fine."
|
| |
The sounds of John Coltrane fill the air as you
assemble the ingredients list projected on the wall, with the Kitchen only
occasionally advising you on where you last put the baking powder. You put
a mixing bowl down on the counter, and look at the wall. The recipe is replaced
with a grinning foot-high character in a tall cook's hat, who points at
the flour. You pick it up. |
| |
"Four cups of flour." You start pouring.
"One cup... two... three... three and half... and stop."
You put the flour back on the counter.
"You can put that away now. You won't be needing it. And it'll make
the place tidy." |
| |
Guess you accidentally engaged the "Mother"
mode. Still, you continue with the rest of the recipe, mixing and stirring.
The Kitchen reminds you of the substition, and suggests you use low-fat
chocolate - a suggestion you cheerfully ignore, despite a twinge of guilt
as it updates the calorie count at the bottom of the page. It's only a matter
of sliding the cake into the pre-heated oven and waiting until the Kitchen
reminds you to take it out. And if you're in the shower when that happens?
No need to worry: your Kitchen will remember to turn the oven off, even
if you don't. |
|
We've discovered a lot of ways to look at human-computer
interaction, and the very concept of data, in the process of working with
Mr. Java and Kitchen Sync. The first is a common enough realization at the
Media Lab: that we exist in a cloud of bits, a set of information about
your current condition. Today we mainly think of bits as perhaps graphics,
webpages, QuickTime movies. In Personal Information Architecture, we go
beyond this definition and see bits as a spectrum, ranging from the fixed
and quantifiable to the fuzzy and intangible. |
| |
For example, I am six foot two inches tall. That's
a constant and relatively unchanging bit. Continuing along our spectrum
of bits, I have a body temperature, pulse and blood pressure that are measurable
and recordable using a variety of sensors. Nearer the other end of the spectrum,
I may be hungry, or want a particular kind of coffee today. These are far
less fixed and easy to measure: hunger is a function of blood sugar, but
goes unnoticed with sufficient levels of adrenaline in the bloodstream.
|
|
In our initial design for Mr. Java, we had thought
about a number of ways to recognise users of the machine. One possibility,
for example, was IR transmitting badges, previously used on the Penguin
Demo to great success. Stuffed penguins wore nametags that emitted a constant
infra-red signature, saying, in effect, "I'm Irv. I'm Irv". When Mort, the
other penguin, received this, Mort and Irv would have a conversation, as
they knew they were facing each other. |
 |
The problem with this is that there's no implicit
context. We wanted to avoid the problem of a coffee machine that spewed
out espresso whenever you walked through the 3rd floor kitchen. In the plans
for Kitchen Sync, there are many tag readers and ways to identify objects:
it's important to know the context in which this is happening. |
| |
Another way to think about the importance of context
is in thinking about sharing bits. Unless you know what you're looking for,
it's hard to figure out whether the stream of information you're looking
at is biometric data from a human being on a bicycle, weather data from
a probe at Base Camp on Everest, or an I Love Lucy rerun. "Bits be bits."
Once bits leave their creating environment, it's important to ensure that
they're implicitly and unambiguously labeled. |
| |
Barcodes
Barcode technology has a number of advantages: it's cheap, and it's widely
available. Commercial products frequently come with barcodes, enabling
easy recognition.
|
| |
However, there are problems with current barcodes
as implemented in the UPC standard - that is to say, the labels on nearly
every product you buy at the grocery store. They don't distinguish between
different iterations of the same product - one can of tomatoes looks like
another can of tomatoes. That's fine at the checkout, but difficult if you're
trying to tell how many cans you have in your larder. If we're trying to
keep track of how old milk is, for example, it's important to be able to
distinguish between two cartons of milk that have the same barcode but were
purchased a week apart. |
 |
One possibility would be for every UPC code to
have two separate parts: an identification portion and a serial number portion.
For example, a particular bottle of apple juice currently has the barcode
2-26284-17513-9. 2-26284 refers to the company who make the product, as
assigned by the UPC council. 17513 is the company's code for "8 fl.
oz. bottle Pressed Apple Juice." 9 is a check digit to ensure the computer
has read the numbers correctly. Expanding this to include a serial number
- thus, say, 226284-17513.0170222 would enable tracking of that particular
bottle's history, including storage, sale, and environmental conditions
during shipping. |
| |
The most important change in barcodes will come
when barcodes are no longer seen as identifying objects in themselves but
as links to information. There is a practical limit on the quantity of information
that can reliably be stored in a physical label space: there is no limit
to the amount of information that can be linked to that label. |
| |
The next step will come when you purchase a product
which has its own individual webpage. A can of beans will come with its
own individual webpage detailing such information as production date, transport
history, and time spent on the shelf, all entered automatically as it moves
along the retail chain. Two apparently identical packets of rice you purchased
on two trips to the supermarket can have entirely different histories of
transport, storage, and origin. This incredible quantity of information
will begin to appear for high end items - a web-accessible history of your
car, say - but as time goes on will continue down the value chain. |
|
RFID
Radio Frequency Identification has the potential to be one of the most
widely used and powerful identifying technologies we have. Tags can be
battery powered or unpowered, and can be purchased in a variety of sizes
and configurations to allow for a wide range of uses. They are contactless,
and require neither touch nor line of sight. In particular, they work
through plastic, wood, and other materials, and can be set up to work
in harsh environmental conditions, where barcodes or less robust equipment
would be unable to function.Our classic example of this is under the spout
of Mr. Java, where a polyurethane-encased reader is regularly subjected
to 245°F espresso. The kitchen is no place for fragile technology.
|
| |
There are a wide variety of RFID tags. The simplest
work in much the same way a barcode does, giving out a single pre-programmed
number when placed in the vicinity of a reader. It's also possible to store
a limited amount of information on the tags themselves. Research is currently
underway to produce tags that report information about their physical environment,
such as temperature or position, in addition to readable and writable storage. |
| |
The current (temporary) argument against RF tags
is that of cost. Compared to a printed barcode, the cost is presently prohibitive
except in harsher environments unsuited to barcodes or more elaborate packaging
systems, such as the EZPass toll-paying system.. Currently, a simple tag
has a lower price limit of approximately ten cents: too much to put on a
packet of cornflakes, but an entirely reasonable way to track the history
of a $300 jacket. A tagged world will arrive, one bit at a time. |
|
The next step after identification is associating
preferences with the object identified, be it person, cup, clothing or penguin.
These preferences fall into two categories: bits and atoms. This is an important
distinction in defining what we're looking to do in Kitchen Sync. Looking
back at what was seen as a kitchen of the future in the sixties, through
the eyes of the Jetsons, we see a kitchen concerned with manipulating atoms:
mechanical hands come out holding frying pans into which other mechanical
hands crack eggs. We're not trying to deal with manipulating atoms, we're
interested in bits. |
| |
Mr. Java, for example, associates two sets of
information with each tag, with each individual's cup. One set is how they
like their bits arranged, and the other set is how they like their atoms
arranged. Arrangements of bits refers to, for example, the latest news from
NPR, or the current weather report, or the sports scores. Mr. Java goes
out across the web and picks up the RealAudio feed of your choice, and plays
it while your coffee is being made. Importantly, we've provided the facility
for users to create their own RealAudio or .au files by linking to a URL,
which could contain their own personalized stock portfolio, or their messages.
This also provides them with the means to track their personal coffee consumption
without Mr. Java storing such personal individual data. |
| |
This concept of preferences for bits and atoms
is a powerful concept that can be extended far outside the realm of the
kitchen. A car could recognise you through your key or key fob, and adjust
the seats and mirrors (atoms) and the radio (bits) to your preferences.
A washing machine could recognise clothing going in through flexible RFID
tags and know you might not like your red T-shirt in with your otherwise
white wash. All of these are examples of simple but powerful operating concept: |
| |
Within a context, recognise and associate preferences.
|
|
The field of domestic media is one that will truly
come to fruition in the next decade. We currently have use of barcode and
RFID technology almost entirely in industrial and commercial settings. As
with the computer, the pager, and the microwave, we can expect this technology
to start to be integrated in to home life. |
| |
We're frequently asked how long it will be until
we see Kitchen Sync technology entering the marketplace. Different pieces
of technology will no doubt take different periods of time to really become
useful. Mr. Java is currently starting to move into the marketplace, with
the help of generous support and interest from EDS and Acorto. MicroChef
/ PC Dinners technology is already arriving in the home in a variety of
forms, notably in the Japanese market. Longer term projects include Counter
Intelligence and CoolIO, which currently appear to be in the ten-year range,
as they require greater investment in an infrastructure and information
base. Both will be effected by the degree of growth of shop-from-home services
such as Peapod. |
| |
We are currently at the Model T stage of computers.
The Model T is still known, seventy years later, for being available in
any colour you wanted, so long as it was black. Computers are currently
available in any size you want, so long as they're a box. You can buy boxes
that sit under your table, on your table, in your bag or in your pocket,
but they're all boxes. |
| |
Kitchen Sync is one example of technology designed
to not be a box but instead be a coffee cup, a counter, a toy. A vast quantity
of research at the Media Lab is engaged in making non-box computers. Kitchen
Sync and the Counter Intelligence SIG show an extremely exciting and viable
area of research in this arena with a great number of both commercial and
research possibilities |
|
My primary acknowledgement is to the members of
the Personal Information Architecture
group for all varieties of help from technical to emotional. Particular
thanks must go to Kristin
Hall for invaluable editing, friendship and support, Niko
Matsakis, without whom none of this would have been possible, and my
research advisor, Professor
Mike Hawley - despite, and perhaps due to, his love for appalling puns
in project names. A note of thanks is also due to Matt Lau for the name
"Counter Intelligence." |
| |
Another serious acknowledgement is due to those
I live, work, rehearse and play with for the amount of time and support
they have given me while I worked at the Lab instead of being with them.
My family and my girlfriend, Allison Wolf, have given me constant, incredible
and invaluable support, and Roadkill
Buffet and my various theatrical endeavors have kept me sane. Without
my friends, I would have and be nothing. |
| |
A great deal of thanks is also due to the sponsors
of the Things That Think consortium
and Counter Intelligence SIG
for providing funding for this research. |
