Recognition-based technology has made substantial advances in the past few years because of enhanced algorithms and faster processing speeds. However, current recognition systems are still not reliable enough to be integrated into user interface designs. A possible solution to this problem is to combine results from existing recognition systems and mutually disambiguate the unreliable sections. Piecing together partial results obtained from each mode of recognition can derive more reliable results. In addition, the results of one recognition system can be used to prepare the other recognition system. We are experimenting with an approach that uses a software agent to integrate off-the-shelf recognition applications via scripting languages. We use a software agent called MARCO (Multimodal Agent for Route Construction) that utilizes multiple recognition systems to assist users in giving directions for urban navigation.