Detecting Regime Changes with a Mobile Robot using Multiple Models

Dani Goldberg and Maja J Mataric

We present an approach to the detection of global environmental regime changes by a mobile robot performing a task. The approach is based on the use of augmented Markov models (AMMs), which are essentially Markov chains having additional statistics associated with states and state transitions. We have developed an algorithm that constructs AMMs on-line and in real-time with little computational and space overhead. We developed AMMs as a general tool for capturing the interaction dynamics between a robot and its environment, and have demonstrated it to be effective in applications such as fault detected and dynamic leader selection. In this paper, we extend AMMs to regime detection, using multiple models to monitor events at different time scales and provide statistics to detect regime changes at those time scales. This approach has been successfully implemented using a physical mobile robot performing a land mine collection task. In the context of this task, we present experimental results, first validating our approach, then demonstrating a more complex proportion-maintaining scenario of the land mine collection task. Finally, we present results using an alternative reward maximization decision criterion in the same task.