Sequence-based place recognition for mobile robots

Inferring ego position by recognizing previously seen places in the world is an essential capability for autonomous mobile systems. Recent advances have addressed increasingly challenging recognition problems, e.g. long-term vision-based localization despite severe appearance changes induced by changing illumination, weather or season. Since robotstypically move continuously through an environment, there is high correlation within consecutive sensory inputs and across similar trajectories. Exploiting this sequential information is a key element of some of the most successful approaches for place recognition in changing environments.

We work on a neurally inspired approach that uses sequences for mobile robot localization. It builds upon Hierarchical Temporal Memory (HTM), an established neuroscientific model of working principles of the human neocortex. HTM features two properties that are interesting for place recognition applications: (1) It relies on sparse distributed representations, which are known to have high representational capacity and high robustness towards noise. (2) It heavily exploits the sequential structure of incoming sensory data.

The above image illustrates place recognition based on a simplified version of HTM's higher order sequence memory. (left) Each frame of the input data sequence is encoded in form of a SDR and provides feed-forward input to the minicolumns. Between subsequent frames, active cells predict the activation of cells in the next time step. Output representation is the set of winner cells. (right) Example similarity matrix for a place recognition experiment with 4 loops (visible as (minor) diagonals with high similarity). The similarities are obtained from SDR overlap of the sparse vector of winner cells.

Publications and Code

Neubert, P., Schubert, S. & Protzel, P. (2019) A neurologically inspired sequence processing model for mobile robot place recognition. In IEEE Robotics and Automation Letters (RA-L) and presentation at Intl. Conf. on Intelligent Robots and Systems (IROS). DOI: 10.1109/LRA.2019.2927096 (Code)
Schubert, S., Neubert, P. & Protzel, P. (2019) Towards combining a neocortex model with entorhinal grid cells for mobile robot localization. In Proc. of European Conference on Mobile Robotics (ECMR). DOI: 10.1109/ECMR.2019.8870939
Neubert, P., Ahmad, S. & Protzel, P. (2018) A Sequence-Based Neuronal Model for Mobile Robot Localization. In Proc. of German Conference on Artificial Intelligence (KI). DOI: 10.1007/978-3-030-00111-7_11 (Code)