Learning Hierarchical Acquisition Functions for Bayesian Optimization

Image by Nils Rottmann

Abstract

Learning control policies in robotic tasks requires a large number of interactions due to small learning rates, bounds on the updates or unknown constraints. In contrast humans can infer protective and safe solutions after a single failure or unexpected observation. In order to reach similar performance, we developed a hierarchical Bayesian optimization algorithm that replicates the cognitive inference and memorization process for avoiding failures in motor control tasks. A Gaussian Process implements the modeling and the sampling of the acquisition function. This enables rapid learning with large learning rates while a mental replay phase ensures that policy regions that led to failures are inhibited during the sampling process. The features of the hierarchical Bayesian optimization method are evaluated in a simulated and physiological humanoid postural balancing task. The method outperforms standard optimization techniques, such as Bayesian Optimization, in the number of interactions to solve the task, in the computational demands and in the frequency of observed failures. Further, we show that our method performs similar to humans for learning the postural balancing task by comparing our simulation results with real human data.

Video of the presentation, Corresponding publication

Date
Location
Las Vegas, USA
Avatar
Nils Rottmann
Team Lead for Robotics & Autonomous Systems

With September 2021, Nils Rottmann started as a Software Developer/Product Owner at the Hako GmbH. He studied Theoretical Mechanical Engineering at the Hamburg University of Technology, Germany and holds a PhD in Robotics from the University of Luebeck, Germany, In his PhD with the title “Smart Sensor, Navigation and Learning Strategies for low-cost lawn care Systems”, he developed low-cost sensor systems and investigated probabilistic learning and modeling approaches. Currenlty, he works as a Team Lead for the robotic section at the Hako GmbH.