Esteban Real (Google DeepMind): Automatically Discovering Learning Algorithms with Hardware Constraints

Abstract:

Can learning emerge from a search process in-silico? Our AutoML-Zero work at Google DeepMind shows that a simple evolutionary search process can automatically discover modern learning techniques from scratch. Without knowledge of machine learning, the method discovers algorithms by combining simple primitives such as additions and multiplications into a functional learning algorithm. In this type of evolutionary search, learning emerges naturally when algorithm survival depends on its performance on multiple tasks. These discovery methods transfer to realistic setups, where we can find novel algorithms for ML optimization and robot adaptation. Importantly, we can meaningfully shape the properties of the discovered algorithm by constraining the environment on which the algorithm evolves. I will speculate that using hardware abstractions as such a constraint is a promising direction for finding new paradigms of neural computation.

Speaker Bio:

Esteban Real is a Research Scientist at Google Deep Mind. Real previously worked at the Google Brain Team within Google Research. Esteban Real focuses on bio-inspired computing, especially automated machine learning (AutoML), the combination of evolutionary computation and learning, and the relationship between natural and artificial neural networks. A selection of Real’s publications can be found in Esteban Real’s Google Scholar page.