– Teams link : https://teams.microsoft.com/l/meetup-join/19%3aMVHWcoJpE9ydRWENhbnWqgLENWmTloOdNUbGwTj3dYs1%40thread.tacv2/1703782252879?context=%7b%22Tid%22%3a%22158716cf-46b9-48ca-8c49-c7bb67e575f3%22%2c%22Oid%22%3a%22955ae724-3c70-4f81-8a92-327e1dc5af57%22%7d
– Titre : Feedback Design Principles for Efficient and Reliable Robot Learning
– Résumé : Truly autonomous robots must rapidly adapt their behavior to new operating conditions. Classical techniques from control leverage first principles models to synthesize robust controllers and efficient adaptation strategies. However, these frameworks often break down in real-world scenarios where models can fail. Data-driven techniques from machine learning such as deep reinforcement learning promise to overcome this challenge. In theory, these paradigms have the ability to learn high-performing controllers directly from real world data, eschewing the need for a first-principles model. However, in current practice they are too data-inefficient and unreliable to see widespread practical deployment. In this talk I will discuss how to fuse these paradigms in a principled way, by embedding design techniques from feedback control into reinforcement learning setups. This approach enables reinforcement learning algorithms to leverage known structures in approximate dynamics models, while maintaining the flexibility to learn from unmodeled dynamics. First, I will discuss principled reward shaping approaches that use feedback design techniques to produce stable controllers. Second, I will discuss how to codesign feedback controllers and policy gradient algorithms to make learning in the real world efficient and reliable. I will show analytically how these solutions lead to inherent robustness guarantees, while empirically reducing the amount of real-world data required by an order of magnitude. Finally I will discuss directions for future work.
– Biographie : Tyler Westenbroek is a postdoctoral scholar at the University of Washington. He completed his Ph.D. in Electrical Engineering and Computer Sciences at UC Berkeley in February 2023, under the supervision of Shankar Sastry. His work aims to develop scalable data-driven tools for controlling complex, high-dimensional robotic systems in the real world. His work leverages techniques from control theory and machine learning, and has appeared in top conferences and journals across robotics, control and machine learning.