Subject Ph. D.

Modeling and simulation of systems are still of critical importance for characterizing the underlying dynamics of time-varying nonlinear system. A recent solution has been to exploit emerging methods from machine learning and deep learning.

To facilitate the fundamental needs of system identification of time varying nonlinear dynamics, physics-informed neural networks (PINN) has emerged as a hybrid deep learning paradigm that imbues machine learning models with physical domain knowledge such as physical laws and constraints emanating from basic Newtonian physics-based laws (Raissi et al. 2019). Within this paradigm, neural networks are trained to solve supervised learning tasks while respecting any given laws of physics described by general nonlinear differential equations (Stiasny et al. 2021).The distinguishing feature of PINN is the synergistic integration of data-driven methods that reflect system dynamics in real-time with the governing physics.

Additionally, Koopman operator based approach for identification of nonlinear dynamics has garnered an extensive attention in the last 5 years (Bevanda et al. 2021). The Koopman operator allows for handling nonlinear systems through a globally linear representation. In general, the operator is infinite- dimensional - necessitating finite approximations - for which there is no overarching framework. Also, Koopman operator theory has long-standing connections to known system-theoretic and dynamical system notions that are not universally recognized. Although there are principled ways of learning such finite approximations, they are in many instances overlooked in favor of, often ill-posed and unstructured methods. When developed in conjunction with Deep neural networks, the so called Deep Koopman operators have shown remarkable results in identifying nonlinear dynamics over finite linear approximations (Lusch et al. 2018; Mauroy and Goncalves 2019).

However, so far, the potential of combining PINNs with Deep Koopman operators for identification of nonlinear dynamical systems has not been explored in a detailed manner. It is believed that leveraging Koopman operator-based approach with PINNs shall result in high-fidelity and reduced models that are well aligned with generally known physics properties as well as "interpretable".

On the other hand, recent advancements in the domain of adaptive dynamic programming and Reinforcement learning (ADP-RL) have led to remarkable results in optimal control design for non-linear systems in the absence of system knowledge (complete or partial) (Kiumarsi et al. 2017). RL is a mature field with well-established mathematical grounds for optimal (sub-optimal) control of non-linear dynamical systems in continuous as well as discrete time where in the optimal control synthesis is largely based upon iterative solution for non-linear Hamilton-Jacobi-bellman equation (HJB) using neural network-based structure (Mu et al. 2016). However, while the existing approaches provide exceptional asymptotic performance, trial-and-error based reinforcement learning suffers from being data inefficient, i.e., requiring an excessive amount of trial data for training. This hampers the training quality as well as leads to semi-global performance. Moreover, there is no insight/explanation about the system model learnt by the RL agents in an implicit manner. Similar is the case with the nonlinear control policy learnt within the RL framework i.e., there is very little or no understanding available about the dynamics of the learnt control policy. These aforementioned factors oblige training with excessive data and training duration (episodes) that are necessarily long.

Very recent efforts have been made by representing the environment dynamics as a linear dynamical system in a high-dimensional space (Han et al. 2020), wherein the Koopman operator theory allows incorporating effective optimal control methods to produce high-quality trials thus accelerating learning (Shi and Meng 2022).

While the first phase of the thesis will target development of PINN enabled Deep Koopman operator-based approaches for nonlinear system identification,

the second phase will investigate the appropriate integration and exploitation of such models within the deep reinforcement learning framework to render

the (sub)optimal control learning process data-efficient and interpretable. Overall, the goal will be to use recent data-driven identification methods and

Koopman theory to build adaptive, real-time models that can be integrated with reinforcement learning control frameworks. The field of application will be robotics.

References

Bevanda, Petar, Stefan Sosnowski, and Sandra Hirche. 2021. "Koopman Operator Dynamical Models: Learning, Analysis and Control." Annual Reviews in Control 52: 197-212.

Kiumarsi, Bahare, Kyriakos G Vamvoudakis, Hamidreza Modares, and Frank L Lewis. 2017. "Optimal and Autonomous Control Using Reinforcement Learning: A Survey." IEEE Transactions on Neural Networks and Learning Systems 29 (6): 2042-62.

Lusch, Bethany, J Nathan Kutz, and Steven L Brunton. 2018. "Deep Learning for Universal Linear Embeddings of Nonlinear Dynamics." Nature Communications 9 (1): 1-10.

Mu, Chaoxu, Zhen Ni, Changyin Sun, and Haibo He. 2016. "Data-Driven Tracking Control with Adaptive Dynamic Programming for a Class of Continuous-Time Nonlinear Systems." IEEE Transactions on Cybernetics 47 (6): 1460-70.

Raissi, Maziar, Paris Perdikaris, and George E Karniadakis. 2019. "Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations." Journal of Computational Physics 378: 686-707.

Shi, Haojie, and Max Q-H Meng. 2022. "Deep Koopman Operator with Control for Nonlinear Systems." IEEE Robotics and Automation Letters 7 (3): 7700-7707.

Stiasny, Jochen, George S Misyris, and Spyros Chatzivasileiadis. 2021. "Physics-Informed Neural Networks for Non-Linear System Identification for Power System Dynamics." In 2021 IEEE Madrid PowerTech, 1-6.