Home Activities Seminars

AI and Natural Sciences
Oct 05 (Wed), 2022
14:00 ~ 16:00
Lee, Jaeyong
Mathematical Sciences at KAIST
[AI]Towards a mathematical understanding of Scientific Machine Learning: Theory, Algorithms, and Applications
Machine learning (ML) has achieved unprecedented empirical success in diverse applications. It now has been applied to solve scientific problems, which has become an emerging field, Scientific Machine Learning (SciML). Many ML techniques, however, are very complex and sophisticated, commonly requiring many trial-and-error and tricks. These result in a lack of robustness and interpretability, which are critical factors for scientific applications. This talk centers around mathematical approaches for SciML, promoting trustworthiness. The first part will present an abstract mathematical framework for physics-informed neural networks (PINNs) - one of the most popular deep learning frameworks for solving PDEs. The framework leverages the stability condition of PDEs and the compatibility of neural networks, providing conditions on which PINNs converge to the solution. The second part is about how to embed physics into neural networks (NNs). I will present a general framework for designing NNs that obey the first and second laws of thermodynamics. The framework not only provides flexible ways of leveraging available physics information but also results in expressive NN architectures. The last, but not least, part is about the training of NNs, one of the biggest challenges in ML. I will present an efficient training method for NNs - Active Neuron Least Squares (ANLS). ANLS is developed from the insight gained from the analysis of gradient descent training. Various numerical examples show that ANLS not only significantly outperforms all other methods, but frequently makes the difference between reaching a good solution of the problem versus not solving the problem at all.