Caltech Logo

CMX Lunch Seminar

Wednesday, December 8, 2021
12:00pm to 1:00pm
Add to Cal
Online Event
On neural network approximations for partial differential equations and convergence analyses for gradient descent optimization methods
Arnulf Jentzen, Presidential Chair Professor / Professor of Applied Mathematics, Mathematics and Computer Science, The Chinese University of Hong Kong, Shenzhen / University of M√ľnster,

In the first part of this talk we show that artificial neural networks (ANNs) with rectified linear unit (ReLU) activation have the fundamental capacity to overcome the curse of dimensionality in the numerical approximation of semilinear heat partial differential equations with Lipschitz continuous nonlinearities. In the second part of this talk we present recent convergence analysis results for gradient descent (GD) optimization methods in the training of ANNs with ReLU activation. Despite the great success of GD type optimization methods in numerical simulations for the training of ANNs with ReLU activation, it remains -- even in the simplest situation of the plain vanilla GD optimization method with random initializations -- an open problem to prove (or disprove) the conjecture that the risk of the GD optimization method converges in the training of ANNs with ReLU activation to zero as the width/depth of the ANNs, the number of independent random initializations, and the number of GD steps increase to infinity. In the second part of this talk we, in particular, present the affirmative answer of this conjecture in the special situation where the probability distribution of the input data is absolutely continuous with respect to the continuous uniform distribution on a compact interval and where the target function under consideration is piecewise linear.

For more information, please contact Jolene Brink by phone at (626)395-2813 or by email at jbrink@caltech.edu or visit CMX Website.