CMX Lunch Seminar

In the first part of this talk we show that artificial neural networks (ANNs) with rectified linear unit (ReLU) activation have the fundamental capacity to overcome the curse of dimensionality in the numerical approximation of semilinear heat partial differential equations with Lipschitz continuous nonlinearities. In the second part of this talk we present recent convergence analysis results for gradient descent (GD) optimization methods in the training of ANNs with ReLU activation. Despite the great success of GD type optimization methods in numerical simulations for the training of ANNs with ReLU activation, it remains -- even in the simplest situation of the plain vanilla GD optimization method with random initializations -- an open problem to prove (or disprove) the conjecture that the risk of the GD optimization method converges in the training of ANNs with ReLU activation to zero as the width/depth of the ANNs, the number of independent random initializations, and the number of GD steps increase to infinity. In the second part of this talk we, in particular, present the affirmative answer of this conjecture in the special situation where the probability distribution of the input data is absolutely continuous with respect to the continuous uniform distribution on a compact interval and where the target function under consideration is piecewise linear.