Why Neural Networks converge to “simple” solutions?

Abstract:

Since 2012, deep neural networks are having an impressive practical success in many domains, yet their theoretical properties are not well understood. I will discuss why does neural network optimization, which based on local greedy steps, tend to converge to:

1) A global minimum, while many local minima exist.

2) A specific “good” global minimum in which the network function is surprisingly “simple” (while many “bad” global minima exist).

Name	Daniel Soudry
Date	10-11-19
Faculty	EE
Title	Why Neural Networks converge to “simple” solutions?
Web page	https://sites.google.com/site/danielsoudry/
Email	daniel.soudry@gmail.com
Study materials	About the deep learning era