Centers & Programs

Seminars

Home Centers & Programs AI and Natural Sciences Seminars

FIELD
AI and Natural Sciences
DATE
Nov 23 (Wed), 2022
TIME
14:00 ~ 16:00
PLACE
7323
SPEAKER
Park, Yeachan
HOST
Lee, Sungyoon
INSTITUTE
AI기초과학센터
TITLE
[AI] Does a local minimum exist in a neural network with arbitrary activation functions?
ABSTRACT
Understanding the loss surface of neural networks is essential to understanding deep learning. However, the existence of bad local minima has not yet been fully confirmed. We look a brief history of searching for a bad local minimum. Then, we investigate the existence of bad local minima in 2-layer and 3-layer neural networks with general smooth activation functions. We use the algebraic nature of the activation function to provide constructive evidence. This shows for the realistic setup where the data $(X,Y)$ has a positive measure. It is hoped that these results will provide a theoretical basis for studies related to local minima and loss surfaces.
FILE