The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects

An implementation for the paper:

The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects

Zhanxing Zhu^*, Jingfeng Wu^*, Bing Yu, Lei Wu, Jinwen Ma

Usage

Two-dimensional experiments

See folder 2Dim.

One hidden layer experiments

See folder OneHiddenLayer.

FashionMNIST

See folder FashionMNIST.

SVHN adn CIFAR-10

See folder SVHN and CIFAR-10.

Note

• Bibtex, Slides, Poster.