Date:
Wed, 03/04/201912:00-13:30
Location:
Danciger B building, Seminar room
Speaker: Guy Gur Ari from google
Abstract:
The large width limit has emerged as a useful tool for studying deep neural networks. We adapt the 't Hooft expansion to this case, allowing us to easily work out scaling laws in large width networks using Feynman diagrams. We apply these techniques to quantities involving deep network observables such as the loss function, the Hessian, and the Neural Tangent Kernel.
Abstract:
The large width limit has emerged as a useful tool for studying deep neural networks. We adapt the 't Hooft expansion to this case, allowing us to easily work out scaling laws in large width networks using Feynman diagrams. We apply these techniques to quantities involving deep network observables such as the loss function, the Hessian, and the Neural Tangent Kernel.