Gradient-Descent

I Spent Years on Adaptive Filters. I Was Already Training Neural Networks.

I spent years implementing LMS-based equalizers and echo cancellers in telecommunications. Only later did I fully appreciate what I had been doing mathematically: the same family of update rules that …