Functional Gradient Descent with Adaptive Representations [R]

Excerpt

Sharing our recent work, now accepted at NeurIPS: Functional Gradient Descent with Adaptive Representations . Functional GD algorithms generally outperform neural nets, but are hard to accurately implement. This is because functional gradients are infinite-dimensional, and therefore must be approximated in practice; but if you approximate them naively, you converge to the wrong place! To rectify this, we formalize a broad class of approximation schemes ("adaptive representations"), which provabl