Gradient descent uses these ideas to visit each variable in an equation and adjust it to minimize the output of the equation. That’s exactly what we want in training our network. If we think of ...

Function estimation/approximation is viewed from the perspective of numerical optimization in function space, rather than parameter space. A connection is made between stagewise additive expansions ...

Giovanni Colombo, Antonio Marigonda, Peter R. Wolenski, The Clarke Generalized Gradient for Functions Whose Epigraph Has Positive Reach, Mathematics of Operations Research, Vol. 38, No. 3 (August 2013 ...