Optimization in Machine Learning

Data Science, AI and ML
#1

Optimization is at the core of Machine Learning. All what Machine Learning/Deep Learning models do is ‘optimize’ the weights of the loss/cost function in order to achieve the optimal solution. And the optimal point is where Global minima lies.

Optimization comes in when you have words like ‘Min’ or ‘max’ of a function f(x) — the Objective Function or the Cost Function. This Objective Function could define anything with respect to the problem you are optimizing. It could be costs for a company, Losses for another or even revenue etc.

This function will be optimal at a specific point X*. This X* is the optimal point. Hence your optimization problem could be — Find X* for which f(X) is minimum/maximum. This can also be written as argmin(f(x)) — argument where the function f(x) is minimum (or argmax(f(x) conversely).