【Convex Optimization】Convex Optimization Basics

【Convex Optimization Basics - YouTube】
https://www.youtube.com/watch?v=oLowhs83aHk

【Convex Sets】Some properties:

1. The empty set ? and  ?d are both convex.
2. Preserved by scaling and translation.
3. Intersections of convex sets are convex.

【Convex Functions】

Some properties:

1. Any local minimum is a global minimum.
2. Where it exists, the Hessian is positive semi-definite.
3. Level sets are convex.
4. a·f(x) + b·g(x) is convex for convex f,g and a,b > 0.
5. max(f(x), g(x)) is convex for convex f(x) and g(x).

【Convex Optimization Terminology】

1. optimization variable
2. objective / cost function
3. inequality constraints
4. equality constraints
5. feasible
6. optimal value
7. optimal point
8. active
9. inactive

【Why Convex Optimization?】

1. Contains various types of problems, e.g., many ML and OR tasks.
2. Repeatability: different runs give the same results.
3. Some convex problems can be solved in polynomial time

• However, lots of important problems aren‘t convex: neural networks, k-means, most Bayesian inference.

【Duality】

The max-min inequality: the max of the minima ≤ the min of the maxima

【Convex Optimization】Convex Optimization Basics

原文：https://www.cnblogs.com/harmanchen/p/15259806.html