Optimization Techniques - W2023- Lecture 11 (Non-Convex Optimization, Sequential Convex Programming)

The course "Optimization Techniques" (ENGG*6140, section 2) at the School of Engineering at the University of Guelph. Instructor: Benyamin Ghojogh Lecture 11 covers non-convex optimization for solving optimization problems in which the cost is a non-convex function and/or the feasibility set of constraints is a non-convex set. We cover both local and global optimization methods where local methods are faster but do not guarantee finding the global optimum solution while global methods are slow but guarantee finding the global optimum solution. The slides are available at: https://bghojogh.github.io/pages/uoguelph/engg-6140-w23/ Very useful optimization courses at Stanford University by Prof. Stephen Boyd: https://www.youtube.com/playlist?list=PL3940DD956CDF0622 Chapters: 0:00 - Explaining the ambiguity in the previous session 3:41 - Where to stop in branch and bound for integer linear programming 11:55 - Non-convex function 15:11 - Non-convex optimization problem 19:47 - Optimization landscape of neural networks is highly non-convex 33:41 - Non-convex optimization categories 37:47 - SCP: convex approximation 43:30 - SCP: convex approximation by Taylor series expansion 48:19 - SCP: convex approximation by particle method 51:48 - SCP: convex approximation by quasi-linearization 56:05 - SCP: formulation of trust region 1:00:54 - SCP: updating trust region 1:22:00 - Branch and bound method 1:41:46 - Important references in non-convex optimization