Understanding scipy.minimize part 1: The BFGS algorithm

A description of how quasi Newton algorithms in general, and in special the BFGS algorithm work. Animations are made with the manimce library. Sources: * Nocedal & Wright: Numerical Optimization Ch. 6 (which also presents a SR1 trust region method) * Dennis & More: Quasi-Newton Methods, Motivation, and Theory, SIAM Review, Vol. 19, No. 1, 1977 (describing the PSB method. The video is based mostly on the derivation in section 7 of this paper.) The actual update formulas of BFGS are not included in the video. These can be found in both sources as well as e.g. Wikipedia.