What automatic differentiation is missing is its ability to differentiate through implicit or iterative solvers. Physics constrained learning (PCL) is the remedy for such a challenge.
See slides here: https://kailaix.github.io/ADCME.jl/dev/assets/Slide/AD.pdf
