[b_tterc_p]

Edit: I think I answered a different question than I believed.

Been a while but my best crack at it: If you’re trying to minimize a function, you can call it the primal function. It will have n inputs and m constraints (like how many of each product should I buy constrained by budget and carrying capacity). You can flip the problem around into its dual formulation. This will be a function with m inputs and n constraints, and it will be a maximization problem.

If you can solve the dual formulation (global max), you know that the primal cannot possibly be lower than the dual. For some types of problems (convex), you can even guarantee that the global max of the dual is the global min of the primal.

The transformation between primal and dual formulations goes both ways, so the primal is the dual of the dual.

[bitL]

I'd add a very important thing - with your primal problem you are trying to e.g. minimize a function and you proceed the way that all your intermediate solutions are feasible. With dual approach you flip the direction, i.e. maximization in this case, but you start in completely infeasible solutions and hope to end up in the very first feasible solution that should be your optimum.

Now how does that relate to automatic differentiation I am not sure either.