View source for Lagrange multipliers

{{MSC|49}}
{{TEX|done}}
[[Category:Analysis]]

The ''Lagrange multipliers'' are variables with the help of which one constructs a
[[Lagrange function|Lagrange function]] for investigating problems on ''conditional extrema''. 

'''Definition'''
If $f, g_1, \ldots, g_m: \mathbb R^n \supset \Omega \to \mathbb R$
are given functions, a conditional extremal point of $f$ under the costraints $g_1, \ldots, g_m$ is an element $x^\star$ with the property that $f (x^\star)$ is the maximum (resp. minimum) value taken by $f$ on the set
\begin{equation}\label{e:constrained_set}
\Sigma :=\{y\in U : g_i (y) = g_i (x^\star) =: b_i\quad \forall i \in \{ 1, \ldots , m\}\}\, .
\end{equation}
If instead $f(x)$ is the maximum (resp. minimum) value taken by $f$ on $\Sigma \cap U$ for some neighborhood $U$ of $x^\star$, then $x^\star$ is called a local conditional extremal point.

The method of Lagrange multipliers gives necessary conditions for local conditional extremal points. More precisely we have the following

'''Theorem 1'''
Assume $\Omega$ is an open set and let $f, g_1, \ldots, g_m: \mathbb R^n \supset \Omega \to \mathbb R$ be $C^1$ functions. If $x^\star$ is a local
conditional extremal point under the constraints $g_1, \ldots , g_m$, then the gradients $\nabla f (x^\star), \nabla g_1 (x^\star), \ldots , \nabla g_m (x^\star)$ are linearly dependent. 

The conclusion above is in fact usually stated when $m<n$ and $\nabla g_1 (x^\star), \ldots, \nabla g_m (x^\star)$ are linearly dependent, i.e. when 
$b = (b_1, \ldots , b_i)$ is  a [[Regular value|regular value]] of the function $g = (g_1, \ldots, g_m)$ (at least if we restrict $g$ to some neighborhood $V$ of $x^\star$). The necessary condition of Theorem 1 can then be translated into the identity
\begin{equation}\label{e:lagrange_m}
\nabla f (x^\star) = \lambda^\star_1 \nabla g_1 (x^\star) + \ldots + \lambda^\star_m \nabla g_m (x^\star)\, .
\end{equation}
The Lagrange multipliers are then the real numbers $\lambda_1, \ldots , \lambda_m$ appearing in \eqref{e:lagrange_m}.
Observe that, if we define the [[Lagrange function]]
\begin{equation}\label{e:lagrange_f}
F(x,\lambda) = f(x) + \sum_{i=1}^m\lambda_i (b_i - g_i(x))\, ,
\end{equation}
then the conditions $x^\star\in \Sigma = \{g=b\}$ and \eqref{e:lagrange_m} are equivalent to the fact that $(x^\star, \lambda^\star)$ is a [[Critical point|critical point]] of $F$. We can therefore summarize the discussion above in the following statement, which in fact can be easily seen to be equivalent to Theorem 1:

'''Theorem 2''' 
Assume that $m<n$, that $b$ is a regular value for $g$ and that $x^\star$ is a local conditional extremal point for $f$ under the constraint $g$ with $g (x^\star) = b$. Then there is $\lambda^\star\in \mathbb R^i$ such that $(x^\star, \lambda^\star)$ is a critical point of $F$.

Observe that under the hypothesis of the latter theorem, the set $\Sigma$ is a $C^1$ submanifold of dimension $i$. Indeed the theorem is usually proved via the [[Implicit function|Implicit function theorem]], reducing it to the usual necessary condition for uncostrained minima of a differentiable function.
Observe also that the coordinates of the point $x^\star = (x_1^\star,\dots,x_n^\star)$ together with the Lagrange multipliers $\lambda^\star = (\lambda_1^\star,\dots,\lambda_m^\star)$ give us $m+n$ real numbers 
which satisfy a system of $m+n$ equations: $m$ equations are indeed given by the constraint $g (x^\star) = b$ and $n$ by the identity \eqref{e:lagrange_m}. 

The Lagrange multipliers $\lambda^\star_i$, $i=1,\dots,m$, have the following interpretation
{{Cite|Ha}}. Suppose that $x^\star$ provides a relative extremum of the function $f$ under the constraints $g$ and set $z^\star = f(x^\star)$. The values of $x^\star$, $\lambda^\star$ and $z^\star$ depend on the values of $b$. Under suitable assumptions such dependence is $C^1$  in some $\varepsilon$-neighbourhood of $g (x^\star)$. Under these assumptions the function $z^\star$ is also continuously differentiable with respect to the $b_i$. The partial derivatives of $z^\star$ with respect to the $b_i$ are equal to the corresponding Lagrange multipliers $\lambda_i^*$, calculated for the given $b=(b_1\dots,b_m)$:
\begin{equation}\label{e:costs}
\frac{\partial z^\star}{\partial b_i} = \lambda_i^\star,\quad i=1,\dots,m\, .
\end{equation}
In applied problems $z$ is often interpreted as profit or cost, and the right-hand sides, $b_i$, as losses of certain resources. Then the absolute value of $\lambda_i^\star$ is the ratio of the unit cost to the unit $i$-th resource. The numbers $\lambda_i^\star$ show how the maximum profit (or maximum cost) changes if the amount of the $i$-th resource is increased by one. This interpretation of the Lagrange multipliers is very useful because it can be extended to the case of constraints in the form of inequalities.

In the calculus of variations suitable versions of the method of Lagrange multipliers have been developed in several infinite-dimensional settings, namely when the sought conditional extremal points are functions and both the cost to be minimized and the constraints are suitable functionals. In this case the vector of Lagrange multipliers might itself be infinite dimensional. 

In the theory of optimal control and in the [[Pontryagin maximum principle|Pontryagin maximum principle]], the Lagrange multipliers are usually called conjugate variables.

====Comments====
The same arguments as used above lead to the interpretation of the Lagrange multipliers $\lambda_i^*$ as sensitivity coefficients (with respect to changes in the $b_j$).


====References====
{|
|-
|valign="top"|{{Ref|Bl}}||valign="top"|  G.A. Bliss,   "Lectures on the calculus of variations", Chicago Univ. Press  (1947)  {{MR|0017881}}  {{ZBL|0036.34401}} 
|-
|valign="top"|{{Ref|Br}}||valign="top"|  A.E. Bryson,   Y.-C. Ho,   "Applied optimal control", Blaisdell  (1969)  {{MR|0446628}}  
|-
|valign="top"|{{Ref|Ha}}||valign="top"|  G.F. Hadley,   "Nonlinear and dynamic programming", Addison-Wesley  (1964)  {{MR|0173543}}  {{ZBL|0179.24601}} 
|-
|valign="top"|{{Ref|Ro}}||valign="top"|  R.T. Rockafellar,   "Convex analysis", Princeton Univ. Press  (1970)  {{MR|0274683}}  {{ZBL|0193.18401}} 
|-
|}