Lagrange multipliers

2020 Mathematics Subject Classification: Primary: 49-XX [MSN][ZBL]

Lagrange multipilers are variables with the help of which one constructs a Lagrange function for investigating problems on conditional extrema. The use of Lagrange multipliers and a Lagrange function makes it possible to obtain in a uniform way necessary optimality conditions in problems on conditional extrema. The method of obtaining necessary conditions in the problem of determining an extremum of a function

$$f(x_1,\dots,x_n)\tag{1}$$ under the constraints

$$g_i(x_1,\dots,x_n) = b_i,\quad i=1,\dots,m,\quad m<n,\tag{2}$$ consisting of the use of Lagrange multipliers $\def\l{\lambda}\l_i$, $i=1,\dots,m$, the construction of the Lagrange function

$$F(x,\l) = f(x) + \sum_{i=1}^m\l_i ( b_i - g_i(x) )$$ and equating its partial derivatives with respect to the $x_i$ and $\l_i$ to zero, is called the Lagrange method. In this method the optimal value $x^* = (x_1^*,\dots,x_n^*)$ is found together with the vector of Lagrange multipliers $\l^* = (\l_1^*,\dots,\l_m^*)$ corresponding to it by solving the system of $m + n$ equations. The Lagrange multipliers $\l^*_i$, $i=1,\dots,m$, have the following interpretation [Ha]. Suppose that $x^*$ provides a relative extremum of the function (1) under the constraints (2): $z^* = f(x^*)$. The values of $x^*$, $\l^*$ and $z^*$ depend on the values of the $b_i$, the right-hand sides of the constraints (2). One has formulated quite general assumptions under which all the $x_j^*$ and $\l_i^*$ are continuously-differentiable functions of the vector $b=(b_1\dots,b_m)$ in some $\varepsilon$-neighbourhood of its value specified in (2). Under these assumptions the function $z^*$ is also continuously differentiable with respect to the $b_i$. The partial derivatives of $z^*$ with respect to the $b_i$ are equal to the corresponding Lagrange multipliers $\l_i^*$, calculated for the given $b=(b_1\dots,b_m)$:

$$\frac{\partial z^*}{\partial b_i} = \l_i^*,\quad i=1,\dots,m.\tag{3}$$ In applied problems $z$ is often interpreted as profit or cost, and the right-hand sides, $b_i$, as losses of certain resources. Then the absolute value of $\l_i^*$ is the ratio of the unit cost to the unit $i$-th resource. The numbers $\l_i^*$ show how the maximum profit (or maximum cost) changes if the amount of the $i$-th resource is increased by one. This interpretation of Lagrange multipliers can be extended to the case of constraints in the form of inequalities and to the case when the variables $x_j$ are subject to the requirement of being non-negative.

In the calculus of variations one conveniently obtains by means of Lagrange multipliers necessary conditions for optimality in the problem on a conditional extremum as necessary conditions for an unconditional extremum of a certain composite functional. Lagrange multipliers in the calculus of variations are not constants, but certain functions. In the theory of optimal control and in the Pontryagin maximum principle, Lagrange multipliers have been called conjugate variables.

Comments

The same arguments as used above lead to the interpretation of the Lagrange multiplier values $\l_i^*$ as sensitivity coefficients (with respect to changes in the $b_j$).

References

[Bl]	G.A. Bliss, "Lectures on the calculus of variations", Chicago Univ. Press (1947) MR0017881 Zbl 0036.34401
[Br]	A.E. Bryson, Y.-C. Ho, "Applied optimal control", Blaisdell (1969) MR0446628
[Ha]	G.F. Hadley, "Nonlinear and dynamic programming", Addison-Wesley (1964) MR0173543 Zbl 0179.24601
[Ro]	R.T. Rockafellar, "Convex analysis", Princeton Univ. Press (1970) MR0274683 Zbl 0193.18401