Pontryagin maximum principle

Relations describing necessary conditions for a strong maximum in a non-classical variational problem in the mathematical theory of optimal control. It was first formulated in 1956 by L.S. Pontryagin .

The proposed formulation of the Pontryagin maximum principle corresponds to the following problem of optimal control. Given a system of ordinary differential equations \begin{equation}\label{eq:1} \dot{x}=f(x,u), \end{equation} where $x\in\mathbb{R}^n$ is a phase vector, $u\in\mathbb{R}^p$ is a control parameter and $f$ is a continuous vector function in the variables $x$, $u$ that is continuously differentiable with respect to $x$. A certain set $U$ of admissible values of the control parameter $u$ in the space $\mathbb{R}^p$ is given; two points $x^0$ and $x^1$ in the phase space $\mathbb{R}^n$ are given; the initial time $t_0$ is fixed. Any piecewise-continuous function $u(t)$, $t_0\leq t\leq t_1$, with values in $U$, is called an admissible control. One says that an admissible control $u=u(t)$ transfers the phase point from the position $x^0$ to the position $x^1$ ($x^0\rightarrow x^1$) if the corresponding solution $x(t)$ of the system \eqref{eq:1} satisfying the initial condition $x(t_0)=x^0$ is defined for all $t\in[t_0,t_1]$ and if $x(t_1)=x^1$. Among all admissible controls transferring the phase point from the position $x^0$ to the position $x^1$ it is required to find an optimal control, i.e. a function $u^*(t)$ for which the functional \begin{equation}\label{eq:2} J=\int\limits_{t_0}^{t_1}f^0(x(t),u(t))dt \end{equation} takes least possible value. Here $f^0(x,u)$ is a given function from the same class as $f(x,u)$, $x(t)$ is the solution of the system \eqref{eq:1} with the initial condition $x(t_0)=x^0$ corresponding to the control $u(t)$, and $t_1$ is the time at which this solution passes through $x^1$. The problem consists of finding a pair consisting of the optimal control $u^*(t)$ and the corresponding optimal trajectory $x^*(t)$ of \eqref{eq:1}.

Let \begin{equation} H(\psi,x,u)=(\psi,\mathbf{f}(x,u)) \end{equation} be a scalar function (Hamiltonian) of the variables $\psi$, $x$, $u$, where $\psi=(\psi_0,\psi^1)\in\mathbb{R}^{n+1}$, $\psi_0\in\mathbb{R}^1$, $\psi^1\in\mathbb{R}^n$, $\mathbf{f}=(f_0,f)$. To the function $H(\psi,x,u)$ corresponds a canonical Hamiltonian system (with respect to $\psi$, $x$) \begin{equation}\label{eq:3} \frac{dx}{dt}=\frac{\partial H}{\partial\psi},\quad\frac{d\psi}{dt}=-\frac{\partial H}{\partial x}, \end{equation} (the first equation in \eqref{eq:3} is the system \eqref{eq:1}). Let \begin{equation} M(\psi,x)=\sup\{H(\psi,x,u)\colon u\in U\}. \end{equation}

The Pontryagin maximum principle states: If $u^*(t)$, $x^*(t)$ ($t\in[t_0,t_1]$) is a solution of the optimal control problem \eqref{eq:1}, \eqref{eq:2} ($x^0\rightarrow x^1$, $u\in U$), then there exists a non-zero absolutely-continuous function $\psi(t)$ such that $\psi(t)$, $x^*(t)$, $u^*(t)$ satisfy system \eqref{eq:3} in $[t_0,t_1]$, such that for almost-all $t\in[t_0,t_1]$ the function $H(\psi(t),x^*(t),u^*(t))$ attains its maximum: \begin{equation}\label{eq:4} H(\psi(t),x^*(t),u^*(t))=M(\psi(t),x^*(t)), \end{equation} and such that at the terminal time $t_1$ the conditions \begin{equation} M(\psi(t_1),x^*(t_1))=0,\quad\psi_0(t_1)\leq 0, \end{equation} are satisfied.

If the functions $\psi(t)$, $x(t)$, $u(t)$ satisfy the relations \eqref{eq:3}, \eqref{eq:4} (i.e. $x(t)$, $u(t)$ are Pontryagin extremal), then the conditions \begin{equation} \mathcal{M}(t)=M(\psi(t),x(t))\equiv const,\quad\psi_0(t)\equiv const, \end{equation} hold.

From the above statement follows the maximum principle for the time-optimal problem ($f^0=1$, $J=t_1-t_0$). This statement admits a natural generalization to non-autonomous systems, problems with variable end-points and problems with restricted phase coordinates ($x(t)\in X$, where $X$ is a closed set in the phase space $\mathbb{R}^n$ satisfying some additional restrictions .

Admitting closed sets $U$, $X$ (in particular, these regions can be determined by systems of non-strict inequalities) makes the problem under consideration non-classical. The fundamental necessary conditions from the classical calculus of variations with ordinary derivative follow from the Pontryagin maximum principle (see and also Weierstrass conditions (for a variational extremum)).

A widely used proof of the above formulation of the Pontryagin maximum principle, based on needle variations (i.e. one considers admissible controls arbitrarily deviating from the optimal one but only on a finite number of small time intervals), consists of linearization of the problem in a neighborhood of the optimal solution, construction of a convex cone of variations of the optimal trajectory, and subsequent application of the theorem on separated convex cones . The corresponding condition is then rewritten in the analytical form \eqref{eq:3}, \eqref{eq:4} in terms of the maximum of the Hamiltonian $H(\psi,x,u)$ of the phase variables $x$, the controls $u$ and the adjoint variables $\psi$, which play the same role as the Lagrange multipliers in the classical calculus of variations. Effective application of the Pontryagin maximum principle often necessitates the solution of a two-point boundary value problem for \eqref{eq:3}.

The most complete solution of the problem of optimal control was obtained in the case of certain linear systems, for which the relations in the Pontryagin maximum principle are not only necessary but also sufficient optimality conditions.

There are numerous generalizations of the Pontryagin maximum principle; for instance, in the direction of more complicated non-classical constraints (including mixed constraints imposed on the controls and phase coordinates, functional and different integral constraints), in studies of the sufficiency of the corresponding constraints, in the consideration of generalized solutions, so-called sliding regimes, systems of differential equations with non-smooth right-hand side, differential inclusions, optimal control problems for discrete systems and systems with an infinite number of degrees of freedom, in particular, described by partial differential equations, equations with an after effect (including equations with a delay), evolution equations in a Banach space, etc. The latter lead to new classes of variations of the corresponding functionals, the introduction of the so-called integral maximum principle, the linearized maximum principle, etc. Rather general classes of variational problems with non-classical constraints (including non-strict inequalities) or with non-smooth functionals are usually called problems of Pontryagin type. The discovery of the Pontryagin maximum principle initiated the development of mathematical optimal control theory. It stimulated new research in the field of differential equations, functional analysis and extremal problems, computational mathematics and other related domains.

Comments

In the Western literature the Pontryagin maximum principle is also simply known as the minimum principle. (Cf. Optimal control, mathematical theory of)

References

2. W.H. Fleming, R.W. Rishel, "Deterministic and stochastic optimal control" , Springer (1975)
3. L. Markus, "Foundations of optimal control theory" , Wiley (1967)
4. L.D. Berkovitz, "Optimal control theory" , Springer (1974)
5. L. Cesari, "Optimization - Theory and applications" , Springer (1983)
6. F. Clarke, "Optimization and nonsmooth analysis" , Wiley (1983)