View source for Pontryagin's maximum principle

'''Pontryagin's maximum principle''' is used in [[optimal control]] theory to find the best possible control for taking a [[dynamical system]] from one state to another, especially in the presence of constraints for the state or input controls. It states that it is [[Necessity and sufficiency|necessary]] for any optimal control along with the optimal state trajectory to solve the so-called Hamiltonian system, which is a two-point [[boundary value problem]], plus a maximum condition of the [[Hamiltonian (control theory)|Hamiltonian]].{{efn|Whether the extreme value is maximum or minimum depends both on the problem and on the sign convention used for defining the Hamiltonian. The normal convention leads to a maximum hence ''maximum principle.''}} These necessary conditions become sufficient under certain convexity conditions on the objective and constraint functions.<ref>{{cite journal |journal=SIAM Journal on Control |volume=4 |year=1966 |issue=1 |pages=139–152 |title=Sufficient Conditions for the Optimal Control of Nonlinear Systems |first=O. L. |last=Mangasarian |authorlink=Olvi L. Mangasarian |doi=10.1137/0304013 }}</ref><ref>{{cite journal |first=Morton I. |last=Kamien |authorlink=Morton Kamien |first2=Nancy L. |last2=Schwartz |title=Sufficient Conditions in Optimal Control Theory |journal=[[Journal of Economic Theory]] |volume=3 |issue=2 |year=1971 |pages=207–214 |doi=10.1016/0022-0531(71)90018-4 }}</ref>

The maximum principle was formulated in 1956 by the Russian mathematician [[Lev Pontryagin]] and his students,<ref>{{cite book |first=V. |last=Boltyanski |first2=H. |last2=Martini |first3=V. |last3=Soltan |title=Geometric Methods and Optimization Problems |location=New York |publisher=Springer |year=1998 |isbn=0-7923-5454-0 |pages=204–227 |chapter=The Maximum Principle – How it came to be? |chapterurl=https://books.google.com/books?id=YD7UBwAAQBAJ&pg=PA204 }}</ref><ref>{{cite journal |first=R. V. |last=Gamkrelidze |title=Discovery of the Maximum Principle |journal=Journal of Dynamical and Control Systems |year=1999 |volume=5 |issue=4 |pages=437–451 |doi=10.1023/A:1021783020548 |postscript= }} Reprinted in {{cite book |editor-first=A. A. |editor-last=Bolibruch |editor-link=Andrei Bolibrukh |editor2-first=Yu. S. |editor2-last=Osipov |editor3-first=Ya. G. |editor3-last=Sinai |displayeditors=1 |title=Mathematical Events of the Twentieth Century |location=Berlin |publisher=Springer |year=2006 |isbn=3-540-23235-4 |pages=85–99 }}</ref> and its initial application was to the maximization of the terminal speed of a rocket.<ref>For first published works, see references in {{cite journal |last=Fuller |first=A. T. |title=Bibliography of Pontryagin's Maximum Principle |journal=J. Electronics & Control |volume=15 |issue=5 |year=1963 |pages=513–517 |doi=10.1080/00207216308937602 }}</ref> The result was derived using ideas from the classical [[calculus of variations]].<ref>{{cite journal |first=E. J. |last=McShane |authorlink=Edward J. McShane |year=1989 |title=The Calculus of Variations from the Beginning Through Optimal Control Theory |journal=SIAM J. Control Optim. |volume=27 |issue=5 |pages=916–939 |doi=10.1137/0327049 }}</ref> After a slight [[Perturbation function|perturbation]] of the optimal control, one considers the first-order term of a Taylor expansion with respect to the perturbation; sending the perturbation to zero leads to a variational inequality from which the maximum principle follows.<ref name="YongZhou">{{cite book |last1=Yong |first1=J. |last2=Zhou |first2=X. Y. |title=Stochastic Controls: Hamiltonian Systems and HJB Equations |chapter=Maximum Principle and Stochastic Hamiltonian Systems |date=1999 |publisher=Springer |location=New York |isbn=0-387-98723-1 |pages=101–156 }}</ref>

Widely regarded as a milestone in optimal control theory, the significance of the maximum principle lies in the fact that maximizing the Hamiltonian is much easier than the original infinite-dimensional control problem; rather than maximizing over a [[function space]], the problem is converted to a [[pointwise]] optimization.<ref>{{cite web |first=Shankar |last=Sastry |title=Lecture Notes 8. Optimal Control and Dynamic Games |date=March 29, 2009 |work= |url=https://inst.eecs.berkeley.edu/~ee291e/sp09/handouts/09-lecture8-rev.pdf }}</ref> A similar logic leads to [[Bellman's principle of optimality]], a related approach to optimal control problems which states that the optimal trajectory remains optimal at intermediate points in time.<ref>{{cite journal |first=X. Y. |last=Zhou |title=Maximum Principle, Dynamic Programming, and their Connection in Deterministic Control |journal=Journal of Optimization Theory and Applications |year=1990 |volume=65 |issue=2 |pages=363–373 |doi=10.1007/BF01102352 }}</ref> The resulting [[Hamilton–Jacobi–Bellman equation]] provides a necessary and sufficient condition for an optimum, and admits [[Hamilton–Jacobi–Bellman equation#Extension to stochastic problems|a straightforward extension]] to stochastic optimal control problems, whereas the maximum principle does not.<ref name="YongZhou" />

==Notation==
In what follows we will be making use of the following notation.
:<math>
\Psi_T(x(T))= \frac{\partial \Psi(x)}{\partial T}|_{x=x(T)} \,
</math>

:<math>
\Psi_x(x(T))=\begin{bmatrix} \frac{\partial
\Psi(x)}{\partial x_1}|_{x=x(T)} & \cdots & \frac{\partial
\Psi(x)}{\partial x_n} |_{x=x(T)}
\end{bmatrix}
</math>

:<math>
H_x(x^*,u^*,\lambda^*,t)=\begin{bmatrix} \frac{\partial H}{\partial x_1}|_{x=x^*,u=u^*,\lambda=\lambda^*}
& \cdots & \frac{\partial H}{\partial x_n}|_{x=x^*,u=u^*,\lambda=\lambda^*}
\end{bmatrix}
</math>

:<math>
L_x(x^*,u^*)=\begin{bmatrix} \frac{\partial L}{\partial x_1}|_{x=x^*,u=u^*}
& \cdots & \frac{\partial L}{\partial x_n}|_{x=x^*,u=u^*}
\end{bmatrix}
</math>

:<math>
f_x(x^*,u^*)=\begin{bmatrix} \frac{\partial f_1}{\partial x_1}|_{x=x^*,u=u^*} & \cdots & \frac{\partial f_1}{\partial x_n}|_{x=x^*,u=u^*} \\
\vdots & \ddots & \vdots \\ \frac{\partial f_n}{\partial x_1}|_{x=x^*,u=u^*} &
\ldots & \frac{\partial f_n}{\partial x_n}|_{x=x^*,u=u^*}
\end{bmatrix}
</math>

==Formal statement of necessary conditions for minimization problem==
Here the necessary conditions are shown for minimization of a functional. Take <math>x</math> to be the state of the [[dynamical system]] with input <math>u</math>, such that
:<math>
\dot{x}=f(x,u), \quad x(0)=x_0, \quad u(t) \in \mathcal{U}, \quad t \in
[0,T]
</math>
where <math>\mathcal{U}</math> is the set of admissible controls and <math>T</math> is the terminal (i.e., final) time of the system. The control <math>u \in \mathcal{U}</math> must be chosen for all <math>t \in [0,T]</math> to minimize the objective functional <math>J</math> which is defined by the application and can be abstracted as

:<math>
J=\Psi(x(T))+\int^T_0 L(x(t),u(t)) \,dt
</math>

The constraints on the system dynamics can be adjoined to the [[Lagrangian mechanics|Lagrangian]] <math>L</math> by introducing time-varying [[Lagrange multipliers|Lagrange multiplier]] vector <math>\lambda</math>, whose elements are called the costates of the system. This motivates the construction of the [[Hamiltonian (control theory)|Hamiltonian]] <math>H</math> defined for all <math>t \in [0,T]</math> by:
:<math>
H(x(t),u(t),\lambda(t),t)=\lambda^{\rm T}(t)f(x(t),u(t))+L(x(t),u(t)) \,
</math>
where <math>\lambda^{\rm T}</math> is the transpose of <math>\lambda</math>.

Pontryagin's minimum principle states that the optimal state trajectory <math>x^*</math>, optimal control <math>u^*</math>, and corresponding Lagrange multiplier vector <math>\lambda^*</math> must  minimize the Hamiltonian <math>H</math> so that

:<math>
(1) \qquad H(x^*(t),u^*(t),\lambda^*(t),t)\leq H(x^*(t),u,\lambda^*(t),t) \,
</math>

for all time <math>t \in [0,T]</math> and for all permissible control inputs <math>u \in \mathcal{U}</math>. It must also be the case that

:<math>
(2) \qquad \Psi_T(x(T))+H(T)=0 \,
</math>

Additionally, the [[costate equations]]

:<math>
(3) \qquad -\dot{\lambda}^{\rm T}(t)=H_x(x^*(t),u^*(t),\lambda(t),t)=\lambda^{\rm T}(t)f_x(x^*(t),u^*(t))+L_x(x^*(t),u^*(t))
</math>

must be satisfied. If the final state <math>x(T)</math> is not fixed (i.e., its differential variation is not zero), it must also be that the terminal costates are such that

:<math>
(4) \qquad \lambda^{\rm T}(T)=\Psi_x(x(T)) \,
</math>

These four conditions in (1)-(4) are the necessary conditions for an optimal control. Note that (4) only applies when <math>x(T)</math> is free. If it is fixed, then this condition is not necessary for an optimum.

==See also==
* [[Lagrange multipliers on Banach spaces]], Lagrangian method in calculus of variations

==Notes==
{{notelist}}

==References==
{{Reflist}}

==Further reading==
* {{cite book |last=Geering |first=H. P. |title=Optimal Control with Engineering Applications |location= |publisher=Springer |year=2007 |isbn=978-3-540-69437-3 }}
* {{cite book |last=Kirk |first=D. E. |title=Optimal Control Theory: An Introduction |location= |publisher=Prentice Hall |year=1970 |isbn=0-486-43484-2 }}
* {{cite book |last=Lee |first=E. B. |first2=L. |last2=Markus |title=Foundations of Optimal Control Theory |url=https://archive.org/details/foundationsofopt0000leee |url-access=registration |location=New York |publisher=Wiley |year=1967 }}
* {{cite book |first=Atle |last=Seierstad |first2=Knut |last2=Sydsæter |authorlink2=Knut Sydsæter |title=Optimal Control Theory with Economic Applications |location=Amsterdam |publisher=North-Holland |year=1987 |isbn=0-444-87923-4 }}

==External links==
* {{springer|title=Pontryagin maximum principle|id=p/p073780}}

{{DEFAULTSORT:Pontryagin's Minimum Principle}}
[[Category:Optimal control]]
[[Category:Principles]]

[[fr:Commande optimale#Principe du maximum]]
[[ru:Оптимальное управление#Принцип максимума Понтрягина]]