San José State University
Department of Economics

applet-magic.com
Thayer Watkins
Silicon Valley
& Tornado Alley
USA

Lev Pontryagin

Pontryagin's Maximum Principle

Pontryagin's Maximum Principle applies to a particular type of problem called a Bolzano Problem. Most optimization problems can be put into the form of a Bolzano problem, but more about that later.

A Bolzano problem involves a number of state variables which can change over time where time t runs from 0 to T. Let us suppose the state variables are X1(t),X2(t),...,Xn(t). We want to maximize


V(T) = c1X1(T) + c2X2(T) + ...+cnXn(T),
 

given that we start at the point X1(0), X2(0),..., Xn(0), and where the coefficients c1, c2, ...,cn are given and T is some definite finite time. We are given so-called steering functions for controlling the changes in the state variables; i.e.,


dX1/dt = f1(X1,X2,.., Xn,u1,u2,..,um)

dX2/dt = f2(X1,X2,.., Xn,u1,u2,..,um)

...........................................................

dXn/dt = fn(X1,X2,.., Xn,u1,u2,..,um)

 

where the variables u1,u2,...,um are functions of time and are called the control variables. The objective is to choose the control variables at each instant of time so as to steer the state variables from their initial values


X1(0), X2(0),...,Xn(0)
 

to some point


X1(T), X2(T),...,Xn(T)
 

where V(T) = c1X1(T) + c2X2(T) + ...cnXn(T) is maximized.

This seems to be a very difficult task. Pontryagin's Maximum Principle provides a neat, systematic solution.

To implement Pontryagin's method one defines a Hamiltonian function


H = φ1f1 + φ2f2 + ... + φnfn
= Σφifi,
 

where the set of adjoint variables φ12,..,φn are such that


i/dt = −∂H/∂Xi

= −Σi φj(∂fi/∂Xi)
 

and φi(T)= ci for i=1,2,..,n. Note that if H does not depend upon Xi then dφi/dt=0 for all t and thus φ(t) is a constant. In physics φi would be said to be conserved.

The optimum value of the control variables at time t are the ones that maximizes H.

This usually means that the optimum uj(t) is such that


∂H/∂uj(t) = 0
which means

ΣiXi(∂fi/∂uj(t)) = 0
for j=1,...,m.
 

An Example of a Problem of Bolzano in economics:

Suppose an individual has a noninterest income of y(t) for 0≤t≤T which can either be consumed or saved at an interest rate of r. The individual wants to choose a consumption program c(t) for 0≤t≤T which will maximize utility


U(T) = ∫0T ln(c(t))exp(-at)dt.
 

The financial assets A(t) of the individual are determined by the differential equation:


dA/dt = y(t) + rA(t) - c(t).
 

There is also a requirement that the financial assets of the individual at the end of his lifespan be nonnegative; i.e.,


A(T)≥0.
 

This can be considered a problem of Bolzano with X1=U and X2=A.

The steering functions are:


dU/dt = ln(c(t))exp(-at)

dA/dt = rA + y(t) - c(t).
 

The objective function is to maximize U(T) subject to the constraint that A(T)≥0. The constraint can be satisfied by making the objective function


V(T) = U(T) + λA(T)
 

and choose λ sufficiently large to insure that A(T)≥0.

The Hamiltonian function is


H(t) = φUln(c)exp(-at) + φA[rA + y - c],
 

where the adjoint variables are labled by the name of the corresponding state variable.

The condition for an optimal c(t) is


φUexp(-at)/c(t) - φA = 0,
or
c(t) = φUexp(-at)/φA.
 

The adjoint variables are defined by


U/dt = -∂H/∂U = 0

A/dt = -∂H/∂A = -φAr.
 

Since dφU/dt = 0 for all t, φU = constant. Therefore since φU(T)=1, φU(t)=1 for all t.

The equation for φA implies that


(1/φA)(dφA)/dt) = -r
 

so


d(ln[φA])/dt = -r
 

and hence, as a result of integrating from t to T,


ln[φA(T)] - ln[φA(t)] = -r(T-t).
 

Since φ(T) = λ,


ln[φA(t)] = ln(λ) + r(T-t)

φA = λexp(r(T-t)).
 

Substituting the values for φU and φA into the condition for an optimal c(t) gives


c(t) = exp(-at)/(λ exp(r(T-t))

or, equivalently,

c(t) = exp(-(a-r)t)/(λ exp(rT)).
 

When this expression for c(t) is substituted into the differential equation

dA/dt = y(t) + rA(t) - c(t)
 

and the equation is solved for A(T), a value of λ can be found to make A(T)=0.

An Example of Finding an Optimal Policy
Using Pontryagin's Maximum Principle

Suppose that when there is no fishing the growth of the fish population in a lake is given by


dP/dt = 0.08P(1-0.000001P),
 

where P is the number of fish.

This equation indicates that

dP/dt = 0 when (1-0.000001P)=0; i.e., when P = 1,000,000.

Suppose that we want to choose a level of consumption of fish C(t) over the period 0 to T which will maximize the utility


U = ∫0T exp(-0.03t)ln(C(t))dt.
 

This can be put into the form of a problem of Bolzano; i.e., maximize U subject to:


dU/dt = exp(-0.03t)ln(C(t))

dP/dt = 0.08P(1-0.000001P)-C(t).
 

and P(T)≥0.

The constraint P(T)≥0 can be replaced by the requirement that

U(T)+λP(T) be maximized.

The Hamiltonian function is


H = φUexp(-0.03t)ln(C(t)) + φP[0.08P(1-0.000001P)-C(t)]
 

and hence


U/dt = 0 and

P/dt = -φP(0.08)(1 - 0.000002P).
 

Since φU(T)=1, φU(t) = 1 for all t.

The second equation implies that


(1/φP)(dφP)/dt = -(0.08)(1 - 0.000002P)

or d(ln(φPP)/dt = -(0.08)(1 - 0.000002P).
 

The optimal C(t) is the one that maximizes H(t). This is achieved where


∂H/∂C(t) = exp(-0.03t)/C(t) - φP = 0.
 

Thus the optimal C(t) is given by


C(t) = exp(-0.03t)/φP.
 

The optimal policy is found by solving backwards from t=T the three equations


C(t) = exp(-0.03t)/φP

dP/dt = 0.08P(1-0.000001P) - C(t)
with P(T)=0

d(ln(φP)/dt = -(0.08)/(1 - 0.000002P)
with φP(T) = λ.
 

An approximate solution is determined using


dP/dt = [P(t)-P(t-h)]/h
so
P(t-h) = P(t) - h(dP/dt).
 

A value of λ is chosen and the value of P(0) is determined. If this value does not equal the given initial value of P then the value of λ is adjusted.


HOME PAGE OF applet-magic
HOME PAGE OF Thayer Watkins