Pontryagin's Maximum Principle applies to a particular type of problem called a Bolzano Problem. Most optimization problems can be put into the form of a Bolzano problem, but more about that later.

A Bolzano problem involves a number of state variables which can change over time where time t runs from 0 to T. Let us suppose the state variables are X₁(t), X₂(t), ..., X_n(t). We want to maximize

V(T) = c₁X₁(T) + c₂X₂(T) + ...+c_nX_n(T),

given that we start at the point X₁(0), X₂(0), ..., X_n(0), and where the coefficients c₁, c₂, ..., c_n are given and T is some definite finite time. We are given so-called steering functions for controlling the changes in the state variables; i.e.,

dX₁/dt = f₁(X₁, X₂, .., X_n, u₁, u₂, .., u_m)

dX₂/dt = f₂(X₁, X₂, .., X_n, u₁, u₂, .., u_m)

...........................................................

dX_n/dt = f_n(X₁, X₂, .., X_n, u₁, u₂, .., u_m)

where the variables u₁, u₂, ..., u_m are functions of time and are called the control variables. The objective is to choose the control variables at each instant of time so as to steer the state variables from their initial values

X₁(0), X₂(0), ..., X_n(0)

to some point

X₁(T), X₂(T), ..., X_n(T)

where V(T) = c₁X₁(T) + c₂X₂(T) + ...c_nX_n(T) is maximized.

This seems to be a very difficult task. Pontryagin's Maximum Principle provides a neat, systematic solution.

To implement Pontryagin's method one defines a Hamiltonian function

H = φ₁f₁ + φ₂f₂ + ... + φ_nf_n
= Σφ_if_i,

where the set of adjoint variables φ₁, φ₂, .., φ_n are such that

dφ_j/dt = −∂H/∂X_j

= −Σ_i φ_i(∂f_i/∂X_j)

and φ_i(T)= c_i for i=1, 2, .., n. Note that if H does not depend upon X_j then dφ_j/dt=0 for all t and thus φ_jj would be said to be conserved.

The optimum value of the control variables at time t are the ones that maximizes H.

This usually means that the optimum u_k(t) is such that

∂H/∂u_k(t) = 0
which means

Σ_iφ_i(∂f_i/∂u_k(t)) = 0
for k=1, ..., m.

unless u_k is constrained, in which case the optimal u_k may be at a limit of its range.

An Example of a Problem of
Bolzano in Economics:

Suppose an individual has a noninterest income of y(t) for 0≤t≤T which can either be consumed or saved at an interest rate of r. The individual wants to choose a consumption program c(t) for 0≤t≤T which will maximize utility

U(T) = ∫₀^T ln(c(t))exp(-at)dt.

The financial assets A(t) of the individual are determined by the differential equation:

dA/dt = y(t) + rA(t) - c(t).

There is also a requirement that the financial assets of the individual at the end of his lifespan be nonnegative; i.e.,

A(T)≥0.

This can be considered a problem of Bolzano with X₁=U and X₂=A.

The steering functions are:

dU/dt = ln(c(t))exp(-at)

dA/dt = rA + y(t) - c(t).

The objective function is to maximize U(T) subject to the constraint that A(T)≥0. The constraint can be satisfied by making the objective function

V(T) = U(T) + λA(T)

and choose λ sufficiently large to insure that A(T)≥0.

The Hamiltonian function is

H(t) = φ_Uln(c)exp(-at) + φ_A[rA + y - c],

where the adjoint variables are labled by the name of the corresponding state variable.

The condition for an optimal c(t) is

φ_Uexp(-at)/c(t) - φ_A = 0,
or
c(t) = φ_Uexp(-at)/φ_A.

The adjoint variables are defined by

dφ_U/dt = -∂H/∂U = 0

dφ_A/dt = -∂H/∂A = -φ_Ar.

Since dφ_U/dt = 0 for all t, φ_U = constant. Therefore since φ_U(T)=1, φ_U(t)=1 for all t.

The equation for φ_A implies that

(1/φ_A)(dφ_A)/dt) = -r

so

d(ln[φ_A])/dt = -r

and hence, as a result of integrating from t to T,

ln[φ_A(T)] - ln[φ_A(t)] = -r(T-t).

Since φ(T) = λ,

ln[φ_A(t)] = ln(λ) + r(T-t)

φ_A = λexp(r(T-t)).

Substituting the values for φ_U and φ_A into the condition for an optimal c(t) gives

c(t) = exp(-at)/(λ exp(r(T-t))

or, equivalently,

c(t) = exp(-(a-r)t)/(λ exp(rT)).

When this expression for c(t) is substituted into the differential equation

dA/dt = y(t) + rA(t) - c(t)

and the equation is solved for A(T), a value of λ can be found to make A(T)=0.

An Example of
Finding an Optimal Policy Using
Pontryagin's Maximum Principle

Suppose that when there is no fishing the growth of the fish population in a lake is given by

dP/dt = 0.08P(1-0.000001P),

where P is the number of fish.

This equation indicates that

dP/dt = 0 when (1-0.000001P)=0; i.e., when P = 1, 000, 000.

Suppose that we want to choose a level of consumption of fish C(t) over the period 0 to T which will maximize the utility

U = ∫₀^T exp(-0.03t)ln(C(t))dt.

This can be put into the form of a problem of Bolzano; i.e., maximize U subject to:

dU/dt = exp(-0.03t)ln(C(t))

dP/dt = 0.08P(1-0.000001P)-C(t).

and P(T)≥0.

The constraint P(T)≥0 can be replaced by the requirement that

U(T)+λP(T) be maximized.

The Hamiltonian function is

H = φ_Uexp(-0.03t)ln(C(t)) + φ_P[0.08P(1-0.000001P)-C(t)]

and hence

dφ_U/dt = 0 and

dφ_P/dt = -φ_P(0.08)(1 - 0.000002P).

Since φ_U(T)=1, φ_U(t) = 1 for all t.

The second equation implies that

(1/φ_P)(dφ_P)/dt = -(0.08)(1 - 0.000002P)

or d(ln(φ_PP)/dt = -(0.08)(1 - 0.000002P).

The optimal C(t) is the one that maximizes H(t). This is achieved where

∂H/∂C(t) = exp(-0.03t)/C(t) - φ_P = 0.

Thus the optimal C(t) is given by

C(t) = exp(-0.03t)/φ_P.

The optimal policy is found by solving backwards from t=T the three equations

C(t) = exp(-0.03t)/φ_P

dP/dt = 0.08P(1-0.000001P) - C(t)
with P(T)=0

d(ln(φ_P)/dt = -(0.08)/(1 - 0.000002P)
with φ_P(T) = λ.

An approximate solution is determined using

dP/dt = [P(t)-P(t-h)]/h
so
P(t-h) = P(t) - h(dP/dt).

A value of λ is chosen and the value of P(0) is determined. If this value does not equal the given initial value of P then the value of λ is adjusted.