applet-magic.com
Thayer Watkins
Silicon Valley
& Tornado Alley
USA

Second Order Conditions
for an Unconstrained
Maximum or Minimum

The familiar second order condition for a relative maximum of a univariate function f(x) at the critical point x=x0 where f'(x)=0 is that the second derivative at that point must be negative;


f"(x) < 0.
 

Likewise the second order condition for a relative minimum is usually stated to be that the second derivative at a critical point must be positive. As will be shown below, this is an oversimplification. But let us consider the generalization of these conditions to multivariate functions.

The second order conditions for a relative maximum of a multivariate function f(x1, x2, ...xn) is most conveniently stated in terms of the properties of the matrix of second derivatives,


S = (∂2f/∂xj∂xi) = fi,j
 

At this point some new terminology must be introduced. A minor of a matrix is the determinant of a submatrix formed by deleting some rows and columns of a matrix. The n-th principal minors of a matrix are the ones formed by deleting all rows and columns except the first n rows and columns. Thus the first principal minor is the determinant of the submatrix consisting of what is left when all but the first row and column are deleted. This is just m1,1 for a matrix M = (mi,j). The n-th principal minor for an nxn matrix is just the determinant of that matrix.

The condition for a relative maximum at a critical point is that the matrix S must negative definite. This will prevail if the principal minors of S alternate in sign., starting with negative values for the first principal minor.

For a minimum the condition is that the matrix S must be positive definite and this will prevail if the principal minors are all positive.

For a bivariate function f(x,y) this means that for for a relative maximum at a critical point it must be that


fxx < 0
and
fxxfyy - fxyfyx > 0.
 

Since fxy = fyx this latter condition is usually stated as


fxxfyy > fxy2.
 

The condition that fyy have the same sign as fxx can be deduced from this condition; i.e. that the product of fxx and fyy must be positive.

The conditions for a relative minimum is that


fxx > 0
and
fxxfyy - fxy2 > 0.
 

Caution: The condition that


fxxfyy > fxy2
 

is a required condition for having either a relative maximum or a relative minimum. Unless this condition prevails it does not matter what the signs of fxx or fyy are.

Refinements

In the previous analysis relative maximum meant that the value of the function at a critical point was strictly greater than the values at nearby points. If relative maximum were taken to mean that the value of the function at the critical point is greater than or equal to the nearby values then the condition on the matrix S is that it is negative semidefinite. Likewise for a relative minimum that is only less than or equal to nearby values the condition on S is that it must positive semidefinite.

Let us go back now to the simple case of a univariate function f(x). What if, at the critical point, f"(x) = 0? Does this mean that the critical point is a point of inflection. Not necessarily; the critical point could still be a relative maximum. In order to tell we would need to look at the next derivatives of the function at the critical point. If the second derivative is going from negative values to positive values (which corresponds to the third derivative, f'"(x) being positive at the point), then is critical point is a point of inflection. It is also a point of inflection is the second derivative is changing from positive values to negative values (the third derivative being negative). But if the second derivative is negative and the goes to zero at the critical point and then becomes negative again (corresponding to the third derivative also being zero at the critical point) then the critical point is a relative maximum. If the second derivative is positive at points near the critical point and zero at the critical point (corresponding to a zero third derivative at the critical point) then the critical point would be a relative minimum. Thus in order to determine the nature of a critical point at which the second derivative is zero we have to look at the value of the third derivative. If it is nonzero then the critical point is a point of inflection. If it is zero it could be a maximum or minimum depending upon the value of the fourth derivative at the critical point.

It might appear that the precise second order conditions for a maximum could be formulated in terms of the next nonzero derivative. However, there is a test case that shows how difficult it is to make the second order conditions precise. Consider the function


f(x) = exp[-1/x2]
 

This is a perfectly reasonable function that has a minimum at x=0. The problem is that all derivatives of this function at x= 0 are zero. The function has a well defined minimum at x=0 but is "infinitely flat" at x=0.


HOME PAGE OF Thayer Watkins
HOME PAGE OF applet-magic.com