1,071 211 28MB
English Pages 818 [837] Year 2015
Table of contents :
Contents
CHAPTER 0 PRELIMINARIES
0.0 Introduction
0.1 Reading mathematics
0.2 Quantifiers and negation
0.3 Set theory
0.4 Functions
0.5 Real numbers
0.6 Infinite sets
0. 7 Complex numbers
CHAPTER 1 VECTORS, MATRICES, AND DERIVATIVES
LO Introduction
1.1 Introducing the actors: Points and vectors
1.2 Introducing the actors: Matrices
1.3 Matrix multiplication as a linear transformation
1.4 The geometry of Rn
1.5 Limits and continuity
1.6 Five big theorems
vii
1
1
4
6
9
17
22
25
32
33
42
56
67
83
104
1. 7 Derivatives in several variables as linear transformations
1.8 Rules for computing derivatives
1.9 The mean value theorem and criteria for differentiability
1.10 Review exercises for Chapter 1
CHAPTER 2 SOLVING EQUATIONS
2.0 Introduction
2.1 The main algorithm: Row reduction
CHAPTER 3 MANIFOLDS, TAYLOR POLYNOMIALS,
QUADRATIC FORMS, AND CURVATURE
3.0 Introduction
3.1 Manifolds
3.2 Tangent spaces
3.3 Taylor polynomials in several variables
3.4 Rules for computing Taylor polynomials
3.5 Quadratic forms
3.6 Classifying critical points of functions
3.7 Constrained critical points and Lagrange multipliers
3.8 Probability and the singular value decomposition
3.9 Geometry of curves and surfaces
3.10 Review exercises for Chapter 3
CHAPTER 4 INTEGRATION
4.0 Introduction
4.1 Defining the integral
4.2 Probability and centers of gravity
4.3 What functions can be integrated
4.4 Measure zero
4.5 Fubini's theorem and iterated integrals
4.6 Numerical methods of integration
4.7 Other pavings
4.8 Determinants
4.9 Volumes and determinants
4.10 The change of variables formula
4.11 Lebesgue integrals
4.12 Review exercises for Chapter 4
CHAPTER.5 VOLUMES OF MANIFOLDS
5.0 Introduction
5.1 Parallelograms and their volumes
5.2 Parametrizations
5.3 Computing volumes of manifolds
5.4 Integration and curvature
5.5 Fractals and fractional dimension
5.6 Review exercises for Chapter 5
CHAPTER 6 FORMS AND VECTOR CALCULUS
6.0 Introduction
6.1 Forms on !Rn
6.2 Integrating form fields over parametrized domains
6.3 Orientation of manifolds
vi Contents
6.4 Integrating forms over oriented manifolds
6.5 Forms in the language of vector calculus
6.6 Boundary orientation
6. 7 The exterior derivative
6.8 Grad, curl, div, and all that
6.9 The pullback
6.10 The generalized Stokes's theorem
6.11 The integral theorems of vector calculus
6.12 Electromagnetism
6.13 Potentials
6.14 Review exercises for Chapter 6
APPENDIX: ANALYSIS
A.O Introduction
A.1 Arithmetic of real numbers
A.2 Cubic and quartic equations
A.3 Two results in topology: Nested compact sets
and HeineBorel
A.4 Proof of the chain rule
A.5 Proof of Kantorovich's theorem
A.6 Proof of Lemma 2.9.5 (superconvergence
A. 7 Proof of differentiability of the inverse function
A.8 Proof of the implicit function theorem
A.9 Proving the equality of crossed partials
A.10 Functions with many vanishing partial derivatives
A.11 Proving rules for Taylor polynomials; big 0 and little o
A.12 Taylor's theorem with remainder
A.13 Proving Theorem 3.5.3 (completing squares
A.14 Classifying constrained critical points
A.15 Geometry of curves and surfaces: Proofs
A.16 Stirling's formula and proof of the central limit theorem
A.17 Proving Fubini's theorem
A.18 Justifying the use of other pavings
A.19 Change of variables formula: A rigorous proof
A.20 Volume 0 and related results
A.21 Lebesgue measure and proofs for Lebesgue integrals
A.22 Computing the exterior derivative
A.23 Proving Stokes's theorem
BIBLIOGRAPHY
PHOTO CREDITS
INDEX
VECTOR CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL FORMS
A Unified Approach STH EDITION
JOHN H. HUBBARD BARBARA BURKE HUBBARD
sgn( a) Span (J"
sup Supp(!) T
TxX tr
v·w vxw lvl
(v) J_
Varf
[x]k
signature of a permutation (Theorem and Definition 4.8.11) span (Definition 2.4.3) standard deviation (Definition 3.8.6). Also denotes permutation. sum (Section 0.1) supremum; least upper bound (Definitions 0.5.1 , 1.6.5) support of a function f (Definition 4.1.2) (tau) torsion (Definition 3.9.14) tangent space to manifold (Definition 3.2.1) trace of a matrix (Definition 1.4.13) dot product of two vectors, (Definition 1.4.1) cross product of two vectors (Definition 1.4.17) length of vector v (Definition 1.4.2) orthogonal complement to subspace spanned by v (proof of Theorem 3.7.15) variance (Definitions 3.8.6) ktruncation (Definition Al.2)
Notation particular to this book
[OJ
matrix with all entries 0 (equation 1. 7.48) equal in the sense of Lebesgue (Definition 4.11 .6)
L
~_ (e.g .,
v i)
A
[Alb] Br(x)
/Jn D N(Ilr)
[Df(a)] IIJ)
D1f
Wxl 8'Af X
r(J) [h]R
0, there exists 8 > 0 such that for all x, y E ~' if \y x\ < 8, then \y 2  x 2 \ < E,'' we have a meaningful mathematical sentence but it is false. (It claims that the squaring function is uniformly continuous, which it is not.)
0.2
Note that in ordinary English, the word "any" can be used to mean either "for all" or "there exists" . The sentence "any execution of an innocent person invalidates the death penalty" means one single execution; the sentence "any fool knows that" means "every fool knows that". Usually in mathematical writing the meaning is clear from context, but not always. The solution is to use language that sounds stilted, but is at least unambiguous.
Quantifiers and negation
5
Even professional mathematicians have to be careful when negating a mathematical statement with several quantifiers. The rules are: 1. The opposite of [For all x, P(x) is true] 0.2.1 [There exists x for which P(x) is not true] .
is
Above, P stands for "property." Symbolically the sentence is written The opposite of ('Vx)P(x)
is
(3x) not P(x).
0.2.2
Another standard notation for (3x) not P(x) is (3x)I not P(x), where the bar I means "such that." 2. The opposite of
Most mathematicians avoid the symbolic notation, instead writing out quantifiers in full, as in formula 0.2.1. But when there is a complicated string of quantifiers, they often use the symbolic notation to avoid ambiguity.
Statements that to the ordinary mortal are false or meaningless are thus accepted as true by mathematicians; if you object, the mathematician will retort, "find me a counterexample."
Notice that when we have a "for all" followed by "there exists" , the thing that exists is allowed to depend on the preceding variable. For instance, when we write "for all € there exists 8", there can be a different 8 for each €. But if we write "there exists 8 such that for all €" , the single 8 has to work for all € .
[There exists x for which P(x) is true] 0.2.3 is
[For all x, P(x) is not true] .
Symbolically the same sentence is written The opposite of (3x)P(x)
is
('Vx) not P(x).
0.2.4
These rules may seem reasonable and simple. Clearly the opposite of the (false) statement "All rational numbers equal l" is the statement "There exists a rational number that does not equal l." However, by the same rules, the statement, "All elevenlegged alligators are orange with blue spots" is true, since if it were false, then there would exist an elevenlegged alligator that is not orange with blue spots. The statement, "All elevenlegged alligators are black with white stripes" is equally true. In addition, mathematical statements are rarely as simple as "All rational numbers equal 1." Often there are many quantifiers, and even the experts have to watch out. At a lecture attended by one of the authors, it was not clear to the audience in what order the lecturer was taking the quantifiers; when he was forced to write down a precise statement, he discovered that he didn't know what he meant and the lecture fell apart. Example 0.2.1 (Order of quantifiers). The statement (' 0)(3y)
such that Ix  YI
< 6 and lf(x)  f(y)I :'.'.'.
€.
Of course one could also negate formula 0.2.5, or any mathematical statement, by putting "not" in the very front, but that is not very useful when you are trying to determine whether a complicated statement is true. You can also reverse some leading quantifiers, then insert a "not" and leave the remainder as it was. Usually getting the not at the end is most useful: you finally come down to a statement that you can check.
A function f is uniformly continuous if for all E > 0, there exists 8 > 0 such that, for all x and ally, if Ix yl < 8, then lf(x)  f(y)I < E. That is, f is uniformly continuous if
(VE> 0)(38 > O)(\fx)(\fy) (Ix  YI< 8
EXERCISE FOR SECTION
b . For all x E R and for all € > 0, there exists § > 0 such that for all y E R, if IY  xi < 6, then IY2  x 2 1 < €. c. For all € > 0, there exists § > 0 such that for all x, y E R, if IY  xi < 6, then ly2  x 2 1< €.
0 .3
The Latin word locus means "place"; its plural is loci.
0.2.7
0.2.1 Negate the following statements: a. Every prime number such that if you divide it by 4 you have a remainder of 1 is the sum of two squares.
Explain why one of these statements is true and the other false: (3 woman W)(V man M)
0.3.1.
lf(x)  f(y)I < E).
0.2
(V man M)(3 woman W)
FIGURE
===?
For the continuous function, we can choose difjerent 8 for different x; for the uniformly continuous function, we start with E and have to find a single 8 that works for all x. For example, the function f(x) = x 2 is continuous but not uniformly continuous: as you choose bigger and bigger x, you will need a smaller 8 if you want the statement Ix  YI < 8 to imply lf(x)  f(y)I < E, because the function keeps climbing more and more steeply. But sin x is uniformly 6. continuous; you can find one 8 that works for all x and all y.
0 .2.2
An artist's image of Euclid.
0.2.6
I Wis the mother of M I Wis the mother of M
SET THEORY
There is nothing new about the concept of a "set" composed of elements such that some property is true. Euclid spoke of geometric loci , a locus being the set of points defined by some property. But historically, mathematicians apparently did not think in terms of sets, and the introduction of set theory was part of a revolution at the end of the nineteenth century that included topology and measure theory; central to this revolution was Cantor's discovery (discussed in Section 0.6) that some infinities are bigger than others.
0.3
Set theory
7
At the level at which we are working, set theory is a language, with a vocabulary consisting of eight words: In spoken mathematics, the symbols E and C often become "in": x E Rn becomes "x in Rn" and U C Rn becomes "U in Rn". Make sure you know whether "in" means element of or subset of. The symbol ff. ("not in") means "not an element of" ; similarly, Y be a function" to indicate that the domain is X, the codomain is Y, and the rule is f. For instance, the function that associates to every real number x the largest integer n ::=; x is a function f : JR > Z, often called "floor". (Evaluated on 4.3, that function returns 4; evaluated on 5.3 it returns 6.) It must be possible to evaluate the function on every element of the domain, and every output (value of the function) must be in the codomain. But it is not necessary that every element of the codomain be a value of the function. We use the word "image" to denote the set of elements in the codomain that are actually reached.
Definition 0.4.2 (Image). The set of all values off is called its image: y is an element of the image of a function f : X > Y if there exists an x EX such that f(x) = y.
10
Chapter 0.
Preliminaries
For example, the image of the squaring function f : JR + JR given by f(x) = x 2 is the nonnegative real numbers; the codomain is R The words function, mapping, and map are synonyms, generally used in different contexts. A function normally returns a number. Mapping is a more recent word; it was first used in topology and geometry and has spread to all parts of mathematics. In higher dimensions, we tend to use the word mapping rather than function. In English it is more natural to say, "John's father" rather than "the father of John". A school of algebraists exists that uses this notation: they write (x)f rather than f(x). The notation f(x) was established by the Swiss mathematician Leonhard Euler (17071783) . He set the notation we use from high school on: sin, cos, and tan for the trigonometric functions are also due to him. When Cantor proposed this function, it was viewed as pathological, but it turns out to be important for understanding Newton's method for complex cubic polynomials. A surprising discovery of the early 1980s was that functions just like it occur everywhere in complex dynamics.
FIGURE 0.4.1.
Not a function: Not well defined at a, not defined at b.
The codomain of a function may be considerably larger than its image. Moreover, you cannot think of a function without knowing its codomain, i.e., without having some idea of what kind of object the function produces. (This is especially important when dealing with vectorvalued functions.) Knowing the image, on the other hand, may be difficult.
What do we mean by rule? The rule used to define a function may be a computational scheme specifiable in finitely many words, but it need not be. If we are measuring the conductivity of copper as a function of temperature, then an element in the domain is a temperature and an element in the codomain is a measure of conductivity; all other variables held constant, each temperature is associated to one and only one measure of conductivity. The "rule" here is "for each temperature, measure the conductivity and write down the result". We can also devise functions where the "rule" is "because I said so". For example, we can devise the function M : [O, 1] + JR that takes every number in the interval [O, 1] that can be written in base 3 without using 1, changes every 2 to a 1, and then considers the result as a number in base 2. If the number written in base 3 must contain a 1, the function M changes every digit after the first 1 to 0, then changes every 2 to 1, and considers the result as a number in base 2. Cantor proposed this function to point out the need for greater precision in a number of theorems, in particular the fundamental theorem of calculus. In other cases the rule may be simply "look it up". Thus to define a function associating to each student in a class his or her final grade, all you need is the final list of grades; you do not need to know how the professor graded and weighted various exams, homeworks, and papers (although you could define such a function, which, if you were given access to the student's work for the year, would allow you to compute his or her final grade). Moreover, if the rule is "look it up in the table", the table need not be finite . One of the fundamental differences between mathematics and virtually everything else is that mathematics routinely deals with the infinite. We are going to be interested in things like the set of all continuous functions f that take an element of JR (i.e., any real number) and return an element of R If we restrict ourselves to functions that are finitely specifiable, then much of what we might want to say about such sets is not true or has quite a different meaning. For instance, any time we want to take the maximum of some infinite set of numbers, we would have to specify a way of finding the maximum. Thus the "rule" in Definition 0.4.l can be virtually anything at all, just so long as every element of the domain (which in most cases contains infinitely many elements) can be associated to one and only one element of the codomain (which in most cases also contains infinitely many elements).
0.4
Functions
11
Definition 0.4.3 emphasizes that it is this result that is crucial; any procedure that arrives at it is acceptable.
Definition 0.4.3 (Set theoretic definition of function). A function f : X + Y is a subset r f c X x Y having the property that for every x EX, there exists a unique y E Y such that (x, y) Er1. FIGURE 0.4.2. A function: Every point on the left goes to only one point on the right. The fact that a function takes you unambiguously from any point in the domain to a single point in the codomain does not mean that you can go unambiguously, or at all, in the reverse direction; here, going backward from d in the codomain takes you to either a or b in the domain, and there is no path from c in the codomain to any point in the domain.
FIGURE 0.4.3. The graph of arcsin. The part in bold is the graph of the "function" arcsin as defined by calculators and computers: the function arcsin : [1, 1] > JR whose rule is "arcsin(x) is the unique angle 8 satisfying 7r /2 s; 8 s; 7r /2 and sin8=x."
Is arcsin a function? Natural domains and other ambiguities We use functions from early childhood, typically with the word "of" or its equivalent: "the price of a book" associates a price to a book; "the father of" associates a man to a person. Yet not all such expressions are true functions in the mathematical sense. Nor are all expressions of the form f(x) = y true functions. As both Definitions 0.4.1 and 0.4.3 express in different words, a function must be defined at every point of the domain (everywhere defined), and for each, it must return a unique element of the codomain (it must be well defined). This is illustrated by Figures 0.4.1 and 0.4.2. "The daughter of", as a "rule" from people to girls and women, is not everywhere defined, because not everyone has a daughter; it is not well defined because some people have more than one daughter. It is not a mathematical function. But "the number of daughters of" is a function from women to numbers: it is everywhere defined and well defined, at a particular time. So is "the biological father of" as a rule from people to men; every person has a biological father, and only one. The mathematical definition of function then seems straightforward and unambiguous. Yet what are we to make of the arcsin "function" key on your calculator? Figure 0.4.3 shows the "graph" of "arcsin". Clearly the argument 1/2 does not return one and only one value in the codomain; arcsin(l/2) = 7r /6 but we also have arcsin(l/2) = 57r /6 and so on. But if you ask your calculator to compute arcsin(l/2) it returns only the answer .523599 ~ 7r /6. The people who programmed the calculator declared "arcsin" to be the function arcsin: [1, 1] +JR whose rule is "arcsin(x) is the unique angle() satisfying 7r /2 ~ () ~ 7r /2 and sin()= x." Remark. In the past, some textbooks spoke of "multivalued functions" that assign different values to the same argument; such a "definition" would allow arcsin to be a function. In his book Calcul Infinitesimal, published in 1980, the French mathematician Jean Dieudonne pointed out that such definitions are meaningless, "for the authors of such texts refrain from giving the least rule for how to perform calculations using these new mathematical objects that they claim to define, which makes the socalled 'definition' unusable." Computers have shown just how right he was. Computers do not tolerate ambiguity. If the "function" assigns more than one value to a single argument, the computer will choose one without telling you that it is making a choice. Computers are in effect redefining certain expressions to be
12
Chapter 0.
Preliminaries
functions. When the authors were in school, J4 was two numbers, +2 and 2. Increasingly, "square root" is taken to mean "positive square root" , because a computer cannot compute if each time it lands on a square root it must consider both positive and negative square roots. !:::,.
Natural domain
When working with complex numbers, choosing a "natural domain" is more difficult . The natural domain is usually ambiguous: a choice of a domain for a formula f is referred to a "choosing a branch of f" . For instance, one speaks of "the branch of Jz defined in Re z > 0, taking positive values on the positive real axis" . Historically, the notion of Riemann surface grew out of trying to find natural domains.
Parentheses denote an open interval and brackets denote a closed one; (a, b) is open, (a, b] is closed: (a , b)
[a,b]
= {x E JR Ia < x < b} = {x E JR la S x Sb}.
We discuss open and closed sets in Section 1.5. We could "define" the natural domain of a formula to consist of those arguments for which a computer does not return an error message.
Often a mathematical function modeling a real system has a codomain considerably larger than the realistic values. We may say that the codomain of the function assigning height in centimeters to children is JR, but clearly many real numbers do not correspond to the height of any child.
Often people refer to functions without specifying the domain or the codomain; they speak of something like "the function ln(l + x)" . When the word "function" is used in this way, there is an implicit domain consisting of all numbers for which the formula makes sense. In the case of ln(l + x) it is the set of numbers x > 1. This default domain is called the formula's natural domain. Discovering the natural domain of a formula can be complicated, and the answer may depend on context. In this book we have tried to be scrupulous about specifying a function 's domain and codomain. Example 0.4.4 (Natural domain). The natural domain of the formula f(x) = lnx is the positive real numbers; the natural domain of the formula f(x) = ,jX is the nonnegative real numbers. The notion of natural domain may depend on context: for both ln x and ,jX we are assuming that the domain and codomain are restricted to be real numbers, not complex numbers. What is the natural domain of the formula
f (x) =
J x2 
3x + 2 ?
0.4.1
This can be evaluated only if x 2  3x + 2 ~ 0, which happens if x ~ 1 or x ~ 2. So the natural domain is ( oo, 1] U [2, oo) . !:::,. Most often, a computer discovers that a number is not in the natural domain of a formula when it lands on an illegal procedure like dividing by 0. When working with computers, failure to be clear about a function's domain can be dangerous. One does not wish a computer to shut down an airplane's engines or the cooling system of a nuclear power plant because an input has been entered that is not in a formula's natural domain. Obviously it would be desirable to know before feeding a formula a number whether the result will be an error message; an active field of computer science research consists of trying to figure out ways to guarantee that a set is in a formula's natural domain.
Latitude in choosing a codomain A function consists of three things: a rule, a domain, and a codomain. A rule comes with a natural domain, but there is no similar notion of natural codomain. The codomain must be at least as big as the image, but it can be a little bigger, or a lot bigger; if JR will do, then so will C, for example. In this sense we can speak of the "choice" of a codomain. Since the codomain is
0.4 Functions
Of course, computers cannot actually compute with real numbers; they compute with approximations to real numbers. The computer language C is if anything more emphatic about specifying the codomain of a function. In C, the first word of a function declaration describes the codomain. The functions at right would be introduced by the lines integer floor( double x); and double floor(double x). The first word indicates the type of output (the word "double" is C's name for a particular encoding of the reals); the second word is the name of the function; and the expression in parentheses describes the type of input.
0.4.4. An onto function, not 11: a and b go to the same point. FIGURE
FIGURE 0.4.5. A function: 11, not onto, no points go to a or to b.
13
part of what defines a particular function, then, strictly speaking, choosing one allowable codomain as opposed to another means creating a different function. But generally, two functions with the same rule and same domain but different codomains behave the same. When we work with computers, the situation is more complicated. We mentioned earlier the floor function f : R + Z that associates to every real number x the largest integer n :=::; x. We could also consider the floor function as a function R + R, since an integer is a real number. If you are working with pen and paper, these two (strictly speaking different) functions will behave the same. But a computer will treat them differently: if you write a computer program to compute them, in Pascal for instance, one will be introduced by the line function floor( x:real) :integer; whereas the other will be introduced by function floor( x:real) :real. These functions are indeed different: in the computer, reals and integers are not stored the same way and cannot be used interchangeably. For instance, you cannot perform a division with remainder unless the divisor is an integer, and if you attempt such a division using the output of the second "floor" function above, you will get a TYPE MISMATCH error.
Existence and uniqueness of solutions Given a function f, is there a solution to the equation f(x) = b, for every b in the codomain? If so, the function is said to be onto, or surjective. "Onto" is thus a way to talk about the existence of solutions. The function "the father of" as a function from people to men is not onto, because not all men are fathers. There is no solution to the equation "The father of x is Mr. Childless". An onto function is shown in Figure 0.4.4. A second question of interest concerns uniqueness of solutions. Given any particular b in the codomain, is there at most one value of x that solves the equation T(x) = b, or might there be many? If for each b there is at most one solution to the equation T(x) = b, the function T is said to be one to one, or injective. The mapping "the father of" is not one to one. There are, in fact, four solutions to the equation "The father of x is John Hubbard". But the function "the twin sibling of", as a function from twins to twins, is one to one: the equation "the twin sibling of x = y" has a unique solution for each y. "One to one" is thus a way to talk about the uniqueness of solutions. A one to one function is shown in Figure 0.4.5. A function T that is both onto and one to one has an inverse function r 1 that undoes it. Because Tis onto, r 1 is everywhere defined; because T is one to one, r 1 is well defined. So r 1 qualifies as a function . To summarize:
Definition 0.4.5 (Onto). A function f: X+ Y is onto (or surjective) if for every y E Y there exists x EX such that f(x) = y.
14 Chapter 0. Preliminaries Thus 1 is onto if every element of the set of arrival (the codomain Y) corresponds to at least one element of the set of departure (the domain X). The inverse function of f is usually called simply "f inverse".
Definition 0.4.6 (One to one). A function f: X+ Y is one to one (or injective) if for every y E Y there is at most one x E X such that
l(x) = y. Thus 1 is one to one if every element of the set of arrival corresponds to at most one element of the set of departure. The horizontal line test to see whether a function is one to one is shown in Figure 0.4.6. y
Definition 0.4. 7 (Invertible). A mapping 1 is invertible (or bijective) if it is both onto and one to one. The inverse function of f is denoted
11.
x
FIGURE 0.4.6. The function graphed above is not one to one. It fails the "horizontal line test": the horizontal dotted line cuts it in three places, showing that three different values of x give the same value of y.
An invertible function can be undone; if f(a) = b, then 1 1 (b) =a. The words "invertible" and "inverse" are particularly appropriate for multiplication; to undo multiplication by a, we multiply by its inverse, 1/a. (But the inverse function of the function l(x) =xis 1 1 (x) = x, so that in this case the inverse of xis x, not 1/x, which is the multiplicative inverse. Usually it is clear from context whether "inverse" means "inverse mapping", as in Definition 0.4.7, or "multiplicative inverse", but sometimes there can be ambiguity.) Example 0.4.8 (One to one; onto). The mapping "the Social Security number of" as a mapping from United States citizens to numbers is not onto because there exist numbers that aren't Social Security numbers. But it is one to one: no two U.S. citizens have the same Social Security number. The mapping l(x) = x 2 from real numbers to real nonnegative numbers is onto because every real nonnegative number has a real square root, but it is not one to one because every real positive number has both a positive and a negative square root. 6. If a function is not invertible, we can still speak of the inverse image of a set under f.
For the map of Example 0.4.10, 1 ( { 1}) = cf>; it is a welldefined set. If we had defined g(x) = x 2 as a map from JR to the nonnegative reals, then g 1 ( { 1}) and g 1 ( { 1, 4, 9, 16}) would not exist.
r
Definition 0.4.9 (Inverse image). Let 1: X+ Y be a function, and let C c Y be a subset of the codomain of 1. Then the inverse image of C under 1, denoted 1 1 (C), consists of those elements x EX such that
l(x) EC. Example 0.4.10 (Inverse image). Let 1 : lR + lR be the (noninvertible) mapping l(x) = x 2 . The inverse image of {1,4,9,16} under 1 is
{4, 3, 2, 2, 3, 4}:
1 1 ({1,4,9,16}) =
{4,3,2,2,3,4}.
6.
0.4.2
0.4 Functions
15
Proposition 0.4.11 (Inverse image of intersection, union). You are asked to prove Proposition 0.4.11 in Exercise 0.4.6.
1. The inverse image of an intersection equals the intersection of the inverse images:
1l (An B)
=
1l (A) n 1l (B).
0.4.3
2. The inverse image of a union equals the union of the inverse images: 0.4.4 It is often easier to understand a composition if one writes it in diagram form; (fog) : A > D can be written A+ B g
c
C+ D. f
A composition is written from left to right but computed from right to left: you apply the mapping g to the argument x and then apply the mapping f to the result. Exercise 0.4.7 provides some practice.
When computers do compositions, it is not quite true that composition is associative. One way of doing the calculation may be more computationally effective than another; because of roundoff errors, the computer may even come up with different answers, depending on where the parentheses are placed.
Composition of mappings Often one wishes to apply, consecutively, more than one mapping. This is known as composition.
Definition 0.4.12 (Composition). If l: C+ D and g: A+ Bare two mappings with BC C, then the composition (! o g) : A+ Dis the mapping given by
(! o g)(x) = l(g(x)).
0.4.5
Note that for the composition l o g to make sense, the codomain of g must be contained in the domain of l.
Example 0.4.13 (Composition of "the father of" and "the mother of"). Consider the following two mappings from the set of persons to the set of persons (alive or dead): F, "the father of", and M, "the mother of". Composing these gives:
= of =
F oM
(the father of the mother of
maternal grandfather of)
M oF
(the mother of the father
paternal grandmother of).
It is clear in this case that composition is associative:
F
o
(F o M) = (F o F)
o
M.
0.4.6
The father of David's maternal grandfather is the same person as the paternal grandfather of David's mother. Of course, it is not commutative: the "father of the mother" is not the "mother of the father".) 6
Example 0.4.14 (Composition of two functions). If l(x) = x  1, and g(x) = x 2 , then
(! o g)(x) = f(g(x)) = x 2

l.
6
0.4.7
Proposition 0.4.15 (Composition is associative). Composition is associative:
(! 0 g)
0
h=
l 0 (g 0
h).
0.4.8
16
Chapter 0.
Preliminaries
Proof. This is simply the computation
((! o g) o h)(x) = (f o g)(h(x)) = f(g(h(x))) whereas (f o (go h)) (x) = f ((go h)(x)) = f (g(h(x))) . 0
Although composition is associative, in many settings,
( (f o g) o
h)
and
(! o (g o h))
correspond to different ways of thinking. The author of a biography . might use "the father of the maternal grandfather" when focusing on the relationship between the subject's grandfather and the grandfather's father, and use "the paternal grandfather of the mother" when focusing on the relationship between the subject's mother and her grandfather.
0.4.9
You may find this proof devoid of content. Composition of mappings is part of our basic thought processes: you use a composition any time you speak of "the this of the that of the other". So the statement that composition is associative may seem too obvious to need proving, and the proof may seem too simple to be a proof.
Proposition 0.4.16 (Composition of onto functions). Let the functions f: B  t C and g : A  t B be onto. Then the composition (fog) is onto. Proposition 0.4.17 (Composition of one to one functions). Let f : B  t C and g : A  t B be one to one. Then the composition (f o g) is one to one. You are asked to prove Propositions 0.4.16 and 0.4.17 in Exercise 0.4.8
EXERCISES FOR SECTION 0.4 0.4.1 Are the following true functions? That is, are they both everywhere defined and well defined? a . "The aunt of" , from people to people. b.
f(x) =~,from real numbers to real numbers.
c. "The capital of", from countries to cities (careful  at least one country, Bolivia, has two capitals.) 0.4.2 a. Make up a nonmathematical function that is onto but not one to one. b . Make up a mathematical function that is onto but not one to one. 0.4.3 a. Make up a nonmathematical function that is bijective (onto and one to one). b. Make up a mathematical function that is bijective. 0.4.4 a. Make up a nonmathematical function that is one to one but not onto. b . Make up a mathematical function that is one to one but not onto. 0.4.5 Given the functions f : A> B, g: B> C, h: A> C, and k: C> A, which of the following compositions are well defined? For those that are, give the domain and codomain of each.
0.4.6
a . fog
b. gof
c. ho f
g. hok
h. kog
i.
Prove Proposition 0.4.11.
k0
f
d. g j.
0
h
f0h
e. ko h k.
f0k
f. h 0 g I. g
0
k
0.5 0.4.7
= x2 f(x) = x 2 ,
1,
b.
g(x)
0 .4 .9
17
Evaluate (fog o h)(x) at x =a for the following.
a. f(x)
0.4.8
Real numbers
g(x)
= 3x,
=x 
3,
= x + 2, for a= 3. h(x) = x  3, for a= 1.
a. Prove Proposition 0.4.16.
h(x)
b. Prove Proposition 0.4.17.
What is the natural domain of Jr?
What is the natural domain of a. In o In? b. In o In o In? c. In composed with itself n times?
0 .4 .10
0.4.11
What subset of JR is the natural domain of the function (1
+ x) 1 / x ?
0.4.12 The function f(x) = x 2 from real numbers to real nonnegative numbers is onto but not one to one. a. Can you make it one to one by changing its domain? By changing its codomain?
b. Can you make it not onto by changing its domain? Its codomain?
0.5 REAL
Showing that all such constructions lead to the same numbers is a fastidious exercise, which we will not pursue.
NUMBERS
Calculus is about limits, continuity, and approximation. These concepts involve real numbers and complex numbers, as opposed to integers and rationals. In this Section (and in Appendix Al), we present real numbers and establish some of their most useful properties. Our approach privileges the writing of numbers in base 10; as such it is a bit unnatural, but we hope you will like our real numbers being exactly the numbers you are used to.
Numbers and their ordering
Real numbers are actually biinfinite decimals with O's to the left: a number like 3.0000 . . . is actually . . . 00003.0000 .... By convention, leading O's are usually omitted. One exception is credit card expiration dates: the month March is 03, not 3.
By definition, the set of real numbers is the set of infinite decimals: expressions like 2.957653920457 .. . , preceded by a plus or a minus sign (often the + is omitted). The number that you think of as 3 is the infinite decimal 3.0000 . . . , ending in all O's. The following identification is vital: a number ending in all 9's is equal to the "rounded up" number ending in all O's: 0.34999999 . . . = 0.350000 . .. .
0.5.1
Also, +.0000 . . . =  .0000 .... Other than these exceptions, there is only one way of writing a real number. Numbers starting with a + sign, except +0.000 . . . , are positive; those starting with a  sign, except 0.00 .. . , are negative. If xis a real number, then x has the same string of digits, but with the opposite sign in front. When a number is written in base 10, the digit in the lOkth position refers to the number of lOks. For instance, 217.4 . . . has 2 hundreds, 1 ten , 7 ones, and 4 tenths, corresponding to 2 in the 102 position, 1 in the 10 1
18
Real numbers can be defined in more elegant ways: Dedekind cuts, for instance (see, for example, M. Spivak, Calculus, second edition, Publish or Perish, 1980, pp. 554572), or Cauchy sequences of rational numbers. One could also mirror the present approach, writing numbers in any base, for instance 2. Since this section is partially motivated by the treatment of floatingpoint numbers on computers, base 2 would seem very natural.
The least upper bound property of the reals is often taken as an axiom; indeed, it characterizes the real numbers, and it lies at the foundation of every theorem in calculus. However, at least with the preceding description of the reals, it is a theorem, not an axiom. The least upper bound sup X is sometimes denoted l.u.b.X; the notation max X is also used, but it suggests to some people that max X E X, which may not be the case.
If a= 29.86 ... , b = 29.73 . .. ,
then
[b]2
< [a]2,
(b]1 < (a]1;
j = 1 is the largest j such that
(b]i < [aJi.
Chapter 0.
Preliminaries
position, and so on. We denote by [x]k the number formed by keeping all digits to the left of and including the lOkth position, setting all others to 0. Thus for a = 5129.359 ... , we have [ab = 5100.00 ... , [a] 0 = 5129.00 ... , [a]2 = 5129.3500 .... If x has two decimal expressions, we define [x]k to be the finite decimal built from the infinite decimal ending in O's; for the number in formula 0.5.1, [x]3 = 0.350; it is not 0.349. Given two different finite numbers x and y, one is always bigger than the other, as follows. If x is positive and y is nonpositive, then x > y. If both are positive, then in their decimal expansions there is a leftmost digit in which they differ; whichever has the larger digit in that position is larger. If both x and y are negative, then x > y if y > x.
Least upper bound Definition 0.5.1 (Upper bound; least upper bound). A number a is an upper bound for a subset X C JR if for every x E X we have x :::; a. A least upper bound, also known as the supremum, is an upper bound b such that for any other upper bound a, we have b :::; a. It is denoted sup X. If X is unbounded above, sup X is defined to be +oo. Definition 0.5.2 (Lower bound; greatest lower bound). A number a is a lower bound for a subset X c JR if for every x E X we have x ;:::: a. A greatest lower bound is a lower bound b such that for any other lower bound a, we have b ;:::: a. The greatest lower bound, or infimum, is denoted inf X. If Xis unbounded below, inf Xis defined to be oo. Theorem 0.5.3 (The real numbers are complete). Every nonempty subset X C JR that has an upper bound has a least upper bound sup X. Every nonempty subset X c JR that has a lower bound has a greatest lower bound inf X. Proof. We will construct successive decimals of supX. Suppose that x E X is an element (which we know exists, since X =/= ) and that a is an upper bound. We will assume that x > 0 (the case x :::; 0 is slightly different). If x =a, we are done: the least upper bound is a. If x =!= a, there is then a largest j such that [x]j < [a]j. There are 10 numbers that have the same kth digit as x for k > j and that have 0 as the kth digit for k < j; consider those that are in [ [x]j, a]. This set is not empty, since [x]j is one of them. Let bj be the largest of these ten numbers such that X n [bj, a] =!= ; such a bj exists, since x E X n [[x]j, a]. Consider the set of numbers in [bj, a] that have the same kth digit as bj for k > j  1, and 0 for k < j  1. This is a nonempty set with at most 10 elements, and bj is one of them (the smallest). Call bjl the largest such that X n [bjl, a] =j=. Such a bjl exists, since if necessary we can choose bj. Keep going this way, defining bj_ 2 , bj_ 3 , and so on, and let b be the
0.5 Real numbers
The procedure we give for proving the existence of b = sup X gives no recipe for finding it. Like the proof of Theorem 1.6.3, this proof is nonconstructive. Example 1.6.4 illustrates the kind of difficulty one might encounter when trying to construct b. The symbol ,__.. ("maps to") describes what a function does to an input; see the margin note about function notation, page 9. Using this notation for sequences is reasonable, since a sequence really is a map from the positive integers to whatever space the sequence lives in. A sequence i ,__.. ai can also be written as (ai) or as (a;) ;eN or even as ai. We used the notation ai ourselves in previous editions, but we have become convinced that i ,__.. ai is best.
number whose nth decimal digit (for all n) is the same as the nth decimal digit of bn. We claim that b = sup X . Indeed, if there exists y E X with y > b, then there is a first k such that the kth digit of y differs from the kth digit of b. This contradicts our assumption that bk was the largest number (out of 10) such that X n [bk, a] =f:. cf>, since using the kth digit of y would give a bigger one. So b is an upper bound. Now suppose that b' < b. If b' is an upper bound for X, then (b', a] n X =cf>. Again there is a first k such that the kth digit of b differs from the kth digit of b' . Then (b', a] n X ::::> [bk, a] n X =f:. cf>. Thus b' is not an upper bound for X. D
Sequences and series A sequence is an infinite list a 1, a2, . . . (of numbers or vectors or matrices ... ). We denote such a list by n f+ an, where n is assumed to be a positive (or sometimes nonnegative) integer. Definition 0.5.4 (Convergent sequence). A sequence n f+ an of real numbers converges to the limit a if for all € > 0, there exists N such that for all n > N, we have la  anl < €. Many important sequences appear as partial sums of series. A series is a sequence whose terms are to be added. If we consider the sequence a 1 , a 2 , . . . as a series, then the associated sequence of partial sums is the sequence s1, s2, ... , where N
SN=
If a series converges, then the
same list of numbers viewed as a sequence must converge to 0. The converse is not true. For example, the harmonic series 1 1 1 +2+3+·· ·
0.5.2
Lan. n=l
For example, if ai
= 1, a2 = 2,
a3
= 3,
and so on, then
s4
= 1+2 + 3 +
4.
Definition 0.5.5 (Convergent series). If the sequence of partial sums of a series has a limit S, the series converges, and its limit is 00
S def~ = L..Jan .
does not converge, although the terms tend to 0. In practice, the index set for a series may vary; for instance, in Example 0.5.6, n goes from 0 to oo, not from 1 to oo. For Fourier series, n goes from oo to oo. But series are usually written with the sum running from 1 to oo.
19
0.5.3
n=l
Example 0.5.6 (Convergent geometric series). If
lrl < 1, then
00
~arn=_a_.
L..J
n=O
For example, 2.020202 ...
0.5.4
lr
= 2 + 2(.01) + 2(.01) 2 + · · · = 1 _
2 (.Ol)
200
= 99·
Indeed, the following subtraction shows that Sn(l  r) =a  arn+l:
Sn ~ a + ar + ar 2 + ar 3 + · · · + arn ar + ar 2 + ar 3 + · · · + arn + arn+l Sn(lr)=a
0.5.5
20
Chapter 0.
Preliminaries
But limn+oo arn+I = 0 when lrl < 1, so we can forget about the arn+I: as n+ oo, we have Sn+ a/(l  r). 6 Theorem 0.5.7: Of course it is also true that a nonincreasing sequence converges if and only if it is bounded. Most sequences are neither nondecreasing nor nonincreasing. In mathematical analysis, problems are usually solved by exhibiting a sequence that converges to the solution. Since we don't know the solution, it is essential to guarantee convergence without knowing the limit. Coming to terms with this was a watershed in the history of mathematics, associated first with a rigorous construction of the real numbers, and later with the definition of the Lebesgue integral, which allows the construction of Banach spaces and Hilbert spaces where "absolute convergence implies convergence", again giving convergence without knowing the limit. The use of these notions is also a watershed in mathematical education: elementary calculus gives solutions exactly, more advanced calculus constructs them as limits. In contrast to the real numbers and the complex numbers, it is impossible to prove that a sequence of rational numbers or algebraic numbers has a rational or algebraic limit without exhibiting it specifically.
Inequality 0.5.6: the sum
Proving convergence The weakness of the definition of a convergent sequence or series is that it involves the limit value; it is hard to see how you will ever be able to prove that a sequence has a limit if you don't know the limit ahead of time. The first result along these lines is Theorem 0.5.7. It and its corollaries underlie all of calculus.
Theorem 0.5. 7. A nondecreasing sequence n converges if and only if it is bounded.
t+
an of real numbers
Proof. If a sequence n t+ an of real numbers converges, it is clearly bounded. If it is bounded, then (by Theorem 0.5.3) it has a least upper bound A. We claim that A is the limit. This means that for any E > 0, there exists N such that if n > N, then Ian  Al < E. Choose E > O; if A  an > E for all n, then A  E is an upper bound for the sequence, contradicting the definition of A. So there is a first N with A  aN < E, and it will do, since when n > N, we must have A an~ A aN < E. 0 Theorem 0.5. 7 has the following consequence:
Theorem 0.5.8 (Absolute convergence implies convergence). If the series of absolute values 00
L an.
converges, then so does the series
n=l
Proof. The series L:::=l (an+ lanl) is a series of nonnegative numbers, so the partial sums bm = L:;~ 1 (an+ lani) are nondecreasing. They are also bounded: m
bm = L(an
n=l
m
m
+ lani) ~
L
2lanl = 2L
n=l
n=l
ex:>
lanl ~
2L
lanl·
0.5.6
n=l
So (by Theorem 0.5.7) mt+ bm is a convergent sequence, and L:::=l an can be represented as the sum of two numbers, each the sum of a convergent series:
00
0
0.5.7
n=I
is bounded by the hypothesis of the theorem.
The intermediate value theorem The intermediate value theorem appears to be obviously true, and is often useful. It follows easily from Theorem 0.5.3 and the definition of continuity.
0.5 One unsuccessful nineteenthcentury definition of continuity stated that a function f is continuous if it satisfies the intermediate value theorem. You are asked in Exercise 0.5.2 to show that this does not coincide with the usual definition (and presumably not with anyone's intuition of what continuity should mean).
Real numbers
21
Theorem 0.5.9 (Intermediate value theorem). If f : [a, b] + JR is a continuous function and c is a number such that f(a) ~ c and f(b) :'.'.'. c, then there exists xo E [a, b] such that f(xo) = c. Proof. Let X be the set of x E [a, b] such that f(x) ~ c. Note that X is nonempty (a is in it) and it has an upper bound, namely b, so that it has a least upper bound, which we call xo. We claim f(xo) = c. Since f is continuous, for any €. > 0, there exists o> 0 such that when lxoxl < o, then lf(xo)f(x)I < €.. If f(xo) > c, we can set€.= f(xo)c, and find a corresponding o. Since xo is a least upper bound for X, there exists x EX such that xo  x < o, so
f(x)
= f(xo) + f(x)
 f(xo) :'.'.'. f(xo)  lf(x)  f(xo)I > c + €. 
€.
= c,
contradicting that x is in X; see Figure 0.5.1. If f (xo) < c, a similar argument shows that there exists > 0 such that f(xo + o/2) < c, contradicting the assumption that xo is an upper bound for X. The only choice left is f(xo) = c. D
o
FIGURE
0.5.1.
The bold intervals in [a, b] are the set X of x such that f(x) ::; c. The point x E X slightly to the left of xo gives rise to f(x) > c, contradicting the definition of X. The point x slightly to the right gives rise to f(x) < c, contradicting xo being an upper bound. Exercise 0.5.1: Exercise 1.6.11 repeats this exercise, with hints. Exercise 0.5.3: By convention, [a, b] implies a ::; b. Exercise 0.5.3 is the onedimensional case of the celebrated Brouwer fixed point theorem, to be discussed in a subsequent volume. In dimension one it is an easy consequence of the intermediate value theorem, but in higher dimensions (even two) it is quite a delicate result. Exercise 0.5.4 illustrates how complicated convergence can be when a series is not absolutely convergent. Exercise 0.5.5 shows that these problem do not arise for absolutely convergent series.
EXERCISES FOR SECTION
0.5
The exercises for Section 0.5 are fairly difficult. 0.5.1 Show that if p is a polynomial of odd degree with real coefficients, then there is a real number c such that p(c) = 0. 0.5.2
a. Show that the function sin l f(x) = { 0 x
if x
=I 0
is not continuous. if x = 0 b. Show that f satisfies the conclusion of the intermediate value theorem: if f(x1) = a1 and f(x2) = a2, then for any number a between a1 and a2, there exists a number x between x1 and x2 such that f(x) =a. 0.5.3 Suppose a ::; b. Show that if c E [a, b] with f(c) = c. 0.5.4
f : [a, b]
+
[a, b] is continuous, there exists
Let for n
= 1, 2, ....
a. Show that the series Lan is convergent. *b. Show that L~=l an= ln2. c. Explain how to rearrange the terms of the series so it converges to 5. d . Explain how to rearrange the terms of the series so that it diverges. 0.5.5 Show that if a series 2:;;"= 1 an is absolutely convergent, then any rearrangement of the series is still convergent and converges to the same limit. Hint: For any € > 0, there exists N such that L~=N+i lanl < €. For any rearrangement 2:;;"= 1 bn of the series, there exists M such that all of a1, . .. , aN appear among b1, ... , bM. Show that I L:;';'=l an  L~=l bnl < €.
22
0. 6
Chapter 0.
Preliminaries
INFINITE SETS
One reason set theory is accorded so much importance is that Georg Cantor (18451918) discovered that two infinite sets need not have the same "number" of elements; there isn't just one infinity. You might think this is just obvious; for example, that clearly there are more whole numbers than even whole numbers. But with the definition Cantor gave, two sets A and B have the same number of elements (the same cardinality) if you can set up a bijective correspondence between them (i.e., a mapping that is one to one and onto). For instance, 0
1
2
3
4
5
6
0 2 4 6 8 10 12 FIGURE
0.6. l.
Georg Cantor (18451918) After thousands of years of philosophical speculation about the infinite, Cantor found a fundamental notion that had been completely overlooked. Recall (Section 0.3) that N is the "natural numbers" 0, 1, 2, ... ; Z is the integers; JR is the real numbers. It would seem likely that JR and JR 2 have different infinities of elements, but that is not the case (see Exercise Al.5).
FIGURE
0.6.2.
Joseph Liouville (18091882)
0.6.1
establishes a bijective correspondence between the natural numbers and the even natural numbers. More generally, any set whose elements you can list has the same cardinality as N: for instance, 0, 1, 1/2, 1/3, 2/3, 1/4, 3/4, 1/5, 2/5, 3/5, 4/5, ...
0.6.2
is the beginning of a list of the rational numbers in [O, l]. But in 1873 Cantor discovered that JR does not have the same cardinality as N: it has a bigger infinity of elements. Indeed, imagine making any infinite list of real numbers, say between 0 and 1, so that written as decimals, your list might look like .154362786453429823763490652367347548757 .. . .987354621943756598673562940657349327658 .. . .229573521903564355423035465523390080742 .. . .104752018746267653209365723689076565787 . . .
0.6.3
.026328560082356835654432879897652377327 ...
Now consider the decimal .18972 ... formed by the diagonal digits (in bold in formula 0.6.3), and modify it (almost any way you want) so that every digit is changed, for instance according to the rule "change 7's to 5's and change anything that is not a 7 to a 7": in this case, your number becomes .77757 . . . . Clearly this last number does not appear in your list: it is not the nth element of the list, because it doesn't have the same nth decimal digit. Infinite sets that can be put in onetoone correspondence with the natural numbers are called countable or countably infinite. Those that cannot are called uncountable; the set JR of real numbers is uncountable.
Existence of transcendental numbers An algebraic number is a root of a polynomial equation with integer coefficients: the rational number p/q is algebraic, since it is a solution of
0.6 qx  p = 0, and so is
Infinite sets
23
J2, since it
is a root of x 2  2 = 0. A number that is not algebraic is called transcendental. In 1851 Joseph Liouville came up with the transcendental number (now called the Liouvillian number)
~
1 L.J 10n.I
= 0.11000100000000000000000100 ... ,
0.6.4
n=l
FIGURE
0.6.3.
Charles Hermite (18221901) For Hermite, there was something scandalous about Cantor's proof of the existence of infinitely many transcendental numbers, which required no computations and virtually no effort and failed to come up with a single example.
the number with 1 in every position corresponding to n! and O's elsewhere. In 1873 Charles Hermite proved a much harder result, that e is transcendental. But Cantor's work on cardinality made it obvious that there must exist uncountably many transcendental numbers: all those real numbers left over when one tries to put the real numbers in onetoone correspondence with the algebraic numbers. Here is one way to show that the algebraic numbers are countable. First list the polynomials a 1 x+a 0 of degree S 1 with integer coefficients satisfying lail S 1, then the polynomials a2x 2 + aix + ao of degree$ 2 with lail $ 2, etc. The list starts
x  1, x + 0, x + 1, 1, 0, 1, x  1, x, x + 1, 2x 2  2x  2, 0.6.5  2x 2  2x  1, 2x 2  2x, 2x2  2x + 1, 2x 2  2x + 2, .... (The polynomial 1 in formula 0.6.5 is 0 · x 1.) Then we go over the list, crossing out repetitions. Next we write a second list, putting first the roots of the first polynomial in formula 0.6.5, then the roots of the second polynomial, etc.; again, go through the list and cross out repetitions. This lists all algebraic numbers, showing that they form a countable set.
Other consequences of different cardinalities Two sets A and B have the same cardinality (denoted A x B) if there exists an invertible mapping A____, B . A set A is countable if Ax N, and it has the cardinality of the continuum if A x R We will say that the cardinality of a set A is at most that of B (denoted A ~ B) if there exists a onetoone map from A to B. The SchroderBernstein theorem, sketched in Exercise 0.6.5, shows that if A~ Band B ~A, then Ax B. The fact that JR. and N have different cardinalities raises all sorts of questions. Are there other infinities besides those of N and JR.? We will see in Proposition 0.6.1 that there are infinitely many. For any set E, we denote by P(E) the set of all subsets of E, called the power set of E. Clearly for any set E there exists a onetoone map f: E ____, P(E); for instance, the map J(a) = {a}. So the cardinality of E is at most that of P(E). In fact, it is strictly less. If E is finite and has n elements, then P(E) has 2n elements, clearly more than E (see Exercise 0.6.2). Proposition 0.6.1 says that this is still true if E is infinite. Proposition 0.6.1. A mapping f: E ____, P(E) is never onto.
24
o.
Chapter 0.
Preliminaries
Proof. Let A c E be the set of x such that x ~ f (x). We will show that A is not in the image of f, and thus that f is not onto. Suppose A is in the image off. Then A must be of the form f(y) for an appropriate y E E. But if y is in A= f(y), this implies that y ~A, and if y is not in A= f(y), this implies that y E A. Thus A is not an element of the image of f, and f is not onto. D Thus JR, P(JR), P(P(JR)), ... all have different cardinalities, each bigger than the last. Exercise 0.6.8 asks you to show that P(N) has the same cardinality as JR.
FIGURE
0.6.4.
Paul Cohen (19342007) Proving the continuum hypothesis was the first in the famous list of 23 problems Hilbert presented to the Second International Congress of Mathematicians in 1900. Cohen and Godel, working some 30 years apart, proved that the problem is unsolvable.
The continuum hypothesis Another natural question is: are there infinite subsets of JR that cannot be put into onetoone correspondence with either JR or N? Cantor's continuum hypothesis says that the answer is no. This statement has been shown to be unsolvable: it is consistent with the other axioms of set theory to assume it is true (Godel, 1938) or false (Paul Cohen, 1965). This means that if there is a contradiction in set theory assuming the continuum hypothesis, then there is a contradiction without assuming it, and if there is a contradiction in set theory assuming that the continuum hypothesis is false, then again there is a contradiction without assuming it is false.
EXERCISES FOR SECTION
0.6
0.6.1 a. Show that the set of rational numbers is countable (i.e., that you can list all rational numbers).
b. Show that the set ]]} of finite decimals is countable. FIGURE
0.6.5 .
Kurt Godel (19061978) is considered the greatest logician of all time. His brother, a doctor, wrote that Godel " . . . believed all his life that he was always right not only in mathematics but also in medicine, so he was a very difficult patient for doctors." Many of the exercises for Section 0.6 are difficult, requiring a lot of imagination.
0.6.2 a. Show that if E is finite and has n elements, then the power set P(E) has 2n elements.
b. Choose a map f : {a, b, c} > P( {a, b, c} ), and compute for that map the set {xix tt f(x)}. Verify that this set is not in the image off. 0.6.3 Show that (1, 1) has the same infinity of points as the reals. Consider the function g(x) = tan(1rx/2).
Hint:
0.6.4 a. How many polynomials of degree d are there with coefficients with absolute value ::::; d?
b. Give an upper bound for the position in the list of formula 0.6.5 of the polynomial x 4  x 3 + 5. c. Give an upper bound for the position of the real cube root of 2 in the list of numbers made from the list of formula 0.6.5. 0.6.5 Let f : A > B and g : B > A be one to one. We will sketch how to construct a mapping h : A > B that is one to one and onto.
0. 7
Complex numbers
25
Let an (f, g )chain be a sequence consisting s alternately of elements of A and B, with the element following an element a EA being f(a) EB, and the element following an element b E B being g(b) EA. Exercise 0.6.5: The existence of such a mapping h is the SchroderBernstein theorem; we owe the proof sketched here to Thierry Gallouet at AixMarseille University.
a. Show that every (f, g )chain can be uniquely continued forever to the right, and to the left can 1. either be uniquely continued forever, or 2. can be continued to an element of A that is not in the image of g, or 3. can be continued to an element of B that is not in the image of f. b. Show that every element of A and every element of B is an element of a unique such maximal (f,g)chain. c. Construct h : A
h(a)
={
f(a) g 1 (a)
+
B by setting if a belongs to a maximal chain of type 1 or 2 above, if a belongs to a maximal chain of type 3.
Show that h : A + B is well defined, one to one, and onto. Exercise 0.6.7: See Section 1.1 for a discussion of the notation IR 2 d. Take A = [1, 1] and B = (1, 1) . It is surprisingly difficult to write a and llr. Hint for part a: Conmapping h : A+ B. Take f : A+ B defined by f(x) = x/2, and g : B+ A sider the "mapping" that takes defined by g(x) = x. (O.a1a2 ... , O.b1b2 . .. ) and returns What elements of [1, 1] belong to chains of type 1, 2, 3? the real number O.a1b1a2b2 .... What map h: [1, 1]+ (1, 1) does the construction in part c give? There are difficulties with numbers that can be written either ending in O's or 9's; make choices 0.6.6 Show that the points of the circle { ( ~) E IR 2 x 2 + y 2 = 1 } have the so as to make the mapping one to same infinity of elements as R Hint: This is easy if you use Bernstein's theorem one, and apply Bernstein's theo(Exercise 0.6.5.) rem. a. Show that [O, 1) x [O, 1) has the same cardinality as [O, 1). Part c: In 1877 Cantor wrote *0.6.7 to Dedekind, proving that points b. Show that IR 2 has the same infinity of elements as R on the interval [O, 1] can be put c. Show that !Rn has the same infinity of elements as IR. in onetoone correspondence with points in !Rn. "I see it, but I don't *0.6.8 Show that the power set P(N) has the same cardinality as IR. believe it!" he wrote. *0.6.9 Is it possible to list (all) the rationals in [O, 1], written as decimals, so that the entries on the diagonal also give a rational number?
I
0.7
COMPLEX NUMBERS
Complex numbers were introduced about 1550 by several Italian mathematicians trying to solve cubic equations. Their work represented the rebirth of mathematics in Europe after a sleep of more than 15 centuries. A complex number is written a + bi, where a and b are real numbers, and addition and multiplication are defined in equations 0.7.1 and 0.7.2. It follows from those rules that i 2 = 1.
26
Chapter 0.
Preliminaries
The complex number a+ ib is often plotted as the point (
b) E JR
2.
The
real number a is called the real part of a + ib, denoted Re (a + ib); the real number b is called the imaginary part, denoted Im (a + ib). The reals JR can be considered as a subset of the complex numbers C, by identifying a E JR with a+ iO EC; such complex numbers are called real, as you might imagine. Real numbers are systematically identified with the real complex numbers, and a + iO is usually denoted simply a. Numbers of the form 0 + ib are called purely imaginary. What complex numbers, if any, are both real and purely imaginary? 2 If we plot a + ib as the point (
b) E JR
2,
what do the purely real numbers correspond to? The
purely imaginary numbers? 3 FIGURE
0.7.1.
The Italian Girolamo Cardano (15011576) used complex numbers (considered "impossible") as a crutch that made it possible to find real roots of real cubic polynomials. But complex numbers turned out to have immense significance in mathematics, and even physics. In quantum mechanics a state of a physical system is given by a "wave function", which is a complexvalued function; realvalued functions will not do. Equation 0.7.2 is not the only possible definition of multiplication. For instance, we could define
(a1
+ ib1)*(a2 + ib2)
= (a1a2) + i(b1b2). But then there would be lots of elements by which one could not divide, since the product of any purely real number and any purely imaginary number would be 0:
(a1
+ iO) * (0 + ib2) = 0.
The four properties concerning addition don't depend on the special nature of complex numbers; we can similarly define addition for ntuples of real numbers, and these rules will still be true.
Arithmetic in at allows us to perlorm
1.2
"When Werner Heisenberg discovered 'matrix' mechanics in 1925, he didn't know what a matrix was (Max Born had to tell him), and neither Heisenberg nor Born knew what to make of the appearance of matrices in the context of the atom."Manfred R. Schroeder, "Number Theory and the Real World," Mathematical Intelligencer, Vol. 7, No. 4. The entry a;,j of a matrix A is the entry at the intersection of the ith row and jth column; it is the entry you find by taking the elevator to the ith floor (row) and walking down the corridor to the jth room. One student objected that here we speak of the ith row and jth column, but in Example 1.2.6 we write that "the ith column of A is Ae;." Do not assume that i has to be a row and j a column. "The ith column of A is Ae;'' is just a convenient way to say "The first column of A is Ae1, the second column is Ae2 , etc." The thing to remember is that the first index of a;,j refers to the row, and the second to the column; thus a3,2 corresponds to the entry of the third row, second column. Another reader pointed out that sometimes we write "n x m matrix" and sometimes "m x n". Again, which is n and which ism doesn't matter, and we could just as well write s x v. The point is that the first letter refers to height and the second to width.
Matrices
43
another operation: matrix multiplication. We will see in Section 1.3 that every linear transformation corresponds to multiplication by a matrix, and (Theorem 1.3.10) that composition of linear transformations corresponds to multiplying together the corresponding matrices. This is one reason matrix multiplication is a natural and important operation; other important applications of matrix multiplication are found in probability theory and graph theory. Matrix multiplication is best learned by example. The simplest way to compute AB is to write B above and to the right of A . Then the product AB fits in the space to the right of A and below B, the (i,j)th entry of AB being the intersection of the ith row of A and the jth column of B, as shown in Example 1.2.3. Note that for AB to exist, the width of A must equal the height of B. The resulting matrix then has the height of A and the width of B.
Example 1.2.3 (Matrix multiplication). The first entry of the first row of AB is obtained by multiplying, one by one, the entries of the first row of A by those of the first column of B, and adding these products together: in equation 1.2.1, (2 x 1) + (1 x 3) = 1. The second entry of the first row is obtained by multiplying the first row of A by the second column of B: (2 x 4) + (1 x 0) = 8. After multiplying the first row of A by all the columns of B, the process is repeated with the second row of A: (3 x 1) + (2 x 3) = 9, and so on: B
4 0
~J
[~ ~]
8 12
6] 2
A
AB
.,._..
1.2.1
Given the matrices
A=[~ ~] B=[~ ~]
~]
1
0
what are the products AB, AC and CD? Check your answers in the footnote below. 1 Now compute BA. What do you notice? What if you try to compute CA ?2 Next we state the formal definition of the process we've just described. If the indices bother you, refer to Figure 1.2.1. i
AB= [OO 51] ;
2 Matrix
1 AC= [ 51 2
11] ;
CD
multiplication is not commutative; BA
Although the product AC exists, CA does not.
=
t
= ;
1] . 00 1
~] i= AB = [ ~
44
Chapter 1.
Vectors, matrices, and derivatives
Definition 1.2.4 (Matrix multiplication). If A is an m x n matrix whose (i, j)th entry is ai,j, and B is an n x p matrix whose (i, j)th entry is bi,j, then C = AB is the m x p matrix with entries n
Definition 1.2.4: Note that the summation is over the inner index k of a;,kbk ,j . Definition 1.2.4 says nothing new, but it provides some practice moving between the concrete (multiplying two particular matrices) and the symbolic (expressing this operation so that it applies to any two matrices of appropriate dimensions, even if the entries are complex numbers or functions, not real numbers.) In linear algebra one is constantly moving from one form of representation to another. For example, as we have seen, a point b in nr can be considered as a single entity, b, or as the ordered list of its coordinates; a matrix A can be thought of as a single entity, A, or as the rectangular array of its entries.
ci,j
=
L ai,kbk,j
1.2.2
k=l
p  ·· ,
n
mi
ith row
l ' 
~~
   
jth
n
col.
FIGURE 1.2.1. The entry Ci,j of the matrix C =AB is the sum of the products of the entries a;,k of the matrix A and the corresponding entry bk,j of the matrix B. The entries a;,k are all in the ith row of A; the first index i is constant, and the second index k varies. The entries bk,j are all in the jth column of B; the first index k varies, and the second index j is constant. Since the width of A equals the height of B, the entries of A and those of B can be paired up exactly.
Remark. Often matrix multiplication is written in a row: [A][B] = [AB] . The format shown in Example 1.2.3 avoids confusion: the product of the ith row of A and the jth column of B lies at the intersection of that row and column. It also avoids recopying matrices when doing repeated multiplications. For example, to multiply A times B times C times D we write
[ B ] [(AB)]
[
C
]
[(AB)C]
[
D
l
1.2.3
[(ABC)D]
Noncommutativity of matrix multiplication As we saw earlier, matrix multiplication is not commutative. It may well be possible to multiply A by B but not B by A. Even if both matrices have
1.2 Matrices
45
the same number of rows and columns, AB will usually not equal BA, as shown in Example 1.2.5. Example 1.2.5 (Matrix multiplication is not commutative) .
is not equal to A
AB
[~ [~ ~J [~
n
n
1.2.4
Multiplying a matrix by a standard basis vector FIGURE 1.2 .2. The ith column of the product AB depends on all the entries of A but only the ith column of B.
Multiplying a matrix A by the standard basis vector ei selects out the ith column of A, as shown in the following example. We will use this fact often. Example 1.2.6 (The ith column of A is Aei )· Below, we show that the second column of A is Ae2:
corresponds to
A
.
D,
AB
1.2.3 . The jth row of the product AB depends on all the entries of B but only the jth row of A . FIGURE
Similarly, the ith column of AB is Abi, where bi is the ith column of B , as shown in Example 1.2.7 and represented in Figure 1.2.2. The jth row of AB is the product of the jth row of A and the matrix B, as shown in Example 1.2.8 and Figure 1.2.3. Example 1.2.7 (The ith column of AB is Abi). The second column of the product AB is the product of A and the second column of B: b2 ,.......
B
1 3
4
0
;J
[~ [; .....A
8 12
6]
[
;J
AB
2
[~]
;J
8 [; [ 12] .._,,,_, .....A Ab2
1.2.5
46
Chapter 1.
Vectors, matrices, and derivatives
Example 1.2.8 . The second row of the product AB is the product of the second row of A and the matrix B : B
;J
[~
[~
;J
4
0
[!
8 12
6] 2
B
[ [3
2]
1
4
3
0
;J
9
12
2]
1.2.6
~
A
AB
Matrix multiplication is associative FIGURE
1.2.4.
Arthur Cayley (18211895) introduced matrices in 1858. Cayley worked as a lawyer until 1863, when he was appointed professor at Cambridge. As professor, he "had to manage on a salary only a fraction of that which he had earned as a skilled lawyer. However, Cayley was very happy to have the chance to devote himself entirely to mathematics." From a biographical sketch by J. J. O'Connor and E. F. Robertson. For more , see the MacTutor History of Mathematics archive at http://wwwhistory .mcs.stand.ac. uk/history /
When multiplying the matrices A, B, and C, we could set up the repeated multiplication as we did in equation 1.2.3, which corresponds to the product (AB)C. We can use another format to get the product A(BC):
[ B ]
[
C
or
[ A ]
[ AB ]
[ c
]
[ B ]
[ (BC)
] . 1.2.7
[ A ]
[ A(BC)]
]
[ (AB)C ]
Is (AB)C the same as A(BC)? In Section 1.3 we see that associativity of matrix multiplication follows from Theorem 1.3.10. Here we give a computational proof. Proposition 1.2.9 (Matrix multiplication is associative). If A is an n x m matrix, B an m x p matrix, and C a p x q matrix, so that (AB)C and A(BC) are both defined, then they are equal:
(AB)C = A(BC).
In his 1858 article on matrices, Cayley stated that matrix multiplication is associative but gave no proof. The impression one gets is that he played around with matrices (mostly 2 x 2 and 3 x 3) to get some feeling for how they behave, without worrying about rigor. Concerning another matrix result, Theorem 4.8.27 (the CayleyHamilton theorem) he verified it for 2 x 2 and 3 x 3 matrices, and stopped there.
1.2.8
Proof. Figure 1.2.5 shows that the (i,j)th entry of both A(BC) and (AB)C depend only on the ith row of A and the jth column of C (but on all the entries of B). Without loss of generality we can assume that A is a line matrix and C is a column matrix (n = q = 1), so that both (AB)C and A(BC) are numbers. Now apply associativity of multiplication of numbers:
(AB)C =
t ..._,,_.., (~akbk,l) c1 Ith entry of AB
=
t~akbk,1C1 = ~ak
(tbk,1ci) = '.....' kth entry of BC
1.2.9
A(BC).
D
1.2
Matrices
47
q
p
c
1
p
q m
Exercise 1.2.24 asks you to show that matrix multiplication is distributive over addition:
A(B + C)
= AB+ AC
(B+C)A
= BA+CA.
B
c
m
,]
, ~ · ~
JL__
I
IP I
B
A
BC
A
n
m
Not all operations are associative. For example, the operation that takes two matrices A , B and gives AB  BA is not associative. The cross product, discussed in Section 1.4, is also not associative.
1.2.5.
This way of writing the matrices corresponds to calculating (AB)C. The ith row of AB depends on the i th row of A and the entire matrix B . RIGHT : This format corresponds to calculating A(BC). The jth column of BC depends on the jth column of C and the entire matrix B. FIGURE
LEFT :
The identity matrix The identity matrix I plays the same role in matrix multiplication as the number 1 in multiplication of numbers: I A = A = AI.
The main diagonal is also called the diagonal. The diagonal from bottom left to top right is the antidiagonal.
Definition 1.2.10 {Identity matrix). The identity matrix In is the n x nmatrix with l's along the main diagonal (the diagonal from top left to bottom right) and O's elsewhere.
Fo 3, how can n vectors be orthogonal, or is that some weird math thing you just can't visualize?" We can't visualize four (or 17, or 98) vectors all perpendicular to each other. But we still find the mental image of a right angle helpful. To determine that two vectors are orthogonal, we check that their dot product is 0. But we think of orthogonal vectors v, w as being at right angles to each other; we don't think V1W1 +· · · +VnWn = 0. We prefer the word "orthogonal" to its synonym "perpendicular". Orthogonal comes from the Greek for "right angle," while perpendicular comes from the Latin for "plumb line," which suggests a vertical line. The word "normal" is also used, both as noun and as adjective, to express a right angle.
Chapter 1.
Vectors, matrices, and derivatives
Example 1.4. 7 (Finding an angle) . What is the angle between the diagonal of a cube and any edge? Let us assume that our cube is the unit cube 0 ::; x, y, z ::; 1, so that the standard basis vectors ei, e2, e3 are edge,, and the vecto< is
ldl = J3,
cl ~ [:] ;, a diagonal.
The length of the diagonal
so the required angle a satisfies
Ci· e~
cos a= ldlle~I Thus a= arccosJ3/3 ~
54.7°.
1
1.4.23
= J3'
6
Corollary 1.4.8. Two vectors are orthogonal (form a right angle) if their dot product is zero. Schwarz's inequality also gives us the triangle inequality: to travel from London to Paris, it is shorter to go across the English Channel than by way of Moscow. We give two variants of the inequality on pages 109 and 113. Theorem 1.4.9 (The triangle inequality). For vectors
Ix+ 'YI :::: IX.I+ l'YI·
x, y E lRn, 1.4.24
Proof. This inequality is proved by the following computation:
lx+'Yl 2 = lxl 2 +2x·'Y+l'Yl 2
lxl 2 +21xll'Yl+l'Yl 2 = (lxl+l'Yl) 2 , 1.4.25
< Schwarz
so that
~I
i
\ l:YI
lx+:YI
I
I
··'
1(·/·
j
...·/. ·/1:::1· x
I FIGURE
1.4. 7.
The triangle inequality:
lx+:YI < lxl+l:YI·
Ix+ 'YI :::: lxl + I.YI. o
The triangle inequality can be interpreted (in the case of strict inequality, not ::;) as the statement that the length of one side of a triangle is less than the sum of the lengths of the other two sides, as shown in Figure 1.4.7.
Measuring matrices The dot product gives a way to measure the length of a vector. We will also need a way to measure the "length" of a matrix (not to be confused with either its height or its width) . There is an obvious way to do this: consider an m x n matrix as a point in JR nm, and use the ordinary dot product. Definition 1.4.10 (The length of a matrix). If A is an n x m matrix, its length IAI is the square root of the sum of the squares of all its entries: n
m
IAl ~ LLaI,j· 2
i=l j=l
1.4.26
1.4
For example, the length 1+4+0+1
= 6.
IAI
of the matrix A
The geometry of nr
= [01 21] is
\1'6,
= [~ ~
!]
What is the length of the matrix B
73
since
?9
If you find double sum notation confusing, equation 1.4.26 can be rewritten
as a single sum:
L
IAl 2 =
aL. (We sum all aI,j for i from 1 ton and j from 1 tom.)
i =l , ... ,n j=l, ... ,m
In some texts, JAJ denotes the determinant of the matrix A. We use 0
1.4.44
Proof. 1. To check that the cross product a x b is orthogonal to a and b, we check that the dot product in each case is zero (Corollary 1.4.8). For instance, a x b is orthogonal to a because Definition 1.4.17 of a'.xb
a t• x hJ ~ [ ;; ] ·
in part 3 of Proposition 1.4.20 by det[i x h,i, b} > o.
and b. (ax b) = 0.
2. Its length lax bl is the area of the parallelogram spanned by a andb. 3. If a and b are not collinear, then det[a, b, a x b] > o.
= det(b, c, i) so we could replace
aand b:
1.4.45
= aia2b3  aia3b2  aia2b3
+ a2a3b1 + aia3b2
 a2a3b1 = 0.
2. The area of the parallelogram spanned by a and bis lallbl sinO, where
0 is the angle between a and b. We know (equation 1.4.7) that cos 0 =
a. b
~
lallbl
=
+ a2b2 + a3b3 Jar+ a~+ a5Jbi + b~ + b5' aib1
;:::::;;==::::;;==::;;:;=:;;;:::==;;:=::::;;:
1.4.46
so we have sin 0 =
J1 
cos 2 0 = 1.4.47
(ar +a~+ a5)(br + b~ + b5)  (a1b1 + a2b2 (ar +a~+ a5)(br + b~ + b5)
+ a3b3) 2
1.4
The geometry of !Rn
79
so that lallbl sinB = V(ar
+a~+ a§}(br + b~ + b~) 
(a1b1
+ a2b2 + a3b3) 2 .
1.4.48
Carrying out the multiplication results in a formula for the area that looks worse than it is: a long string of terms too big to fit on this page under one square root sign. That's a good excuse for omitting it here. But after cancellations we have the following on the right side:
1.4.49 which conveniently gives us Area = lallbl sinB = y'(a1b2  a2b1) 2 + (a1b3  a3b1) 2 + (a2b3  a3b2) 2
1.4.50
=la x bl .
3. By Proposition 1.4.19, det[a x b,a, bJ =(ax b) . (a x b) =lax bl 2 . Proposition 1.4.21: The word "parallelepiped" seems to have fallen into disuse. It is simply a possibly slanted box: a box with six faces, each of which is a parallelogram; opposite faces are equal.
Thus det[a x
b,a,b]
b ¥= 0, so
det[a x
a x
1.4.51
2 0, but by part 2 if a , b are not collinear then
b, a, b] > 0.
D
Proposition 1.4.21 (Determinant in JR3 ) . 1. If p is the parallelepiped spanned by three vectors
a, b, c, then
volume of P =la . (bx c)I = I det[a, b,c]J. 2. If a, b, c are coplanar, then det[a, b, c] = 0.
3. If e1,
e2, e3 are drawn the standard way, so they satisfy the righthand rule, then det[a, b, c] > 0 if a, b, c (in that order) satisfy the righthand rule.
FIGURE
IR 3 ,
1.4.12.
In the standard basis vectors, drawn the usual way, "fit" the right hand but not the left hand. Any three vectors that are not coplanar and "fit" the right hand like this are said to satisfy the righthand rule.
In part 3, note that three different permutations of the vectors give the same result: a , b, c satisfy the righthand rule if and only if b, c, a and c, a, b satisfy the righthand rule. (These are two instances of even permutations , which we will explore in Section 4.8.)
Proof of Proposition 1.4.21. 1. The proof is illustrated by Figure 1.4.13. The volume is height x area of the base. The base is the parallelogram spanned by band c; its area is lb x cl . The height is the distance from a to the plane spanned by b and c. Let B be the angle between a and the line through the origin perpendicular to the plane spanned by b and c; the height is then Jal cosB. This gives Volume of parallelepiped =lb x cllal cosB =a. (bx c)I.
1.4.52
2. If a, b, care coplanar' then the volume of the parallelepiped they span is 0, so by part 1 det[a, b, c] = 0.
80
z
Chapter 1.
Vectors, matrices, and derivatives
3. If a, b, care not coplanar, you can rotate JR3 to place them at i!' b'' 2' where a' is a positive multiple of ei, the vector b' is in the (x, y )plane with positive ycoordinate (i.e., counterclockwise from ei), and the zcoordinate of c' is nonzero. While being rotated, the vectors never become coplanar so the determinant never vanishes, and det[a, b, c] = det[i!, b', c'J. Then y
i
a'
x
det[a, b, c] = 0, and the other hand if c3 < 0. See Figure 1.4.12. !:::, The relationships between algebra and geometry form a constant theme of mathematics. Table 1.4.14 summarizes those discussed in this section. Correspondence between Algebra and Geometry Algebra Geometry
Operation dot product determinant of 2 x 2 matrix
v ·w
JR3 .
~1
= v2(a·v)a is a linear
b. What is Ta(a)? If v is orthogonal to a, what is Ta(v)? Can you give a name to Ta? c. Write the matrix M of T 8 (in terms of a, b, c, of course). What can you say of M 2 ? 1.4.28 Let M be an n x m matrix. Show that IMl 2 = tr( MT M), where tr is the trace (sum of diagonal entries; see Definition 4.8.14).
1.5 LIMITS AND CONTINUITY Integrals, derivatives, series, approximations: calculus is all about convergence and limits. It could easily be argued that these notions are the
84
The inventors of calculus in the seventeenth century did not have rigorous definitions of limits and continuity; these were achieved only in the 1870s. Rigor is ultimately necessary in mathematics, but it does not always come first, as Archimedes acknowledged in a manuscript discovered in 1906. In it Archimedes reveals that his deepest results were found using dubious arguments, and only later proved rigorously, because "it is of course easier to supply the proof when we have previously acquired some knowledge of the questions by the method, than it is to find it without any previous knowledge." (We found this story in John Stillwell's Mathematics and Its History, Springer Verlag, 1997.)
FIGURE
1.5.1.
An open set includes none of the fence; however close a point in the open set is to the fence, you can always surround it with a ball of other points in the open set.
Chapter 1.
Vectors, matrices, and derivatives
hardest and deepest of all of mathematics; mathematicians struggled for two hundred years to come up with correct definitions. More students have foundered on these definitions than on anything else in calculus: the combination of Greek letters, precise order of quantifiers, and inequalities is a hefty obstacle. Fortunately, these notions do not become more difficult in several variables than they are in one variable. Working through a few examples will help you understand what the definitions mean, but a proper appreciation can probably only come from use; we hope you have already started on this path in onevariable calculus.
Open and closed sets In mathematics we often need to speak of an open set U; whenever we want to approach points of a set U from every side, U must be open. Think of a set or subset as your property, surrounded by a fence. The set is open if the entire fence belongs to your neighbor. As long as you stay on your property, you can get closer and closer to the fence, but you can never reach it; no matter how close you are to your neighbor's property, there is an epsilonthin buffer zone of your property between you and it. The set is closed if you own the fence. Now, if you sit on your fence, there is nothing between you and your neighbor's property. If you move even an epsilon further, you will be trespassing. What if some of the fence belongs to you and some belongs to your neighbor? Then the set is neither open nor closed. To state this in proper mathematical language, we first need to define an open ball. Imagine a balloon of radius r, centered around a point x. The open ball of radius r around x consists of all points y inside the balloon, but not the skin of the balloon itself. We use a subscript to indicate the radius of a ball B; the argument gives the center of the ball: B2(a) is a ball of radius 2 centered at the point a.
Definition 1.5.1 (Open ball). For any x E IR.n and any r > 0, the open ball of radius r around x is the subset
Br(x) ~ {y E IR.n such that be
Ix yl < r}.
1.5.l
Note that Ix  YI must be less than r for the ball to be open; it cannot = r. In dimension 1, a ball is an interval.
Definition 1.5.2 (Open set of IR.n). A subset UC IR.n is open if for every point x E U, there exists r > 0 such that the open ball Br (x) is contained in U. An open set is shown in Figure 1.5.1, a closed set in Figure 1.5.2. FIGURE
1.5.2.
A closed set includes its fence .
Example 1.5.3 (Open sets). 1. If a < b, then the interval (a, b) = { x E JR. I a < x < b } is open. Indeed, if x E (a, b), set r = min { x  a, b  x}. Both these numbers
1.5 Limits and continuity
Parentheses denote an open interval and brackets denote a closed one: (a, b) is open, [a, b] is closed. Sometimes backwards brackets are used to denote an open interval: ]a,b[= (a,b). Open and closed subsets of nr are special; you shouldn't expect a subset to be either. For instance, the set of rationals Q C JR is neither open nor closed: every neighborhood of a rational number contains irrational numbers, and every neighborhood of an irrational number contains rational numbers.
D
I~
FIGURE 1.5.3. The natural domain of
is neither open nor closed. It includes the yaxis (with the origin removed) but not the xaxis.
85
are strictly positive, since a < x < b, and so is their minimum. Then the ball { y I IY xi< r} is a subset of (a,b). 2. The rectangle
(a, b) x (c, d) = { (
~)
E IR 2
Ia
0 in any direction possible in your particular !Rn, without leaving U. Thus an open subset U C !Rn is necessarily ndimensional. In contrast, a plane in IR3 cannot be open; a flatworm living in the plane cannot lift its head out of the plane. A line in a plane, or in space, cannot be open; a "lineworm" living in a line that is a subset of a plane or of space can neither wiggle from side to side nor lift its head. A closed ndimensional subset of IRn is "thick" in its interior (if the interior isn't empty) but not on its fence. If an ndimensional subset of !Rn is neither closed nor open, then it is "thick" everywhere except on the part of fence that it owns.
Before discussing the crucial topics of this section  limits and continuity we need to introduce some more vocabulary. We will use the word neighborhood often; it is handy when we want to describe a region around a point without requiring it to be open; a neighborhood contains an open ball but need not be open itself.
Definition 1.5.7 {Neighborhood). A neighborhood of a point x E !Rn is a subset X C !Rn such that there exists E > 0 with Be(x) C X. Most often, we deal with sets that are neither open nor closed. But every set is contained in a smallest closed set, called its closure, and every set contains a biggest open set, called its interior. (Exercise 1.5.4 asks you to show that these characterizations of closure and interior are equivalent to the following definitions.)
Definition 1.5.8 (Closure). If A is a subset of !Rn, the closure of A, denoted A, is the set of x E !Rn such that for all r > 0, 1.5.6
Definition 1.5.9 (Interior). If A is a subset of !Rn, the interior of A, 0
denoted A, is the set of x E !Rn such that there exists r > 0 such that
Br(x) CA.
1.5.7
The closure of a closed set is itself; the interior of an open set is itself. We spoke of the "fence" of a set when we defined closed and open sets informally. The technical term is boundary. A closed set contains its boundary; an open set contains none of its boundary. We used the word "fence" because we think it is easier to think of owning a fence than owning a boundary. But "boundary" is generally more appropriate. The boundary of the rational numbers is all of JR, which would be difficult to imagine as a picket fence.
Definition 1.5.10 (Boundary of subset). The boundary of a subset A c !Rn, denoted aA, consists of those points x E !Rn such that every neighborhood of x intersects both A and the complement of A. Equation 1.5.8: Remember that U denotes "union": A U B is the set of elements of either A or B or both. The notation of set theory is discussed in Section 0.3.
The closure of A is thus A plus its boundary; its interior is A minus its boundary: 0
A=AuaA
A= AaA.
and
1.5.8
The boundary is the closure minus the interior: 
0
aA =AA.
1.5.9
1.5 Limits and continuity Exercise 1.18 asks you to show that the closure of U is the subset of nr made up of all limits of sequences in U that converge in Rn.
Exercise 1.5.6 asks you to show that aA is the intersection of the closure of A and the closure of the complement of A.
87
Exercise 1.5.4 asks you to justify equations 1.5.8 and 1.5.9.
Examples 1.5.11 (Closure, interior, and boundary). 1. The sets (0, 1), [O, 1], [O, 1), and (0, 1] all have the same closure, interior, and boundary: the closure is [O, 1], the interior is (0, 1), and the boundary consists of the two points 0 and l.
2. The sets { (
~)
E JR 2
Ix
2
+ y2 < 1 }
and
{ ( ~) E JR 2
Ix
2
+ y2
:::;
1}
both have the same closure, interior, and boundary: the closure is the disc of equation x 2 + y 2 :::; 1, the interior is the disc of equation x 2 + y 2 < 1, and the boundary is the circle of equation x 2 + y 2 = l. 3. The closure of the rational numbers Q C JR is all of JR: the intersection of every neighborhood of every real number with Q is not empty. The interior is empty, and the boundary is JR: every neighborhood of every real number contains both rationals and irrationals. 4. The closure of the open unit disc with the origin removed: FIGURE 1.5.4.
The shaded region is U in Example 1.5.11, part 5. You can approach the origin from this region, but only in rather special ways.
U= { (
~)
E JR 2
I 0 < x 2 + y2 < 1 }
1.5.10
is the closed unit disc; the interior is itself, since it is open; and its boundary consists of the unit circle and the origin. 5. The closure of the region U between two parabolas touching at the origin, shown in Figure 1.5.4: 1.5.11 is the region given by
1.5.12 0
in particular, it contains the origin. The set is open, so U boundary is the union of the two parabolas. 6
= U.
The
Convergence and limits FIGURE 1.5.5. Karl Weierstrass (18151897) At his father's insistence, Weierstrass studied law, finance, and economics, but he refused to take his final exams. Instead he became a teacher, teaching mathematics, history, physical education and handwriting. In 1854 an article on Abelian functions brought him to the notice of the mathematical world.
Limits of sequences and limits of functions are the fundamental construct of calculus, as you will already have seen in first year calculus: both derivatives and integrals are defined as limits. Nothing can be done in calculus without limits. The notion of limit is complicated; historically, coming up with a correct definition took 200 years. It wasn't until Karl Weierstrass (18151897) wrote an incontestably correct definition that calculus became a rigorous subject.11 A major stumbling block to writing an unambiguous definition 11 Many great mathematicians wrote correct definitions of limits before Weierstrass: Newton, Euler, Cauchy, among others. However, Weierstrass was the first to show that his definition, which is the modern definition, provides an adequate foundation for analysis.
88
Chapter 1.
Vectors, matrices, and derivatives
was understanding in what order quantifiers should be taken. (You may wish to review the discussion of quantifiers in Section 0.2.) We will define two notions of limits: limits of sequences and limits of functions. Limits of sequences are simpler; for limits of functions one needs to think carefully about domains. Unless we state explicitly that a sequence is finite, it is infinite. Definition 1.5.12: Recall from Section 0.2 that 3 means "there exists" ; \;/ means "for all" . In words, the sequence i >+ ai converges to a if for all € > 0 there exists M such that when m > M, then lam  al < €. Saying that i >+ ai converges to 0 is the same as saying that lim;~oo jail= O; we can show that a sequence of vectors converges to 0 by showing that their lengths converge to 0. Note that this is not true if the sequence converges to a i= 0.
Infinite decimals are limits of convergent sequences. If ao = 3,
a2
= 3.1, = 3.14,
an
= 7r to n
ai
decimal places,
how large must M be so that if n ~ M, then Ian  trl < 10 3 ? The answer is M = 3: 7r 
3.141
= .0005926 ....
The same argument holds for any real number.
Definition 1.5.12 (Convergent sequence; limit of sequence). A sequence i ~ 3i of points in IRn converges to a E IRn if \/€
> 0, 3M I m > M
=}
la.n  al < €.
1.5.13
We then call a the limit of the sequence. Exactly the same definition applies to a sequence of vectors: just replace a in Definition 1.5.12 by a, and substitute the word "vector" for "point". Convergence in !Rn is just n separate convergences in JR:
Proposition 1.5.13 (Convergence in terms of coordinates) . A sequence m ~ a.n with a.n E JRn converges to a if and only if each coordinate converges; i.e., if for all j with 1 :::; j :::; n the jth coordinate (ai)j of 3i converges to ai, the jth coordinate of the limit a. The proof is a good setting for understanding how the (t:, M)game is played (where M is the M of Definition 1.5.12). You should imagine that your opponent gives you an epsilon and challenges you to find an M that works: an M such that when m > M, then l(am)j  ail < t: . You get Brownie points for style for finding a small M, but it is not necessary in order to win the game.
Proof. Let us first see the easy direction: that m
~:.[::~~.:::::~~ul:~
~
a.n converges to a
.: [: •::.::::t(e(~::J, c)on::::::,::y (am)n
you have a teammate who knows how to play the game for the sequence m ~ a.n, and you hand her the epsilon you just got. She promptly hands you back an M with the guarantee that when m > M, then lam  al < t: (since the sequence m ~ am is convergent). The length of the vector a.n  a is
lam  al=
V((amh  ai)
2
+ · · · + ((am)n  an) 2 ,
1.5.14
so you give that M to your challenger, with the argument that 1.5.15 He promptly concedes defeat.
1.5
Limits and continuity
89
Now we have to show that convergence of the coordinate sequences implies convergence of the sequence m f+ am. Again the challenger hands you an E > 0. This time you have n teammates; each knows how the play the game for a single convergent coordinate sequence m f+ (am )j. After a bit of thought and scribbling on a piece of paper, you pass along i:/ .Jii, to each of them. They dutifully return to you cards containing numbers M 1 , ... , Mn, with the guarantee that
l(am)j  ajl
Mj.
1.5.16
You sort through the cards and choose the one with the largest number, M
= max{M1, ... ,Mn},
1.5.17
which you pass on to the challenger with the following message: if m > M, then m > Mj for each j = 1 · · · = n, so l(am)j  ajl < i:/ .Jii,, so
This is typical of all proofs involving convergence and limits: you are given an € and challenged to come up with a o (or M or whatever) such that a certain quantity is less than €. Your "challenger" can give you any € > 0 he likes; statements concerning limits and continuity are of the form "for all epsilon, there exists . . . " .
1.5.18 0
The scribbling you did was to figure out that handing i:/ .Jii, to your teammates would work. What if you can't figure out how to "slice up" E so that the final answer will be precisely i:? Just work directly with E and see where it takes you. If you use E instead of i::/.Jii, in inequalities 1.5.16 and 1.5.18, you will end up with
lam  al
0. By the definition of the limit, there exists Mi such that lam  al < Eo when m > Mi, and M2 such that lam  bl < Eo when m > M 2. Set M =max{ Mi, M2}· If m > M, then by the triangle inequality, la bl= l(a  am)+ (am  b)I
~la  aml +lam  bl< 2Eo = ...___.., "'...' i(j) when k > j.
and the subsequence is 1
1
1
ai(l)
ai(2)
ai(3)
2 ' 4 ' 6 , .. . we see that i(l) = 2, since 1/2 is the second entry of the original sequence. Similarly,
i(2)
= 4,
i(3)
= 6, . . ..
The proof of Proposition 1.5.19 is left as Exercise 1.5.18, largely to provide practice with the language.
You might take all the even terms, or all the odd terms, or all those whose index is a prime, etc. Of course, any sequence is a subsequence of itself. Sometimes the subsequence j f+ ai(j) is denoted ai(l), ai(2), . . . or ai1, a i 2> · · · ·
Proposition 1.5.19 (Subsequence of convergent sequence converges). If a sequence k f+ ak converges to a, then any subsequence of the sequence converges to the same limit a.
Limits of functions Limits like limx ...... xo f (x) can only be defined if you can approach Xo by points where f can be evaluated. Thus when xo is in the closure of the domain off, it makes sense to ask whether the limit limx ...... xo f(x) exists. Of course, this includes the case when xo is in the domain of f; in that case for the limit to exist we must have lim f(x) x~x 0
= f (xo).
1.5.24
But the interesting case is when x 0 is in the closure of the domain but not in the domain. For example, does it make sense to speak of lim(l + x) 1 f x?
x>O
1.5.25
92
Chapter 1.
Vectors, matrices, and derivatives
Yes; although we cannot evaluate the function at 0, the natural domain of the function includes (1, 0) U (0, oo), and 0 is in the closure of that set. 12 Definition 1.5.20: Recall (Definition 1.5.8) that if c nr' its closure X is the set of x E Ilr such that for all r > 0,
x
Definition 1.5.20 {Limit of a function). Let X be a subset of JR.n, and xo a point in X. A function f : X + JR.m has the limit a at Xo: lim f(x) =a
Br(x) n X :f.
0 as € > 0.
1.5.26
X+Xo
if for all
E
> 0 there exists o> 0 such that for all x Ix  xol
0
which is some number that varies between 0 and 1/e, and which depends on k (i.e., on the particular parabola along which you are approaching the origin; see Exercise 1.6.6). Thus if you approach the origin in different ways, the limits may be different. 6 A mapping f: !Rn+ !Rm is an "!Rmvalued" mapping; its argument is in JRn and its values are in JRm . Such a mapping is sometimes written in terms
of the coordinate functions. For example, the mapping f : JR2 Often lRmvalued mappings are called "vectorvalued" mappings, but usually we think of their values as points rather than vectors. We denote a mapping whose values are points in JRm with a boldface letter without arrow: f. Sometimes we do want to think of the values of a mapping ]Rn + ]Rm as vectors: when we are thinking of vector fields. We denote a vector field with an arrow: F or f.
f (
where
~)
= (
2~ : xy
)
can be written
Ji ( ~) = xy, h ( ~) = x 2 y,
and
f = (
+
JR3 ,
j~ ) ,
1.5.36
h
h ( ~) = x y.
Proposition 1.5.25 (Convergence by coordinates). Suppose U C JR",
f
~
cu ' ~ JR~, U
and X0 E U
Then lim f(x) = X+Xo
(aam·:l)
if and only if lim fi(x) = ai, i = 1, ... , m. X+Xo
Theorem 1.5.26 (Limits of functions). Let Ube a subset of!Rn and let f and g be functions U + !Rm and h a function U + R 1. If limx>xo f(x) and limx>xo g(x) exist, then limx_,x 0 (f exists, and lim (f + g)(x)
In Theorem 1.5.26, when we specify that limx~xo f(x) exists, this means that xo is in the closure of U . It does not have to be in U .
X+Xo
= lim f(x) + lim g(x). X+Xo
+ g)(x) 1.5.37
X+Xo
2. If limx+xo f(x) and limx>xo h(x) exist, then limx_,x 0 (hf)(x) exists, and lim (hf)(x) = lim h(x) lim f(x). X+Xo
x+Xo
1.5.38
X+Xo
3. If limx>xo f(x) exists, and limx>xo h(x) exists and is different from 0, then limx>xo(f )(x) exists, and lim x>xo
(!) (x) h
= limx>xo f(x) .
limx+xo h(x)
1. 5 .39
1.5
Limits and continuity
95
4. Set (f · g)(x) ~r f(x) · g(x). If limxxo f(x) and limxxo g(x) exist, then limxx0 (f · g)(x) exists, and lim (f · g)(x) = lim f(x) · lim g(x). X+Xo
X+Xo
X+Xo
1.5.40
5. Iff is bounded on U (i.e., there exists R such that jf(x)J ::; R for all x EU), and if limxxo h(x) = 0, then lim (hf)(x) = 0.
1.5.41
X+Xo
6. Iflimxxo f(x) = 0 and his bounded, then lim (hf)(x)
xxo
= 0.
1.5.42
Proof of Proposition 1.5.25. Let's go through the picturesque description again. For the implication => ("only if"), the challenger hands you an E > 0. You give it to a teammate who returns a o with the guarantee that when Ix  xol < oand f(x) is defined, then lf(x)  al < E. You pass on the same o, and ai, to the challenger, with the explanation lfi(x)  ail ::; lf(x)  al
+ sin 10m is a sequence in a compact set, and therefore (by Theorem 1.6.3) contains a convergent subsequence, we can't begin to "locate" that subsequence. We can't even say whether it is in [1,0) or [0,1).
If instead of writing Xm
= sin 10m we write 10m 10m = sin 27r 27r , i.e., a= 2 7r , Xm
1.6.6
we see, as stated above, that Xm is positive when the fractional part of 10m/(27r) is less than 1/2. So if a convergent subsequence of the sequence m >+sin 10m is contained in the box [O, 1), an infinite number of 10m /(27r) must have a fractional part that is less than 1/2. This will ensure that we have infinitely many Xm =sin 10m in [O, 1). For any single Xm, it is enough to know that the first digit of the fractional part of 10m/(27r) is 0, 1, 2, 3 or 4: knowing the first digit after the decimal point tells you whether the fractional part is less than or greater than 1/2. Since multiplying by 10m just moves the decimal point to the right by m, asking whether the fractional part of 10m/(27r) is in [O, 1/2) is really asking whether the (m + l)st digit of 2~ = .1591549 ... is 0, 1, 2, 3, or 4; saying that the Xm have a limit in [O, 1/2) is the same as saying that at least one of the digits 0, 1, 2, 3, 4 appears infinitely many times in the decimal expansion of l/27r. (Figure 1.6.3 illustrates the more refined question, whether the fractional part of 10m/(27r) is in [0.7,0.8).) Note that we are not saying that all the 10m /(27r) must have the decimal point followed by 0, 1, 2, 3, or 4! Clearly they don't. We are not interested in all the Xmi we just want to know that we can find a subsequence of the sequence m >+ Xm that converges to something inside the box [O, 1). For example, x 1 is not in [O, 1), since 10 x .1591549 . . . = 1.591549 ... ; the fractional part starts with 5. Nor is x 2 , since the fractional part starts with 9: 102 x .1591549 ... = 15.91549 .... But x 3 is in [O, 1), since the fractional part of 103 x .1591549 ... = 159.1549 . .. starts with 1. Everyone believes that the digits 0, 1, 2, 3, 4 appear infinitely many times in the decimal expansion of 2~ . Indeed, it is widely believed that 7r
108
Chapter 1.
Vectors, matrices, and derivatives
is a normal number (every digit appears roughly 1/lOth of the time, every pair of digits appears roughly 1/lOOth of the time, etc.). The first 4 billion digits of 7r have been computed and appear to bear out this conjecture. It is also widely believed that .J7r is normal. Still, no one knows how to prove it; as far as we know it is conceivable that all the digits after the 10 billionth are 6, 7, and 8. Thus, even choosing the first box Bo requires some Godlike ability to "see" this whole infinite sequence, when there is no obvious way to do it. 6
Definition 1.6.5: The least upper bound is defined in Definition 0.5.1; of course it only exists if the set of f(c) has an upper bound; in that case the function is said to be bounded above. The word "maximum" is sometimes used to denote a point in the domain and sometimes to denote a point in the codomain. We will use "maximum value" for the point M in the codomain and "maximum" for a point b in the domain such that J(b) = M . Some people use "maximum" to denote a least upper bound, rather than a least upper bound that is achieved. The greatest lower bound is defined in Definition 0.5.2: a number a is a lower bound for a subset X C JR if for every x E X we have x ;:::: a; a greatest lower bound is a lower bound b such that for any other lower bound a, we have b ;:::: a. Some people use the words "greatest lower bound" and "minimum" interchangeably. On the open set (0, 1) the greatest lower bound of f(x) = x 2 is 0, and f has no minimum value. On the closed set [O, 1], the number 0 is both the infimum of f and its minimum value.
Theorem 1.6.3 is nonconstructive: it proves that something exists but gives not the slightest hint of how to find it. Many mathematicians of the end of the nineteenth century were deeply disturbed by this type of proof; even today, a school of mathematicians called constructivists reject this sort of thinking. They demand that in order for a number to be determined, one give a rule for computing the successive decimals. Constructivists are pretty scarce these days; we have never met one. But we have a certain sympathy with their views and much prefer proofs that involve effectively computable algorithms, at least implicitly.
Continuous functions on compact sets We can now explore some consequences of Theorem 1.6.3. One is that a continuous function defined on a compact subset has both a point at which it has a maximum value and a point at which it has a minimum value. In the definitions below, C is a subset of ~n.
Definition 1.6.5 (Supremum). A number Mis the supremum of a function f : C+ ~if Mis the least upper bound of the values off (i.e., of the set of numbers f (c) for all c E C). It is denoted M = supcEC f (c). Definition 1.6.6 (Maximum value, maximum). A number Mis the maximum value of a function f : C + ~ if it is the supremum of f and there exists b E C such that f(b) = M. The point bis called a maximum of f. For example, on the open set (0, 1) the least upper bound of f(x) = x 2 is 1, and f has no maximum. On the closed set [O, 2], 4 is both the supremum and the maximum value off; the maximum is 2.
Definition 1.6. 7 (Infimum). A number mis the infimum of a function f : C + ~ if m is the greatest lower bound of the set of values off. It is denoted m = infcEC f. Definition 1.6.8 (Minimum value; minimum) . A number mis the minimum value of a function f defined on a set C C ~n if it is the infimum and if there exists c E C such that f (c) = m. The point c is then a minimum of the function.
1.6 Five big theorems
109
Theorem 1.6.9 enables you to go from local knowledge  that the function is continuous  to global knowledge: existence of a maximum and a minimum for the entire function. Note that C is compact. Recall that "compact" means "closed and bounded".
Theorem 1.6.9: One student objected, "what is there to prove?" It might seem obvious that the graph of a function on [O, 1] that you can draw must have a maximum. But Theorem 1.6.9 also talks about functions defined on compact subsets of JR 17 , about which we have no intuition. Moreover, it isn't quite so clear as you might think, even for functions on
[O, l].
Exercise 1.6.2 asks you to show that if A C !Rn is any noncompact subset, then there always is a continuous unbounded function on A.
By the triangle inequality, if l/(x)  /(b)I < e, then l/(x)I < l/(b)I + €: l/(x)I = l/(x)  /(b) + /(b)I :::; lf(x)  /(b)I + IJ(b)I < e + l/(b)I.
Theorem 1.6.9 (Existence of minima and maxima). Let Cc !Rn be a compact subset, and let f : C  JR be a continuous function. Then there exists a point a E C such that /(a) 2'. f (x) for all x E C, and a point b EC such that f(b) ~ f(x) for all x EC. Examples 1.6.10 (Functions with no maxima). Consider the function
f(x) = {
0
when x = 0
~
otherwise,
1.6.7
defined on the compact set [O, 1]. As x+ 0, we see that f(x) blows up to infinity; the function does not have a maximum value (or even a supremum); it is not bounded. This function is not continuous, so Theorem 1.6.9 does not apply to it. The function f(x) = 1/x defined on (0, 1] is continuous but has no maximum; this time the problem is that (0, 1] is not closed, hence not compact. The function f(x) = x defined on all of JR has no maximum (or minimum); this time the problem is that JR is not bounded, hence not compact. Now consider the function f(x) = lx~I' defined on the rational numbers Q in the closed interval [O, 2]. Restricting the domain this way avoids the potential trouble spot at J2, where lx1v'2i would blow up to infinity, but the function is not guaranteed to have a maximum because the rational numbers are not closed in R In some sense, Theorem 1.6.9 says that JR (and more generally !Rn) has no holes, unlike Q, which does (the irrational numbers). 6.
Proof of Theorem 1.6.9. We will prove the statement for the maximum. The proof is by contradiction. Assume f is unbounded. Then for any integer N, no matter how large, there exists a point XN E C such that lf(xN)I > N. By Theorem 1.6.3, the sequence N ~ XN must contain a convergent subsequence j ~ XN(j) which converges to some point b E C. Since f is continuous at b, for any f, there exists a 8 > 0 such that when Ix  bl < 8, then lf(x)  /(b)I < i:; that is, lf(x)I < lf(b)I + E. Since j ~ XN(j) converges to b , we have lxN(j)  bl < 8 for j sufficiently large. But as soon as N(j) > lf(b)I + E, we have
lf(XN(j))I > N(j) > lf(b)I + E,
1.6.8
a contradiction. Therefore, the set of values of f is bounded, which means that f has a supremum M. Next we want to show is that f has a maximum: that there exists a point a EC such that /(a)= M. There is a sequence i ~xi such that .lim
i+OO
f (xi) = M.
1.6.9
110
Chapter 1.
Vectors, matrices, and derivatives
We can again extract a convergent subsequence m ~ Xi(m) that converges to some point a EC. Then, since a= limm+ooXi(m)• f(a) = lim f(xi(m)) = M. m+oo The proof for the minimum works the same way.
1.6.10
0
Theorem 1.6.11 is another result that allows us to get global information from local information using compactness. Theorem 1.6.11 will be essential in Chapter 4.
Theorem 1.6.11. Let X C lRn be compact. A continuous function J : X + JR is uniformly continuous. Proof. Uniform continuity says:
(VE> 0)(38 > 0) (Ix yl < 8
===?
l/(x)  f(y)I
0 and sequences i ~ xi, i ~ Yi such that .lim IXi  Yil = 0, i+oo
but for all i we have
lf(xi)  f(Yi)I 2".:
1.6.12
E.
Since X is compact, we can extract a subsequence j ~ Xi; that converges to some point a E X. Since limi+oo lxi  Yil = 0, the sequence j ~ Yi; also converges to a. By hypothesis, f is continuous at a, so there exists 8 > 0 such that Ix  al< 8
lf(x)  f(a)I
0 if h < O;
.
'( )  r
g c  h~
i.e.,
is simultaneously ::::; 0 and 2".: 0, so it is 0.
0
g(c + h)  g(c) h 1.6.16
1.6 Obviously a car cannot jump from 59 mph to 61 mph without passing through 60 mph, but note that the mean value theorem does not require that the derivative be continuous. Example 1.9.4 describes a function with a discontinuous derivative that oscillates infinitely many times between 1 and 1 (see also Exercise 0.5.2) . It follows from the mean value theorem that for functions of one variable, either the derivative is good (continuous) or it is very, very bad (oscillating wildly). It can't simply jump from one value to another, skipping a value in between. More precisely, the derivative of a differentiable function of one variable satisfies the intermediate value property; this is the object of Exercise 1.6. 7.
Five big theorems
111
Our next result is the mean value theorem, without which practically nothing in differential calculus can be proved. The mean value theorem says that you can't drive 60 miles in an hour without going exactly 60 mph at one instant at least: the average change in f over the interval (a, b) is the derivative off at some point c E (a, b). This is pretty obvious (but still not so easy to prove) if the velocity is continuous; but if the velocity is not assumed to be continuous, it isn't so clear.
Theorem 1.6.13 (Mean value theorem). IE f : [a, b] + ~ is continuous, and f is differentiable on (a, b), then there exists c E (a, b) such that f'(c) = f(b)  f(a). ba
1.6.17
The special case where f(a) = f(b) = 0 is called Rolle's theorem. Note that f is defined on the closed and bounded interval [a, b], but we must specify the open interval (a, b) when we talk about where f is differentiable. 15 If we think that f measures position as a function of time, then the right side of equation 1.6.17 measures average speed over the time interval b  a.
Proof. Think of f as representing distance traveled (by a car or, as in Figure 1.6.4, by a hare). The distance the hare travels in the time interval b  a is f(b)  f(a), so its average speed is m=
f(b)  f(a) b . a
1.6.18
The function g represents the steady progress of a tortoise starting at f(a) and constantly maintaining that average speed (alternatively, a car set on cruise control):
= f(a) + m(x  a). The function h measures the distance between f and g: g(x)
FIGURE
1.6.4.
A race between hare and tortoise ends in a dead heat. The function f represents the progress of the hare, starting at time a and ending at time b. He speeds ahead, overshoots the mark, and returns. Slowandsteady tortoise is represented by g(x) = f(a) + m(x  a)
h(x)
= f(x)
 g(x)
= f(x)
+ m(x  a)). = h(b) = 0. (The
 (f(a)
1.6.19
1.6.20
It is a continuous function on [a, b], and h(a) hare and the tortoise start together and finish in a dead heat.) If his 0 everywhere, then f(x) = g(x) = f(a) + m(x  a) has derivative m everywhere, so the theorem is true. If h is not 0 everywhere, then it must take on positive values or negative values somewhere, so (by Theorem 1.6.9) it must have a point where it achieves a positive maximum value or a point where it achieves a negative minimum value, or both. Let c be such a point; then c E (a, b), so his differentiable at c, and by Proposition 1.6.12, h'(c) = 0. 15 0ne could have a left and right derivative at the endpoints, but we are not assuming that such onesided derivatives exist.
112
Chapter 1.
Vectors, matrices, and derivatives
Since h'(x) = J'(x)  m (the terms f(a) and ma in equation 1.6.20 are constant), this gives 0 = h'(c) = f'(c)  m. 0
The fundamental theorem of algebra In equation 1.6.21, the coefficient of zk is 1. A polynomial whose highestdegree term has coefficient 1 is said to be monic. Of course, any polynomial can be put into monic form by dividing through by the appropriate number. Using monic polynomials simplifies statements and proofs. Even if the coefficients an are real, the fundamental theorem of algebra does not guarantee that the polynomial has any real roots. The roots may be complex. For instance, x 2 + 1 = 0 has no real roots.
The fundamental theorem of algebra is one of the most important results of all mathematics, with a history going back to the Greeks and Babylonians. It was not proved satisfactorily until about 1830. The theorem asserts that every complex polynomial of degree k has at least one complex root. It follows that every polynomial of degree k has k complex roots, counted with multiplicity; this is discussed at the end of this section (Corollary 1.6.15). We will also see that every real polynomial can be factored as a product of real polynomials of degree 1 or 2.
Theorem 1.6.14 (Fundamental theorem of algebra). Let
p(z)
=
zk + ak1zkl
+ · · · + ao
1.6.21
be a polynomial of degree k > 0 with complex coefficients. Then p has a root: there exists a complex number z0 such that p(zo) = 0. When k = 1, this is clear: the unique root is zo = ao. When k = 2, the famous quadratic formula tells you that the roots are a1
FIGURE 1.6.5. Niels Henrik Abel, born in 1802, assumed responsibility for his younger brother and sister after the death of their alcoholic father in 1820. For years he struggled against poverty and illness, trying to obtain a position that would allow him to marry his fiancee; he died from tuberculosis at the age of 26, without learning that he had been appointed professor in Berlin.
±Jar  4ao
1.6.22
2 This was known to the Greeks and Babylonians. The cases k = 3 and k = 4 were solved in the sixteenth century by Cardano and others; their solutions are presented in Appendix A2. For the next two centuries, an intense search failed to find anything analogous for equations of higher degree. Finally, around 1830, two young mathematicians with tragic personal histories, the Norwegian Niels Henrik Abel and the Frenchman Evariste Galois, proved that no analogous formulas exist in degrees 5 and higher. 16 Again, these discoveries opened new fields in mathematics. Several mathematicians (Laplace, d'Alembert, Gauss) had earlier come to suspect that the fundamental theorem was true, and tried their hands at proving it; d'Alembert's argument was published in 1746, while Gauss gave five "proofs", the first in 1799 and the last in about 1810. In the absence of topological tools, their proofs were necessarily short on rigor, and the criticism each heaped on his competitors does not reflect well on any of them. Our proof illustrates the kind of thing we meant when we said, in the beginning of Section 1.4, that calculus is about "some terms being dominant or negligible compared with other terms". 16 Although there is no formula for solving higherdegree polynomials, there is an enormous literature on how to search for solutions.
1.6 The absolute value of a complex number z = x + iy is (Def2 + y 2 • It inition 0.7.3) lzl = wouldn't make sense to look for a minimum of p, since complex numbers aren't ordered.
Jx
Unlike the quadratic formula and Cardano's formulas, this proof does not provide a recipe to find a root. This is a serious problem: one very often needs to solve polynomials, and to this day there is no really satisfactory way to do it.
FIGURE
1.6.6.
Born illegitimate in 1717, Jean Le Rond d 'Alembert was abandoned by his mother, a former nun; his father sought him out and placed him with a foster mother. Inequality 1.6.24: The first inequality in the first line is the triangle inequality, which can be stated as
la+ bl 2:: lal  lbl, since
lal = la+ b  bl s la+ bl + I  bl = la+ bl + lbl. Here, a+ bis p(z) .
Five big theorems
113
Proof of Theorem 1.6.14. We want to show that there exists a number z such that p(z) = 0. The proof requires thinking about complex numbers geometrically, as discussed in Section 0.7, in the subsection "complex numbers in polar coordinates" . The polynomial pis a complexvalued function on 1, otherwise lzkl is small rather than large. We must also require that lzl be large as compared to the coefficients ak l, ... , ao . After a bit of fiddling, we find that setting A= max{lak11, ... , laol} and R = max{(k + l)A, l} works: if lzl 2'.'. R, then lp(z)I 2'.'. lzlk  (lak1llzlkl + · · · + laol) 2'.'. lzlk  kAlzlkl = lzlk 1(1zl  kA) 2'.'. lzl  kA 2'.'. R kA 2'.'. A 2'.'. laol·
1.6.24
In the first inequality in the second line, we can drop lzlkl because by hypothesis lzl 2'.'. R 2'.'. 1, and because we know that lzl  kA 2'.'. 0, since 17 Using Theorem 1.6.9 to prove existence of a minimum is what d' Alembert and Gauss did not know how to do; that is why their proofs were not rigorous. Topology was invented in large part to make these kinds of arguments possible.
114
Chapter 1.
Vectors, matrices, and derivatives
lzl  kA ~ R kA ~ (k + l)A kA =A~ 0. This last computation also justifies the remaining inequalities. Thus any z outside the disc of radius R gives lp(z)I ~ laol Therefore p has a global minimum on 0 that has a nonzero coefficient. 18 We rewrite q as follows:
( r( cos 8+i sin 8)) k
= rk (cos kB + i
Five big theorems
+bJuj + bJ+luJ+l
+ .. · + uk = p(z) =
dog.
1.6.28
leash
We will show  this is the crux of the proof  that for small p, the leash is shorter than the distance from man to flagpole, so at some point the dog is closer to the origin (the doghouse) than is the flagpole: lp(z)I < lp(zo)I. But this is impossible, since lp(zo)I is the minimum value of p.
1.6.9. The flagpole is p(zo); the dog is p(z). We have shown that there is a point zo at which IPI has a minimum. We assume that p(zo) =I 0 and show FIGURE
a contradiction: if lul is small, the leash is shorter than the radius lbi lpi of the circle on which the man is walking, so at some point the dog will be closer to the doghouse than the distance from flagpole to doghouse: lp(z)I < lp(zo)I. Since IPI has a minimum at z0 , this is impossible, so our assumption p(zo) =I 0 is false. the smallest such power will be j = 1; for example, if we had q(u) = u 4 + 2u 2 + 3u + 10, that term, which we call bjui, would be 3u. If we had q(u) = u 5 + 2u 4 + 5u3 + 1, that term would be 5u3 . 18 Usually
116
1.6.10. When the man is between the doghouse and flagpole, the dog is constrained to the dark shaded disc. If inequality 1.6.30 is satisfied, the dog is then always closer to the doghouse than the flagpole is to the doghouse. FIGURE
Inequality 1.6.31:
Chapter 1.
Vectors, matrices, and derivatives
Let us justify this description. If we write u in terms of length p and polar angle 8, the number z = zo+u in the domain turns in a circle of radius p around z0 as (} goes from 0 to 271". What is happening in the codomain? De Moivre's formula says that the man representing the number b0 + bjuj travels in a circle ofradius lbj I~ around the flagpole (i.e., around b0 ). Thus if lbjl~ < lbol (which is true as long as we choose p < lbo/bjl 1 /J), then for some values of 0, the man will be between b0 and 0: on the line segment joining 0 to bo. (In fact, there are precisely j such values of O; Exercise 1.6.9 asks you to find them.) But it is not enough to know where the man is; we need to know whether the dog representing p( z) is ever closer to the origin than is the flagpole (see Figure 1.6.10). This will happen if the leash is shorter than the distance lbj I~ from the man to the flagpole:
1.6.29 which will happen if we take p > 0 sufficiently small and set lul = p. To see this, set B = max{lbJ+il, ... , lbkl = 1} and choose p satisfying
Note that
lbi I is working in our favor (it is
in the numerator, so the bigger it is, the easier it is to satisfy the inequality), and that lbi+d, ... , lbkl are working against us (via B, in the denominator). This is reasonable; if any of these coefficients is very big, we would expect to have to take p very small to compensate. Long division of polynomials reduces finding all the roots of a polynomial to finding one. Unfortunately, this division is quite unstable numerically. If you know a polynomial with a certain precision, you know the result of the division with considerably less precision, and if you need to do this many times to find all the roots, these roots will be less and less accurate. So other schemes are needed to find roots of polynomials of large degree (k > 10).
distance man to pole
length of leash
p If we set lul
lbJ+1u
= p so
1·+1
1 1 < min { (k _lb·I j)B, lbb~ 1 /j ,
that luli < lulH 1 for i > j k
.
·+1
+···+u l::;(kJ)Bp1
length of leash
1
}
1.6.30
.
+ 1, we have .
.
=(kJ)Bpp1
1.6.31
Thus when the man is between 0 and the flagpole, the dog, which represents the point p( z), is closer to 0 than is the flagpole. That is, lp(z)I < lbol = lp(zo)I . This is impossible, because we proved that lp(zo)I is the minimum value of our function. Therefore, our assumption that p(zo) ¥ 0 is false. D
Corollary 1.6.15 (Polynomial of degree k has k roots). Every complex polynomial p(z) = zk + ak_ 1zkl + · · · + ao with k > 0 can be written 1.6.32
with the Cj all distinct. This expression is unique up to permutation of the factors. The number kj is called the multiplicity of the root Cj . Thus Corollary 1.6.15 asserts that every polynomial of degree k has exactly k complex roots, counted with multiplicity.
1.6
When Gauss gave his first proof of the fundamental theorem of algebra, in 1799, he stated the theorem in terms of real polynomials, in the form of Corollary 1.6.16: complex numbers were sufficiently disreputable that stating the theorem in its present form might have cast doubt on the respectability of the result. When he gave his fifth proof, some 12 years later, complex numbers were completely acceptable. The change was probably due to the realization that a complex number can be depicted geometrically as a point in the plane.
Five big theorems
117
Proof. Let k be the largest degree of a monic polynomial p that divides p and is a product of polynomials of degree 1, so that we can write p(z) = p(z)q(z), where q(z) is a polynomial of degree k  k. (Conceivably, k might be 0, and p might be the constant polynomial 1. The proof shows that this is not the case.) If k < k, then by Theorem 1.6.14, there exists a number c such that q(c) = 0, and we can w.:_ite q(z) = (z  c)q(z), where q(z) is a monic polynomial of degree k  k  1. Now
p(z) = p(z)q(z) = (ji(z)(z  c))q(z),
1.6.33
and we see that ji(z)(z  c) is a polynomial of degree k + 1 that factors as a product of factors of degree 1 and divides p. So k is not the largest degree of such a polynomial: our assumption k < k is wrong. Therefore k = k (i.e., p is the product of factors of degree 1). We can collect the factors corresponding to the same root, and write p as
p(z) = (z  c1)k 1
• ••• •
(z 
c,.,,)km,
with the
Cj
distinct.
1.6.34
Now we show that this expression is unique. Suppose we can write
p(z) = (z  ci) k 1
• • •
(z  Cm) k m = (z  cI1 ) k'1
• • •
(z 
I
Cm')
k'
m'.
1.6.35
Cancel all the common factors on both sides of the equation above. If the two factorizations are different, a factor zc.; will appear on the left and not on the right. But that means that the polynomial on the left vanishes at Ci and the one on the right doesn't; since the polynomials are equal this is a contradiction, and the two factorizations are really the same. D Long division of polynomials gives the following result: for any polynomials p1, p2, there exist unique polynomials q and r such that p1
= p2q + r,
with degr < degp2. In particular, if p(a) = 0, then we can factor p as p(z) = (z  a)q(z).
Indeed, there exists r of degree 0 (i.e., a constant) such that p(z)
= (z 
a)q(z) + r;
evaluating at z = a, we find 0 = 0 · q(a) + r,
i.e., r = 0.
Lemma 1.6.17 is an application of equations 0.7.6.
Corollary 1.6.16 (Factoring real polynomials). Every real polynomial of degree k > 0 can be factored a.s a product of real polynomials of degree 1 or 2. Proof. We will start as above: let k be the largest degree of a real monic polynomial p that divides p and which is a product of real polynomials of degree 1 or 2, so that we_can ~rite p(z) = p(z)q(z), where q(z) is a real polynomial of degree k  k. If k < k, then by Theorem 1.6.14, there exists a number c such that q(c) = 0. There are two cases to consider. If c is real, we can wri~ q(z) = (z  c)q(z), where q(z) is a real monic polynomial of degree k  k  1. Now
p(z) = p(z)q(z) = (ii(z)(z  c))q(z),
1.6.36
and we see that ji(z)(z  c) is a real polynomial of degree k + 1 that factors as a product of real polynomials of degree 1 and 2 and divides p. This is a contradiction and shows that k = k (i.e., that p is the product of real polynomials of degree 1 or 2). The case where c is not real requires a separate lemma.
Lemma 1.6.17. If q is a real polynomial, and c E C is a root of q, then c is also a root of q.
118
Chapter 1.
Vectors, matrices, and derivatives
Proof. In essence, the proof is the following sequence of equalities:
q(c)
= q(c) = O= 0.
1.6.37
We need to justify the first equality. The polynomial is a sum of monomials of the form azJ with a real. Since b1 + b2 = b1 + b2, it is enough to prove the result for each monomial. This follows from
a(z)l = a(zJ) = azJ,
1.6.38
where we use the fact that a is real in the form a= a.
D Lemma 1.6.17
Thus if c is not real, then q(c) = 0, and
(z  c)(z  c)
=
z2

(c + c)z +cc= z 2

2Re (c)z
+ lcl 2
1.6.39
is a real polynomial that divides q. So we can write q(z) = (zc)(zc)q(z), where q(z) is a real monic polynomial of degree k  2. Since
= p(z)q(z) = (p(z)(z 2  2Re (c)z + lcl 2 ))q(z), 1.6.40 p(z)(z 2  2Re (c)z + lcl 2 ) is a real polynomial of degree k + 2
p(z)
we see that that factors as a product of real polynomials of c;!_egree 1 and 2 and divides p. Again this is a contradiction and shows that k = k. D Corollary 1.6.16
EXERCISES FOR SECTION 1.6 1.6.1 Show that a set is bounded if it is contained in a ball centered anywhere; it does not have to be centered at the origin. 1.6.2 Let A C Rn be a subset that is not compact. Show that there exists a continuous unbounded function on A. 1.6.3 Set z = x+iy, where x, y E JR. Show that the polynomial p(z) has no roots. Why doesn't this contradict Theorem 1.6.14?
= 1 +x 2 y 2
1.6.4 Find, with justification, a number R such that there is a root of p(z) = z 5 + 4z 3 + 3iz  3 in the disc lzl :=:; R. (You may use that a minimum of IPI is a root of p.) 1.6.5 Consider the polynomial p(z) = z 6 + 4z 4 + z a. Find R such that lz 6 1 > jq(z)I when lzl > R .
+ 2 = z 6 + q(z).
b. Find a number Ri such that you are sure that the minimum of at a point z such that lzl < Ri.
IPI
occurs
1.6.6 a. Let f : IR+ R be the function f(x) = lxlelxl. Show that it has an absolute maximum (i.e., not merely local) at some x > 0.
b. Where is the maximum of the function? What is the maximum value? c. Show that the image off is [O, 1/e). 1.6. 7 Show that if f is differentiable on a neighborhood of [a, b), and we have f'(a) < m < f'(b), then there exists c E (a,b) such that f'(c) = m.
1.7
Derivatives in several variables as linear transformations
119
1.6.8 Consider Example 1.6.4. What properties do the digits of 1/(27r) have to satisfy in order for the Xm to have a limit between 0.7 and 0.8?
Exercise 1.6.9 concerns the proof of Theorem 1.6.14. You will need Proposition 0.7.7.
1.6.9 If a, b EC, a,b i= 0 and j ~ 1, find po> 0 such that if 0 < p O
h>Oh
::fa
0.
1.7.12
Such a function L does not qualify as the derivative of f. But if we have another linear function, D, such that the difference is smaller than linear, say 3h2 , then 1 (!(a+ h)  f(a)  D(h)) lim h
h>O
= h>O lim h1 3h 2 = lim 3h = 0. h>O
6
The following computation shows that Definition 1.7.5 is just a way of restating Definition 1. 7.1: J'(a) by Def. 1.7. 1
lim .!_((!(a+ h)  f (a)) h>O h
[J'(a)Jh)
•
= lim (f(a h>O
= J'(a) 
+ h)  f(a)  f'(a)h) h
J'(a)
h
= 0.
1.7.13
Moreover, the linear function h i. f'(a)h is the only linear function satisfying equation 1.7.11. Indeed, any linear function of one variable can be written h i. mh, and
! f
o = l~ ~((!(a+ h) 
f(a))  mh) 1.7.14
= lim (f(a + h)  f(a) _ mh) = J'(a) _ m h
h.O
h
'
so we have f'(a) = m . The point of rewriting the definition of the derivative is that with Definition 1.7.5, we can divide by lhl rather than h; the number m = f'(a) is also the unique number such that FIGURE
1.7.1.
The mapping
takes the shaded square in the square at top to the shaded area at bottom.
t:>f
linear function of h
l~ l~I ( (J(a + h) f(a)) 
(JTa)h} ) = 0.
1.7.15
It doesn't matter if the limit changes sign, since the limit is O; a number close to 0 is close to 0 whether it is positive or negative. Therefore, we can generalize equation 1.7.15 to mappings in higher dimensions, like the one in Figure 1.7.l. As in the case of functions of one variable, the key to understanding derivatives of functions of several variables is to think that the increment to the function (the output) is approximately a linear function of the increment to the variable (the input): the increment ~f
= f(a +ii)  f(a)
is approximately a linear function of the increment
1.7.16
ii.
1. 7
Derivatives in several variables as linear transformations
125
In the onedimensional case, D..f is well approximated by the linear function h ~ f'(a)h. We saw in Section 1.3 that every linear transformation is given by a matrix; the linear transformation h ~ f'(a)h is given by multiplication by the 1 x 1 matrix [!' (a)]. For a mapping f : U > !Rm, where U C !Rn is open, the role of this 1 x 1 matrix is played by an m x n matrix composed of the n partial derivatives of the mapping at a . This matrix is called the Jacobian matrix of the mapping f; we denote it [Jf(a)]. Note that in the Jacobian matrix we write the components of f from top to bottom, and the variables from left to right. The first column gives the partial derivatives with respect to the first variable; the second column gives the partial derivatives with respect to the second variable, and so on.
Definition 1.7.7 (Jacobian matrix). Let Ube an open subset of!Rn. The Jacobian matrix of a function f : U > !Rm is the m x n matrix composed of the n partial derivatives of f evaluated at a: Dif1(a) [Jf(a)) ~ [
: Difm(a)
Dn~1(a)]
=
Y
(sin(~y+y)) x2 y2
[Jr(;)]~ [cos~+Y) cos~;•)].
FIGURE
1.7.2.
Carl Jacobi (18041851) was 12 years old when he finished the last year of secondary school, but had to wait until 1821 before entering the University of Berlin, which did not accept students younger than 16. He died after contracting flu and smallpox.
1.7.17
Dnfm(a)
Example 1.7.8. TheJacobianmatrixoff(x)
What IB the Jacobian mat IR and g : U > !Rm are differentiable at a, then so is and the derivative is given by [D(fg)(a)]v =/(a) [Dg(a)]v + ([D/(a)]v) g(a).
.._.,..,____ R
'.'
.._.,..,
JR
Rm
]Rm
1.8.3
f : U > IR and g : U > !Rm are differentiable at a, and /(a) =/: 0, then so is g/ f, and the derivative is given by
6. If
[D
1.8. l. Gottfried Leibniz (16461716) Parts 5 and 7 of Theorem 1.8.1 are versions of Leibniz's rule. Leibniz (or Leibnitz) was both philosopher and mathematician. He wrote the first book on calculus and invented the notation for derivatives and integrals still used today. Contention between Leibniz and Newton for who should get credit for calculus contributed to a schism between English and continental mathematics that lasted into the 20th century. FIGURE
Equation 1.8.3: Note that the terms on the right belong to the indicated spaces, and therefore the whole expression makes sense; it is the sum of two vectors in Rm, each of which is the product of a vector in Rm and a number. Note that (Df(a)]v is the product of a line matrix and a vector, hence it is a number. Proof, part 2 of Theorem 1.8.1: We used the variable v in the statement of the theorem, because we were thinking of it as a random vector. In the proof the variable naturally is an increment, so we use h, which we tend to use for increments.
( ~) f
( )] = [Dg(a)]v _ ([D/(a)]v) (g(a)) a v /(a) (f(a))2 .
1.8.4
7. If f, g : U > !Rm are both differentiable at a, then so is the dot product f · g: U> IR, and (as in one dimension) [D(f · g)(a)]v = [Df(a)]v · g(a)
+

1.8.5
f(a) · [Dg(a)]V .
..__._... .._.,.., ......._,
We saw in Corollary 1.5.31 that polynomials are continuous, and rational functions are continuous wherever the denominator does not vanish. It follows from Theorem 1.8. l that they are also differentiable. Corollary 1.8.2 (Differentiability of polynomials and rational functions). l. Any polynomial function !Rn > IR is differentiable on all of !Rn.
2. Any rational function is differentiable on the subset of !Rn where the denominator does not vanish. Proof of Theorem 1.8.1. Proving most parts of Theorem 1.8.1 is straightforward; parts 5 and 6 are a bit tricky. l. If f is a constant function, then f(a+h) = f(a), so the derivative [Df(a)] is the zero linear transformation: 1  ) =lim=;0=0, 1 lim=;(f(a+h)f(a)[O]h ii.o lhl ......._, ii.o lhl
1.8.6
[Df(a)]h
where [OJ is a zero matrix. 2. Since f is linear, f(a + h) = f(a) + f(h) , so
1(f(a + h)   f(a)  f(h) ) = 0. lim =iio lhl
1.8.7
It follows that [Df(a)] = f, i.e., for every h E !Rn we have [Df(a)]h = f(h) . 3. The assumption that f is differentiable can be written
. __!_ hm _ ii.o lhl
((
fi(a_+ . h) )  (
fi~a) . )  [ [Dfi_(a)]h .
/m(a. + h)
fm (a) 0
[Df~(a)]h
l)
[~] . 0
1.8.8
1.8
The assumption that
Ji, ... , fm
Rules for computing derivatives
139
are differentiable can be written
1.8.9
The expressions on the left sides are equal by Proposition 1.5.25. 4. Functions are added point by point, so we can separate out f and g:
(f + g)(a + h)  (f + g)(a)  ([Df(a)) + [Dg(a)j)h
1.8.10
= (f(a + h)  f(a)  [Df(a)Jh) + (g(a + h)  g(a)  [Dg(a)Jh). Now divide by lhl, and take the limit as Jhl
+
0.
The right side gives
0 + 0 = 0, so the left side does too. In the second line of equation 1.8.11 we added and subtracted f(a)g(a+ii); to go from the second equality to the third, we added and subtracted g(a +
5. We need to show both that Jg is differentiable and that its derivative is given by equation 1.8.3. By part 3, we may assume that m = 1 (i.e., that g = g is scalar valued). Then [D(fg)(a)]h, according to Thm. 1.8.1
ii)[Df(a)]ii.
How did we know to do this? We began with the fact that f and g are differentiable; that's really all we had to work with, so clearly we needed to use that information. Thus we wanted to end up with something of the form
!im__;_ (Ug)(a + h)  (fg)(a) /(a)([Dg(a))h)  ([D/(a))h)g(a)) Jhl
h+O
= !im _.;_ (f(a + h)g(a + h) f(a)g(a + h) + f(a)g(a + h)f(a)g(a) h+O lhl 0
 ([D/(a)Jh)g(a)  (!Dg(a)Jh)/(a))
f(a +ii)  f(a)  [Df(a)]ii and g(a +ii)
 g(a)  [Dg(a)]ii.
So we looked for quantities we could add and subtract in order to arrive at something that includes those quantities. (The ones we chose to add and subtract are not the only possibilities.)
= !im _.;_ ((f(a + h)  f(a)) (g(a + h)) + f(a) (g(a + h)  g(a)) h+O JhJ  ([D/(a)Jh)g(a)  ([Dg(a)Jh)f(a)) =
1.8.11
. (/(a+ h)  /(a) lim _  [D/(a))h) g (a+ h) ii+O JhJ + !im_(g(a + h)  g(a))
[D/~a))h lhl
h+0
+ lim /(a) h+0
(g(a + h)  g(a)  [Dg(a)Jh) _
!hi
0+0+0= 0.
By Theorem 1.5.29, part 5, the first limit is 0: by the definition of the derivative off, the first factor has limit 0, and the second factor g(a + h) is bounded in a neighborhood of h = O; see the margin.
140
Chapter 1. Vectors, matrices, and derivatives
We again use Theorem 1.5.29, part 5, to show that the second limit is + 0 (since g is differentiable, hence
0: g(a + h)  g(a) goes to 0 as ii
continuous, at a) and the second factor [D~~~)Jii is bounded, since
[D/(a))h lhl
Prop. 1.4.11
'?

i[D/(a)JI :~:
1.8.12
l[D/(a)JI.
=
The third limit is 0 by the definition of the derivative of g (and the fact that /(a) is constant). 6. Applying part 5 to the function that
1/f,
( )] = ID( ~) f a v
we see that it is enough to show
[D/(a)]v
_
1.8.13
(f(a))2 .
This is seen as follows: 1 (
lhl =
[D/(a)]h) 1 (/(a)  f(a + h) [D/(a)]h) 1 1 /(a+ h)  f(a) + (/(a) ) 2 = lhl /(a+ h)/(a) + (/(a) ) 2
~ (/(a)  /(a+ ii)+ [D/(a)]ii) _ : (/(a)  f(a +ii) _ /(a)  !(a+ h)) (/(a)) 2
lhl =
1
(/(a)) 2
(/(a)) 2
lhl
/(a+ h)(/(a))
(/(a)  f(a + h) + [D/(a)]h) __1_ /(a)  f(a + h) (1 _ 1 ) lhl /(a) lhl /(a) /(a+ h) . limit as
h>0 is 0 by
bounded
def. of deriv.
limit as
h>O is
o
To see why we label one term in the last line as bounded, note that since f is differentiable,
[D/(a)](h) 1ii1 ...___._.,
. (f(a+ii)f(a) 1Im iio lhl
) =0.
1.8.14
bounded; see eq. 1.8.12
If the first term in the parentheses were unbounded, then the sum would be unbounded, so the limit could not be zero.
7. We do not need to prove that f · g is differentiable, since it is the sum of products of differentiable functions, thus differentiable by parts 4 and 5. So we just need to compute the Jacobian matrix: Equation 1.8.15: The second equality uses part 4: f·g is the sum of the figi, so the derivative of the sum is the sum of the derivatives. The third equality uses part 5. Exercise 1.8.6 sketches a more conceptual proof of parts 5 and 7.

[D(f · g)(a)]h
def.prod. of dot
=
(5)
[n
(
n
8 figi
)
4  ( )
n

(a) Jh = 8[D(figi)(a))h
n
2: ([Dfi(a)]h)gi(a) + /i(a) ([Dgi(a)]ii)
1.8.15
i=l def. of dot prod. ([
=
Df ( a )] h) ·g ( a )
+
f(a) · ([Dg(a)]h).
D
1.8
The chain rule Some physicists claim that the chain rule is the most important theorem in all mathematics. For the composition f o g to be well defined, the codomain of g and the domain of f must be the same (i.e., V in the theorem). In terms of matrix multiplication, this is equivalent to saying that the width of [Df(g(a))] must equal the height of [Dg(a)] for the multiplication to be possible.
Rules for computing derivatives
141
One rule for differentiation is so fundamental that it deserves a subsection of its own: the chain rule, which states that the derivative of a composition is the composition of the derivatives. It is proved in Appendix A4. Theorem 1.8.3 (Chain rule). Let UC Rn, V C Rm be open sets, let g : U + V and f : V + JR.P be mappings, and let a be a point of U. If g is differentiable at a and f is differentiable at g(a), then the composition f o g is differentiable at a, and its derivative is given by [D(f o g)(a)] = [Df (g(a))] o [Dg(a)].
1.8.16
The chain rule is illustrated by Figure 1.8.2. [Df(b)J[Dg(a)]w = [D(f o g)(a)]w _ [Dg(a)](w)
One motivation for discussing the relationship between matrix multiplication and composition of linear transformations at the beginning of this chapter was to have these tools available now. In coordinates, and using matrix multiplication, the chain rule states that Di(f o g)i(a) m
=
L
Dkfi ( g(a)) Djgk(a).
k=l
As a statement, this form of the chain rule is a disaster: it turns a fundamental, transparent statement into a messy formula, the proof of which seems to be a computational miracle.
u [Df(b)][Dg(a)]v = [D(f o g)(a)]v FIGURE 1.8.2. The function g maps a point a E U to a point g(a) E V. The function f maps the point g(a) = b to the point f(b). The derivative of g maps the vector v to [Dg(a)](v). The derivative off o g maps v to [Df(b)][Dg(a)]v, which, of course, is identical to [Df (g(a))][Dg(a)]V.
In practice, when using the chain rule, most often we represent linear transformations by their matrices, and we compute the right side of equation 1.8.16 by multiplying the matrices together: [D(f o g)(a)] = [Df (g(a))][Dg(a)].
1.8.17
Remark. Note that the equality (f o g)(a) = f(g(a)) does not mean you can stick a D in front of each and claim that [D(f o g)(a)] equals [Df(g(a))]; this is wrong. Remember that [D(f o g)(a)] is the derivative of fog evaluated at the point a; [Df(g(a))] is the derivative off evaluated at the point g(a). 6 Example 1.8.4 (The derivative of a composition). Define g : JR. f : JR.3 + JR by
and
f
m
~ x' +y2 + z';
g(t)
~ (::) .
+
JR.3
1.8.18
142
Chapter 1.
Vectors, matrices, and derivatives
G) ]~
The dedvative off i' the 1 x 3 matiis [ D f
[2", 2y, 2z); evaluated
t
at g(t) it is [2t, 2t 2 , 2t3 ]. The derivative of g at
is [Dg(t)] = [
[D(fog)(t)] = [D/(g(t))][Dg(t)] = [2t,2t 2 ,2t3 ]
[
;t 3t
2
Note that the dimensions are right: the composition f o g goes from JR to JR, so its derivative is a number.
l
it
3t2
l·
so
= 2t+4t3 +6t 5 .
In this case it is also easy to compute the derivative of the composition directly: since (! o g) (t) = t 2 + t 4 + t 6 , we have [D(f o g)(t)] = (t 2 + t 4 + t 6 )' = 2t + 4t3 + 6t 5 •
6.
Example 1.8.5 (Composition of a function with itself).
Ul 3
~R
3
be given by f
G) ~ (~;
1.8.19
Let
3 ) What is the derivative of
f o f, evaluated at ( : ) ? We need to compute
1.8.20
We
m m(D
use~~ tffioe, evaluating it at
Sinoe [ Df
G)l
[D(fof)m
[~ ; n
and at f
equation 1.8.20 give.
l [~ ~ ~] rn ~ ~] ~ rn ~ ~l ·
1821
Now compute the derivative of the composition directly and check in the footnote below. 24 6. Example 1.8.6 (Composition of linear transformations). Here is a case where it is easier to think of the derivative as a linear transformation 24 Since
f of ( : ) z
=f
compositionat (:)is z
(
xyz~ 3)
= ( xyz 2
[y~2y2
: :2
2x
~y;z2

3) , the derivative of the
2xy 6
2y
2xyz06z];at 0
(~),it is[~2 2~ ~]· 1 0
1.8
Example 1.8.6: Equation 1.7.46 says that the derivative of the "squaring function" f is
=
...._..,
Equation 1.7.52 says that the derivative of the inverse function for matrices is
chain rule
[DJ(A)]H = A 1 HA 1 .
eq. 1.7.46
In the second line of equation 1.8.22, g(A) = A 1 plays the role of A, and A 1 HA 1 plays the role of H. Notice the interesting way this result is related to the onevariable computation: if f(x) = x 2 , then 3 . Notice also how J'(x) = much easier this computation is, using the chain rule, than the proof of Proposition 1.7.18, without the chain rule.
2x
Exercise 1.32 asks you to compute the derivatives of the maps A>+ A 3 and A>+ An.
143
than as a matrix, and it is easier to think of the chain rule as concerning a composition of linear transformations rather than a product of matrices. If A and Haren x n matrices, and f(A) = A2 and g(A) = A 1 , then (! o g) (A) = A 2 • Below we compute the derivative of f o g:
[D(f o g)(A)]H
[Df(A)]H =AH+ HA.
Rules for computing derivatives
...._..,
= [D/(A 1 )] ..._,,,._... (A 1 HA 1 )
[D/(g(A))] [Dg(A)]H
'v"
new increment H for DJ
A 1 (A 1 HA 1 )
eq. 1.7.52
+ (A 1 HA 1 )A 1 1.8.22
EXERCISES FOR SECTION
1.8
1.8.1 Let f : R 3 + R, f : R 2 + R 3 , g : R 2 + R, g : R 3 +R2 be differentiable functions. Which of the following compositions make sense? For each that does, what are the dimensions of its derivative?
a.
fog
b.
gof
c.
gof
d.
fog
e.
f of
f.
f
[D(gof) b. Lot f
gofat
f
~) ~ x' +i/ +2z' and g(t) ~ (~),what
a. Giwn tho functioM f (
1.8.2 ffi
0
m]1
G) ~ (''.~ z) ~d
g (:)
~ a' + b'. WhatIB tho dorivativo of
G)1 f (:) = sin( e"'Y) differentiable at ( 8)?
1.8.3
Is
1.8.4
a. What compositions can you form using the following functions?
i.f ( : ) z
=x +y ii.g(b) =2a+b iii.f(t)= (;t) 2
2
2
f
iv.g(~) = (~~~) filny
b. Compute these compositions. c. Compute their derivatives, both by using the chain rule and directly from the compositions. 1.8.5 The following "proof" of part 5 of Theorem 1.8.l is correct as far as it goes, but it is not a complete proof. Why not?
144
Chapter 1.
Vectors, matrices, and derivatives
"Proof": By part 3 of Theorem 1.8.1, we may assume that m = 1 (i.e., that g = g is scalar valued). Then Jaco bi an matrix of f g
[DJg(a)]h = [(Difg)(a), ... , (Dnfg)(a)] h = [f(a)(D1g)(a)
+ (D1f)(a)g(a), ... , f(a)(Dng)(a) + (Dnf)(a)g(a)] h in one variable, (fg)'=fg'+f'g
= f(a)[(D1g)(a), ... , (Dng)(a)]h + [(D1f)(a), ... , (Dnf)(a)]g(a)h Jacobian matrix off
Jacobian matrix of g
= f(a)([Dg(a)Jh) + (lDf(a)Jh)g(a). 1.8.6 a. Prove the rule for differentiating dot products (part 7 of Theorem 1.8.1) directly from the definition of the derivative.
b. Let U E R 3 be open. Show by a similar argument that if f, g : U + R 3 are both differentiable at a, then so is the cross product f x g : U + R 3 . Find the formula for this derivative. 1.8. 7
Consider the function
What is the derivative of the function t,...... f ( 1(t))? 1.8.8 True or false? Justify your answer. If f : R 2 + R 2 is a differentiable function with
f (
8)
=
0)
and
[Df (
8)]
=
Dn'
there is no differentiable mapping g: R 2 + R 2 with
1.8.9
Let '{): R+ R be any differentiable function. Show that the function
satisfies the equation
~Dif ( ~) + ~D2f ( ~) = :
2
f (
~).
1.8.10 a. Show that if a function f : R 2 + R can be written '{J(x 2 some differentiable function '{) : R + R, then it satisfies
Exercise 1.8.10, hint for part b: What is the "partial derivative of f with respect to the polar angle (}"?
xD2f  yDif
+ y2 )
for
= 0. +
*b. Show the converse: every function satisfying xD2f  yDif written '{J(x 2 + y 2 ) for some function'{): R+ R.
=0
can be
1.9 1.8.11
Show that if
r,o : JR
JR, then
+
1.8.12
f ( ~)
= r,o
Criteria for differentiability
(~ ~ t)
145
for some differentiable function
xDif + yD2f = 0. True or false? Explain your answers.
a. If f : JR 2 + JR2 is differentiable and [Df(O)] is not invertible, then there is no differentiable function g : JR2 + JR2 such that
(go f)(x) = x. b. Differentiable functions have continuous partial derivatives. 1.8.13 Let U C Mat (n, n) be the set of matrices A such that A + A 2 is invertible. Compute the derivative of the map F : U + Mat (n, n) given by F(A) =(A+ A 2 ) 1 .
1.9 THE MEAN VALUE THEOREM AND CRITERIA FOR DIFFERENTIABILITY I turn with terror and horror from this lamentable scourge of continuous functions with no derivatives.Charles Hermite, in a letter to Thomas Stieltjes, 1893
In this section we discuss two applications of the mean value theorem. The first extends that theorem to functions of several variables, and the second gives a criterion for determining when a function is differentiable.
The mean value theorem for functions of several variables The derivative measures the difference of the values of functions at different points. For functions of one variable, the mean value theorem (Theorem 1.6.13) says that if f : [a, b] + JR is continuous, and f is differentiable on (a, b), then there exists c E (a, b) such that
f(b)  f(a) = J'(c)(b a). Theorem 1.9.1: The segment [a, b] is the image of the map
t
>+
(1  t)a +th,
forO::;t::;l.
1.9.1
The analogous statement in several variables is the following.
Theorem 1.9.1 (Mean value theorem for functions of several variables). Let U C Rn be open, let f : U + JR be differentiable, and let the segment [a, h] joining a to b be contained in U. Then there exists c 0 E [a, b] such that +
f(b)  /(a)= [D/(co)](b  a).
1.9.2
146
Chapter 1.
Vectors, matrices, and derivatives
Corollary 1.9.2. If f is a function as defined in Theorem 1.9.1, then lf(b)  f(a)I ::; ( sup cE[a,b]
Why do we write inequality 1.9.3 with the sup, rather than

Jf(b)  f(a)J::; J[Df(c)]J Jb  aJ, which of course is also true? Using the sup means that we do not need to know the value of c in order to relate how fast f is changing to its derivative; we can run through all c E [a, h] and choose the one where the derivative is greatest. This will be useful in Section 2.8 when we discuss Lipschitz ratios.
I
[Df(c)J
I) lb~
al.
1.9.3
Proof of Corollary 1.9.2. This follows immediately from Theorem 1.9.1 and Proposition 1.4.11. D Proof of Theorem 1.9.1. Ast varies from 0 to 1, the point (1 t)a +th moves from a to b. Consider the mapping g(t) = f((l  t)a +th). By the chain rule, g is differentiable, and by the onevariable mean value theorem, there exists to such that
g(l)  g(O) = g'(to)(l  0) = g'(to).
1.9.4
Set co = (1  to)a + tob. By Proposition 1.7.14, we can express g'(t0 ) in terms of the derivative off: . g(to '(t ) g O = 1Im
+ s) 
s>O
g(to)
S
= lim /(co+ s((b~a))  /(co) = (Df(co)](b~a). s>0
1.9.5
S
So equation 1.9.4 reads >
f(b)  /(a)= [D/(co)](b  a). FIGURE
1.9.1.
The graph of the function f defined in equation 1.9. 7 is made up of straight lines through the origin, so if you leave the origin in any direction, the directional derivative in that direction certainly exists. Both axes are among the lines making up the graph, so the directional derivatives in those directions are 0. But clearly there is no tangent plane to the graph at the origin.
D
1.9.6
Differentiability and pathological functions Most often the Jacobian matrix of a function is its derivative. But as we mentioned in Section 1.7, there are exceptions. It is possible for all partial derivatives of f to exist, and even all directional derivatives, and yet for f not to be differentiable! In such a case the Jacobian matrix exists but does not represent the derivative. Example 1.9.3 (Nondifferentiable function with Jacobian matrix). This happens even for the innocentlooking function
f
(Yx) = x+yy2 2
x2
1.9. 7
shown in Figure 1.9.1. Actually, we should write this function as
if(:)#(~) if(:)=(~).
1.9.8
You have probably learned to be suspicious of functions that are defined by different formulas for different values of the variable. In this case, the
1.9
Criteria for differentiability
8)
147
8),
value at ( is really natural, in the sense that as ( ~ ) approaches ( the function f approaches 0. This is not one of those functions whose value takes a sudden jump; indeed, f is continuous everywhere. Away from the origin, this is obvious by Corollary 1.5.31: away from the origin, f is a rational function whose denominator does not vanish. So we can compute
8) .
both its partial derivatives at any point ( ~ ) # ( That f is continuous at the origin requires a little checking, as follows. If x 2 + y 2 = r 2 , then lxl $ r and IYI $ r so lx2 yl $ r 3 • Therefore, and
"Identically" means "at every point."
lim
(:)(~)
f (~)
= 0.
1.9.9
So f is continuous at the origin. Moreover, f vanishes identically on both axes, so both partial derivatives of f vanish at the origin. So far, f looks perfectly civilized: it is continuous, and both partial derivatives exist everywhere. But consider the derivative in the direction of the vector [ ~ ] :
. 1((8)+t[~])1(8)
t3 1 hm = lim  = 1.9.10 to t to 2t3 2 This is not what we get when we compute the same directional derivative If we change the function of Example 1.9.3, replacing the x 2 y in the numerator of x2y x2 +y2
by xy, then the resulting function, which we'll call g, will not be continuous at the origin. If x = y, then g = 1/2 no matter how close (
~)
have
gets to the origin: we then
by multiplying the Jacobian matrix of f by the vector [ ~] , as on the right side of equation 1.7.38:
[ Dif
(8) ,D2f (8)] [n = [O,O] [n = 0.
1.9.11
Jacobian matrix [.Jf(O)]
Thus, by Proposition 1.7.14,
f is not differentiable.
6
Things can get worse. The function we just discussed is continuous, but it is possible for all directional derivatives of a function to exist, and yet for the function not to be continuous, or even bounded in a neighborhood of 0. For instance, the function discussed in Example 1.5.24 is not continuous in a neighborhood of the origin; if we redefine it to be 0 at the origin, then all directional derivatives would exist everywhere, but the function would not be continuous. Exercise 1.9.2 provides another example. Thus knowing that a function has partial derivatives or directional derivatives does not tell you either that the function is differentiable or even that it is continuous. Even knowing that a function is differentiable tells you less than you might think. The function in Example 1.9.4 has a positive derivative at x although it does not increase in a neighborhood of x!
Example 1.9.4 (A differentiable yet pathological function). Consider the function f : R + R defined by
f(x) =
~ +x 2 sin~,
1.9.12
148
Chapter 1.
Vectors, matrices, and derivatives
a variant of which is shown in Figure 1.9.2. To be precise, one should add f(O) = 0, since sin 1/x is not defined there, but this was the only reasonable value, since lim x 2 sin ~ = 0.
x+0
1.9.13
X
Moreover, we will see that the function f is differentiable at the origin. This is one case where you must use the definition of the derivative as a limit; you cannot use the rules for computing derivatives blindly. In fact, let's try. We find FIGURE
1.9.2.
Graph of the function
f(x)
=~+
6x 2
sin~
The derivative of f does not have a limit at the origin, but the curve still has slope 1/2 there.
. 1 2 ( cos1) (  1 ) =+2xsincos. 1 1 1 l.9.14 f '( x ) =21 +2xsm+x 2
x
x
This formula is certainly correct for x when x + 0. Indeed,
x
2
"I 0,
x
x
but f'(x) doesn't have a limit
1 2xs1n. 1) = 1 . ( + 11m 2 X 2
x+O
1.9.15
does exist, but cos 1/x oscillates infinitely many times between 1 and 1. So f' will oscillate from a value near 1/2 to a value near 3/2. This does not mean that f isn't differentiable at 0. We can compute the derivative at 0 using the definition of the derivative:
f' (0) =
l~ ~ ( O; 1
h + (0 + h )2 sin 0 1
~ h) = l~ ~ ( ~ + h 2 sin ~)
1
1 9 16
l" h . = 2 + h~ sm h = 2'
··
since (Theorem 1.5.26, part 5) limh+O h sin* exists, and indeed vanishes. Finally, we can see that although the derivative at 0 is positive, the function is not increasing in any neighborhood of 0, since in any interval arbitrarily close to 0 the derivative takes negative values; as we saw above, it oscillates from a value near 1/2 to a value near 3/2. 6. This is very bad. Our whole point is that locally, the function should behave like its best linear approximation, and in this case it emphatically does not. We could easily make up examples in several variables where the same thing occurs: where the function is differentiable, so that the Jacobian matrix represents the derivative, but where that derivative doesn't tell you much. Of course we don't claim that derivatives are worthless. The problem in these pathological cases is that the partial derivatives of the function are not continuous. Example 1.9.5 (Discontinuous derivatives). Let us go back to the function of Example 1.9.3:
if(:)1(~) if(~)=(~).
1.9.17
1.9
Criteria for differentiability
149
which we saw is not differentiable although its Jacobian matrix exists. Both partial derivatives are 0 at the origin. Away from the origin  that
g)  then Dif (x) = (x 2 + y 2)(2xy) 
is, if ( ~ ) =F (
y
(x2
x 2y(2x)
+ y2)2
=
2xy3 (x2 + y2)2 1.9.18
These partial derivatives are not continuous at the origin, as you will see if you approach the origin from any direction other than one of the axes. For example, if you compute the first partial derivative at the point ( ~) of the diagonal, you find the limit
.
l~ Dif
(t) t
=
2t4 (2t2)2
1
= 2'
1.9.19
which is not the value of
Dif
(8) = 0.
1.9.20
In the case of the differentiable but pathological function of Example 1.9.4, the discontinuities are worse. This is a function of one variable, so (as discussed in Section 1.6) the only kind of discontinuities its derivative can have are wild oscillations. Indeed, f'(x) = + 2xsin ~  cos~. and cos l/x oscillates infinitely many times between 1 and 1. !:::..
!
The moral of the story: Only study continuously differentiable functions.
Definition 1.9.6 {Continuously differentiable function). A function is continuously differentiable on U C JR.n if all its partial derivatives exist and are continuous on U. Such a function is known as a G 1 function. This definition can be generalized; in Section 3.3 we will need functions that are "smoother" than G 1 .
Definition 1.9.7 (GP function). A GP function on U c JR.n is a function that is p times continuously differentiable: all of its partial derivatives up to order p exist and are continuous on U. If you come across a function that is not continuously differentiable, you should be aware that none of the usual tools of calculus can be relied upon. Each such function is an outlaw, obeying none of the standard theorems.
Theorem 1.9.8 guarantees that a "continuously differentiable" function is indeed differentiable: from the continuity of its partial derivatives, one can infer the existence of its derivative. Most often, the criterion of Theorem 1.9.8 is the tool used to determine whether a function is differentiable.
Theorem 1.9.8 {Criterion for differentiability). If U is an open subset of JR.n' and f : u  JR.m is a G 1 mapping, then f is differentiable on U, and its derivative is given by its Jacobian matrix.
150
Chapter 1.
Vectors, matrices, and derivatives
A function that meets this criterion is not only differentiable; it is also guaranteed not to be pathological. By "not pathological" we mean that locally, its derivative is a reliable guide to its behavior. Note that the last part of Theorem 1.9.8  "and its derivative is given by its Jacobian matrix"  is obvious; if a function is differentiable, Theorem 1.7.10 tells us that its derivative is given by its Jacobian matrix. So the point is to prove that the function is differentiable. In equation 1.9.21 we use the interval (a, a + h), rather than (a, b), so instead of the equation
J'(c) = f(b)  f(a) ba of the mean value theorem we have statement J'(c) = f(a
+ h) 
Proof of Theorem 1.9.8. This is an application of Theorem 1.9.1, the mean value theorem. What we need to show is that 1 ( f(a !im~:;
lhl
h+O
  f(a)  [Jf(a)]h ) = 0. + h)
First, note that by part 3 of Theorem 1.8.1, it is enough to prove it when m = 1 (i.e., for f : U  t ~). Next write
f(a),
f(a+h)f(a)=f
h
(::~:)/(:)
or
an+hn hf'(c)
= f(a + h) 
f(a).
1.9.21
1.9.22
an
in expanded form, subtracting and adding inner terms:
f(a + h)  f(a) =
f
(:::~:)/ an+hn
(.,.';\,) +!
an+hn ______... subtracted
[::!~:]/ [•a~ha] an +hn
an +hn
______... added
+···±···+! [
~~,
)!
an+hn
[I]. an
By the mean value theorem, the ith term is
f
ai
ai
ai
a2
a2
a2
ai +hi ai+l + hi+1
!
an+hn
ai
=hiDd
bi
ai+l + hi+1
ai+l + hi+1
an+hn
an +hn
1.9.23
ith term
for some bi E [ai, ai +hi]: there is some point bi in the interval [ai, ai +hi] such that the partial derivative Dd at bi gives the average change of the
1.9 Criteria for differentiability
151
function f over that interval, when all variables except the ith are kept constant. Since f has n variables, we need to find such a point for every i from 1 ton. We will call these points Ci:
n
ai+I
f(a+h)f(a) = 'L,hiDd(ci)·
this gives
bi
Ci=
+ hi+1
i=l
Thus we find that n
f(a+ h)  f(a)
L, Dd(a)hi = L, hi(Dd(ci) 
Dd(a)).
1.9.24
i=l
So far we haven't used the hypothesis that the partial derivatives Dd are continuous. Now we do. Since Dd is continuous, and since ci tends to a as h+ 0, we see that the theorem is true:
2::?= 1 D;f(a)h; ,."..
The inequality in the second line of inequality 1.9.25 comes from the fact that lhil/lhl ~ 1.
!im IJ(a + h) h+O
!(~) 
[J/(a)]h I :::;; !im
lhl
t
h+O i=l
l':l IDd(ci) lhl
Dd(a)I
n
:::;; !im_'L,IDd(ci)  Dd(a)I = 0.
D
1.9.25
h+O i=l
Example 1.9.9. Here we work out the preceding computation when scalarvalued function on JR.2:
f is a 1.9.26
0
=I (
:~ ! ~~)1 (a2;
= h1Di!(a2 ~ h2)
h 2 ) +I ( a2 ; h2
+h2D2f(~~)
EXERCISES FOR SECTION 1.9.1
Show that the function
f : JR.2
= h1Dif(c1) + h2D2f(c2).
1.9 +
)1 (~~)
JR given by
t:,,.
152
Exercise 1.9.2: Remember, sometimes you have to use the definition of the derivative, rather than the rules for computing derivatives.
Chapter 1.
Vectors, matrices, and derivatives
if(:)#(~) if(:)=(~) is differentiable at every point of R 2 • a. Show that for
1.9.2
The functions g and h for Exercise 1.9.2, parts b and c:
all directional derivatives exist, but that
f is not differentiable at the origin.
*b. Show that the function g defined in the margin has directional derivatives at every point but is not continuous. *c. Show that the function h defined in the margin has directional derivatives at every point but is not bounded in a neighborhood of 0. Consider the function
1.9.3
f : R2
+
R given by if(:)#
Exercise 1.9.3, part c: You may find the following fact useful:
Isin xi
if(:) = (~).
:'.5 lxl for all x E R.
This follows from the mean value theorem:
Isin xi=
11"' costdtl
:'.511"' 1 dtl = lxl.
G)
a. What does it mean to say that
f is differentiable at ( ~) ?
b. Show that both partial derivatives Dif them. c. Is
(~) and D2f (~) exist, and compute
f differentiable at (~)?
1.10 REVIEW EXERCISES FOR CHAPTER 1 1.1 Which of the following lines are subspaces of R 2 (or Rn)? For any that are not, why not? a.
1.2
y
= 2x 
5
b.
y = 2x+ 1
c.
y=
5x
2
For what values of a and b do the matrices
A=[~ ~]
and
B=
[!
~]
satisfy AB = BA? 1.3
Show that if A and Bare upper triangular n x n matrices, so is AB.
1.10
1.4
a. Show that the rule associating a complex number z
matrix Tz Exercise 1.4: If you are not comfortable with complex numbers, please read Section 0.7.
Review exercises for Chapter 1
= [ _~ ~]
153
= a + if3 to the 2 x 2
satisfies
Tz 1 +z 2
= Tz 1 + Tz 2
and Tz 1 z 2
= Tz 1 Tz 2 •
b. What is the inverse of the matrix Tz? How is it related to 1/ z? c. Find a 2 x 2 matrix whose square is minus the identity. 1.5 Suppose all the edges of a graph are oriented by an arrow on them. We allow for twoway streets. Define the oriented adjacency matrix to be the square matrix with both rows and columns labeled by the vertices, where the (i, j)th entry is m if there are m oriented edges leading from vertex i to vertex j. What are the oriented adjacency matrices of the graphs below?
1.6
Are the following maps linear? If so, give their matrices.
a. [
:~
X3
]
I+ [ X2 ]
b. [
X4
X4
:~
X3
]
X2X4
I+ [
Xl
+ X3 ]
X4
a. Show that the function defined in Example 1.5.24 is continuous. b. Extend f to be 0 at the origin. Show that then all directional derivatives exist at the origin (although the function is not continuous there). 1. 7
1.8 Let B be a k x n matrix, A an n x n matrix, and C a n x m matrix, so that the product BAG is defined. Show that if IAI < 1, the series
BC+ BAG+ BA 2 C
+ BA 3 C · · ·
converges in Mat (k, m) to B(I  A) 1 C. 1. 9 a. Is there a linear transformation T : R 4 are satisfied?
+
R 3 such that all of the following
If so, what is its matrix?
b. Let S be a transformation such that the equations of part a are satisfied,
~d, m
additirm,
s
m~ m],
s lin?
1.10 Find the matrix for the transformation from R 3 around the yaxis.
+
R 3 that rotates by 30°
154
Chapter 1.
Vectors, matrices, and derivatives
a. What are the matrices of the linear transformations S, T : R 3 corresponding to reflection in the planes of equation x = y and y = z? 1.11
+
R3
b. What are the matrices of the compositions S o T and T o S? c. What relation is there between the matrices in part b? d. Can you name the linear transformations S o T and T o S? 1.12
Let A be a 2 x 2 matrix. If we identify the set of 2 x 2 matrices with R 4
by idont;fy;ng [:
: ] with [;]. what is the angle between A and A  '? Unde
oo On? 1.17
Prove the following statements for closed subsets of Rn:
a. Any intersection of closed sets is closed. b. A finite union of closed sets is closed. c. An infinite union of closed sets is not necessarily closed. 1.18 Show that U (the closure of U) is the subset of Rn made up of all limits of sequences in U that converge in Rn.
1.10 1.19
Review exercises for Chapter 1
155
Consider the function
You are given the choice of five directions: ei, e2, e3, v1
t2] ,
= [ 11 1/../2
v2
=
[11°.n] .
1/../2
(Note that these vectors all have length 1.) You are to move in one of those directions. In which direction should you start out a. if you want zy 2 to be increasing as slowly as possible? b. if you want 2x 2  y 2 to be increasing as fast as possible? c. if you want 2x 2  y 2 to be decreasing as fast as possible? 1.20 Let h: JR + JR be a 0 1 function, periodic of period 211', and define the function f : JR 2 + JR by
f (rc?s9) = rh(9). rsm9
a. Show that f is a continuous realvalued function on JR 2. b. Show that f is differentiable on JR 2  {O}. c. Show that all directional derivatives of f exist at 0 if and only if
h(9)
= h(9 + 11')
for all 9.
d. Show that f is differentiable at 0 if and only if h(9) = acos9 + bsin9 for some numbers a and b. 1.21
a.
*d.
. l1m
(:)~(~) lim
(:) ~(~)
State whether the following limits exist, and prove it.
x +y x2 y2
b.
lim
(:)~(~)
(x2
+ y2)2
c.
x+y
(x 2 + y 2 )(ln jxyl), defined when xy
lim
(:)~(~)
Prove Theorems 1.5.29 and 1.5.30.
1.23
Let an E Rn be the vector whose entries are
= [ ~: ~] , a2 = [ ~: ~~] , and so on.
+ y2 )
¥ 0
1.22
a1
(x 2 + y 2 ) ln(x 2
11'
and e, to n places:
How large does n have to be so that
so that
1.24
Set Am
of matrices m 1.25
=
f+
cos . m9n sin m9n ] . For what numbers fJ does the sequence  sin mu cos mu Am converge? When does it have a convergent subsequence? [
Find a number R for which you can prove that the polynomial
p(z) = z 10 + 2z9 has a root for
lzl < R.
+ 3z8 + · · · + lOz + 11
Explain your reasoning.
156 1.26
Chapter 1.
Vectors, matrices, and derivatives
What is the derivative of the function
I : Jr
+ Rn given by the formula
I (x) = lxl 2 x? 1.27 Using Definition 1.7.1, show that 0, but that VX4 is.
v'x2 and ?'x2 are not differentiable at
1.28 a. Show that the mapping Mat (n, n) + Mat (n, n), A tiable. Compute its derivative.
1+
A 3 is differen
b. Compute the derivative of the mapping Mat (n, n) +Mat (n, n), A 1.29
c. 1(:)=
{A :
+y
{ •f(""l
b. What is
:
+y
if(:)#(~) if(:)=(~)
tt2 )
~)
1+
~)
both of which you should know how to differentiate.
the area is increasing the fastest?
~)
from (
l ) so that as you start
out, the area is increasing the fastest?
f. Find a point at which A is not differentiable. t2
1.31
a.
What is the derivative of the function l(t) = {
lt
fort>l? Exercise 1.32: It's a lot easier to think of this as the composition of A 1+ A 3 and A 1+ A 1 and to apply the chain rule than to compute the derivative directly.
Ix + YI
from ( _ ~ ) so that as you start out,
and
1Y s +d~n s'
=
[DA ( l)] v maximal; that is, in what
v E R2 is
e. In what direction should you move ( (
I (:)
?
if(:)#(~) if(:)=(~)
direction should you begin moving (
1+ (
b.
8)
[DA ( J )] [~] ?
c. For what unit vector
t
for any integer k ~ 1.
Ak,
Which of the following functions are differentiable at (
a. 1(:)=
Hint for Exercise 1.31: Think of the composition of
1+
b. When is 1.32
I increasing or decreasing?
Let A be an n x n matrix, as in Example 1.8.6.
a. Compute the derivative of the map A
1+
A 3 .
b. Compute the derivative of the map A
1+
An.
d~
s+sms
, defined
1.10
Review exercises for Chapter 1
157
Let U C Mat (n, n) be the set of matrices A such that the matrix AAT +AT A is invertible. Compute the derivative of the map F : U + Mat (n, n)
1.33
given by
1.34 Consider the function defined on R 2 and given by the formula in the margin. a. Show that both partial derivatives exist everywhere. b. Where is 1.35
f
differentiable?
Consider the function on R 3 defined by xyz x4 +y4 + z4
Function for Exercise 1.34 0
a. Show that all partial derivatives exist everywhere. b. Where is
f
differentiable?
1.36 a. Show that if an n x n matrix A is strictly upper triangular or strictly lower triangular, then An = [OJ. b. Show that (I  A) is invertible. Starred exercises are difficult; exercises with two stars are particularly challenging.
c. Show that if all the entries of A are nonnegative, then each entry of the matrix (I  A) 1 is nonnegative. *1.37
What 2 x 2 matrices A satisfy a. A2
= 0,
b. A2
= I,
c. A 2 =I?
**1.38 (This is very hard.) In the Singapore public garden, there is a statue consisting of a spherical stone ball, with diameter perhaps 1.3 m, weighing at least a ton. This ball is placed in a semispherical stone cup, which it fits almost exactly; moreover, there is a water jet at the bottom of the cup, so the stone is suspended on a film of water, making the friction of the ball with the cup almost O; it is easy to set it in motion, and it keeps rotating in whatever way you start it for a long time. Suppose now you are given access to this ball only near the top, so that you can push it to make it rotate around any horizontal axis, but you don't have enough of a grip to make it turn around the vertical axis. Can you make it rotate around the vertical axis anyway?
N
The telescope of Exercise 1.39. The angle a is the azimuth; /3 is the elevation. North is denoted N.
*1.39 Suppose a telescope is mounted on an equatorial mount, as shown in the margin. This means that mounted on a vertical axis that can pivot there is a Ushaped holder with a bar through its ends, which can rotate also, and the telescope is mounted on the bar. The angle which the horizontal direction perpendicular to the plane of the U makes with north (the angle labeled a in the picture in the margin) is called the azimuth, and the angle which the telescope makes with the horizontal (labeled /3) is called the elevation. Center your system of coordinates at the center of the bar holding the telescope (which doesn't move either when you change either the azimuth or the elevation),
158
Chapter 1.
Vectors, matrices, and derivatives
and suppose that the xaxis points north, the yaxis points west, and the zaxis points up. Suppose, moreover, that the telescope is in position azimuth Bo and elevation i!otal unknowns. We read their values from the first three rows of [A IOJ (remembering that a solution for Ax= 0 is also a solution for Ax= 0): 1 0 2 1  0 1 1 1 [A IOJ = [ 0 00 0 0 0 0 0 which gives
is in the kernel of A, since Aw = 0, so it should be possible to express w as a linear combination of V1 and v2. Indeed it is: w = 3v1 v2.
Solving equations
0 0] 0 0 0 , 0 0
l
X1+2X3 +
that is,
=
X5
=0.
X4 
+x3 
X4
X3
= 0
x4 = X5
X1 = 2X3 X2
x2
0
2.5.4
= 0,
X4
2.5.5
So for v 1, where x 3 = 1 and X4 = 0, the first entry is X1 = 1, the second is 1 and the fifth is O; the corresponding entries for v2 are 3,  2, and 0:
2.5.6
2.5
Kernels, images, and the dimension formula
These two vectors form a basis of the kernel of A.
6
n
Now find a basis for the image and kernel of
A=[:
1 1
3 0
1
2
:l·
which mw "'duces to
checking your answer in the footnote
195
rn
0
1
l
1 0
0
2.5.7
below. 10
Proof of Theorem 2.5.4 {A basis for the image). Let
A= [a'.1 ... a'.m].
1. The pivotal columns of A (in fact, all columns of A) are in the image, since Aei = ~.
2. The pivotal columns are linearly independent, by Theorem 2.4.5. 3. The pivotal columns span the image, since each nonpivotal column is a linear combination of the preceding pivotal ones. Suppose the kth column of A is nonpivotal. View the first k columns of A as an augmented matrix, i.e., try to express the kth column as a linear combination of the earlier ones. Row reduce the submatrix of A consisting of the first k columns, which is the same thing as considering the first k columns of A. Since the kth column is nonpivotal, there is no pivotal 1 in the last column, so it is possible to express the kth column of A as a linear combination of the earlier ones, and in fact the entries of the kth column of A tell us how to express it as a linear combination of the earlier pivotal columns. D
Proof of Theorem 2.5.6 {A basis for the kernel). 1. By definition,
Av\
=
0, so vi
E ker A.
2. The vi are linearly independent, since exactly one has a nonzero number in each position corresponding to nonpivotal unknown. 3. The vi span the kernel means that any x satisfying Ax= 0 can be written as a linear combination of the vi. Suppose Ax= 0. We can construct a vector w= Xk 1 v1 + · · · + Xkp Vp that has the s_:i.me entry Xk; in the nonpivotal column ki as does x. Since Avi = O, we have Aw= 0. But for each value of the nonpivotal variables, there is a unique vector x such that Ax= 0. Therefore, x = w. D
'°The
~~rn m,Hl·rn
fom a bafils
'°' ~
Un.ge; [
il
= IB a bMIB o; th;
for the kernel. The third column of A is nonpivotal, so for the[v~ct~r of the kernel we set x 3 = 1. The rowreduced matrix i.e,
X1
+ X3 = 0, X2 + X3 = 0, and X4 = 0.
[AIO]
is
0 0
This gives
X1
1 0
= 1, X2 =
b~lis
1 0 0 1 1.
0 , 0
196
Chapter 2.
Solving equations
Uniqueness and existence: the dimension formula
Recall (Proposition and Definition 2.4.21) that the dimension of a subspace of IRn is the number of elements of any basis of the subspace. It is denoted dim. The dimension of the kernel of a linear transformation is called its nullity, but we rarely use that word.
The dimension formula says there is a conservation law concerning the kernel and the image. Since kernels are a way to talk about uniqueness of solutions to linear equations, and images are a way to talk about existence of solutions, this means that saying something about uniqueness says something about existence. We will state this precisely in the case of square matrices in Corollary 2.5.10. The rank of a matrix (i.e., the number of linearly independent columns) is the most important number to associate to it. The power of linear algebra comes from Corollary 2.5.10. We give examples concerning interpolation and partial fractions later in this section. See also Exercises 2.5.17 and 2.37, which deduce major mathematical results from this corollary.
Theorems 2.5.4 and 2.5.6 tell us that if T is a linear transformation given by a matrix [T], then the dimension of its image is the number of pivotal columns of [T], while the dimension of its kernel is the number of nonpivotal columns. Since the total number of columns of [T] is the dimension of the domain of T (a matrix n wide takes as input a vector in Rn), the sum of the dimension of the image and the dimension of the kernel equals the dimension of the domain. This statement, known as the dimension formula, is deceptively simple; it provides much of the power of linear algebra. Because it is so important, we state it formally here.
Theorem 2.5.8 (Dimension formula). Let T: Rn+ Rm be a linear transformation. Then dim(ker T)
+ dim(img T)
= n,
the dimension of the domain. 2.5.8
Definition 2.5.9 (Rank). The dimension of the image of a linear transformation is called its rank. If T is a linear transformation represented by a 3 x 4 matrix [T] with rank 2, what is its domain and codomain? What is the dimension of its kernel? Is it onto? Check your answers in the footnote below. 11 The most important case of the dimension formula is when the domain and codomain have the same dimension: then existence of solutions can be deduced from uniqueness. It is remarkable that knowing that T(X.) = 0 has a unique solution guarantees existence of solutions for all T(x) = b! This is, of course, an elaboration of Theorem 2.2.1 (and of Theorem 2.4.5). But those theorems depend on knowing a matrix. Corollary 2.5.10 can be applied when there is no matrix to write down, as we will see in Example 2.5.14 and Exercises 2.5.12, 2.5.17, and 2.37.
Corollary 2.5.10 (Deducing existence from uniqueness). Let T: Rn +Rn be a linear transformation. Then the equation T(X.) = b has a solution for evezy b E Rn if and only if the only solution to T(X.) = 0 is x = 0, (i.e., if the kernel has dimension 0). Since Corollary 2.5.10 is an "if and only if" statement, it can also be used to deduce uniqueness from existence, when the domain and codomain of a transformation have the same dimension; in practice this is not quite so useful. It is often easier to prove that T(X.) = 0 has a unique solution than it is to construct a solution of T(X.) = b. 11 The
domain of Tis IR4 ; the codomain is IR3 . The dimension of its kernel is 2, since here dim(ker T) + dim(img T) = n becomes dim(ker T) + 2 = 4. The transformation is not onto, since a basis for IR3 must have three basis elements.
2.5
Kernels, images, and the dimension formula
197
Proof. Saying that T(x) = b has a solution for every b E Rn means that Rn is the image of T, so dim imgT = n, which is equivalent to dim ker (T) = 0. D The following result is really quite surprising. Proposition 2.5.11: A matrix A and its transpose AT have the same rank.
Proposition 2.5.11. Let A be an m x n matrix. Then the number of linearly independent columns of A equals the number of linearly independent rows. One way to understand this result is to think of constraints on the kernel of A. Think of A as the m x n matrix made up of its rows:
A=
l
[ ==~~. == .
2.5.9
AmThen the kernel of A is a subspace of Rn; it is made up of the vectors x satisfying the linear constraints Aix = 0, ... , Ami= 0. Think of adding in these constraints one at a time. Each time you add one constraint (i.e., one row Ai), you cut down the dimension of the kernel by 1. But this is only true if the new constraint is genuinely new, not a consequence of the previous ones (i.e., if Ai is linearly independent from Ai, ... , Ai_ 1 ). Let us call the number of linearly independent rows Ai the row rank of A. The argument above leads to the formula dimker A= n  row rank(A).
2.5.10
The dimension formula says exactly that dimker A= n  rank(A),
We defined linear combinations in terms of linear combinations of vectors, but (as we will see in Section 2.6) the same definition can apply to linear combinations of other objects, such as matrices, functions, etc. In this proof we are applying it to row matrices.
2.5.11
so the rank of A and the row rank of A should be equal. This argument isn't quite rigorous: it used the intuitively plausible but unjustified "Each time you add one constraint, you cut down the dimension of the kernel by 1." This is true and not hard to prove, but the following argument is shorter (and interesting too).
Proof. Call the span of the columns of a matrix A the column space of A and the span of the rows its row space. The rows of A are linear combinations of the rows of A, and vice versa since row operations are reversible, so if A row reduces to A, the row space of A and of A coincide. The rows of A that contain pivotal l's are a basis of the row space of A: the other rows are zero so they definitely don't contribute to the row space, and the pivotal rows of A are linearly independent, since all the other entries in a column containing a pivotal 1 are 0. So the dimension of the row space of A is the number of pivotal l's of A, which we have seen is the dimension of the column space of A. D
198
FIGURE
2.5.1.
Carl Friedrich Gauss (17771855) Often thought of as the greatest mathematician of all times, Gauss was also deeply involved in computations; he invented row reduction (also called Gaussian elimination), the Fast Fourier transform, Gaussian quadrature, and much more. We explore Cramer's formulas in Exercise 4.8.18. In group 1, statements 2 and 5 are just the definition of "onto", for any map. In group 2, statement 2 is just the definition of "one to one", for any map. The other equivalences depend on A being a linear transformation .
Statement 4 in group 1: If an m x n matrix spans Rm it has m linearly independent columns, so it follows from Proposition 2.5.11 that all m rows must be linearly independent. Within each group, if one statement is true, all are; if one statement is false, all are. But if A is a square matrix, then if any statement in either group is true, then all 16 statements are true; if any statement is false, then all 16 statements are false.
Chapter 2.
Solving equations
Remark. Proposition 2.5.11 gives us the statement we wanted in Section 2.4: the number of linearly independent equations in a system of linear equations Ax = b equals the number of pivotal columns of A. Basing linear algebra on row reduction can be seen as a return to Euler's way of thinking. It is, as Euler said, immediately apparent why you can't determine x and y from the two equations 3x  2y = 5 and 4y = 6x  10. (In the original, "La raison de cet accident saute d'abord aux yeux": the reason for this accident leaps to the eyes.) When the linear dependence of a system of linear equations no longer leaps to the eyes, row reduction can make it obvious. Unfortunately for the history of mathematics, in the same year 1750 that Euler wrote his analysis, Gabriel Cramer published a treatment of linear equations based on determinants, which rapidly took hold, overshadowing the approach begun by Euler. Today, the rise of the computer, with emphasis on computationally effective schemes, has refocused attention on row reduction as an approach to linear algebra. 6. In linear algebra there are many different ways of saying the same thing; we summarize some below in Table 2.5.2. Equivalent statements about a linear transformation A : !Rn 1. 2. 3. 4. 5. 6. 7. 8.
!Rm
A is onto (also known as surjective). The image of A is !Rm. The columns of A span !Rm. The rows of A are linearly independent. For every b E !Rm there exists a solution to the equation Ax = The dimension of img(A) (also known as the rank of A) ism. The rowreduced matrix A has no row containing all zeros. The rowreduced matrix A has a pivotal 1 in every row.
Equivalent statements about a linear transformation A : !Rn 1. 2. 3. 4. 5. 6. 7. 8.
+
+
b.
!Rm
A is one to one (also called injective). Ax = b has a solution, it is unique. The columns of A are linearly independent. ker A= {O}. The only solution to the equation Ax = 0 is x = 0. The dimension of ker A is 0. The rowreduced matrix A has no nonpivotal column. The rowreduced matrix A has a pivotal 1 in every column.
If the equation
TABLE
2.5.2.
Now let us see two examples of the power of Corollary 2.5.10 (the dimension formula in the case where the domain and codomain of T have the
2.5
The problem of reproducing a polynomial from its samples long engaged mathematicians; around 1805 Gauss hit on the algorithm now known as the fast Fourier transform while trying to determine the position of an asteroid from sampled positions, when the orbits are assumed to be given by a trigonometric polynomial.
199
Kernels, images, and the dimension formula
same dimension). One is related to signal processing, the other to partial fractions.
Interpolation and the dimension formula In the mideighteenth century, the Italian Joseph Lagrange (17361813) developed what is now known as the Lagrange interpolation formula: an explicit formula for reconstituting a polynomial of degree at most k from k + 1 samples (Figure 2.5.3 shows that this is not an effective way to approximate a function). Exercise 2.5.18 asks you to find Lagrange's formula. Here we will use the dimension formula to show that such a formula must exist. Let Pk be the space of polynomials of degree ::::; k. Given k + 1 numbers Co, ... , Ck, can we find a polynomial p E Pk such that
p(O) =Co 2.5.12 Interpolation is fundamental in the communications and computer industry. Usually one does not want to reconstitute a polynomial from sampled values. Rather one wants to find a function that fits a data set; or, in the other direction, to encode a function as n sampled values (for example, to store sampled values of music on a CD and then use these values to reproduce the original sound waves). In 1885 Weierstrass proved that a continuous function on a finite interval of JR can be uniformly approximated as closely as one wants by a polynomial. But if such a function is sampled n times for n even moderately big (more than 10, for example), Lagrange interpolation is a dreadful way to attempt such an approximation. Bernstein polynomials (introduced in 1911 by Sergei Bernstein to give a constructive proof of Weierstrass's theorem) are far superior; they are used when a computer draws Bezier curves.
p(k) =Ck? Our tactic will be to consider the linear transformation that takes the polynomial p and returns the sampled values p(O), ... ,p(k); we will then use the dimension formula to show that this transformation is invertible. Consider the linear transformation Tk : Pk t JRk+l given by 2.5.13
10
1
1
~~
FIGURE
2.5.3. LEFT: The function x 2
+1l/lO'
between x
=
1 and x
=
l.
RIGHT: The unique polynomial of degree 20 reconstituted from the values of the function at the 21 points 1, .9, ... , .9, 1 (the marked dots), using Lagrange interpolation. Near the boundary of the domain of interpolation, Lagrange interpolation is a terrible way to approximate a function.
200
Since ker Tk = {O}, the transformation is one to one; since the domain and codomain of Tk both are of dimension k + 1, Corollary 2.5.10 says that since the kernel is zero, Tk is onto. Thus it is invertible.
Constructing T2 in Example 2.5.12: The ith column of a matrix T is Tei; here, the standard basis vectors ei' e2' ea are replaced by the polynomials 1, x, and x 2 , since ei is identified to the polynomial 1+Ox+ Ox 2 , e2 is identified to 0 + x + Ox 2 , and ea is identified to 0 + 0 + x 2 • Thus the first column of T2 is
T,(!)
~
Chapter 2.
Solving equations
The kernel of Tk is the space of polynomials p E Pk that vanish at 0, 1, ... , k. But a polynomial of degree :::; k cannot have k + 1 roots, unless it is the zero polynomial. So the kernel of Tk is {O}. The dimension formula then tells us that its image is all of JRk+ 1 : dim img Tk + dim ker Tk =
2.5.14
=k+l. dim. of domain of Tk
In particular, Tk is invertible. This gives the result we want: there is a transformation r;; 1 : JRk+l ~Pk such that for the sampled values eo, ... , Ck we have 1
Tk
: [Col .
= ao + a1x + · · · + akx k ,
2.5.15
p(x)
Ck
where pis the unique polynomial with p(O) =
eo, p(l) =
ci, . . . ,
p(k) =Ck·
rn;
the constant polynomial 1 is always 1. The second is 1 [11 01 OJ
2.5.16
1 2 4
since the polynomial x x = 0, 1 when x = when x = 2. For the polynomial x 2 is 0 at and 4 at 2.
is 0 when 1, and 2 third, the 0, 1 at 1,
Exercise 2.5.10 asks you to show that there exist numbers co, ... , Ck such that
r
Jo
0
p(t) dt =
equivalent to the polynomialp(x) = a+bx+cx 2 evaluated at x = 0, x = 1, and x = 2. The inverse of T2 is 02 1 so if we choose (for instance)
Co= 1, c =4, 1
0 1/2 1/2 c2
l
2.5.17
'
= 9, we get 1+2x + x 2 :
~ 1~2] [!]9 [;]1 [ 3~2 1/2 1/2
L Cip(i) k
2.5.18
1
i=O
for all polynomials p E Pk. Thus "sampling" (evaluating) a polynomial p E Pk at k + 1 points, giving appropriate weights Ci to each sample, and adding the results, gives the same information as using an integral to sum the values of the polynomial at all points in a given interval.
which corresponds to the polynomial p(x) = 1+2x + x 2 • Indeed, 1+2. 0 + 0 = 1, _____... 1+2·1+1 = 4, _____... 1+2. 2 + 2 = 9. _____... 2
p(O)
2
2
p(l)
!::::.
p(2)
Remark. Our discussion of interpolation deals with the special case where the k+ 1 numbers eo, ... , Ck are the values of a polynomial evaluated at the specific numbers 0, 1, ... , k. The result is true when the polynomial is evaluated at any distinct k + 1 points x 0 , .•. , Xk: there is a unique polynomial such that p(xo) = eo, p(x1) = c1, ... ,p(xk) =ck. !:::.
2.5
Kernels, images, and the dimension formula
201
Partial fractions
In equation 2.5.19, we are requiring that the ai be distinct. For example, although
p(x)
= x(x 
l)(x  l)(x  1)
can be written p(x) = x(x  l)(x  1) 2 ,
this is not an allowable decomposition for use in Proposition 2.5.13, while p(x) = x(x  1) 3 is. It follows from Proposition 2.5.13 that you can integrate all
rational functions, i.e., ratios of two polynomials. You can integrate each term of the form q(x) (x  a)n
with deg q < n in equation 2.5.20, as follows. Substitute x = (x  a) +a in q and multiply out to see that there exist numbers bn1, ... , bo such that q(x) = bn1(x 
atl + · · · + bo.
Then q(x) (x a)n
+ · · · + bo
(x a)n
= bn1 + ... + (x  a)
"I
bo . (x  a)n
/i     1
1
(x  a)n  n  1 (x  a)nl
if n=l, then
j
(x
~a)
P (x) = (x  ai)n1
• • •
(x  ak)nk
2.5.19
be a polynomial of degree n = ni + · · · + nk with the ai distinct and let q be any polynomial of degree< n. Then the rational function q/p can be written uniquely as a sum of simpler terms, called partial fractions: 2.5.20 with each qi a polynomial of degree
< ni.
Note that by the fundamental theorem of algebra, every polynomial can be written as a product of powers of degree 1 polynomials with the ai distinct, as in equation 2.5.19. For example,
x 3  2x 2 + x
+ l)(x 
1), with ai
= 1, a2 = 1; ni = n2 = 1, son= 2
= x(x 1) 2, with ai = O,a2 =
l;ni
=
l,n2 = 2, son= 3.
Of course, finding the ai means finding the roots of the polynomial, which may be very difficult.
Example 2.5.14 (Partial fractions). When
1, then
dx
Proposition 2.5.13 (Partial fractions). Let
x 2  1 = (x
bn1(x  a)nl
If n
When d' Alembert proved the fundamental theorem of algebra, his motivation was to be able to decompose rational functions into partial fractions so that they could be explicitly integrated; that is why he called his paper Recherches sur le calcul integral ("Enquiries into integral calculus"). You will presumably have run into partial fractions when studying methods of integration. Recall that decomposition into partial fractions is done by writing undetermined coefficients for the numerators; multiplying out then leads to a system of linear equations for these coefficients. Proposition 2.5.13 proves that such a system of equations always has a unique solution. As in the case of interpolation, the dimension formula, in its special case Corollary 2.5.10, will be essential. The claim of partial fractions is the following:
=In Ix 
al + c
+c'
q(x)
= 2x + 3
and
p(x)
= x2 
1,
2.5.21
Proposition 2.5.13 says that there exist polynomials qi and q2 of degree less than 1 (i.e., numbers, which we will call Ao and Bo, the subscript indicating that they are coefficients of the term of degree 0) such that 2x +3 2
x l
Bo Ao + . x+l
xl
2.5.22
If q(x) = x 3 1 and p(x) = (x + 1) 2 (x 1) 2 , then the proposition says that there exist two polynomials of degree 1, qi = Aix +Ao and q2 = Bix +Bo,
202
Chapter 2.
Solving equations
such that
x 3 1 Aix+Ao Bix+Bo (x + 1) 2(x  1) 2 = (x + 1)2 + (x  1)2 ·
2.5.23
In simple cases, it's clear how to proceed. In equation 2.5.22, to find Ao and Bo, we multiply out to get a common denominator:
2x + 3 = ~ + ~ = Ao(x  1) + Bo(x + 1) = (Ao+ Bo)x +(Bo  Ao) x2  1 x+ 1 x 1 x2  1 x2  1 ' so that we get two linear equations in two unknowns: Ao+Bo=3 Ao +Bo= 2,
that is, the constants
Bo =
5
2,
Ao =
1
2.
2.5.24
We can think of the system of linear equations on the left side of equation 2.5.24 as the matrix multiplication 2.5.25 What is the analogous matrix multiplication for equation 2.5.23? 12
!::::.
What about the general case? If we put the right side of equation 2.5.20 on a common denominator, we see that Q(x)/p(x) is equal to Qi (x)(xa2)n 2 • • ·(xak)nk +Q2(x)(xa1)n 1 (xa3)n 3 • • • (xak)nk + · · · +qk(x)(xa1)n 1 · · · (xaki)nki (x  a1)n1(x  a2)n2 .. ·(x  ak)nk 2.5.26 As we did in our simpler cases, we could write this as a system of linear equations for the coefficients of the Qi and solve by row reduction. But except in the simplest cases, computing the matrix would be a big job. Worse, how do we know that the resulting system of equations has solutions? Might we invest a lot of work only to discover that the equations are inconsistent? i 2
x 3 (Ai
Multiplying out, we get
+Bi)+ x 2 (2Ai +Ao+ 2Bi +Bo)+ x(Ai  2Ao +Bi+ 2Bo) +Ao+ Bo (x + 1)2(x  1)2
so
Ao+Bo=1
(coefficient of term of degree 0)
= 0
(coefficient of term of degree 1)
Ai  2Ao + Bi + 2Bo
2Ai +Ao + 2Bi + Bo =
0
(coefficient of term of degree 2)
Ai+ B1 =
1
(coefficient of term of degree 3);
that is,
1 0
[l
2 1 1 2 0 1
2.5
Kernels, images, and the dimension formula
203
Proposition 2.5.13 assures us that there will always be a solution, and Corollary 2.5.10 provides the key. Corollary 2.5.10: If
T:llr+nr is a linear transformation, the equation T(5?} = b has a solution for any b E !Rn if and only if the only solution to the equation T(x) = is x = We are thinking of the transformation T both as the matrix that takes the coefficients of the qi and returns the coefficients of q, and as the linear function that takes qi, ... , qk and returns the polynomial q.
o
o.
Proof of Proposition 2.5.13 (Partial fractions). The matrix we would get following the above procedure would necessarily be n x n. This matrix gives a linear transformation that has as its input a vector whose entries are the coefficients of qi, ... , qk. There are n such coefficients in all: each polynomial qi has degree< ni, so it is specified by its coefficients for terms of degree 0 through (ni  1) and the sum of the ni equals n. (Note that some coefficients may be 0, for instance if qj has degree < nj  1.) It has as its output a vector giving the n coefficients of q (since q is of degree < n, it is specified by its coefficients for the terms of degree 0, ... , n  1.) Thus we can think of the matrix as a linear transformation T : R.n + R.n, and by Corollary 2.5.10, Proposition 2.5.13 is true if and only if the only solution of T(qi,. .. ,qk) = 0 is qi=···= qk = 0. This will follow from Lemma 2.5.15. Lemma 2.5.15. If qi
:f 0 is a polynomial of degree< . 1im
x+a;
That is, if qi
:f 0,
I
ni, then
qi(X) I = oo. (x  ai)n;
2.5.27
then qi(x)/(x  ai)n; blows up to infinity.
Proof of Lemma 2.5.15. For values of x very close to ai, the denominator (x ai)n; gets very small; if all goes well the entire term then gets very big. But we have to make sure that the numerator does not get small equally fast. Let us make the change of variables u = x  ai, so that qi(x)
=
qi(u + ai},
Then we have The numerator in equation 2.5.29 is of degree< ni, while the denominator is of degree ni.
which we will denote
liji(u)I
. 1im   . = oo un,
u+O
Indeed, if qi
:f 0,
then Qi  (u ) qi
Qi(u).
...J.O . i"f qi r
:f 0, and there exists a number = amu m + · · · + an;lUn·1 '
2.5.28
2.5.29 m < ni such that
2.5.30
with am :f 0. (This am is the first nonzero coefficient; as u + 0, the term amum is bigger than all the other terms.) Dividing by un;, we can write Qi(u) 1 un; = un;m (am+ ... },
2.5.31
where the dots ... represent terms containing u to a positive power, since m < ni. In particular, as
u+ 0,
1 I Iun;m 
+ 00
and (am+···) +am.
2.5.32
Thus as x+ ai, the term qi(x)/(x  ai)n; blows up to infinity: the denominator gets smaller and smaller while the numerator tends to am :f 0. D Lemma 2.5.15
204
Chapter 2.
Solving equations
We also have to make sure that as x tends to ai and (xai)n; gets small, the other terms in equation 2.5.20 don't compensate. Suppose qi # 0. For all the other terms % , j # i, the rational functions
2.5.33
(x  aj)n;' The proof of Proposition 2.5.13 really put linear algebra to work. Even after translating the problem into linear algebra, via the linear transformation T, the answer was not clear, but the dimension formula made the result apparent.
have the finite limits qj(ai)/(ai  air; as x ~ ai, and therefore the sum
q(x) p(x)
qi(x) (x  ai)n1
+ ... +
qk(x) (x  ak)nk
2.5.34
which includes the term (x1!.:~~~n, with infinite limit as x ~ ai, has infinite limit as x ~ ai and q cannot vanish identically. Thus T(q 1 , .•. , qk) # 0 if some qi is nonzero, and we can conclude  without computing matrices or solving systems of equations  that Proposition 2.5.13 is correct: for any polynomial p of degree n and q of degree < n, the rational function q / p can be written uniquely as a sum of partial fractions. D Table 2.5.4 summarizes four different ways to think about and speak of linear transformations.
Understanding a linear transformation T: JR_n Property of T
Solutions to T(x) =
b
~ JR_m
Matrix of T
Geometry
One to one (injective)
Unique if exist
All columns linearly independent
ker T = {O}
Onto (surjective)
Exist for every b E IR.m
All rows linearly independent Columns span IR.m
image= codomain
Tis square Invertible (11 and onto)
ker T = {O} and image= codomain
(n=m) All exist and are unique
and all columns are linearly independent
TABLE 2.5.4. Row reduction applied to the matrix of T is the key to understanding the properties of T and solutions to the set of linear equations T(x) = b.
EXERCISES FOR SECTION Vectors for Exercise 2.5.1, part a
2. 5
~ ~ ~i
a. For the matrix A= [; 1 0 2 2 in the margin (if any) are in the kernel of A? 2.5.1
'
which of the vectors v1, v2, V3
Kernels, images, and the dimension formula
2.5
205
b. Which vectors have the right height to be in the kernel of T? To be in its image? Can you find a nonzero element of its kernel?
T
=
3 2 1]  = [1]  = [~]1 '  = [1] [122 11 1 101 3 2 1 0
3
0
'
W1
2
'
W2
W3
0
2 1 2
0
0 2.5.2 Let the linear transformation T : llr > !Rm be onto. Are the following statements true or false? a. The columns of T span !Rn. b. T has rank m. c. For any vector v E !Rm, there exists a solution to TX. = v. d. For any vector v E !Rn, there exists a solution to TX. = v. e. T has nullity n. f. The kernel of T is {O}. g. For any vector v E !Rm, there exists a unique solution to TX. = v.
2.5.3 Let T be a linear transformation. Connect each item in the column at left with all synonymous items in the column at right: dimker T dim domain T rank T
dim codomain T
nullity T
dim image T
rank T + nullity T
no. of linearly independent columns of T no. of pivotal columns of T no. of nonpivotal columns of T
2.5.4 Let T be an n x m matrix such that the rowreduced matrix T has at least one row containing all O's. What can you deduce about the rank of T? What if T has precisely one row containing all O's? Justify your statements.
a.
[~
~] '
1
2
b.
[:
1
2 1
4
~]
[; !] 1
c.
2 3
Matrices for Exercise 2.5.6
A~[:
2.5.5
Prove Proposition 2.5.2. (This is a "turn the crank" proof.)
2.5.6 For each of the matrices in the margin, find a basis for the kernel and a basis for the image. 2.5. 7 Let n be the rank of the matrix A shown in the margin. a. What is n? b. Is there more than one way to choose n linearly independent columns of A? c. Is it possible to choose three columns of A that are not linearly independent? d. What does this say about solutions to the system of equations
+ x2 + 3x3 + X4 = 1 X1 X2 + X5 = 1 X1 + X2 + X3 + X6 = 1 ?
2x1 1 1 1
3 1 0 0 1 0
01 "] 0 0 1
Matrix for Exercise 2.5. 7
2.5.8 True or false? (Justify your answer.) Let be linear transformations. Then fog= 0
implies
f : !Rm > !Rk and g : !Rn > !Rm
imgg = ker f.
Solving equations
Chapter 2.
206 2.5.9
Let P2 be the space of polynomials of degree :5, 2, identified with R 3 by
identifying a
+ bx + "'' to
( :) .
a. Write the matrix of the linear transformation T : P2
P2 given by
>
+ x 2 p"(x).
(T(p))(x) = xp'(x)
b. Find a basis for the image and the kernel of T. 2.5.10
Exercise 2.5.10 is very realistic. When integrating a function numerically you are always going to be taking a weighted average of the values of the function at some points. Here we explain how to choose the weights so that the formula for approximating integrals is exact for polynomials of degree :5, k.
;J al
b
b
for all polynomials p E Pk.
Make a sketch, in the (a, b)plane, of the sets where the kernels of 2.5.11 the matrices at left have dimension 0, 1, 2, 3. Indicate on the same sketch the dimensions of the images. 2.5.12 Decompose the following into partial fractions, as requested, being explicit in each case about the system of linear equations involved and showing that its matrix is invertible:
2.5.13
x+x 2 (x + l)(x + 2)(x + 3) x+x 3 (x + 1)2(x  1)3
C B A ++. x+2 x+3 x+l
as
as
Ax+B (x+1)2
+ Cx 2 +Dx+F (x1)3
a. For what value of a can you not write
Ao
x 1
a
(x
Matrices for Exercise 2.5.11
L Cip(i) i=O
a
b. Write
1 2 B= [ a b a
k
fb },, p(t) dt =
a. Write
A=[!
co, ... , Ck such that
Show that for any a< b, there exist numbers
B1x +Bo
?
+ l)(x 2 +ax+ 5) = x + 1 + x 2 +ax+ 5 ·
b. Why does this not contradict Proposition 2.5.13? 2.5.14 a. Letf(x)=x+Ax2 +Bx3 • Findapolynomialg(x)=x+ax2 +,Bx 3 such that g(f(x))  xis a polynomial starting with terms of degree 4.
Exercise 2.5.17: The polynomial p constructed in this exercise is called the Lagrange interpolation polynomial; it "interpolates" between the assigned values. Consider the map from the space of Pn of polynomials of degree n to Rn+i given by p ._.
[p(~o)]
·
p(xn) You need to show that this map is onto; by Corollary 2.5.10 it is enough to show that its kernel is
{O}.
b. Show that if f(x) = x + 2::~= 2 aixi is a polynomial, then there exists a unique polynomial k
g(x) = x +
L bixi with go f(x) = x + xk+lp(x) for some polynomial p. i=2
2.5.15 Show that if A and Bare n x n matrices, and AB is invertible, then A and B are invertible. 2.5.16 Let S: Rn > Rm and T: Rm that rank(T o S) :5, min( rank T, rank S). *2.5.17
>
RP be linear transformations. Show
a. Find a polynomial p(x) =a+ bx+ cx 2 of degree 2 such that
p(O) = 1,
p(l) = 4,
and p(3) = 2.
b. Show that if xo, ... , Xn are n + 1 distinct points in R, and ao, ... , an are any numbers, there exists a unique polynomial of degree n such that p(xi) = ai for each i = 0, ... ,n. c. Let the Xi and ai be as in part b, and let bo, ... , bn be any numbers. Find a number k such that there exists a unique polynomial of degree k with
p(xi) = ai
and
p'(xi) =bi
for all i = 0, ... ,n.
2.6
Abstract vector spaces
207
2.5.18 a. Prove that given any distinct points xo, ... , Xk E Rand any numbers co, ... , Ck E R, the polynomial p (x ) =
~
I1ui(x 
Xj)
i=O
rr#i(Xi 
Xj)
(Lagrange interpolation formula)
~Ci="'~,..,
is a polynomial of degree at most k such that p( Xi)
= Ci.
b. For k = 1, k = 2, and k = 3, write the matrix of the transformation
Exercise 2.5.19, part a: Remember, the ith column of the matrix is the image of the ith basis vector. Part c: If you multiply out fa, you get something of the form Pa(x) xn
What is the degree of Pa? What must Pa satisfy if a E ker Hn?
( : ) 2.5.19
~
p, whe {v} enables us to move from the concrete world of Rn to the abstract world of a vector space V. Definition 2.6.12 ("Concrete to abstract" function cI>{v})· Let {v} = V1, .•. , Vn be a finite, ordered collection of n vectors in a vector space V. The concrete to abstract function cI> {v} is the linear transformation cI>{v} : Rn. V that translates from Rn to V:
This uses Theorem 1.3.4: the ith column of [T] is T(e;). For instance, 4>{v} ( [
4>{v}
~])
2.6.12
= lv1 + Ov2 = V1
([~]) = Ov1 + lv2 = v2.
Example 2.6.13 (Concrete to abstract function). Let P 2 be the space of polynomials of degree at most 2, with basis v 1 = 1, v2 = x, v3 = x 2. Then {v}
( [:])
~ a,+ a,x + a,x
2
idenUfies R 3 with P,.
6
Example 2.6.14. In practice, the "abstract" vector space Vis often Rn, with a new basis. If V
v2 =
[_
= R 2 and the set {v} consists of v\ = [ ~] and
~ ] , then 2.6.13
212
Chapter 2.
Solving equations
the point with coordinates (a, b) in the basis In Example 2.6.14,
~{v} = U
standard basis:
n
2.6.14
For instance, the point with coordinates (2, 3) in the new basis {v} is the point with coordinates (5, 1) in the standard basis:
because
~{v}(e1) = v\ = [n
~{v}{e2) = v2 = [~]. Proposition 2.6.15 says that any vector space with a basis is "just like" Jr: if {v} is a basis of V, then the linear transformation ~{v} : Rn > V is invertible, and its inverse allows us to identify V with Rn:
~{0,(a,v, + ... + a,,v.)~ [
v\, v2, equals [: ~ ~] in the
ll
u n [i] [n
..____,,,___..,
=
means
~{v} ([i]) = 2v1+3v2 = [; ~ i]. 6
~{v}
Proposition 2.6.15 restates the notions of linear independence, span, and basis in terms of ~ {v}. Proposition 2.6.15 (Linear independence, span, and basis). Let {v} =vi, ... , Vn be vectors in a vector space V, and let ~{v}: Rn  t V be the associated concretetoabstract transformation. Then 1. The set {v} is linearly independent if and only if~ {v} is one to
one. 2. The set {v} spans V if and only if~ {v} is onto. 3. The set {v} is a basis of V if and only if~ { v} is invertible.
This allows us to replace questions about V with questions about the coordinates in ]Rn.
Proof. 1. Definition 2.6.10 says that vi, ... , Vn are linearly independent if and only if n
n
Lai Vi= Lbivi i=l
implies
ai
=bi, a2
=
b2, ... , an= bn,
2.6.15
i=l
that is, if and only if ~{v} is injective (one to one). 2. Definition 2.6.9 says that
{v}=v1, ... ,Vn
2.6.16
span V if and only if any vector v EV is a linear combination ofvi, ... , Vn: 2.6.17
In other words,
~{v}
is surjective (onto).
3. Putting these together, v 1 , ... , Vn is a basis if and only if it is linearly independent and spans V (i.e., if ~{v} is invertible). D
Matrix with respect to a basis and change of basis Proposition and Definition 2.6.16 says we can use matrices to describe linear transformations between finitedimensional vector spaces, once bases have been chosen. The change of basis matrix described in Proposition and
2.6
Abstract vector spaces
213
Definition 2.6.17 then allows us to express a vector in one basis in terms of its expression in another basis. The change of basis formula (Theorem 2.6.20) allows us to relate a linear transformation T: V + W written in one pair of bases to the same linear transformation written in another pair of bases.
Note that equation 2.6.18 is not a matrix multiplication; the summation is over the first index of the p, not the second (see Definition 1.2.4). It is equivalent to the "matrix multiplication" [Tv1 ... Tvn] = [w1 ... Wm][T]{v},{w}·
If V is a subset of Rn and W a subset of Rm, this is a true matrix multiplication. If V and W
are abstract vector spaces, then [Tv1 ... Tvn] and [w1 ... Wm] are not matrices, but the multiplication makes sense anyway; each Tvk is a linear combination of the vectors W1, ... , Wm.
Proposition and Definition 2.6.16 {Matrix with respect to bases) Let V and W be 11.nitedimensional vector spaces, with {v} =vi, ... , Vn a basis ofV and {w} = wi, ... , Wm a basis ofW. Let T: V+ W be a linear transformation. Then the matrix (T]{v},{w} of T with respect to the bases {v} and {w} is the m x n matrix with entries ti,j where m
2.6.18
Tvk = L:>e.kwe. £=1
Equivalently, if ~{v}:
Rn+ V
and
~{w}: Rm+
W
2.6.19
are the associated "concrete to abstract" linear transformations, then the matrix of T with respect to the bases {v} and {w} is (T]{v},{w} = ~; 1
o To ~v·
2.6.20
Note that the map ~; 1 o To ~v in equation 2.6.20 is a linear transformation Rn + Rm, and hence has a matrix. In particular, the matrix of a linear transformation Rn + JRm as defined in Theorem 1.3.4 is simply the matrix with respect to the standard basis {en} in the domain and the standard basis {em} in the codomain. In this case, ~{en} is then x n identity matrix, and ~{L} them x m identity matrix.
Proof. The kth column of ~; 1 o To ~vis (~; 1 o To ~v)ek, so compute Charles V (15001558) was reputed to have said: "I speak Spanish to God, Italian to women, French to men, and German to my horse." JeanPierre Kahane relates that in his youth he was taught that both French and German were suited to mathematics, but that English was not. Today, French mathematicians writing for publication probably write in English as often as they do in French.
m
~; 1 o To ~vek = ~; 1 o Tvk m
=
~;1
L te,kwe
= L:te,k~; 1 we = L:te,kee = l=l
2.6.21
l=l m
D
l=l
A basis gives a name to a vector living in an abstract vector space, but a vector has many embodiments, depending on the choice of basis for V, just as book, livre, Buch all mean the same thing. The change of basis matrix (Pv'w] is the dictionary that allows you to translate any vector written in the basis {v'} to the same vector written in the basis {v }. We will see how useful it can be to change bases when we discuss eigenvectors, eigenvalues, and diagonalization in Section 2. 7.
214
As in equation 2.6.18, equation 2.6.22 is not a matrix multiplication. But it can be written as [v~ ... v~] = [v1 ... Vn][Pv'v]i
the multiplication makes sense. We already saw the change of basis matrix in the proof of Proposition and Definition 2.4.21, when we showed that every basis of a subset E c Rn has the same number of elements; the matrices A and B there are change of basis matrices.
Chapter 2.
Solving equations
Proposition and Definition 2.6.17 {Change of basis matrix). Let V be an ndimensional vector space. Given two bases {v}, {v'} ofV, we can express each vector in {v'} in terms of the vectors in {v}: n
v'i = P1,iV1
+ P2,iV2 + · · · + Pn,iVn,
i.e.,
V1i = LPj,iVj.
2.6.22
j=l
The change of basis matrix fPv'+v] is then
P~,nl. : ,
P1,1 fPv 1 +v] ~ [
'.
Pn,1
2.6.23
Pn,n
the ith column of this matrix consists of the coefficients of the ith basis vector of {v'}, written in terms ofvi, ... , Vn· If n
[Pv'+v]a =
b,
then
n
Laiv~ = Lbivi. i=l
2.6.24
j=l
The change of basis matrix [Pv'+v] is the matrix of the linear transformation cJ.>{;} o cI>{v'}: 2.6.25
Thus to translate from basis {v'} to basis {v}, we write each basis vector of {v'} as a linear combination of the vectors of {v}. These coefficients are the entries of the change of basis matrix [Pv' +v J. When writing the matrix, make sure you write the coefficients for each basis vector as a column, not a row: the coefficients used to write v~ in terms of the vectors {v} form the ith column of [Pv'+v]·
Proof.
The following computation shows that [Pv'+v]a
'wi=l "°"'n aivi = 'wj=l "°"'n bjVj .·
= b
implies
1
taiv~ = tai tPj,iYJ = t
(tPJ,iai)
v3 = t([Pv'+v]a) 3v3
=
tb3v3.
To see that [Pv'+v] is the matrix of cJ.>{;} o cI>{v'}• just check what the image of ei is under cJ.>{;} ocI>{v'}· We have cI>{v'}(ei) = v'i, and cJ.>{;} then returns the coordinates of v'i with respect to the (old) basis vectors v 3. D
In diagram form this gives Rn {v'} '\,
lPv 1 vJ
+
v
Rn /q,1. {v}
2.6.26
Think of the change of basis matrix as a foreign language dictionary; the existence of such a dictionary implies the existence of a world of objects
2.6 Abstract vector spaces
215
and ideas that can be assigned different names in different languages:

Fr.Eng. Dictionary
Names in French "Name to ~ obj." function.
Note that unlike R3 , for which the "obvious" basis is the standard basis vectors, the subspace V C R3 in Example 2.6.18 does not come with a distinguished basis. We found these bases by randomly choosing two linearly independent vectors with three entries that sum to O; any two vectors that meet those requirements form a basis for V. The requirement "entries that sum to O" is why we can get away with only two basis vectors, although Vis a subset of R3 ; the 2dimensional structure of V is built into the choice of basis vectors.
2.6.27
/' "Obj. to name" function
Objects & Ideas
Example 2.6.18 {Change of basis matrix). Suppose Vis the plane of equation x + y + z = 0 in JR3 . A basis in V consists of two basis vectors (since V is a plane) each with three entries (since V is a subspace of JR3 ). Suppose our bases consist of 2.6.28 basis {v}
basis {v'}
What is the change of basis matrix [Pv'+v]? Since vi = v1 + v2, the coordinates of vi expressed in terms of v 1 and v 2 are p 1,1 = 1 and P2,1 = 1, giving the first column of the matrix. Similarly, v~ = v1 + 2v2, giving coordinates P1,2 = 1 and P2,2 = 2. So the change of basis matrix is
[Pv'+v] =
The tricky point when changing bases is not getting confused about what direction you are going. We adopted the notation [Pv'+v] in hopes that it will help you remember that this matrix translates a vector written in {v'} to the corresponding vector written in {v}.
Names in English
If, fo, example,
w=
[ _:
l
[Pl,l P2,1
P1,2J P2,2
= [1
1
1]2 .
2.6.29
mthe standa
~
n. Show
Mat (2, 2) be the linear trans
formation "multiplication on the left by A", i.e., LA : B t> AB. Similarly, define RA : Mat (2, 2) > Mat (2, 2} by RA : B t> BA. Identify Mat (2, 2) with IR4 , choosing as the basis {v} of Mat (2, 2)
a. Show that and
[LA]{v},{v} =
[ ~ ~ o~b o~bl .
2. 7 b. Compute
IRAI and ILAI
in terms of
Eigenvectors and eigenvalues
219
IAI.
2.6.6 a. As in Exercise 2.6.5, find the matrices for the linear transformations LA : Mat (3, 3) + Mat (3, 3) and RA : Mat (3, 3) + Mat (3, 3) when A is a 3 x 3 matrix. b. Repeat when A is an n x n matrix. c. In then x n case, compute
IRAI
and
ILAI
in terms of
IAI.
2.6. 7 Let V be the vector space of C 1 functions on (0, 1). Which of the following are subspaces of V? Exercise 2.6.8: By "identified to IR 3 via the coefficients" we mean that
p(x) =a+ bx+ cx 2 E P2
a. {!EV
I f(x)
c. { f EV
I f(x) =
= J'(x)
+ 1}
(f'(x)) 2
b. {!EV
2.6.8 Let P2 be the space of polynomials of degree at most two, identified to IR3 via the coefficients. Consider the mapping T : P2 + P2 given by
+ 2p(x). Verify that Tis linear, i.e., that T(ap1 + bp2) = aT(p1) + bT(p2 ). Choose the basis of P2 consisting of the polynomials p1 (x) = 1, P2 (x) = x, T(p)(x)
is identified to a.
= (x 2 + l)p"(x) 
b. p3(x) = x 2 • Denote by .ivi, then pl APei = pl Avi = pl AiVi = pl AiPei = Aipl Pei= Aiei,
!
2.7.16
A1 w we have
P' AP
~
[
2.7.17 0
224
Chapter 2.
Solving equations
3. Recall that vi =Pei. If p 1AP is a diagonal matrix with diagonal entries Ai, then the columns of P are eigenvectors of A and the Ai are the corresponding eigenvalues:
D 0
When do eigenbases exist? We say "essentially unique" because if you multiply each eigenvector of an eigenbasis by some number the result will still be an eigenbasis. The converse of Theorem 2.7.7 is not true; k eigenvectors can be linearly independent even if some of the eigenvectors share the same eigenvalue. For instance, every basis of en is an eigenbasis for the identity, but the identity has only one eigenvalue: the number 1.
Since the .A are all distinct, .A;  A; # 0, and the ai cannot all be zero because if they were, equation 2.7.18 would give Vj = 0, but vi cannot be zero, since it is an eigenvector.
When do eigenbases exist and how do we find them? Might a linear transformation have lots of eigenvectors and eigenvalues? Theorem 2. 7. 7 says that if an eigenbasis exists with distinct eigenvalues, it is essentially unique.
Theorem 2. 7. 7 (Eigenvectors with distinct eigenvalues are linearly independent). If A: V + V is a linear transformation, and vi, ... , Vk are eigenvectors of A with distinct eigenvalues Ai, ... , Ak, then v1, ... , vk are linearly independent.
In particular, if dim V = n, there are at most n eigenvalues. Proof. We will prove this by contradiction. If v 1, ... , vk are not linearly independent, then there is a first vector Vj that is a linear combination of the earlier ones. Thus we can write 2.7.18
where at least one coefficient is not zero, say ai. Apply Ajl  A to both sides to get 2.7.19
=al (Ajl  A)v1
+ · · · + aj1 (Ajl 
= al(,\i  Al)v1
+ · · · + aj1(Aj  Aj1)Vjl·
A)vj1
By hypothesis, the Ai are all different, so the coefficients ak(AjAk) can't all be zero. Suppose the ith term ai(Aj  Ai)vi is the last nonzero term; move it to the other side of the equality and divide out by the (nonzero) coefficient ai(Ai  Ai)· This expresses some vi with i < j as a linear combination of the vi, ... , Vjli which contradicts the assumption that vi is the first vector that is a linear combination of the earlier linearly independent ones. D
Procedure for finding eigenvectors Our procedure for finding eigenvectors will require knowing how to evaluate polynomials on matrices. Let p(t) = ao +alt+···+ am1tml + tm be a
2.7 Eigenvectors and eigenvalues
225
polynomial in one variable. Then we can write the matrix
p(A) = aol + alA + · · · + am1Aml +Am. Procedure for finding eigenvectors: The matrix A can have real entries, and wcan be real, but to use the fundamental theorem of algebra we must allow the root >. to be complex. We might worry that there might be lots of such polynomials for different choices of w, and that these polynomials might be unrelated. This is not the case. For instance, if for some w we find a polynomial p of degree n with distinct roots, then by Theorem 2.7.7 these roots are all the eigenvalues, so starting with any w1 will lead to a polynomial p1 whose roots are among those of p.
This is a new and powerful tool: it establishes a link between row reduction (linear dependence, kernels, etc.) and polynomials (factoring, roots, long division, the fundamental theorem of algebra). We now show how to find at least one eigenvector for any n x n complex matrix A. For any nonzero w E en, there is a smallest m such that Amw is a linear combination of w, Aw, ... , Am 1w: there exist numbers ao, ... , am1 such that
aow + alAw + · · · + am1Am 1w
+ Amw = 0.
q(A) = bol + biA + · · · + Ami
for appropriate bo, ... , b=1, and the coefficient of Aml is 1. Thus q(A)w =bow+ bi Aw +···+Am1w, is a linear combination of the m vectors w, Aw, ... , Am 1 w.
2.7.21
Then we can use the coefficients to define a polynomial:
p(t) ~r ao +alt+···+ am1tml
+ tm
2.7.22
satisfies p(A)w = 0, and it is the lowest degree nonzero polynomial with this property. By the fundamental theorem of algebra (Theorem 1.6.14), p has at least one root >., so we can write
p(t) = (t  >.)q(t) Equation 2.7.23: The polynomial q is obtained by dividing p by t  >., so deg q = m  1. Thus
2.7.20
2.7.23
for some polynomial q of degree m  1. Define v = q(A)w. Then (A>.J)v = (A>.I)(q(A)w) = ( (A>.J)(q(A)) )w = p(A)w =
0, 2.7.24
so Av= >.v. To see that vis an eigenvector with eigenvalue>., we still need to check that vi= 0. Since m > 0 is the smallest integer such that Amw is a linear combination of w, Aw, ... , Am 1w, the vectors w, ... , Amlw are linearly independent. Since the coefficient of Amlw is not 0 (see the margin note for equation 2.7.23), vi= 0. This procedure will yield one eigenvector for each distinct root of p. How can we find the polynomial p? That is exactly the kind of problem row reduction handles well. Consider the matrix
2.7.25 This matrix has n + 1 columns and n rows, so if you row reduce it to M, then M will certainly have a nonpivotal column. Let the (m+ l)st column be the first nonpivotal column, and consider the first m + 1 columns of M as an augmented matrix to solve the system of linear equations
2.7.26 The matrix row reduces to 1 0 0 1
0 0 0 0
0 0
bo bi
1 bm1 0 0
2.7.27
226 Example 2.7.8: Below we compute Ae1, A 2 e1, and A 3 e1:
[ ~]) H n [!] q1(t) =
t2 
= A 2 e1 
and by Theorem 2.2.1, the bi are the solution of our system of equations:
bow+ biAw + · · · + bm_ 1 Am 1w = Amw.
}A'e,
Example 2. 7 .8 {Finding an eigenbasis). Let A : ~3 transformation A
+ 3e1
~ [fl •[ii +, m .
=
+
1 1OJ , and for w use e
1 2 [ 1 0 1
1.
~3 be the linear
We want to find
1
the first vector in the list ei, Ae1 , A 2 ei, A3 e 1 that is a linear combination of the preceding vectors. Because of the zero entries in e1 and Ae1 , the first three vectors are clearly linearly independent, but the fourth must then be a linear combination of the first three. We row reduce
[100 10 321 945] 1
4t + 3,
4Ae1
2.7.28
Thus pis given by p(t) = bo  bit  · · ·  bmitml + tm.
we have qi(A)e1
Solving equations
e1
H :J Hl)Ae, [: :J Hl)A'•,
1 2 1 1 2 1 1 2 1 Equation 2. 7.30: Since
Chapter 2.
o o 3ol '
1 [0 1 0
to get
0 0 1
2.7.29
4
which tells us that Oe1  3Ae1 + 4A 2 e 1 = A3 e 1 . Thus pis the polynomial p(t) = t 3  4t 2 + 3t, with roots 0, 1, and 3. 1. (.X
= 0). Write p(t)
=
t(t 2

4t + 3) ~ tq1 (t) to find the eigenvector 2.7.30
In the eigenbasis
rn .[fl .[~i
2. (.X = 1). Write p(t) eigenvector
=
(t  l)(t 2
3t) ~ (t  l)q2 (t) to find the

the linear transformation
A~ [i ~: is written set P=
[ o~
n
o1 0
[~ ~ 1
p' AP
1
~ rn ~ ~] ·
2.7.31
3. (.X = 3). Write p(t) = (t  3)(t2 eigenvector

t) ~ (t  3)q3 (t) to find the
2.7.32 It follows from equation 2.7.24 that the first eigenvector has eigenvalue 0, the second has eigenvalue 1, and the third eigenvalue 3. (It is also easy to check this directly.) 6.
Remark. The procedure we describe for finding eigenvectors is not practical even for matrices as small as 5 x 5. For one thing, it requires finding roots of polynomials without saying how to do it; for another, row reduction is too unstable for the polynomial defined in equation 2.7.22 to be accurately known. In practice, the QR algorithm is used. It iterates directly in the space of matrices. Although not foolproof, it is the method of choice. 6.
2. 7
Eigenvectors and eigenvalues
227
Criterion for the existence of an eigenbasis When A is an n x n complex matrix and the polynomial p has degree n and the roots are simple, the above procedure (using a single standard basis vector) will find an eigenbasis. This is the usual case. But exceptionally, there can be complications. If degp < n, then we will have to repeat the procedure with a different vector. But what if the roots aren't simple? Theorem 2.7.9 says that in that case, there is no eigenbasis. Let Pi denote the polynomial constructed as in equations 2.7.21 and 2. 7.22, where w= ei, for i = 1, ... , n. Thus Pi is the lowest degree nonzero polynomial satisfying Pi(A)ei = 0. Note that it is possible for the Pi not to have simple roots. Ex
ercise 2.7.1 asks you to show that for the matrix [ ~
~] , the poly
nomial p1 is p1(t) = (t1) 2. We construct Pi by finding the smallest mi such that Am; ei is a linear combination of
so Span (Ei) contains m; linearly independent vectors. See equations 2. 7.21 and 2. 7.22.
Theorem 2.7.9. Let A be an n x n complex matrix. There exists an eigenbasis of en for A if and only if all the roots of all the Pi are simple. Proof. First assume that all the roots of all the Pi are simple, and denote by mi the degree of Pi· Let Ei be the span of ei, Aei, A 2 ei, .... It should be clear that this is a vector subspace of en of dimension mi (see the margin note). Since E 1 U · · · U En contains the standard basis vectors, we have Span (E1 U · · · U En)= en. The procedure described above yields for each root >..i,j of Pi an eigenvector vi,j, and since vi,1' ... , Vi,m; are mi linearly independent vectors in the space Ei of dimension mi, they span Ei. Thus the set n
mi
U LJ{vi,j}
2.7.33
i=lj=l
If A s a real matrix, you might want a real basis. If any eigenvalues are complex, there is no real eigenbasis, but often the real and imaginary parts of eigenvectors will serve the same purposes as eigenvectors. If Aui = µui withµ f/:. JR, then Aui = ji.fi.j, and the vectors Uj and fi.i are linearly independent in en, by Theorem 2.7.7. Moreover, the vectors Uj
Uj
i
are two linearly independent elements of !Rn in Span (Uj, Uj). See Exercise 2.14.
spans all of en, and we can select a basis of en from it. This proves the direction (all roots simple ==> there exists a basis of eigenvectors). For the converse, let >.. 1 , ... , >..k be the distinct eigenvalues of A. Set k
p(t)
=II (t  Aj),
2.7.34
j=l
so that p has simple roots. We will show that for all i, 2.7.35 This will imply that all Pi divide p, hence all Pi have simple roots. Set ij = ker (A  >..jl), so that the elements of ij are precisely the eigenvectors of A with eigenvalue .Ai. Since V1 u · · · U Vk contains all the eigenvectors, our hypothesis that there is a basis of eigenvectors says that Vi U · · · U Vk spans en. Thus each ei is a linear combination of elements in the ij: we can write 2.7.36
228
By Definition 2. 7.2, dim Vj is the multiplicity of Aj as an eigenvalue of A. If k < n, then at least one Vj will satisfy dim Vj > 1, and will contain dim Vj > 1 elements of the eigenbasis. The corresponding Wi,j will then be a linear combination of these eigenvectors.
all
Solving equations
Since by definition Vj = ker (A  Ajl), we have (A  >..1,I)wi 1, from 1 to k, so p(A)(ei) = 0: ,
e
p(A)(ei)
=
(rr
(A  >.if)'
J=l
)
(t
=
t
= Ofor
ai,twi)
l=l
)
p(A)
Equation 2.7.37: Note that when A is a square matrix, since A commutes with itself and all its powers. If this were not the case, then in equation 2.7.37 we would not be able to change the order of matrices in the product when we go from the first line to the second, putting the term (A  >..11) at the end.
Chapter 2.
2.7.37 p(A)w;,t, factored
ai,l
((II (A  Ajl)) (A  >..1,I)wi,l) = 0.
l=l
~
j=f.l
0
It follows that Pi divides p with remainder 0. Indeed, using long division we can write p = Piqi + ri, where degree Ti < degree Pi· By definition, Pi(A)ei = 0, so by equation 2.7.37,
0 = p(A)ei = Pi(A)qi(A)ei + Ti(A)ei = 0 + Ti(A)ei. 2.7.38 Thus Ti(A)ei = 0. Since degree Ti 0 if all entries ai,j satisfy ai,j > 0. Exercise 2.7.7 extends the PerronFrobenius theorem, weakening the hypothesis A > 0 to A ~ 0 and adding the hypothesis that there exists k such that Ak >0.
Theorem 2.7.10: The vector v (and any multiple of v, thus with the same eigenvalue) is called the leading eigenvector.
Lemma 2.7.11: We write
v S w with v ¥ w rather than v < w because some entries of v and w may coincide.
Theorem 2.7.10 (PerronFrobenius theorem). If A is a real n x n matrix such that A > 0, there exists a unique real eigenvector v > 0 with lvl = 1. This eigenvector has a simple real eigenvalue >.. > 0, called the leading or dominant eigenvalue. Any other eigenvalue µ E C of A satisfies lµI < >... Further, v can be found by iteration: for any w with w ;::: 0, we have
v=
limkoo Ak(w)/IAk(w)I.
Lemma 2.7.11 is key to the proof.
Lemma 2.7.11. If A> 0 and 0 Proof. In order to have v :f.
sv sw
with v :f. w, then Av< Aw.
w, at least one entry of w must be larger than the corresponding entry of v. That entry will be multiplied by some strictly positive entry of A > 0, leading to the strict inequality in the product. D
2. 7 Eigenvectors and eigenvalues
Formula 2.7.39: For instance, set A = [
~ ~] and w = [ ~ ] .
Then Aw' = [
~] , so
g(w) = inf{3, 7} = 3 and g : (w)w
= [ ~]
~ [ ~ J.
The "finitely many" in the second paragraph of the proof refers, in a neighborhood of any w E .6., to those Aw /wi for which Wi =I 0. The terms where Wi = 0 are all arbitrarily large in a sufficiently small neighborhood of w, so that they cannot contribute to the infimum. If A > 0, then AT > 0.
We have v > O since Av= µv and Av > O; we have vi > 0 since ATv1 = >.1v1 and ATv1 > 0.
229
Proof of Theorem 2.7.10. Let QC !Rn be the "quadrant" w 2: 0, and let a be the set of unit vectors in Q. If w E a, then w 2: 0 and w:/: 0, so (by Lemma 2.7.11) Aw> O. Consider the function g : a + JR given by . ....
. f{(Aw)i (Awh , , ... ,(Aw)n}.  ,
g.w~m
W1
W2
2.7.39
Wn
then g(w)w:::; Aw for all w E a, and g(w) is the largest number for which this is true. Since g is an infimum of finitely many continuous functions a + JR, the function g is continuous; since a is compact, g achieves its maximum at some v E a. Let us see that v is an eigenvector of A with eigenvalue >. ~ g(v). By contradiction, suppose that g(v)v:/: Av. By Lemma 2.7.11, g(v)v :5 Av and g(v)v:/: Av imply g(v)Av = Ag(v)v < AAv.
2.1.40
Since the inequality g(v)Av < AAv is strict, this contradicts the hypothesis that vis an element of a at which g achieves its maximum: g(Av) is the largest number such that g(Av)Av:::; AAv, so g(Av) > g(v). Thus Av = g(v)v: we have found an eigenvector v > 0, with real eigenvalue >. = g(v) > 0. Exactly the same argument shows that there exists a vector V1 E a such that AT V1 = A1V1, i.e., vi A = >.1vr Let v Ea be an eigenvector of A with eigenvalueµ. Then , +T+T +T2.7.41 A1V1 v=v+TA1 v=v1 µv=µv 1 v. Since (see the margin) vi > O and v > 0, we have viv:/: 0, soµ= .A1. Since vis any eigenvector of A in a, it could be v, soµ=>.. Further, vis the unique eigenvector of A in a: if v' :/: v were another such eigenvector, then the restriction of A to the subspace spanned by v and v' would be 0
>.id, contradicting A(a) c Q (which is true since A> 0). Now let w E a be any vector in a satisfying w :/: v. Then we have g(w)w:::; Aw and g(w)w:/: Aw (since vis the unique eigenvector of A in a), so (again using Lemma 2.7.11), g(w)Aw = A(g(w))w < A(Aw), so g(Aw) > g(w). Earlier we showed that v exists, using a nonconstructive proof. Here we show how to construct it.
2.7.42
Thus for any w E a with w :f. v, the sequence k ~ g(Akw) is strictly increasing, and so (since a is compact) the sequence k ~ Akw/IAkwl converges to v. We will now show that any other eigenvalue 'f/ E e of A satisfies 1111 < .A. Let ii1, ... , Um E en be the eigenvectors of A linearly independent from v with eigenvalues 'f/i satisfying l"lil 2: .A. Then Span (v, iii, ... , ilm) c en intersects !Rn in a real subspace of dimension m + 1, since the nonreal eigenvectors come in conjugate pairs. If m > 0, there are elements w :/: v in a with w E Span (v, ii1, ... , iim): they can be written m
which gives
Akw = a.Akv +
L ai'f/fili, i=l
2.7.43
230
The patent for the PageRank algorithm was awarded to Stanford University, which granted Google a longterm license for 1.8 million shares of Google stock. Stanford sold the stock in 2005 for $336 million.
Chapter 2.
Solving equations
and our assumption l1Jil ~ .X means that the sequence k tt Akw/IAkwl cannot converge to v. Since the sequence does converge to v, this shows that all the other eigenvalues TJ "#A of A satisfy ITJI < .X. Exercise 2.7.8 asks you to show that A is a simple eigenvalue; this completes the proof. D A striking application of the PerronFrobenius theorem is the PageRank algorithm Google uses to rank the importance of web pages, in order to return relevant search results. Example 2.7.12 (Google's PageRank). Google's PageRank algorithm uses the PerronFrobenius theorem to find a vector whose entries are the "importance" of each web page: Xi gives the importance of the ith page. Importance is defined as having lots of links from other pages ( "backlinks"), with a backlink from an important page weighted more heavily than one from a less important page. Each page is given one "vote", split evenly between all the pages to which it links: if the jth page links to eight pages, the contribution of a link to a recipient page is xi /8; if it links to only one page, the contribution to the recipient page is xi. Suppose there are n webpages (some huge number) and construct the "weighted adjacency matrix" A of the corresponding graph: ai,j = 0 if web page j does not link to web page i, and ai,i = I/ki if j does link to web page i, and there are in all ki links from web page j; see Figure 2.7.2, where n = 5. Putting 1/kj rather than 1 reflects "one page, one vote". Then the multiplication Ax = says that Xi (the importance of web page i) is the sum of the weighted links from all other web pages. In order to guarantee that such a vector exists with eigenvalue 1, we need to modify the matrix A. At least in the original version of PageRank, this was done roughly as follows. Those columns that contain any nonzero entries now sum to 1, but there may be columns of O's, corresponding to web pages that do not link out; we replace all entries of these columns by 1/n; now all columns sum to 1. Next, A will still have entries that are 0, but the PerronFrobenius theorem requires A> 0. 14 To remedy this, we define G = (1  m)A + mB, where Bis the matrix all of whose entries are 1/n, and mis some number satisfying 0 < m < 1; apparently the original value of m was .15. An entry of A that was 0 becomes an entry m/n of G, and an entry of A that was I/kj becomes the entry (1  m)/ki + m/n of G. Now we have G > 0, and the entries of each column of G sum to 1. It follows that the leading eigenvalue of GT is 1, since the vector Rew that is n high with all l's satisfies w > 0 and is an eigenvector of GT, since GT w = w. Our proof of PerronFrobenius shows that the leading eigenvalue of G is also 1. Now the PerronFrobenius theorem tells us that G has an eigenvector x (unique up to multiples) with eigenvalue 1: Gx = x. Moreover, x can be found by iteration. !:::.
x
FIGURE 2.7.2. A directed graph representing five web pages. The corresponding matrix A is
[~ ~ ~
1 0 0 0 0 0 The first column A=
~j~ ~i
0 0 0 0 0 1 0 1/3 0 of matrix G is
x
14
Alternatively, we could use A 2:: 0 as long as there exists k with Ak > 0.
2. 7
Eigenvectors and eigenvalues
231
EXERCISES FOR SECTION 2.7 2. 7.1
Show that for A= [ ~
~], the polynomial p1 is p1(t) =
(t  1)2, so A
admits no eigenbasis. Let C00 (R) denote the vector space of infinitely differentiable functions on R Consider the linear transformation D: C00 (R) + C"°(R) given by D(f) = f'. 2. 7 .2
Part a of Exercise 2.7.2 shows that eigenvalues are not so special in infinitedimensional vector spaces: it is possible to find linear transformations such that every number is an eigenvalue. In part b, bo + bie"' + · · · + bkekx
=0
means that bo+b1e"'+· · ·+bkek:i: is the zero function: the zero element of the vector space C00 (R).
a. Show that ea"' is an eigenvector of D, with eigenvalue a. b. Show that for any integer k
> 0,
bo + bie"' + · · · + bkekx = 0 ~ bo =bi=···= bk= 0. 2.7.3 a. Let P3 be the space of polynomials of degree 3. Find the matrix of the linear transformation T(p) = p + xp" with respect to the basis P1(x)=l, p2(x)=l+x, p3(x)=l+x+x2, p 4 (x)=l+x+x 2 +x3 • b. Let Pk be the space of polynomials of degree k. Find the matrix of the linear transformation T(p) = p + xp' with respect to the basis p1(x) = 1, P2(x) = 1 + x, ... , Pi(x) = 1 + x + · · · + xil.
~~:·~:.:~:~:~::·[~~.~ an n x; dli:::~ matrix with Ai, ... , An on
x~ [i ~~
:]
Matrix for Exercise 2.7.4, part b.
0
et>'n
b. For an n x n matrix A, let eA be defined as in Exercise 1.5.10. Compute ex, where X is the matrix in the margin. c. Let x be a vector in R 3 and consider the function f : R + R defined by f(t) = ietxxl. For what values of x does f(t) tend to oo as t+ oo? Remain bounded as t+ oo? Tend to 0 as t+ oo?
2. 7.5 Let n ~ bn be the "Fibonaccilike" sequence defined by bo = bi = 1 and bn+l = 2bn + bn1 for n 2 2.
[ o0
a. Find a formula for bn analogous to equation 2.7.l. b. How many digits does b1000 have? What are the leading digits?
3 1]
2
2 3
c. Do the same for the sequence n Cn+l = Cn + Cn1 + Cn2, for n 2 3.
0 3
'v' A
[
22 12 15 6
23 18 19 7
10 16 13 4
98] 38 58 25
B
Matrices for Exercise 2.7.6
~ Cn
defined by
Co
=
c1 = c2
=
1 and
2. 7.6 For each matrix A and Bin the margin, tell whether it admits an eigenbasis and if so, find it. 2. 7. 7 a. Let A be a square matrix. Show that if A 2 0 (if every entry of A is 2 0) , then A has an eigenvector v satisfying v 2 0 ( every entry of vis 2 0) .
b. Prove the "improved" PerronFrobenius theorem: that if A 2 0 and An > 0 for some n 2 1, there exists an eigenvector v > 0 with simple eigenvalue A > 0, and any other eigenvalue µ E IC of A satisfies JµJ < A.
2. 7.8 Complete the proof of Theorem 2. 7.10 by showing that the eigenvalue A is simple.
232
2.8
Chapter 2.
Solving equations
NEWTON'S METHOD When John Hubbard was teaching first year calculus in France in 1976, he wanted to include some numerical content. Computers for undergraduates did not exist, but Newton's method to solve cubic polynomials just about fit into the 50 steps of program and eight memory registers available with programmable calculators, so he used that as his main example. But what should the initial guess be'? He assumed that although he didn't know where to start, the experts surely did. It took some time to discover that no one knew anything about the global behavior of Newton's method. A natural thing to do was to color each point of the complex plane according to what root (if any) starting at that point led to. (This was before color screens and printers, so he printed some character at every point of some grid: x and 0, for example.} The resulting printouts were the first pictures of fractals arising from complex dynamical systems, with its archetype the Mandelbrot set.
FIGURE
2.8.l.
Isaac Newton ( 16431 727)
Newton's method requires an initial guess ao. How do you choose it? You might have a good reason to think that nearby there is a solution, for instance, because lf(ao)I is small. In good cases you can then prove that the scheme works. Or it might be wishful thinking: you know roughly what solution you want. Or you might pull your guess out of thin air and start with a collection of initial guesses a 0 , hoping that at least one will converge. In some cases, this is just a hope. Newton's method:
a1
= ao  [Df(ao)] 1f(ao),
= a1 a3 = a2 a2
Theorem 2.2.l gives a quite complete understanding of linear equations. In practice, one often wants to solve nonlinear equations. This is a genuinely hard problem; the usual response is to apply Newton's method and hope for the best. This hope is sometimes justified on theoretical grounds and actually works much more often than any theory explains. Definition 2.8.1 (Newton's method). Let f be a differentiable map from U to !Rn, where U is an open subset of!R.n. Newton's method consists of starting with some guess ao for a solution of f(x) = 0. Then linearize the equation at ao: replace the increment to the function, f(x)  f(ao), by a linear function of the increment, [Df(ao)](x  ao). Now solve the corresponding linear equation: f(ao)
+ [Df(ao)](x 
ao) =
0.
2.8.1
This is a system of n linear equations in n unknowns. We can write it as [Df(ao)](x  ao)
'."' 'v'
= f(ao). ...__,_....
2.8.2
A
If [Df(ao)] is invertible, which will usually be the case, then
x
= ao  [Df(ao)]lf(ao) .
2.8.3
Call this solution a 1 , use it as your new "guess", and solve
[Df(ai)] 1f(ai)
2.8.4
[Df(a2)r 1f(a2)
calling the solution a2, and so on. The hope is that a1 is a better approximation to a root than a 0 , and that the sequence i 1+ ai converges to a root of the equation.
2.8
Newton's method
233
Newton's method is illustrated by Figure 2.8.2. Definition 2.8.1: The arrow over f indicates that elements of the codomain are vectors; a.o is a point and f(a.o) is a vector. In equation 2.8.1, the derivative [Df(a.o)] is a matrix, and the increment to the variable, x  ao, is a vector. Thus on the left side we have the addition of two vectors (elements of the codomain). In a1
equation
x
(setting
= x), point in domain
a1
2.8.3
y
increment in domain
= ~ [Df(a.o)t 1 f(a.o).
~
vector
in codomain point minus vector equals point
Equation 2.8.3 explains the theory behind Newton's method but it is not the way Newton's method is used to solve equations, since finding [Df(a.o)] 1 is computationally expensive. Instead row reduction is used to solve equation 2.8.2 for x  ao; adding a.o to the solution gives the new a1. (Partial row reduction and back substitution, discussed in Exercise 2.2.11, are more effective yet.) When Newton's method is used, most of the computational time is spent doing row operations. Example 2.8.2: Exercise 2.8.4 asks you to find the corresponding formula for nth roots.
FIGURE 2.8.2. Newton's method: We start with a 0 , and draw the tangent to the curve at the point with xcoordinate a 0 • The point where that tangent to intersects the xaxis is ai. Now we draw the tangent to the curve at the point with xcoordinate ai. That tangent ti intersects the xaxis at a2 .... Each time we calculate an+l from an we are calculating the intersection with the xaxis of the line tangent to the curve at the point with xcoordinate an.
Remark. The domain and codomain of i are usually different spaces, with different units. For example, the units of U might be temperature and the units of the codomain might be volume, with i measuring volume as a function of temperature. In a processing plant i might take n inputs (wheat, oats, fruit, sugar, hours of labor, kilowatt hours of electricity) and produce n different kinds of cereal. The only requirement is that there must be as many equations as unknowns: the dimensions of the two spaces must be equal. !:::.. Example 2.8.2 (Finding a square root). How do calculators compute the square root of a positive number b? They apply Newton's method to the equation f(x) = x 2  b = 0. In this case, this means the following: choose ao and plug it into equation 2.8.3. Our equation is in one variable, so we can replace [Df(ao)] by f'(a 0 ) = 2a0 , as shown in equation 2.8.5: ai
= ao 
1
 ( a02 2ao

b)
Newton's method
1(
= 
2
ao
b)
+ ao
.
2.8.5
~
divide and average
This method is sometimes taught in middle school under the name divide and average. The motivation for divide and average is the following: let a be a first guess at ../b. If your guess is too big (i.e., if a > ../b), then b/a will be too small, and the average of the two will be better than the original guess. This seemingly naive explanation is quite solid and can easily be turned into a proof that Newton's method works in this case.
234
Chapter 2.
Solving equations
Suppose first that ao > Vb; then we want to show that Since ai = !(ao + b/ao), this comes down to showing
b
2.8.7
0.
The right inequality follows immediately from the inequality b < a5, hence b2 /a5
< a5:
2.8.8 FIGURE
2.8.3.
Augustin Louis Cauchy (17891857) proved the convergence of Newton's method in one dimension. Cauchy is known both for his great accomplishments  he wrote 789 papers on a wide range of subjects  and for his difficult relations with his colleagues, in particular his mistreatment of the young and impoverished Abel. "Cauchy is mad and there is nothing that can be done about him, although, right now, he is the only one who knows how mathematics should be done," Abel wrote in 1826.
Recall from first year calculus (or from Theorem 0.5. 7) that if a decreasing sequence is bounded below, it converges. Hence the ai converge. The limit a must satisfy
a= Hm ai+ 1 = Hm i>oo
i>oo
.!2 (ai + .!!..) ai
What if you choose 0 < ao
2
2
2.8.9
Vb:
b2
4a0 < 4b < a0 + 2b + 2 . ao
2.8.10
'....."'
4a?
We get the right inequality using the same argument used in equation 2.8.7:
~, a
(ao 
2 . since subtracting 2b from both sides gives 0 < ..!2...) ao Then the same argument as before shows that a 2 < a 1 . This "divide and average" method can be interpreted geometrically in terms of Newton's method, as was shown in Figure 2.8.2. Each time we calculate an+l from an we are calculating the intersection with the xaxis
2b
< a5 +
0
of the line tangent to the parabola y = x 2

bat ( a't":._ b) .
6.
There aren't many cases where Newton's method is really well understood far away from the roots; Example 2.8.3 shows one of the problems that can arise. (There are many others.) Example 2.8.3 (A case where Newton's method doesn't work). Let's apply Newton's method to the equation
x 3 x+
J2 =0,
2
2.8.11
2.8
Newton's method
starting at x = 0 (i.e., our "guess" ao is 0). The derivative is J'(x) so J'(O) = 1 and /(0) = ./2/2, giving
ai
= ao 
Since ai Don't be too discouraged by this example. Most of the time Newton's method does work. It is the best method available for solving nonlinear equations.
1
f'(ao/(ao)
= ao 
a3a0 +fl 3a~ _ 1 2
fl
= 3x2 1,
.j2
= 0 +} = 2·
2.8.12
= ./2/2, we have f'(a 1 ) = 1/2, and 2.8.13
We're back to where we started, at a0 = 0. If we continue, we'll bounce back and forth between Vf and 0, never converging to any root:
This uses equation 0.5.4 concerning the sum of a geometric series: If lrl < 1, then 00
""""'
L...J arn
n=O
= 1a r·
We can substitute 3€2 for r in that equation because € 2 is small.
235
2.8.14 Now let's try starting at some small E > 0. We have f'(i:) = 3i:2 1, and f(i:) = i:3  E + ./2/2, giving al= E3€21_1(€3E+
~)
= i:+ 113€2(€3E+
~)·
2.8.15
1
Now we can treat the factor as the sum of the geometric series 1 3i:2 2 4 (1+3i: + 9i: + · · · ). This gives ./2) (1 + 3€2 + 9€4 + · · · ) . al = E + ( E3  E + 2 Does it seem like cheating to ignore terms that are smaller than € 2 , ignore the remainder, or throw out all the terms with € 2 ? Remember (introduction to Section 1.4) that calculus is about "some terms being dominant or negligible compared to other terms." Ignoring these negligible terms will be justified when we get to Section 3.4 on Taylor rules.
2.8.16
Now we ignore terms that are smaller than i: 2 , getting
ai
=
i:+ (V:i:)(1+3i:2)+ remainder 2.8.17
. TV2 + 3./2€2 2 + remamder. Ignoring the remainder, and repeating the process, we get
2.8.18 This looks unpleasant; let's throw out all the terms with i: 2 . We get
If we continue, we'll bounce between a region around ~ and a region around 0, getting closer and closer to these points each time.
2V2  2V2 = so that a2 = 0 + ci: 2, where c is a constant. We started at 0 + E and we've been sent back to 0 + ci:2!
0, 2.8.19
236 History of the Kantorovich theorem: Cauchy proved the special case of dimension 1. As far as we know, no further work on the subject was done until 1932, when Alexander Ostrowski published a proof in the 2dimensional case. Ostrowski claimed a student proved the general case in his thesis in 1939, but that student was killed in World War II, and we know of no one who has seen this thesis. Kantorovich approached the problem from a different point of view, that of nonlinear problems in Banach spaces, important in economics (and in many other fields). He proved the general case in a paper in the 1940s; it was included in a book on functional analysis published in 1959. The proof we give would work in the infinitedimensional setting of Banach spaces.
Chapter 2.
Solving equations
We're not getting anywhere; does that mean there are no roots? Not at all. 1s Let's try once more, with a0 = 1. We have ai
= ao 
a6  ao + .il 3a 2 0

1
2

2a6  .il 3a512 ·
2.8.20
A computer or programmable calculator can be programmed to keep iterating this formula. It's slightly more tedious with a simple scientific calculator; with the one the authors have at hand, we enter "l +/ Min" to put 1 in the memory ("MR") and then (2 x MR x MR x MR2Jdiv2)div(3 x MR x MR1). We get ai = 1.35355 ... ; entering that in memory by pushing on the "Min" (or "memory in") key, we repeat the process to get a2
= 1.26032 .. .
a4
1.25116 .. .
as
a3 =
= 1.25107 . . . = 1.25107 ... .
2.8.21
It's then simple to confirm that as is indeed a root, to the limits of precision of the calculator or computer. /:::; Does Newton's method depend on starting with a lucky guess? Luck sometimes enters into it; with a fast computer one can afford to try out several guesses and see if one converges. But how do we really know that solutions are converging? Checking by plugging a root into the equation isn't entirely convincing, because of roundoff errors. Any statement that guarantees that you can find solutions to nonlinear equations in any generality at all is bound to be tremendously important. Kantorovich's theorem (Theorem 2.8.13) guarantees that under appropriate circumstances Newton's method converges. Even stating the theorem is difficult, but the effort will pay off. In addition, Newton's method gives a practical algorithm for finding implicit and inverse functions. Kantorovich's theorem proves that these algorithms work. We now lay the groundwork for stating the theorem.
Lipschitz conditions FIGURE
2.8.4.
Rudolf Lipschitz (18321903) Lipschitz's interpretation of Riemann's differential geometry in terms of mechanical laws has been credited with contributing to the development of Einstein's special theory of relativity.
Imagine an airplane beginning its approach to its destination, its altitude represented by f. If it loses altitude gradually, the derivative f' allows one to approximate the function very well; if you know how high the airplane is at the moment t and what its derivative is at t, you can get a good idea of how high the airplane will be at the moment t + h :
f(t
+ h) ~
f(t)
+ f'(t)h.
2.8.22
But if the airplane suddenly loses power and starts plummeting to earth, the derivative changes abruptly: the derivative of f at t will no longer be a reliable gauge of the airplane's altitude a few seconds later. 15 0f course not. All odddegree polynomials have real roots by the intermediate value theorem, Theorem 0.5.9.
2.8
Newton's method
237
The natural way to limit how fast the derivative can change is to bound the second derivative; you probably ran into this when studying Taylor's theorem with remainder. In one variable this is a good idea. If you put an appropriate limit to !" at t, then the airplane will not suddenly change altitude. Bounding the second derivative of an airplane's altitude function is indeed a pilot's primary goal, except in rare emergencies. To guarantee that Newton's method starting at a certain point will converge to a root, we need an explicit bound on how good an approximation is to
[Df(xo))h
In Definition 2.8.4, note that the domain and the codomain of f need not have the same dimension. But when we use this definition in the Kantorovich theorem, the dimensions will have to be the same.
A Lipschitz ratio M is often called a Lipschitz constant. But M is not a true constant; it depends on the problem at hand. In addition, a mapping will almost always have different M at different points or on different regions. When there is a single Lipschitz ratio that works on all of Rn, we will call it a global Lipschitz ratio.
f(xo
+ h) 
2.8.23
f(xo).
As in the case of the airplane, we will need some assumption on how fast the derivative of i changes. In several variables there are lots of second derivatives, so bounding the second derivatives is complicated; see Proposition 2.8.9. Here we adopt a different approach: demanding that the derivative of f satisfy a Lipschitz condition. 16 Definition 2.8.4 (Lipschitz condition for a derivative). Let U c ]Rn be open and let f : U t ]Rm be a differentiable mapping. The derivative [Df(x)) satisfies a Lipschitz condition on a subset V CU with Lipschitz ratio M if for all x, y E V
I[Df(x)) 
[Df(y)J
distance between derivatives
lxyl. I :5 M ...__.,
2.8.24
distance between points
Note that a function whose derivative satisfies a Lipschitz condition is certainly continuously differentiable. Indeed, requiring that the derivative of a function be Lipschitz is close to demanding that the function be twice continuously differentiable; see Proposition 2.8.9. Example 2.8.5 (Lipschitz ratio: a simple case). Consider the mapping f : JR2 t JR 2 f (
~~) = (:i ~ =~)
with derivative
[nf(~~)] = [ 2 ~ 1 ~x 2 ]
.
2.8.25
Given two points x and y,
2(x2 y2)] 0 .
2.8.26
The length of this matrix is
2J(x1  Y1) 2 + (x2 
Y2) 2 =
21 [xi  Yi] I = 2lx  yJ, X2 y2
2.8.27
16 More generally, a function f is Lipschitz if lf(x)  f(y)I :5 Mix  YI· For a function of one variable, being Lipschitz means that the chords joining pairs of points have bounded slope. The function lxl is Lipschitz with Lipschitz ratio 1, but it is not differentiable.
238
Chapter 2.
Solving equations
so M = 2 is a Lipschitz ratio for [Df]. But this example is misleading: there is usually no Lipschitz ratio valid on the entire space. !::,.
y
Example 2.8.6 (Lipschitz ratio: a more complicated case). Consider the mapping f : JR2 + JR 2 given by x
f (
;~ ) = ( =~ ~
:: ),
with derivative [Df (
;~ ) J = [ 3!~
3x~] 1 .
Given two points x and y, we have
FIGURE
[nr(;~) J  [nr(~~) J = [ 3 (x~ ~ Yn
2.8.5.
Inequalities 2.8.30 and 2.8.31 say that when (
~~ )
and (
~~ )
are in the shaded region (which extends forever on the sides where the boundary is not marked), then 3A is a Lipschitz ratio for [Df(x)]. In formula 2.8.34 we translate this statement into a condition on the points x
= ( ~; )
and y
= ( ~; ) .
3(x~ y~)]
0
,
2.8.28
and taking the length gives
I[Df ( ;~ ) ]
 [Df ( ~~ ) ] I
2.8.29
Clearly, for this quantity to be bounded we have to put some restrictions on our variables. If we set 2.8.30 as shown in Figure 2.8.5, we have 2.8.31 that is, 3A is a Lipschitz ratio for [Df(x)]. When are the inequalities 2.8.30 satisfied? We could just say that it is satisfied when it is satisfied; in what sense can we be more explicit? But the requirement in equation 2.8.30 describes some more or less unimaginable region in JR4 . (Keep in mind that inequality 2.8.31 concerns points x with coordinates x 1 , x 2 and y with coordinates y 1 , y2, not the points of Figure 2.8.5, which have coordinates xi,y1 and x2,Y2 respectively.) Moreover, in many settings, what we really want is a ball of radius R such that when two points are in the ball, the Lipschitz condition is satisfied:
Inequality 2.8.33: We are using the fact that for any two numbers a and b, we always have
(a+ b) 2
:::;
2(a2
+ b2 ),
+ x~
If we require that lxl 2 = x~
sup{ (x1 + Y1) 2, (x2
since
(a+b) 2
l[Df(x)]  [Df(y)JI ~ 3Alx yl when lxl ~Rand IYI ~ R.
:::;
(a+b) 2 +(ab) 2
= 2(a2 + b2 ).
2.8.32
+ y~
~ A 2 /4, then
+ Y2) 2} ~ 2(x~ + y~ + x~ + y~) = 2(lxl 2 + IYl 2 ) ~ A 2 ·
2.8.33
~ A 2 /4 and IYl 2 = y~
Thus we can assert that if
lxl,IYI~~.
then
l[Df(x))[Df(y)Jl~3Alxyl.
6
2.8.34
2.8 Newton's method
239
Computing Lipschitz ratios using second partial derivatives
Higher partial derivatives are so important in scientific applications of mathematics that it seems mildly scandalous to slip them in here, just to solve a computational problem. But in our experience students have such trouble computing Lipschitz ratios  each problem seeming to demand a new trick  that we feel it worthwhile to give a "recipe" for an easier approach. We discuss higher partial derivatives in Sections 3.3 and 3.4. Different notations for partial derivatives exist: Di(Dif)(a)
=
a2 J
8xj8Xi (a)
= fx;x; (a). As usual, we specify the point a at which the derivative is evaluated.
Most students can probably follow the computation in Example 2.8.6 line by line, but even well above average students will probably feel that the tricks used are beyond anything they can come up with on their own. The manipulation of inequalities is a hard skill to acquire, and a hard skill to teach. Here we show that you can compute Lipschitz ratios using second partial derivatives, but it still involves evaluating suprema, and there is no systematic way to do so. (In the case of polynomial functions, there is a trick that requires no evaluation of sups; see Example 2.8.12.)
Definition 2.8.7 (Second partial derivative). Let UC Rn be open, and let f : U + R be a differentiable function. If the function Dif is itself differentiable, then its partial derivative with respect to the jth variable,
D;(Dif), is called a second partial derivative of f.
Example 2.8.8 (Second partial derivative). Let f be the function
Im ~
2x + xy3 +2yz2 • Then D,(D1/)
Similarly, D3(D2f) (:)
=
m
~ D2~ ~ 3y2
D3~ = 4z.
!::::,.
Dd
Z
DU, we can denote D2(D2/) by DU, and For the function f ( ~) = xy 2 +sin x, what are DU, DU, Dl (D2f),
We can denote D 1 (Dif) by so on.
and D2(Dif) ? 17 Proposition 2.8.9 says that the derivative of f is Lipschitz if f is of class C 2 • It shows how to construct a Lipschitz ratio from the supremum of each second partial derivative. We give the proof after a couple of examples. The proposition would be true for a function going from an open subset of Rn to Rm., but it would be slightly more complicated to state; the index i would have a different range than j and k.
Proposition 2.8.9 (Derivative of a C 2 mapping is Lipschitz). Let U c Rn be an open ball, and f : U + Rn a C 2 mapping. If 2.8.35
for any x E U and for all triples of indices 1 :::;; i, j, k < n, then for u,vEU, 1/2
l[Df(u))  [Df(v)JI:::;;
(
)
1 ~j;~n (Ci,;,k) 2
ju  vj.
17 DU = D1(y 2 +cosx) = sinx, D~f = D2(2xy) = 2x, D1(D2f) = D1(2xy) = 2y, and D2(D1f) = D2(y 2 + cosx)
= 2y.
2.8.36
240
Chapter 2.
Solving equations
Example 2.8.10 (Redoing Example 2.8.6). Let's see how much easier it is to find a Lipschitz ratio in Example 2.8.6 using second partial derivatives. First we compute the first and second derivatives, for Ji = x 1  x~ and h = x~ + x2: Di/1
= 1;
D2f1
= 3x~;
D1h
= 3x~;
D2f2
= l.
2.8.37
This gives
= 0; D1D2f1 = D2Dif1 = O; D2D2/1 = 6x2 D1D1h = 6x1; D1D2f2 = D2D1h = O; D2D2f2 = 0. D1Dif1
If lxl ::; A/2, then lx1 I ::; A/2,
since lx1I::;
Jx~ +x~.
The same is true of x2 •
2.8.38
If lxl, IYI ::; ~, we have lx1 I ::; A/2 and lx2 I ::; A/2, so
ID2D2/1I::; 3A
= _.,...., c1,2,2
and
ID1D1hl::; 3A
= _.,...., c2,1,1 bound for
ID1D1hl
with all others 0, so Inequality 2.8.40: Earlier we got 3A, a better result. A blunderbuss method guaranteed to work in all cases is unlikely to give as good results as techniques adapted to the problem at hand. But the second partial derivative method gives results that are often good enough.
c~.2.2 + c~.1.1 = 3Av'2.
2.8.39
Thus we have 2.8.40
l[Df(x)]  [Df(y)JI::; 3v'2Alx yl.
Using second partial derivatives, recompute the Lipschitz ratio of Example 2.8.5. Do you get the same answer? 18 Example 2.8.11 (Finding a Lipschitz ratio using second derivatives: a second example). Let us find a Lipschitz ratio for the derivative of the function F (
~) = ( si:o~~r) ,
for !xi
< 2, IYI < 2.
2.8.41
We compute
= D2D2F1 = D2D1F1 = D1D2F1 =  sin(x + y), 2.8.42 D1D1F2 = y2 cos(xy), D2D1F2 = D1D2F2 = (sin(xy) +yxcos(xy)), D1D1F1
D2D2F2 = x2 cosxy. Inequality 2.8.44: By fiddling with the trigonometry, one can get the v'86 down to ../78 ~ 8.8, but the advantage of Proposition 2.8.9 is that it gives a more or less systematic way to compute Lipschitz ratios.
Since Isin I and Icos I are bounded by 1, if we set lxl ID1D1F1I
< 2, IYI < 2, this gives
= ID1D2F1I = ID2D1F1I = ID2D2F1I::;
ID1D1F2I, ID2D2F2I::; 4,
1
2.8.43
ID2D1F2I = ID1D2F2I::; 5.
So for lxl < 2, IYI < 2, we have a Lipschitz ratio
M ::;
J 4 + 16 + 16 + 25 + 25 = J86 < 9.3;
that is, l[DF(u)] [DF(v)JI::; 9.3luvl.
2.8.44
D
18 The only nonzero second partials are D1D1/2
= 2 and D2D2fi = 2, bounded by 2 and 2, so the method using second partial derivatives gives J2 2 + (2) 2 = 2v'2.
2.8
Inequality 2.8.45 uses the fact that for any function g (in our case, Difi), lg(a + h)
 g(a)I
I
:::; ( sup [Dg(a +th)] tE[0,1]
I) Ihi;
remember that
n
s
ID;/;(a +Ii)  D;/;(a)I
(t, 0, starting with ao < O? Hint: Make a careful drawing, focusing on what happens if ao = 0 or if ai = 0, etc.
lf(x)  f(y)I :5 Clx yl.
2.8.4 a. Find the formula an+l = g(an) to compute the kth root of a number by Newton's method.
(Of course a Lipschitz mapping is continuous; it is better than continuous.)
2.8.3
a. Show that the function !xi is Lipschitz with Lipschitz ratio 1.
b. Show that the function
y'jXf is not Lipschitz.
b. Interpret this formula as a weighted average. 2.8.5 a. Compute by hand the number 9 1 / 3 to six decimals, using Newton's method, starting at ao = 2.
b. Find the relevant quantities ho, ai, M of Kantorovich's theorem in this case. c. Prove that Newton's method does converge. (You are allowed to use Kantorovich's theorem, of course.)
f (
~) = ( :~ =~ =~i)
Map for Exercise 2.8.6
2.8.6 a. Find a global Lipschitz ratio for the derivative of the map f: IR.2 in the margin.
b. Do one step of Newton's method to solve f ( point (
~)
= (
8),
>
IR.2
starting at the
!) .
c. Find a disc that you are sure contains a root. 2.8. 7
In Exercise 2.8.7 we advocate using a program like MATLAB (NEWTON.M), but it is not too cumbersome for a calculator.
a.
Consider the system of equations {
cosx+y=l.1
x + cos(x + y)
= 0.9
Carry out four steps of Newton's method, starting at (
.
8).
How many
decimals change between the third and the fourth step? b. Are the conditions of Kantorovich's theorem satisfied at the first step? At the second step?
2.8
Newton's method
251
2.8.8 Use the MATLAB program NEWTON.M 25 (or the equivalent) to solve the following systems of equations.
Exercise 2.8.8: "Superconvergence" is defined with precision in Section 2.9. In Example 2.8.17, we defined it informally. "Does Newton's method appear to superconverge?" means "does the number of correct decimals appear to double at every step?".
a.
b.
x2

y + sin(x  y) = 2 y 2 x=3
x3

starting at ( 22)' (22)
y + sin(x  y) = 5 y 2 x
=3
starting at (
~)'
(n
Does Newton's method appear to superconverge? In all cases, determine the numbers that appear in Kantorovich's theorem, and check whether the theorem guarantees convergence.
x+y 2 =a { y+z 2 =b
2.8.9 Find a number f > 0 such that the set of equations in the margin has a unique solution near 0 when lal, lbl, lei < f. 2.8.10
Do one step of Newton's method to solve the system of equations x+cosy1.1=0
z +x 2 = c
x2
Equations for Exercise 2.8.9

sin y + 0.1 = 0
starting at ao = (
8).
2.8.11 a. Write one step of Newton's method to solve x 5 at xo = 2.

x  6 = 0, starting
b. Prove that this Newton's method converges. 2.8.12
a. Do one step of Newton's method to solve the equations y  x 2 + 8 + cos x = 0
xy2+9+2cosy=O
starting at
(~~)=(~)·
b. Does Newton's method converge? You may find it useful to know that 9.86 is quite close to 10.
7r 2 :::::l
c. Find a number R
< 1 such that there exists a root of the equations in the
disc of radius R around ( Exercise 2.8.13, part a: The blunderbuss (partial derivative) approach works, but a direct calculation gives a better result.
~) .
2.8.13 a. Find a global Lipschitz ratio for the derivative of the mapping F : IR2 + IR 2 given by F (
x) y
= ( x 2 + y 2  2x  15) . xyxy
b. Do one step of Newton's method to solve F (
~)
= (
8)
starting at (
f).
c. Find and sketch a disc in IR2 which you are sure contains a root. Exercise 2.8.14, part b: Hint: Let
f (a)
=
~ (a+ ak~l) ·
Show that f(b 1fk) = b1fk; compute J'(b 1fk).
*2.8.14 a. Prove that if you compute W by Newton's method, as in Exercise 2.8.4, choosing ao > 0 and b > 0, then the sequence n 1> an converges to the positive kth root.
b. Show that this would not be true if you chose ao > 0 and b > 0 and defined the sequence n 1> an recursively by the divide and average algorithm:
an+l = 25 found
~ (an + a;_
1)
at matrixeditions.com/Programs.html
·
252
Chapter 2.
Solving equations
c. Use Newton's method and "divide and average" (and a calculator or computer) to compute {12, starting at ao = 2. What can you say about the speeds of convergence? (A drawing is recommended, as computing cube roots is considerably harder than computing square roots.) Exercise 2.8.15: Note that this is a quadratic equation in y, z, .X. Finding eigenvalues always gives rise to quadratic equations.
2.8.15
l[l [l
Find r > 0 such that Newton's method to solve 2
a a2 [ 1 +a a 0
O a 3
y1 z
=A
y1 , z
is sure to converge to a solution for Exercise 2.8.16: Try adding variables u1 = x 2 , u2 = x 4 = u~, as in Example 2.8.12.
.Xo
= 1,
lal < r.
2.8.16 Find r > 0 such that if you solve x 6 + x 5  x = a by Newton's method, starting at 0, you are sure to converge to a solution for lal < r.
2.9
SUPERCONVERGENCE
If Newton's method gave only the linear rate of convergence guaranteed by Kantorovich's theorem, it would be of limited interest. People use it because if we require < 1/2 in inequality 2.8.53 rather than ~ 1/2, it
Superconvergence explains why Newton's method is the favorite scheme for solving equations. If at each step Newton's method only halved the distance between guess and root, a number of simpler algorithms (bisection, for example) would work just as well. When Newton's method works at all, it starts superconverging soon. As a rule of thumb, if Newton's method hasn't converged to a root in seven or eight steps, you've chosen a poor initial condition.
superconverges. We will also see that if we use a different way to measure matrices: the norm rather than the length, then the hypotheses of Kantorovich's theorem are easier to satisfy, so the theorem applies to more systems of equations. Example 2.9.1 (Slow convergence). Let us solve f(x) = (x  1) 2 = 0 by Newton's method, starting at a0 = 0. As Exercise 2.9.1 asks you to show, the best Lipschitz ratio for f' is 2, so
If (ao) 11 (!' (ao)
r
1
I2 M
= 1. (
~)
2 •
2
=
~,
2.9.1
and Theorem 2.8.13 guarantees that Newton's method will converge to the unique root a= 1, which is on the boundary of U1 = (0, 1). The exercise also asks you to check that hn = 1;2n+ 1 , so an = 1  1/2n. Thus at each step Ihn I is exactly half the preceding Ihn1 I The proof of the Kantorovich theorem in Appendix A5 shows that this is exactly the minimal rate of convergence guaranteed. 6 Example 2.9.1 is both true and squarely misleading. Newton's method usually converges much, much faster than in Example 2.9.1. If the product in inequality 2.8.53 is strictly less than 1/2:
lf(ao)ll[Df(ao)J 1 12 M
= k
0,
• The left halfcircle is the graph of
v'l 
x2
is the graph of
\/1 
y2
Jl  y2
Thus near the points where x, y # 0, the equation x 2 + y2 = 1 represents x as a function of y and y as a function of x. Near the two points where y = 0 it only represents x as a function of y, as shown in Figure 2.10.10; near x = 0 it only represents y as a 'function of x. 6 Example 2.10.13 (Implicit function theorem does not guarantee absence of implicit function). Do the equations
(x 2.10.10. The unit circle is the solution settox 2 +y 21=0. Forx,y>O, the solution set is the graph of both y'l  y 2 and v'l  x 2, one expressing x in terms of y, the other y in terms of x. In the third quadrant, it is the graph of both v1f=Y2 and v'l  x2.
~)
the solution set is not
the graph of y'l  y 2 because the same value of y gives two different values of x. Near (
B)
it is not
the graph of v'l  x2 because the same value of x gives two different values of y.
0
yt=O
FIGURE
Near (
+ t) 2 =
exp""' hnplicitly x and y in terms oft in a ne;gbbo,hood of (
m(':"':_'t) ;, [
~) ? The hn
plicit function theorem does not guarantee the existence of an implicit func
tion' the derivative ofF
which at
m is [
~
2 (x
~rt) ~
~ ~ _ ~ l·That dedvative is not onto R
n,
But there is an implicit function g, defined by g(t)
that F ( g;t) )
~ ( g) in a neigbbmbood of (
tion is differentiable.
=
2".',+ t)],
2.
(D, and such
and this hnplicit func
6
The full statement of the implicit function theorem In Section 3.1 we will see that the short version of the implicit function theorem is enough to tell us when an equation defines a smooth curve, surface, or higherdimensional analogue. The short version requires only that the mapping be C 1 , i.e., that the derivative be continuous. But to get a bound on the domain of the implicit function (which we need to compute implicit functions), we must say how the derivative is continuous. Thus, as was the case for the inverse function theorem, in the long version, we replace the condition that the derivative be continuous by a more demanding condition, requiring that it be Lipschitz. Figure 2.10.11 illustrates the theorem.
270
Since [DF(c)] is onto, it is possible to order the variables in the domain so that the matrix consisting of the first n  k columns of [DF{c)] row reduces to the identity. {There may be more than one way to achieve this.) Then the variables corresponding to those n  k columns are the pivotal unknowns; the variables corresponding to the remaining k columns are the nonpivotal unknowns. Equation 2.10.25: The [OJ is the k x (n  k) zero matrix; Ik is the k x k identity matrix. So L is n x n. If it weren't square, it could not be invertible. Exercise 2.5 justifies the statement that L is invertible. The implicit function theorem could be stated using a weaker continuity condition than the derivative being Lipschitz; see Exercise A5.l in Appendix A5. You get what you pay for: with the weaker condition, you get a smaller domain for the implicit function.
The domain BR {b) of g has dimension k; its codomain has dimension n  k. It takes as input values of the k active variables and gives as output the values of the n  k passive variables. Equation 2.10.29, which tells us how to compute the derivative of an implicit function, is important; we will use it often.
Chapter 2.
Solving equations
Theorem 2.10.14 (The implicit function theorem). Let W be an open neighborhood of c E JR.n, and let F : W  t JR.nk be a differentiable function, with F(c) = 0 and [DF(c)] onto. Order the variables in the domain so that the matrix consisting of the first n  k columns of [DF(c)] row reduces to the identity. Set
~) , where the entries of a correspond to the n  k pivotal unknowns, and the entries ofb correspond to the k nonpivotal (active) unknowns. Then there exists a unique continuously differentiable mapping g from a neighborhood ofb to a neighborhood of a such that F (;) = 0 expresses the first n  k variables as g applied to the last k variables: x = g(y). To specify the domain of g, let L be the n x n matrix c = (
L= [[D1F(c),..[O]DnkF(c)]
[Dnk+iF(ck .. ,DnF(c)]]. 2 _10 _25
Then Lis invertible. Now find a number R > 0 satisfying the following hypotheses: 1. The ball Wo with radius 2RIL 1 I centered at c is contained in W.
2. For u, v in Wo the derivative satisfies the Lipschitz condition
l[DF(u)]  [DF(v)JI::::;
2 Rl~_ 112 1u vi.
2.10.26
Then there exists a unique continuously differentiable mapping 2.10.27
such that
g(b) =a and F ( g~)) = 0 for ally
E
BR(b).
2.10.28
By the chain rule, the derivative of this implicit function g at b is
[Dg(b)] = [D1F(c), ... , DnkF(c)] 1[Dnk+1F(c), ... , DnF(c)]. partial deriv. for the
nk pivotal variables
partial deriv. for the k nonpivotal variables
2.10.29
Remarks. 1. The k variables b determine the n  k variables a, both for the derivative and for the function. This assumption that we express the first n  k variables in terms of the last k is a convenience; in practice the question of what to express in terms of what depends on context. Usually, if the derivative of F is onto Rnk (i.e., has n  k linearly independent columns), there will be more than one way to choose these columns (see Exercise 2.5.7). For any choice of n  k linearly independent columns, the implicit function theorem guarantees that you can express the corresponding variables implicitly in terms of the others.
2.10
The inverse and implicit function theorems
271
y
b
R
R
a
x
FIGURE 2.10.11. In a neighborhood of(:) the equation F = 0 expresses x implicitly as a function of y. That neighborhood is contained in the ball Woof radius r = 2RIL 1 1. But the whole curve is not the graph of a function f expressing x as a function of y; near y = b, there are four such functions.
The inverse function theorem is the special case of the implicit function theorem when we have n = 2k variables, the passive kdimensional variable x and the active kdimensional variable y, and where our original equation is
f(x) y
= o;
shows that every point in Wo that is a solution to F (;) form (
g~)).
F(~ )·
f : JRk
= 0 is of the
Thus the intersection of W 0 with the graph of g is the
intersection of W 0 with the solution set of F (;) function
i.e., we can separate out y from
+
=
0. (The graph of a
JRnk exists in a space whose dimension is n.)
6.
Example 2.10.15 {The unit circle and the implicit function theorem). In Example 2.10.12 we showed that if a# 0, then in some neighborhood of ( the equation x 2 + y 2  1 = 0 implicitly expresses x as a function of y. Let's see what the strong version of the implicit function theorem says about the domain of this implicit function . The matrix L of equation 2.10.25 is
b),
Equation 2.10.30: In the lower right corner of L we have the number 1, not the identity matrix I; our function F goes from IR 2 to IR, son = m = 1, and the 1 x 1 identity matrix is the number 1. Equation 2.10.33: Note the way the radius R of the interval around b shrinks, without ever disappearing, as a
2. The proof of the implicit function theorem given in Appendix A8
>
0. At the points (
~)
and ( _ ~) , the equation x2
+
y2
1=0
does not express x in terms of y, but it does express x in terms of y when a is arbitrarily close to 0.
L = [2oa
2b] 1 ,
and
L1=_.!_[l 2a
0
2b] 2a ·
2.10.30
So we have
IL 1 1= The derivative of F ( ~)
2 ~al \/i + 4a
= x 2 + y2 
2
+ 4b2 = ~ .
2.10.31
1 is Lipschitz with Lipschitz ratio 2:
I[nF( ~~)]  [nF( ~~)]I= l[2u1 
2v1, 2u2  2v2]I
= 2l[u1  v1,
u2 
v2]I :::; 2Ju 
2.10.32
vi,
so (by equation 2.10.26) we can satisfy condition 2 if R satisfies i.e.,
1
R = 4IL1l2
a2
5
2.10.33
272
Chapter 2.
Solving equations
We then see that Wo is the ball of radius 2RIL1I = 2a2 VS= 5 2lal
M. vs'
2.10.34
since W is all of R 2 , condition 1 is satisfied. Therefore, for all (
b) with
a =/: 0, the equation + 1 = 0 expresses x (in the interval of radius lal/VS around a) as a function of y (in BR(b), the interval of radius a 2/5 around b). Of course we don't need the implicit function theorem to understand the unit circle; we know that x = ±.jf=Y2. But let's pretend we don't, and x2
Of course, there are two possible x. One is found by starting Newton's method at a, the other by starting at a.
y2 
b)
go further. The implicit function theorem says that if ( is a root of the 2 2 2 equation x + y  1 = 0, then for any y within a /5 of b, we can find the corresponding x by starting with the guess x 0 =a and applying Newton's method, iterating
F(~n) = Xn
Xn+l
0 0 1
1 0 0
0
1
0
1
0
1
0
0
0
0 0
0 0
0 0
1
0
0
1
r~
1
0
0
0 0 0
1
0
0
1
0 0
0
1
~]
L1 in Example 2.10.16
(
D 1F
Xn
)
=Xn
x; + y
y
2
1
2.10.35
2Xn
(In equation 2.10.35 we write 1/D 1F rather than (D 1F) 1 because D 1 F is a 1 x 1 matrix, i.e., a number.) !::::. Example 2.10.16 (An implicit function in several variables). In what neighborhood of c ~
L in Example 2.10.16
0

o 0) (
{ x 2 y =a
0
do the equations
z 2 x =0
0 0
(
y 2  z = b determine
~) llnplicitly as a function of a and b? H+ 2p  S(p)  T(p)?
0 2
corresponds to
0 0 3 0
2.17 Let ai, ... , ak, bi, ... , bk be any 2k numbers. Show that there exists a unique polynomial p of degree at most 2k  1 such that p( n) = an, p' (n) = bn for all integers n with 1 ~ n ~ k. In other words, show that the values of p and p' at 1, ... , k determine p. 2.18
Hint for Exercise 2.17: You should use the fact that a polynomial p of degree d such that p(n) = p'(n) = 0 can be written p(x) = (xn) 2 q(x) for some polynomial q of degree d  2.
A square n x n matrix P such that P 2 =Pis called a projection.
a. Show that P is a projection if and only if I  P is a projection. Show that if P is invertible, then P is the identity. b. Let Vi = img P and Vi = ker P. Show that any vector v E written uniquely v = v1 + v2 with V1 E Vi and v2 E Vi. c. Show that there exist a basis v1, ... , Vn of that
~n
Hint for Exercise 2.18, part b: Pv).
v = Pii + (v 
Exercise 2.19: Recall that C2 is the space of C 2 (twice continuously differentiable) functions.
o, Piik+2 =
can be
~
n such
and a number k
Pv1 = v1, Pv2 = v2, ... , Pvk = vk Pvk+1 =
:nr
and
o, ... ,Pvn = o.
*d. Show that if Pi and P2 are projections such that PiP2 = [O], then Q = Pi + P2  (P2Pi) is a projection, ker Q = ker Pi n ker H, and the image of Q is the space spanned by the image of Pi and the image of H. 2.19 Show that the transformation T: C 2 (~)+ C(~) given by formula 2.6.8 in Example 2.6. 7 is a linear transformation.
Denote by .c(Mat(n,n),Mat(n,n)) the space of linear transformations from Mat (n, n) to Mat (n, n). 2.20
a. Show that .C (Mat (n, n), Mat (n, n)) is a vector space and that it is finite dimensional. What is its dimension? b. Prove that for any A E Mat (n, n), the transformations
LA, RA : Mat (n, n) + Mat (n, n) given by LA(B) =AB and RA(B) =BA are linear transformations.
280
Chapter 2.
Solving equations
c. Let ML C .C (Mat (n, n), Mat (n, n)) be the set offunctions of the form LA. Show that it is a subspace of
.c(Mat (n, n), Mat (n, n)). What is its dimension?
d. Show that there are linear transformations T: Mat (2, 2) +Mat (2, 2) that cannot be written as LA +RB for any two matrices A, B E Mat (2, 2). Can you find an explicit one? 2.21 Show that in a vector space of dimension n, more than n vectors are never linearly independent, and fewer than n vectors never span. 2.22 Suppose we use the same operator T : P2 + P2 as in Exercise 2.6.8, but choose instead to work with the basis
q1(x)=x 2,
q2(x)=x 2 +x,
q3(x)=x 2 +x+l.
Now what is the matrix iP{q\ o To 'P{q}? We thank Tan Lei and Alexandre Bartlet for Exercise 2.23.
F (
~)
= ( ~~;(x ~y~)~y:)
Map for Exercise 2.24
2.23 Let A be a k x n matrix. Show that if you row reduce the augmented matrix [ATIIn] to get [AT iF], the nonzero columns of A form a basis for the image of A, and the nonzero columns of B form a basis for the kernel of A. 2.24 a. Find a global Lipschitz ratio for the derivative of the map F defined in the margin. b. Do one step of Newton's method to solve F (
~ ) (
·8)
= (
~), starting
g).
at (
c. Can you be sure that Newton's method converges? Exercise 2.25: Note that
[2 !] 3
= [8 I],
i.e.,
2.25
Using Newton's method, solve the equation A3 =
[~ ~ ~i0 2 8
[08 08 0OJ 0 0 8
Exercise 2.26: The computation really does require you to row reduce a 4 x 4 matrix.
2.26 Set
Consider the map F : Mat (2, 2) +Mat (2, 2) given by F(A) = A 2 + A 1 .
Ao= [ ~ ~]
and
Bo = F(Ao),
Ur=
and define
{BE Mat (2, 2) I IB  Bol < r}.
Do there exist r > 0 and a differentiable mapping G : Ur + Mat (2, 2) such that F(G(B)) = B for every BE Ur? 2.27 a. Find a global Lipschitz ratio for the derivative of the mapping f : R 2 + R 2 given in the margin.
r(~) = (::=~=~) Map for Exercise 2.27
b. Do one step of Newton's method to solve f (
~)
= (
g)
starting at (
~).
c. Find and sketch a disc in R 2 which you are sure contains a root. 2.28 There are other plausible ways to measure matrices other than the length and the norm; for example, we could declare the size IAI of a matrix A to be the largest absolute value of an entry. In this case, IA+ Bl :::; IAI + IBI, but the statement IA:XI :::; IAll:XI (where l:XI is the ordinary length of a vector) is false. Find an € so that it is false for
Exercise 2.29: The norm llAll of a matrix A is defined in Section 2.9 (Definition 2.9.6).
0 , A=[~0 ~0 +El 0 1
2.29
Show that
llAll = llATll·
and
x~
m.
2.11 2.30
In Example 2.10.9 we found that M
the function f (
~) = ( si;J~:~)).
Review exercises for Chapter 2
281
= 2v'2 is a global Lipschitz ratio for
What Lipschitz ratio do you get using the
method of second partial derivatives? Using that Lipschitz ratio, what minimum domain do you get for the inverse function at f ( 2.31
a. True or false? The equation sin(xyz)
dilferentiable function of y and z ,,..., the poffit
~)
?
m('n =z
expresses x implicitly as a
b. True or false? The equation sin(xyz) = z expresses z implicitly as a differentiable function of x and y near the same point. Exercise 2.32: You may use the fact that if
S : Mat (2, 2)
+ Mat (2, 2)
is the squaring map
S(A)
= A2 ,
2.32
Exercise 2.35: There are many "right" answers to this question, so try to think of a few.
F([~ ~]) 2.33
g:
=
True or false? There exist r
> 0 and a differentiable map
Br([~ ~D+ Mat(2,2)
2.34
such that
g
([~ ~D = [~
_i]
~]).
Given three vectors [ :: ] , [ ~:] , [ ~:] in R 2 , show that there exist vectors
a~ [:] •b ~ ml, c~ [:] in R' zucb th~ 1a.1 2 = lhl 2 = lcl 2 = 1 if and only if V,
2.35
~ [ :: ]
and Vz
and
a. b = a. c = b . c = 0
~ [ ~] ~ of unit length and orthogonal.
Imagine that, when constructing a Newton sequence
Xn+i
Exercise 2.36: The answer to part d depends on whether you choose the additive identity to be {O} or allow it to be whatever is appropriate for the particular subset you are looking at. In the latter case you might land on quotient spaces.
and
+ Mat (2, 2) with [~ _i], and (F(A))2=A.
and (g(A))2 =A for all A E Br ( [ ~ Dierk Schleicher contributed Exercise 2.34. Geometrically, the condition given is the condition that there exists a unit cube with a vertex at the origin such that the three sides emanating from the origin are a, b, and c.
~]
a C 1 mapping F : U
then
(DS(A)]B =AB+ BA.
True or false? There exist a neighborhood U C Mat (2, 2) of [ ~
= Xn 
[Df(xn)r 1 r(xn),
you happen upon a noninvertible matrix (Df(xn)]. What should you do? Suggest ways to deal with the situation. 2.36 Let V be a vector space, and denote by P* (V) the set of nonempty subsets of V. Define + :P* (V) x P* (V) P* (V) by
+
A
+B
~r { a + b I a E A, b E B }
+
and scalar multiplication R x P* (V) P* (V) by aA ~ { aa I a E A } . a. Show that + is associative: (A+ B) + C =A+ (B + C) and that {O} is a an additive identity. b. Show that a(A + B) = aA + aB, IA = A, and (a,B)A = a(,BA), for all a,,B ER. c. Is P* (V) a vector space with these operations?
282
Chapter 2.
Solving equations
*d. Does P* (V) have subsets that are vector spaces with these operations? *2.37 This exercise gives a proof of Bezout's theorem. Let p 1 and p 2 be polynomials of degree k1 and k2 respectively, and consider the mapping
T: (q1,q2) Exercise 2.37, part a: It may be easier to work over the complex numbers. Relatively prime: with no common factors.
f+
p1q1 + p2q2,
where q1 is a polynomial of degree at most k2  1 and q2 is a polynomial of degree at most k1  1, so that P1 Q1 + p2 q2 is of degree ~ k1 + k2  1. Note that the space of such (q1, q2) is of dimension k1 + k2, and the space of polynomials of degree at most k1 + k2  1 is also of dimension k1 + k2. a. Show that ker T = {O} if and only if p1 and p2 are relatively prime. b. Use Corollary 2.5.10 to prove Bezout's identity: if p1, p2 are relatively prime, then there exist unique Q1 and q2 of degree at most k2  1 and k 1  1 such that P1Q1 + P2Q2 = 1.
**2.38
Let A be an n x n .n
i.>.k  >.1·1>m>0 
for some number m. Let B be an n x n matrix. Find a number R, depending on m, such that if IBI < R, then Newton's method will converge if it is used to solve
(A+ B)x = µx, Exercise 2.39: For instance, if
n
= 4, then
, :[~ ~ !~] J,:
[~ ~ ~ ~]
Of course if an n x n matrix A has rank n, then A = QJnpl just says that A = QP 1 , i.e., A is invertible with inverse PQ 1 ; see Proposition 1.2.15.
starting at Xo 2.39
for
x satisfying lxl 2 = 1,
= ek' µo = Ak.
Prove that any n x n matrix A of rank k can be written A
where P and Q are invertible and Jk = [ ~
~], where Ik
= QJkP 1 ,
is the k x k identity
matrix. 2.40
Let a sequence of integers ao, a 1, a 2, ... be defined inductively by ao = 1, a1 = 0,
and
an = 2anl + an2 for n 2:'.: 2.
a. Find a matrix M such that ( an+l) an+2 terms of powers of M.
=M
( an ) . Express ( an ) . an+l ln an+l
b. Find a linear relation between I, M, M 2 , and use it to find the eigenvalues ofM. c. Find a matrix P such that pl MP is diagonal. d. Compute Mn in terms of powers of numbers. Use the result to find a formula for an. 2.41 Exercise 2.2.11 asked you to show that using row reduction to solve n equations inn unknowns takes n 3 +n 2 /2n/2 operations, where a single addition, multiplication, or division counts as one operation. How many operations are needed to compute the inverse of an n x n matrix A? To perform the matrix multiplication A 1 6?
3 Manifolds, Taylor polynomials, quadratic forms, and curvature Thomson [Lord Kelvin] had predicted the problems of the first [transatlantic] cable by mathematics. On the basis of the same mathematics he now promised the company a rate of eight or even 12 words a minute. Half a million pounds was being staked on the correctness of a partial differential equation.T. W. Korner, Fourier Analysis
3.0 When a computer calculates sines, it does not look up the answer in some mammoth table of sines; stored in the computer is a polynomial that very well approximates sin x for x in some particular range. Specifically, it uses a formula very close to equation 3.4.6:
+ agx 3 + asx 5 + a7 x 7 + agx 9 + a11x 11 + c::(x),
sinx = x
where the coefficients are
= .1666666664 = .0083333315 a1 = .0001984090
a3
as
ag
=
.0000027526
an= .0000000239.
When
lxl ::; 7r /2,
the error c::(x) is guaranteed to be less than 2 x 10 9 , good enough for a calculator that computes to eight significant digits. The computer needs only to remember five coefficients and do a bit of arithmetic to replace a huge table of sines.
INTRODUCTION
This chapter is something of a grab bag. The various themes are related, but the relationship is not immediately apparent. We begin with two sections on geometry. In Section 3.1 we use the implicit function theorem to define smooth curves, smooth surfaces, and more general kdimensional "surfaces" in Rn, called manifolds. In Section 3.2 we discuss linear approximations to manifolds: tangent spaces. We switch gears in Section 3.3, where we use higher partial derivatives to construct the Taylor polynomial of a function in several variables. We saw in Section 1. 7 how to approximate a nonlinear function by its derivative; here we see that we can better approximate Ck functions using Taylor polynomials when k ~ 2. This is useful, since polynomials, unlike sines, cosines, exponentials, square roots, logarithms, ... can actually be computed using arithmetic. Computing Taylor polynomials by calculating higher partial derivatives can be quite unpleasant; in Section 3.4 we show how to compute them by combining Taylor polynomials of simpler functions. In Sections 3.5 and 3.6 we take a brief detour, introducing quadratic forms and seeing how to classify them according to their "signature" : if we consider the seconddegree terms of a function's Taylor polynomial as a quadratic form, its signature usually tells us whether at a point where the derivative vanishes the function has a minimum value, a maximum value, or some kind of saddle, like a mountain pass. In Section 3.7 we use Lagrange multipliers to find extrema of a function restricted to some manifold MC Rn; we use Lagrange multipliers to prove the spectral theorem. In Section 3.8 we introduce finite probability spaces, and show how the singular value decomposition (a consequence of the spectral theorem) gives rise to principal component analysis, of immense importance in statistics. In Section 3.9 we give a brief introduction to the vast and important subject of the geometry of curves and surfaces, using the higherdegree approximations provided by Taylor polynomials: the curvature of a curve or surface depends on the quadratic terms of the functions defining it, and the torsion of a space curve depends on the cubic terms. 283
284
3.1
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
MANIFOLDS Everyone knows what a curve is, until he has studied enough mathematics to become confused through the countless number of possible exceptionsFelix Klein
These familiar objects are by no means simple: already, the theory of soap bubbles is a difficult topic, with a complicated partial differential equation controlling the shape of the film.
In this section we introduce one more actor in multivariable calculus. So far, our mappings have been first linear, then nonlinear with good linear approximations. But the domain and codomain of our mappings have been "flat" open subsets of JRn. Now we want to allow "nonlinear" ]Rn's, called smooth manifolds. Manifolds are a generalization of the familiar curves and surfaces of every day experience. A onedimensional manifold is a smooth curve; a twodimensional manifold is a smooth surface. Smooth curves are idealizations of things like telephone wires or a tangled garden hose. Particularly beautiful smooth surfaces are produced when you blow soap bubbles that wobble and slowly vibrate as they drift through the air. Other examples are shown in Figure 3.1.2.
FIGURE 3.1.1. Felix Klein (18491925) Klein's work in geometry "has become so much a part of our present mathematical thinking that it is hard for us to realise the novelty of his results."From a biographical sketch by J. O'Connor and E. F. Robertson. Klein was also instrumental in developing Mathematische Annalen into one of the most prestigious mathematical journals. FIG URE 3 .1. 2. Four surfaces in JR 3 . The top two are graphs of functions . The bottom two are locally graphs of functions. All four qualify as smooth surfaces (twodimensional manifolds) under Definition 3.1.2.
We will define smooth manifolds mathematically, excluding some objects that we might think of as smooth: a figure eight, for example. We will see how to use the implicit function theorem to tell whether the locus defined by an equation is a smooth manifold. Finally, we will compare knowing a manifold by equations, and knowing it by a parametrization.
3.1
Manifolds
285
Smooth manifolds in Rn When is a subset X C Rn a smooth manifold? Our definition is based on the notion of a graph. It is convenient to denote a point in the graph of a function f·Rk + Rnk as ( x ) with · f(x) ' x E Rk and f(x) E Rnk. But this presupposes that the "active" variables are the first variables, which is a problem, since we usually cannot use the same active variables at all points of the manifold. How then might we describe a point z in the graph of a function f : Rk + Rnk, where the variables of the domain of f have indices ii, ... , ik and the variables of the codomain have indices j1, ... , ink? Here is an accurate if heavyhanded approach, using the "concrete to abstract" linear transformation of Definition 2.6.12. Set x
=(
~1
)
Xk
,
y
=(
~1
)
Ynk
with f(x) = y. Define d (d for the domain of f) and c (c for codomain) by
+ · · · + Xkeik c(Y) = y1eii + ·· · + Ynkeink,
d(x) =
+
Rnk is
the set of points ( ; ) E Rn such that f(x) = y. It is denoted r(f). Thus the graph of a function f lives in a space whose dimension is the sum of the dimensions of the domain and codomain of f. Traditionally we graph functions f : R + R with the horizontal xaxis corresponding to the input, and the vertical axis corresponding to the output (values of !). Note that the graph of such a function is a subset of R 2 . For example, the graph of f(x) = x 2 consists of the points
( J(x))
E R 2 , i.e., the points ( : 2 ) .
The top two surfaces shown in Figure 3.1.2 are graphs of functions from R2
to R: the surface on the left is the graph of
on the right is the graph of
f ( ~ ) = x 2 + y4 •
f ( ~)
= x 3  2xy 2 ; that
Although we depict these
graphs on a fl.at piece of paper, they are actually subsets of R 3 . The first consists of the points (
: x3  2xy2
) , the second of the points (
:
x2
+ y4
) .
Definition 3.1.2 says that if a function f : Rk + Rnk is C1, then its graph is a smooth ndimensional manifold in Rn. Thus the top two graphs shown in Figure 3.1.2 are twodimensional manifolds in R 3 . But the torus and helicoid shown in Figure 3.1.2 are also twodimensional manifolds. Neither is the graph of a single function expressing one variable in terms of the other two. But both are locally graphs of functions.
X1ei 1
where the e's are standard basis vectors in Rn. Then the graph of f is the set of z such that
z
Definition 3.1.1 (Graph}. The graph of a function f: Rk
= d(x) + c(f(x)).
Since the function f of Definition 3.1.2 is C 1 ' its domain must be open. If f is GP rather than C 1 , then the manifold is a GP manifold.
Definition 3.1.2 (Smooth manifold in Rn). A subset MC Rn is a smooth kdimensional manifold if locally it is the graph of a C 1 mapping f expressing n  k variables as functions of the other k variables. With this definition, which depends on chosen coordinates, it isn't obvious that if you rotate a smooth manifold it is still smooth. We will see in Theorem 3.1.16 that it is. Generally, "smooth" means "as many times differentiable as is relevant to the problem at hand". In this and the next section, it means of class C 1 . When speaking of smooth manifolds, we often omit the word smooth. 1 "Locally" means that every point x E M has a neighborhood U C Rn such that Mn U (the part of M in U) is the graph of a mapping expressing 1 Some authors use "smooth" to mean 0 00 : "infinitely many times differentiable". For our purposes this is overkill.
286
Chapter 3. Manifolds, Taylor polynomials, quadratic forms, curvature
n  k of the coordinates of each point in M n U in terms of the other A manifold M embedded in Rn, denoted M C Rn, is sometimes called a submanifold of Rn. Strictly speaking, it should not be referred to simply as a "manifold", which can mean an abstract manifold, not embedded in any space. The manifolds in this book are all submanifolds of Rn for some n. Especially in higher dimensions, making some kind of global sense of a patchwork of graphs of functions can be quite challenging; a mathematician trying to picture a manifold is rather like a blindfolded person who has never met or seen a picture of an elephant seeking to identify one by patting first an ear, then the trunk or a leg. It is a subject full of open questions, some fully as interesting and demanding as, for example, Fermat's last theorem, whose solution after more than three centuries aroused such passionate interest . Threedimensional and fourdimensional manifolds are of particular interest, in part because of applications in representing spacetime.
k. This may sound like an unwelcome complication, but if we omitted the word "locally" then we would exclude from our definition most interesting manifolds. We already saw that neither the torus nor the helicoid of Figure 3.1.2 is the graph of a single function expressing one variable as a function of the other two. Even such a simple curve as the unit circle is not the graph of a single function expressing one variable in terms of the other. In Figure 3.1.3 we show another smooth curve that would not qualify as a manifold if we required it to be the graph of a single function expressing one variable in terms of the other; the caption justifies our claim that this curve is a smooth curve. y
J
x
FIGURE 3.1.3. Above, I and Ii are intervals on the xaxis; J and Ji are intervals on the yaxis. The darkened part of the curve in the shaded rectangle I x J is the graph of a function expressing x EI as a function of y E J, and the darkened part of the curve in Ii x J 1 is the graph of a function expressing y E J1 as a function of x E Ii. (By decreasing the size of J1 a bit, we could also think of the part of the curve in Ii x J1 as the graph of a function expressing x E Ii as a function of y E Ji.) But we cannot think of the darkened part of the curve in I x J as the graph of a function expressing y E J as a function of x E /; there are values of x that give two different values of y, and others that give none, so such a "function" is not well defined.
Example 3.1.3 (Graph of smooth function is smooth manifold).
FIGURE 3.1.4. The graph of f(x) = lxl is not a smooth curve.
The graph of any smooth function is a smooth manifold. The curve of equation y = x 2 is a onedimensional manifold: the graph of y as the function f(x) = x 2 . The curve of equation x = y 2 is also a onedimensional manifold: the graph of a function representing x as a function of y. Each surface at the top of Figure 3.1.2 is the graph of a function representing z as a function of x and y. !:::,.
Example 3.1.4 (Graphs that are not smooth manifolds). The graph + JR, f(x) = lxl, shown in Figure 3.1.4, is not a
of the function f : JR
3.1
Manifolds
287
smooth curve; it is the graph of the function f expressing y as a function of x, of course, but f is not differentiable. Nor is it the graph of a function g expressing x as a function of y, since in a neighborhood of 0 the same value of y gives sometimes two values of x, sometimes none. In contrast, the graph of the function f(x) = x 113 , shown in Figure 3.1.5, is a smooth curve; f is not differentiable at the origin, but the curve is the graph of the function x = g(y) = y 3 , which is differentiable. The set X C IR2 of equation xy = 0 (the union of the two axes), shown on the left in Figure 3.1.6, is not a smooth curve; in any neighborhood of there are infinitely many y corresponding to x = 0, and infinitely many x corresponding toy= 0, so it isn't a graph of a function either way. But X  { ( is a smooth curve  even though it consists of four distinct /::,. pieces. A manifold need not be connected.
(8),
g) }
FIGURE
3.1.5.
The graph of f(x) = x 1 l 3 is a smooth curve: although f is not differentiable at the origin, the function g(y) = y 3 is.
+1FIGURE
LEFT:
3.1.6.
The graph of the x and
y axes is not a smooth curve.
The graph of the axes minus the origin is a smooth curve. RIGHT:
Example 3.1.6: Many students find it hard to call the sphere of equation
x 2 +y2
+ z2 = 1
two dimensional. But when we say that Chicago is "at" x latitude and y longitude, we treat the surface of the earth as two dimensional.
Example 3.1.5 (Unit circle). The unit circle of equation x 2 +y2 = 1 is a smooth curve. Here we need the graphs of four functions to cover the entire circle: the unit circle is only locally the graph of a function. As discussed in Example 2.10.12, the upper half of the circle, made up of points ( ~) with y > 0, is the graph of the function v'l  x 2 expressing y in terms of x, the lower half is the graph of v'l  x 2 , the right half is the graph of the function y 2 expressing x in terms of y, and the left half is the graph 2 ofJly . /::,.
Jl 
Using Definition 3.1.2 to show that a set is a manifold can be quite tedious, even for something as simple (and as obviously smooth) as the unit sphere. In Example 3.1.6 we do it the hard way, working directly from the definition. We will see in Theorem 3.1.10 that there is an easier way. Example 3.1.6 (Surface in JR3 ). By Definition 3.1.2, a subset Sc JR3 is a smooth surface (twodimensional manifold) in !R3 if locally it is the graph of a C 1 mapping expressing one variable as a function of the other two variables. That is, S is a smooth surface if for every point a = (
~)
E S,
there are neighborhoods I of a, J of b, and K of c, and either a differentiable mapping • f: Ix J
K, • g: Ix K  t J, • h: J x K  t I, t
i.e., z as a function of (x, y) or i.e., y as a function of (x, z) or i.e., x as a function of (y, z),
such that Sn (Ix J x K) is the graph off, g, or h. For instance, the unit sphere
s' "" {
m
'"ch that x 2 + y 2 + z2 ~
1}
3.1.1
288
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
is a smooth surface. Let Dx,y = { (
~)
such that x 2
+ y 2 < 1}
3.1.2
be the unit disc in the (x, y)plane and JRt the part of the zaxis where z > 0. Then The graph of a function
f: R+ R2 lives in R 3 . If x determines y, z, we would write this as
f(x)= (!i(x)) h(x)
consists of points
·the graph '
(1i(x)). h(x)
82
n (Dx,y
x JR;)
3.1.3
is the graph of the function Dx,y + JRt expressing z as Jl  x2  y2. This shows that 8 2 is a surface near every point where z > 0, and considering z = Jl  x 2  y 2 should convince you that 8 2 is also a smooth surface near any point where z < 0. Exercise 3.1.4 asks you to consider the case z = 0. 6.
Example 3.1.7 (Smooth curve in JR3 ). For smooth curves in JR 2 and smooth surfaces in JR3 , one variable is expressed as a function of the other variable or variables. For curves in space, we have two variables expressed as a function of the other variable: a space curve is a onedimensional manifold in JR 3 ' so it is locally the graph of a C 1 mapping expressing n  k = 2 variables as functions of the remaining variable. !:::,. Examples of curves and surfaces in JR 3 tend to be misleadingly simple: circles, spheres, tori, all not too hard to draw on paper. But both curves and surfaces can be phenomenally complicated. If you put a ball of yarn in a washing machine you will have a fantastic mess. This is the natural state of a curve in JR 3 . If you think of the surface of the yarn as a surface in JR3 , you will see that surfaces in JR3 can be at least as complicated.
Higherdimensional manifolds
FIGURE
3.1.7.
One possible position of four linked rods, of lengths li, l2, l3, and l4, restricted to a plane. Such fourbar mechanisms are used to convert rotational motion into backandforth rocking motion (to operate the windshield wipers on a car, for example).
An open subset U C !Rn is the simplest ndimensional manifold; it is the graph of the zero function taking points in U to the origin, which is the only point of the vector space {O}. (The graph lives in !Rn x {O} = !Rn.) In particular, !Rn itself is an ndimensional manifold. The next example is far more elaborate. The set X 2 it describes is a fourdimensional manifold in JR 8 ; locally, it is the graph of a function expressing four variables in terms of four other variables.
Example 3.1.8 (Linked rods). Linkages of rods are everywhere, in mechanics (consider a railway bridge or the Eiffel tower), in biology (the skeleton), in robotics, in chemistry. One of the simplest examples is formed of four rigid rods, with assigned lengths li, l2, la, l4 > 0, connected by universal joints that can achieve any position to form a quadrilateral, as shown in Figure 3.1.7. What is the set X 2 of positions the linkage can achieve if the points are restricted to a plane? Or the set X 3 of positions the linkage can achieve if the points are allowed to move in space? (In order to guarantee that our
3.1
Manifolds
289
sets are not empty, we will require that each rod be shorter than the sum of the lengths of the other three.) These sets are easy to describe by equations. For X2 we have X2
= the set of all (xi, x2, x 3, X4) E (JR.2) 4 such that lx1  x2I = li, lx2  x3I = l2, lx3  x4I = [3,
3.1.4 lx4  xii = l4.
Thus X2 is a subset of JR 8 . Another way of saying this is that X 2 is the subset defined by the equation f(x) = 0, where f : (JR 2 ) 4 + JR4 is the mapping
Equation 3.1.5: This description is remarkably concise and remarkably uninformative. It isn't even clear how many dimensions X2 and X 3 have; this is typical when you know a set by equations. If you object that you cannot visualize this manifold, you have our sympathy. Precisely for this reason, it gives a good idea of the kind of problem that comes up: you have a collection of equations defining some set but you have no idea what the set looks like.
Even picturing a twodimensional manifold based on an equation can be hard; we expect you to be able to visualize the surface given by x 2 + y 2 + z 2 = 1, but would you immediately realize that the equation z = x 3  2xy 2 represents the surface shown in the upper left corner of Figure 3.1.2?
Exercise 3.1.18 asks you to determine how many positions are possible using X1 and the two angles above  again, except in a few cases. Exercise 3.1.15 asks you to describe X2 and X3 when
li = h
+ h + l4.
r((xi) (x2 ) (x3 ) (x4 YI ' Y2 ' Y3 ' Y4
))=
.......... .......... .......... .......X1
x2
x3
x4
(x2  X1) 2 + (Y2 y1) 2 l~l [
(x3x2) 2 +(y3y2) 2 z~
(x4  x3) 2 + (y4  y3) 2 
l~
.
315 . .
(x1  X4)2 + (Y1  y4)2  l~
(The Xi have two coordinates, because the points are restricted to a plane.) Similarly, the set X3 of positions in space is also described by equation 3.1.4, if we take xi E JR3 ; X 3 is a subset of!R 12 . Of course, to make equations corresponding to equation 3.1.5 we would have to add a third entry to the Xi, and instead of writing (x2  x1) 2 + (Y2  Y1) 2  l~ we would need to write (x2  x1) 2 + (Y2 y1) 2 + (z2  z1) 2  l~. Can we express some of the xi as functions of the others? You should feel, on physical grounds, that if the linkage is sitting on the floor, you can move two opposite connectors any way you like and the linkage will follow in a unique way. This is not quite to say that x2 and X4 are a function of x1 and x3 (or that x1 and x3 are a function of x2 and x4). This isn't true, as is suggested by Figure 3.1.8.
FIGURE 3.1.8. Two possible positions of a linkage with the same x1 and x 3 are shown in solid and dotted lines. The other two are x1, x2, X3, x~ and x1, x~,x3, X4.
Usually, x 1 and x 3 determine either no positions of the linkage (if x1 and X3 are farther apart than l 1 + l2 or l3 + l4) or exactly four (if a few other conditions are met; see Exercise 3.1.17). But x2 and X4 are locally functions of x 1, x3. For a given x 1 and x3 four positions are possible. If the
290
FIGURE
3.1.9.
If three vertices are aligned, the end vertices {here, X1 and X3) cannot move freely. For instance, they can't moved in the directions of the arrows without stretching the rods. Saying that X2 is a 4dimensional manifold doesn't tell us much about what it "looks like". For instance, it doesn't tell us whether X2 is connected. A subset U C Rn is connected if given any two points xo, x1 E U, there exists a continuous map 'Y : [O, 1] + U with y(O) = xo and y(l) = x1. In other words, U is connected if there is a continuous path joining any two of its points. This very intuitive definition of connectedness is more properly known as path connectedness; it is the definition used in this book. (In topology, connectedness means something slightly different, but for open subsets of !in and for manifolds, the two notions of connectedness coincide.) A manifold need not be connected, as we saw already in the case of smooth curves, in Figure 3.1.6. In higher dimensions, determining whether a set is connected may be difficult. For example, as of this writing we don't know for what lengths of bars X2 is or isn't connected.
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
rods are in one of these positions, and you move x1 and x3 a small amount, then there will be only one possible position of the linkage, and hence of x 2 and X4, near the starting position. Even this isn't always true: if any three vertices are aligned (as shown in Figure 3.1.9, for instance, or if one rod is folded. back against another), then the two end vertices cannot be used as parameters (as the "active" variables that determine the values of the other variables). For instance, if xi, x2, and X3 are aligned, then you cannot move x1 and x3 arbitrarily, since the rods cannot be stretched. But locally the position is a function of x2 and X4. There are many other possibilities. For instance, we could choose x 2 and x4 as the variables that locally determine x 1 and x 3 , again making X 2 locally a graph. Or we could use the coordinates of x 1 (two numbers), the polar angle of the first rod with the horizontal line passing through x 1 (one number), and the angle between the first and the second (one number): four numbers in all, the same number we get using the coordinates of x 1 and X3. 2 Thus the set X2 of example 3.1.8 is a 4manifold in JR8 : locally it is the graph of a function expressing four variables (two coordinates each for two points) in terms of four other variables (the coordinates of the other two points or some other choice). It doesn't have to be the same function everywhere. In most neighborhoods, X2 is the graph of a function of x 1 and x 3 , but we saw that this is not true when xi, x 2 , and x 3 are aligned; near such points, X2 is the graph of a function expressing x 1 and X3 in terms of x2 and x 4 .3 6. The next example shows that when modeling a situation or object as a higherdimensional manifold, the choice of dimension may depend on the problem to be solved. Example 3.1.9 {A bicycle as a higherdimensional manifold). Consider the configuration space (set of positions) of a bicycle as a kdimensional manifold. What number is k? To specify a position, you must say where the center of gravity is (three numbers) and specify the orientation of the bicycle, say the direction in which the horizontal bar is pointing (two numbers) and the angle that the bar supporting the saddle makes with the vertical (one number). Then you need to give the angle that the handlebars make with the body of the bicycle (one number) and the angle that the wheels make with the forks (two numbers). The angle that the pedals make is another number. Presumably the position of the chain is determined by the position of the pedals, so it does not represent an extra free parameter. a system is said to have four degrees of freedom. some choices of lengths, X 2 is no longer a manifold in a neighborhood of some positions: if all four lengths are equal, then X2 is not a manifold near the position where it is folded flat. 2 Such
3 For
3.1
The "configuration space" of a system is the set of positions. The "phase space" is the set of positions and velocities. Although visualizing the 12dimensional configuration space of a bicycle as a single "12dimensional surfacelike thing" is presumably impossible, learning to ride a bicycle is not. In some sense, knowing how to ride a bicycle is a substitute for visualization; it gives a feel for what "12dimensionality" really is. Though there are parts that the rider might miss. For instance, he might think that the positions of the wheels can be ignored: who cares whether the air valves are up or down, except when pumping up the tires? But those variables are actually essential because of their derivatives, which measure how fast the tires are spinning. The angular momentum of the wheels is key to the stability of the motion. Theorem 3.1.10 is correct as stated, but the proof is based on the implicit function theorem, and our proof of the strong version of the implicit function theorem requires that the derivative of F be Lipschitz.
291
For most purposes, this should be enough: 10 numbers. Well, maybe the position of the breakpulls is important too, say 12 numbers in all. The space of positions should be a 12dimensional manifold. But there was a choice made of what to describe and what to ignore. W€ ignored the positions of the balls in the ball bearings, which might be irrelevant to a bicycle rider but very relevant to an engineer trying to minimize friction. We also ignored the vibrations of the rods, which should be imperceptible to a rider but of central importance to an engineer studying metal fatigue. When modeling a real system, this issue always comes up: What variables are relevant? Keep too many, and the model will be awkward and incomprehensible to the point of uselessness; keep too few, and your description may be wrong in essential ways. Note that if a brake cable snaps, the configuration space suddenly gains a dimension, since the position of the brake calipers and the corresponding brake pull suddenly become uncorrelated. The rider will definitely be aware !::::. of this extra dimension.
Using the implicit function theorem to identify manifolds Already for something as simple as the unit circle, using Definition 3.1.2 to show that it qualifies as a smooth curve was a bit tedious. Yet that was an easy case, where we know what the curve looks like, and where the functions can be written down explicitly. More typical is the question, "is the locus defined by x 8 + 2x3 + y + y 5 = 1 a smooth curve?" With the help of a computer, you might be able to visualize such a locus, but using Definition 3.1.2 to determine whether or not it qualifies as a smooth curve would be difficult indeed; you would have to solve for x in terms of y or for y in terms of x, which means solving equations of degree 5 or 8. Using that definition to determine whether a locus is a higherdimensional manifold is more challenging yet. Fortunately, the implicit function theorem gives a way to determine that a locus given by an equation or set of equations is a smooth manifold. In Theorem 3.1.10 below, the condition that [DF(a)] be onto is the crucial condition of the implicit function theorem. Theorem 3.1.10 (Showing that a locus is a smooth manifold). 1. Let U c !Rn be open, and let F : U Let M be a subset of!Rn such that
+
!Rnk be a C 1 mapping.
Mn U = { z EU I F(z) = o}. We prove Theorem 3.1.10 after Example 3.1.15.
Manifolds
3.1.6
If [DF(z)] is onto for every z E Mn U, then Mn U is a smooth
kdimensional manifold embedded in !Rn. If every z E M is in such a U, then Mis a kdimensional manifold. 2. Conversely, if M is a smooth kdimensional manifold embedded in !Rn, then every point z E M has a neighborhood U C !Rn such that there exists a C 1 mapping F : U+ JRnk with [DF(z)] onto and Mn U = { y I F(y) = 0 }.
292
Algebraic geometers call the locus of equation F (
~)
= (x2
+ y2 
1)2
a double circle. In algebraic geometry, it is a different locus from the unit circle.
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
Remark. We cannot use Theorem 3.1.10 to determine that a locus is not a smooth manifold; a locus defined by F(z) = 0 may be a smooth manifold even though [DF(z)] is not onto. For example, the set defined
by F ( ~) = 0, where F ( ~) = (x 2 + y 2  1) 2 , is the unit circle "counted twice"; it is a smooth manifold, but the derivative of F is not onto, since 3.1.7
More generally, if F(z) = 0 defines a manifold, then (F(z)) 2 the same manifold, but its derivative is always 0. !:::.
= 0 defines
Example 3.1.11 (Determining that a locus is a smooth curve). For a onedimensional manifold in JR2 (i.e., a plane curve), the function of Theorem 3.1.10 goes from an open subset of IR 2 to JR, and the requirement that [DF(z)] be onto for every z EM is equivalent to requiring that Example 3.1.11: Here, saying that
3.1.8
For instance, we have no idea what the locus Xe defined by is onto means that any real number can be expressed as a linear combination etD1F(a) + (JD2F(a) for some
(3)
E
JR2 .
Thus,
the following statements mean the same thing: For all a E M, 1. [DF(a)] is onto. 2. [DF(a)] =J [O, O]. 3. At least one of D1F(a) or D2F(a) is not 0. More generally, for an (n  1)dimensional manifold in any !Rn, !Rnk = JR, and saying that
[DF(z)]
= [D1F(z), ... , Dn(z)]
is onto is equivalent to saying that at least one partial derivative is not 0.
x8
+ 2x3 + y + y 5 =
3.1.9
C
looks like, but we know it is a smooth curve for all c, since the derivative of the function F ( ~)
= x 8 + 2x 3 + y + y 5  c is 3.1.10
and the second entry is never 0.
!:::.
Example 3.1.12 (Smooth curve: second example).
function F ( ~) =
x4
+
y4
+
x2 
y2 .
Consider the
We have
[oF(~)J =L4x 3 j2~,~y 3 :2y)= [2x(2x2 +1), 2y(2y2 1)]. 3.1.11 D1F
D2F
The only places where both partials vanish are (
~) , ( ±l~J2), where
F has values 0 and 1/4. Thus the locus of equation x 4 + y 4 + x 2  y 2 = c is a smooth curve for any number c "I 0 and c "# 1/4.
Figure 3.1.10 illustrates this. The locus of equation F (
~) = 1/4
consists of two points; that of equation F ( ~) = 0 is a figure eight. The others really are things one would want to call smooth curves. !:::. Remark. Note that the function F in the equation F ( ~)
= c is of a
different species than the functions f(x) = JI  x2 and g(y) = JI  y2 used to show that the unit circle is a smooth curve. The function F plays the role of F in the implicit function theorem; the functions f and g play
3.1
Manifolds
293
the role of the implicit function g in the implicit function theorem. If the conditions of Theorem 3.1.10 are met, then F(x) = c implicitly expresses x in terms of y or y in terms of x: the locus is the graph of the implicit function g or the implicit function f. Li. A locus defined by an equation of the form F ( ~) = c is called a level curve. One way to imagine such a locus is to think of cutting the surface that is the graph of F by the plane z = c, as shown in Figure 3.1.11.
FIGURE 3.1.10. The locus of equation
x4
+ y4 + x2 
y2
= 1/4
consists of the two points on the yaxis where y = ±1/v'2; it is not a smooth curve (but it is a zerodimensional manifold). Nor is the figure eight a smooth curve: near the origin it looks like two intersecting lines; to make it a smooth curve we would have to take out the point where the lines intersect. The other curves are smooth curves. In addition, by our definition the empty set (the case here if c < 1/4) is also a smooth curve! Allowing the empty set to be a smooth curve makes a number of statements simpler. The arrows on the lines are an artifact of the drawing program.
FIGURE 3.1.11. The surface that is the graph of F (
~) = x 2 
0.2x 4

y 2 is
sliced horizontally by setting z equal to three different constants. The intersection of the surface and the plane z = c used to slice it is known as a level set or level
curve. This intersection is the same as the locus of equation F (
~) = c.
If we "sliced" the surface in Figure 3.1.11 by the plane z = c where c is a maximum value of F, we would get a point, not a smooth curve. If we sliced it at a "saddle point" (also a point where the derivative of F is 0, so that the tangent plane is horizontal), we would get a figure eight, not a smooth curve. (Saddle points are discussed in Section 3.6.) Example 3.1.13 (Smooth surface in JR 3 ). When we use Theorem 3.1.10 to determine whether a locus is a smooth surface in JR 3 , the function F goes from an open subset of JR 3 to JR. In this case the requirement that [DF(z)] be onto for every z E M is equivalent to requiring that for every z E M, we have [DF(z)] =f. [O, 0, 0]; at least one partial derivative must not vanish. For instance, consider the set X defined by the equation
Fm ~
sin(x +yz)
~ 0.
3.1.12
294
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
Is it a smooth surface? The derivative is
You should be impressed by Example 3.1.13. The implicit function theorem is hard to prove, but the work pays off. Without having any idea what the set defined by equation 3.1.12 might look like, we were able to determine, with hardly any effort, that it is a smooth surface. Figuring out what the surface looks like  or even whether the set is empty  is another matter. Exercise 3.1.6 asks you to figure out what it looks like in this case, but usually this kind of thing can be quite hard indeed.
[ DF
(~)] [~,~·~]. =
D1F
D2F
3.1.13
DaF
On X, by definition, sin(a+bc) = 0, so cos(a+bc) =I 0, so Xis a smooth surface. (In this case, the first partial derivative is never 0, so at all points of the surface, the surface is the graph of a function representing x locally as a function of y and z.) 6.
Example 3.1.14 {Smooth curves in JR3 ). Theorem 3.1.10 suggests that a natural way to think of a curve C in JR3 is as the intersection of two surfaces. Here n = 3, k = l. If surfaces 8 1 and 8 2 are given by equations F1(z) = 0 and F2(z) = 0, each function going from JR3 to JR, then C = 81 n 82 is given by the equation F(z) = 0, where
F(z)
= (
~~~=~)
is a mapping from U
~ JR2 , with Uc JR3 open.
Thus [DF(z)] is a 2 x 3 matrix: For the 2 x 3 matrix [DF(z)] to be onto, the two rows must be linearly independent (see the equivalent statements about onto linear transformations in Section 2.5). Geometrically, this means that at z, the tangent plane to the surface 81 and the tangent plane to the surface 82 must be distinct, i.e., the surfaces must not be tangent to each other at z. In that case, the intersection of the two tangent planes is the tangent line to the curve.
3.1.14 The first row of the matrix is the derivative [DF1 (z)]. If it is onto (if any of the three entries of that row is nonzero), then 8 1 is a smooth surface. Similarly, if the derivative [DF2(z)] is onto, then 82 is a smooth surface. Thus if only one entry of the matrix is nonzero, the curve is the intersection of two surfaces, at least one of which is smooth. If any entry of the first row is nonzero and any entry of the second row is nonzero, then the curve is the intersection of two smooth surfaces. But the intersection of two smooth surfaces is not necessarily a smooth curve. For instance, consider the intersection of the surfaces given by F1 = 0 and F2 = 0 where
F.(;:) =
za zl
and
F, (:) =
za z,z2
3.1.15
1 ], so the curve given by F(z) = 0 is the 1 intersection of two smooth surfaces. But at the origin the derivative is not onto JR 2 , so we cannot conclude that the curve is smooth. 6. Then [DF(z)] = [
O
z2
 2z2 Z1
Example 3.1.15 {Checking that the linkage space is a manifold). In Example 3.1.8, X 2 is the set of positions of four rigid rods restricted to
3.1
Manifolds
295
the plane. This locus is given by the equation X1 Y1
X2 Y2 X3 Y3 X4
f{z) = f
3.1.16
Y4 Each partial derivative at right is a vector with four entries. For example,
The derivative is composed of the eight partial derivatives (in the second line we label the partial derivatives explicitly by the names of the variables): ~
~
~
~
[Df{z)] = [D1f{z), D2f{z), D3f{z), D4f{z), Dsf{z), D6f{z), D1f{z), D 8 f(z)] ~
= and so on. Equation 3.1.17: We had to put the matrix on two lines to make it fit. The second line contains the last four columns of the matrix.
~
X1
X2
X3
are aligned, then the first two columns of equation 3.1.18 cannot be linearly independent: Y1 y2 is necessarily a multiple of x1  x2, and Y2  y3 is a multiple of x2  x3.
~
~
Computing the partial derivatives gives 2{x1  x2) [Df{z)] =
2{y1  Y2)
0
[
0
O
O
2{x4  X1)
2(y4  Y1)
0 2(x2  X3) 2(X3  X4) 0
FIGURE 3.1.12. If the points
~
[D,,, 1 f{z), Dy 1 f{z), D,,, 2 f{z), Dy2 f{z), D,,, 3 f{z), Dy3 f{z), D,,, 4 f{z), Dy4 f(z)]. 2(x1  x2) 2(x2  X3) 0 0
0 2(y2  y3) 2(y3  y4) 0
2(y1 y2) 2(y20y3) 3.1.17 0
0 0 2(X3  X4) 2(x4  xi)
Since f is a mapping from JR 8 to JR4, the derivative [Df(z)] is onto if four of its columns are linearly independent. For instance, here you can never use the first four, or the last four, because in both cases there is a row of zeros. How about the third, fourth, seventh, and eighth, i.e., the points X2 = ( ~~ ) , X4 = ( ~!)? These work as long as the corresponding columns of the matrix 0 [2(x1 x2) 2(y1 y2) 2(x2  X3) 2(y2  y3) 0 3.1.18 0 2(X3  X4) y,) 0 0 0 2(X4  X1) 2(y4  Y1)
2(!13~
D,, 2 f(z)
DY2f(z)
D,, 4 f(z)
l
Dy 4 f(z)
are linearly independent. The first two columns are linearly independent precisely when x1, x2, and X3 are not aligned as they are in Figure 3.1.12, and the last two are linearly independent when x 3, x 4, and x 1 are not aligned. The same argument holds for the first, second, fifth, and sixth columns, corresponding to x 1 and x 3 . Thus you can use the positions of opposite points to locally parametrize X 2 , as long as the other two points are aligned with neither of the two opposite points. The points are never all four in line, unless either one length is the sum of the other three, or
296
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
li +l2 = l3 +l4, or l2 +l3 = l4 +li. In all other cases, X 2 is a manifold, and even in these last two cases, it is a manifold except perhaps at the positions /:::,. where all four rods are aligned. Proof of Theorem 3.1.10. 1. This follows immediately from the implicit function theorem.
2. If M is a smooth kdimensional manifold embedded in JR.n, then locally (in a neighborhood of any fixed point c E M) it is the graph of a C 1 mapping f expressing n  k variables Zj 1 , ..• , Zjnk as functions of the other k variables Zi 1 , ••• , Zik. Set
When the n  k passive variables are the first n  k variables of z, we have z = (;).
X=
ZJi )
Zj~k .._,._......., (
passive variables
and
y=
GJ 
3.1.19
active variables
The manifold is then the locus of equation F(z) = x  f(y) 0. The derivative is onto JR.nk because it has n  k pivotal (linearly independent) columns, corresponding to the n  k passive variables. D
Manifolds are independent of coordinates Our definition of a manifold has a serious weakness: it makes explicit reference to a particular coordinate system. Here we remove this restriction: Corollary 3.1.17 says that smooth manifolds can be rotated, translated, and even linearly distorted and remain smooth manifolds. It follows from Theorem 3.1.16, which says a great deal more: the inverse image of a smooth manifold by an arbitrary C 1 mappings with surjective derivative is still a smooth manifold.
In Theorem 3.1.16, r 1 is not an inverse mapping; since f goes from !Rn to !Rm, such an inverse map does not exist when n # m. By C 1 (M) we denote the inverse image: the set of points x E Ile such that f(x) is in M (see Definition 0.4.9 and Proposition 0.4.11).
Theorem 3.1.16 (Inverse image of manifold by a C 1 mapping). Let M C JR.m beakdimensional manifold, U an open subset oflR.n, and f: U+ JR.ma C 1 mapping whose derivative [Df(x)) is surjective at every x E f 1 (M). Then the inverse image f 1 (M) is a submanifold oflR.n of dimension k + n  m. Proof. Let x be a point of r 1 (M). By part 2 of Theorem 3.1.10, there exists a neighborhood V of f(x) such that Mn V is defined by the equation F(y) = 0, where F: V+ JR.mk is a C 1 mapping whose derivative [DF(y)) is onto for every y EM n V. Then W ~ f 1 (V) is an open neighborhood of x, and f 1 (M) n Wis defined by the equation F o f = 0. By the chain rule, [D(F o f)(x)] = [DF(f(x))][Df(x)],
3.1.20
and both [DF(f(x))] and [Df(x)] are surjective for every x E f 1 (M) n W, so we have verified the condition of part 1 of Theorem 3.1.10. Still by part 1
3.1
Manifolds
297
of Theorem 3.1.10, the dimension off 1 (M) is the dimension of the domain of F o f minus the dimension of the codomain, i.e., n  (m  k). D Direct images are more troublesome. In Corollary 3.1.17 we take direct images by invertible mappings, so that the direct image is the inverse image by the inverse. Proof of Corollary 3.1.17: Set f = g 1 , so that
f{y)
= A 1 (y 
c).
Then g( M) = r 1 ( M), and Theorem 3.1.16 applies.
Corollary 3.1.17 (Manifolds are independent of coordinates). Let g : Rn  Rn be a mapping of the form g(x)
=
Ax+c,
3.1.21
where A is an invertible matrix. If M C Rn is a smooth kdimensional manifold, then the direct image g(M) is also a smooth kdimensional manifold. Thus our definition of a manifold, which appeared to be tied to the coordinate system, is in fact independent of the choice of coordinates. In particular, if you rotate or translate a smooth manifold, the result is still a smooth manifold.
Parametrizations of manifolds
FIGURE 3.1.13. A curve in the plane, known by the parametrization
t
( f>
x=t2 sint)
y = 6 sin t cos t
·
So far we have considered manifolds as loci defined by sets of equations. Technically, such equations describe the manifold completely. In practice (as we saw in equation 3.1.5) such a description is not satisfying; the information is not in a form that can be understood as a picture of the manifold. There is another way to think of manifolds: parametrizations. It is generally far easier to get a picture of a curve or surface if you know it by a parametrization than if you know it by equations. It will generally take a computer milliseconds to compute the coordinates of enough points to give you a good picture of a parametrized curve or surface. When the word "parametrize" is used loosely, we can say that any map f : Rn  Rm parametrizes its image by the domain variable in Rn. The map 9:
FIGURE
3.1.14.
A curve in space, k(n~~:t b)y the parametrization t
fo>
s~ t
.
( t 2  sint )
t f+
3.1.22
6sintcost
parametrizes the curve shown in Figure 3.1.13. As the parameter t varies, a point in R 2 is chosen; g parametrizes the curve made up of all points in the image of g. Note that often when thinking of functions as parametrizations, the "maps to" notation f+ is used, but we could also write this parametriza. as g(t) = ( t 2 sint) t1on . t cos t .  6 sm The mappings t
~~::)
~)
~:~)
and ( f+ ( : : parametrize the at v2 cosu space curve shown in Figure 3.1.14 and the surface of Figure 3.1.15. The f+
(
298
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
most famous parametrization of surfaces parametrizes the unit sphere in IR. 3 by latitude u and longitude v:
(~)
>+
(
cosucosv)
cos~sinv
3.1.23
.
smu Using a computer, you can easily produce as many "parametrized" curves and surfaces as you like. If you fill in the blanks of t
>+ (
=),where each
blank represents a function of t, and ask a computer to plot it, it will draw a curve in the plane. H you fill in the blanks of ( FIGURE
3.1.15.
The surface "parametrized" by
( ~)
>+
u 3 cosv) ( u2 + v2 . v 2 cosu
Note that this is not a smooth surface; it intersects itself.
~) ~
(
=),
where
each blank represents a function of u and v, the computer will draw you a surface in JR. 3 . However, if we want a mapping to parametrize a manifold, we must be much more demanding. In particular, such a mapping must be one to one; we cannot allow our manifolds to intersect themselves, like the surface in Figure 3.1.15. Definition 3.1.18 (Parametrization of a manifold). A parametrization of a kdimensional manifold M C IR.n is a mapping 'Y : U C JR.k + M satisfying the following conditions:
In Chapters 5 and 6 we will need to parametrize our manifolds before we can integrate over them. Fortunately, for the purposes of integration we will be able to "relax" our definition of a parametrization, disregarding such trouble spots as the endpoints of the tubing or the curve going from the North Pole to the South Pole. Choosing even a local parametrization that is well adapted to the problem at hand is a difficult and important skill and exceedingly difficult to teach. When parametrizing a surface, it sometimes helps to have a computer draw the surface for you; then ask yourself whether there is some way to identify a point on the surface by coordinates (for example, using sine and cosine or latitude and longitude).
1. U is open.
2. 'Y is C 1 , one to one, and onto M. 3. [D1(u)] is one to one for every u E U. We will give some examples, but first, some comments. 1. Finding a parametrization for a manifold that you know by equations
is very hard, and often impossible. Just as it is rare for a manifold to be the graph of a single function, it is rare to find a map 'Y that meets the criteria of Definition 3.1.18 and parametrizes the entire manifold. A circle is not like an open interval: if you bend a strip of tubing into a circle, the two endpoints become a single point. A cylinder is not like an open subspace of the plane: if you roll up a piece of paper into a cylinder, two edges become a single line. Neither parametrization is one to one. The problem for the sphere is worse. The parametrization by latitude and longitude (equation 3.1.23) satisfies Definition 3.1.18 only if we remove the curve going from the North Pole to the South Pole through Greenwich (for example). There is no general rule for solving such problems. The one easy case is when a manifold is the graph of a single function. For example, the surface in the upper left of Figure 3.1.2
3.1 is the graph of the function parametrized by
f ( ~)
= x3

Manifolds
299
2xy 2 . This surface is
3.1.24 Whenever a manifold is the graph of a function f(x) William Thurston (19462012), arguably the best geometer of the twentieth century, said that the right way to know a manifold is from the inside. Imagine yourself inside the manifold, aiming a flashlight first at one spot, then another, through a mist or dust that makes the beam of light visible. If you point the flashlight straight ahead, will you see anything? Will anything be reflected back? Or will you see the light to your side? This approach is illustrated by the video "Not Knot", produced by The Geometry Center, University of Minnesota, and available from CRCPress.com.
One program that draws parametrized curves and surfaces is Graphing Calculator by Pacific Tech (www.pacifict.com), version 2.2 or higher. (It also draws surfaces given implicitly by an equation, which is much harder.)
parametrized by the mapping x
1+ (
y, it is
f~)).
2. Even proving that a candidate parametrization is one to one can be very hard. An example concerns minimal surfaces (a minimal surface is one that locally minimizes surface area among surfaces with the same boundary; the study of such surfaces was inspired by soap films and water droplets). Weierstrass found a very clever way to "parametrize" all minimal surfaces, but in the more than one hundred years since these mappings were proposed, no one has succeeded in giving a general method for checking that they are indeed one to one. Just showing that one such mapping is a parametrization is quite a feat. 3. Checking that a derivative is one to one is easy, and locally, the derivative is a good guide to the mapping it approximates. Thus if U is an open subset of ~k and '"'( : U ~ ~n is a smooth mapping such that [D1(u)] is one to one, then it is not necessarily true that 1(U) is a smooth kdimensional manifold. But it is true locally: if U is small enough, then the image of the corresponding '"'( will be a smooth manifold. Exercise 3.1.20 sketches how to prove this.
Example 3.1.19 (Parametrization of a curve). In the case of a curve, the open subset U c ~k of Definition 3.1.18 becomes an open interval I c R Think of I as a time interval; as you travel along the curve, the parametrization tells you where you are on the curve at a given time, as shown in Figure 3.1.16. In this interpretation, 1'(t) is the velocity vector; it is tangent to the curve at 1(t) and its length is the speed at which you are traveling at time t.
l
f)
t=4
R
_t=6 .
'
~
~t=l
t=5
t=O
FIGURE 3.1.16. We imagine a parametrized curve as an ant taking a walk in the plane or in space. The parametrization tells where the ant is at any given time.
300 When you get to Yenikiiy look at your mileage reading, then drive on towards Sariyer for exactly four more miles.Parametrizing the road to Sariyer by arc length (Eric Ambler, The Light of Day).
Tangents are discussed in detail in Section 3.2.
When a curve is parametrized, the requirement that 7' (t) =/:. 0 could also be stated in terms of linear independence: a single vector ii is linearly independent if ii =j:. 0.
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
Curves can also be parametrized by arc length (see Definition 3.9.5). The odometer in a car parametrizes curves by arc length. The requirement that [D7(u)] be one to one means '?'(t) :fa 0: since ,Y(t) = [D7(t)] is an n x 1 column matrix (i.e., vector), the linear transformation given by this matrix is one to one exactly when '?'(t) :fa 0. The function ')' : (0, 7r)
+
C defined by 7( 0) = (
~~~:)
parametrizes
the upper halfof the unit circle. As defined on the open interval (0, 7r), the function is bijective, and '?'(O) = (  ~~~:)
# ( ~).
Note that according
to Definition 3.1.18, this mapping is not a parametrization of the entire circle, since in order to make it one to one we must restrict its domain to (0, 27r); unfortunately, this restriction misses the point (
B).
6.
Example 3.1.20 (Parametrization of a surface). In the case of a surface in IR3 , the U of Definition 3.1.18 is an open subset of JR 2 , and [D7(u)] is a 3 x 2 matrix. Saying that it is one to one is the same as saying that D1 'Y and Dn are linearly independent. For instance, the mapping 'Y : (
0) r+
cp
(cosOcoscp) sin ~ cos cp smcp
3.1.25
defined on U = (0, 7r) x (0, 7r /2) parametrizes the quarter of the unit sphere x 2 + y 2 + z 2 = 1 where y, z > 0, which we will denote by M. The map 'Y is of class C 1 and the derivative Since the mapping 'Y : U + M in Example 3.1.20 is one to one and onto, it is invertible. This may seem surprising, since an element of U has two entries and an element of M has three. But both domain and codomain are two dimensional. It takes two numbers, not three, to specify a point in M. The key point is that the codomain of 'Y is not IR3 , or any open subset of IR3 ; it is a subset of the 2sphere.
[D7 (
! )J =
[
cos cp sin 0 cos 0 cos cp 0
 sin cp cos OJ  sin cp sin 0 coscp
3.1.26
is one to one because the entries in the third row ensure that the two columns are linearly independent. To check that 'Y is one to one and onto M, first note that the image of 'Y is part of the unit sphere, since cos 2 0 cos 2 cp + sin 2 0 cos 2 cp + sin2 cp = 1.
3.1.27
Since z E (0, 1), there exists a unique cp E (0,71"/2) such that sincp = z, and since x E (1, 1), there then exists a unique 0 E (0, 7r) such that x x x 3.1.28 cosO=   = ~== coscp JI  z2 +y2
Jx2
For 0 E (0,7r), we have sinO
> 0, soy= sinOcoscp > 0.
6.
Parametrizations versus equations If you know a curve by a global parametrization, it is easy to find points of the curve but difficult to check whether a given point is on the curve. If
you know the curve by an equation, the opposite is true: it may be difficult to find points of the curve, but easy to check whether a point is on the curve.
3.1 Manifolds
301
For example, given the parametrization
.t
'Y .
tt (
cos3 t  3 sin t cos t )
t2  t5
'
3.1.29
you can find a point by substituting some value of t, like t = 0 or t = 1.
b)
But checking whether a point ( is on the curve would require showing that there is a solution to the set of nonlinear equations
= cos3 t  3 sin t cos t b = t2  t5.
a
3.1.30
Now suppose you are given the equation
y + sinxy + cos(x + y) = 0,
3.1.31
which defines a different curve. You could check whether a given point is on the curve by inserting the values for x and y in the equation, but it's not clear how you would go about finding a point of the curve, i.e., solving one nonlinear equation in two variables. You might try fixing one variable and using Newton's method, but it isn't clear how to choose a starting point where convergence is guaranteed. For instance, you might ask whether the curve crosses the yaxis, i.e., set x = 0 and try to solve the equation y + cosy = 0. This has a solution by the intermediate value theorem: y + cosy is positive when y > 1 and negative when y < 1. So you might think that using Newton's method starting at y = 0 should converge to a root, but inequality 2.8.53 of Kantorovich's theorem is not satisfied. But starting at y
= tr/4 gives Mlf(yo)~ ~ 0.027 < ~(/'(Yo))
Relation between differential calculus and linear algebra Knowing a manifold by an equation F(z) = 0 is analogous to (and harder than) knowing the kernel of a linear transformation. Knowing a manifold by a parametrization is analogous to (and harder than) knowing the image of a linear transformation. This continues a theme of this book: each construction and technique in linear algebra has a (harder) counterpart in the nonlinear world of differential calculus, as summarized in Table 3.1.17.
Linear Algebra
Algorithms
Algebra
Geometry
row reduction
Theorem 2.2.6 Theorem 2.2.1
subspaces (fl.at) kernels images manifolds (curvy)
Differential Newton's method inverse function theorem Calculus implicit function theorem
defining manifolds by equations defining manifolds by a parametrizations
TABLE 3.1.17 Correspondences: linear algebra and differential calculus.
302
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
3.1
EXERCISES FOR SECTION
Exercise 3.1.1: Unless otherwise stated, the unit sphere is always centered at the origin. It has radius 1.
3.1.1
Use Theorem 3.1.10 to show that the unit sphere is a smooth surface.
3.1.2
Show that the set { (
3.1.3
Show that every straight line in the plane is a smooth curve.
sina = 0 if and only if a for some integer k.
=
x2 
Hint for Exercise 3.1.8, part d: See Figure 3.9.2.
I x + x 2 + y2 = 2 }
is a smooth curve.
y2 ,
±)1 
x2 
and
z2
± y'l y2 
z 2.
3.1.5 a. For what values of c is the set of equation Xe = x 2 + y3 = c a smooth curve? b. Sketch these curves for a representative sample of values of c.
3.1.6
What do"' the •urlace of
, i.e., a1 x 1 , where I = (2, 3) and a( 2 ,3) = 3. The higher partial derivative D~ D~p is written D( 2 ,3 )P· Since I= (2, 3), we have I! = 2!3! = 12. Proposition 3.3.11 says
a1
=
;,D1p(O);
here,
;,D(
2,3 )p(O)
= ~~ = 3,
which is indeed a( 2 ,3 )·
What if the multiexponent I for the higher partial derivatives is different from the multiexponent J for x? As mentioned in the proof of Proposition 3.3.11, the result is 0. For example, if we take D~D~ of the polynomial p = 3x~x~, so that I = (2, 2) and J = (2, 3), we get 36x2 ; evaluated at 0, this becomes 0. If any index of I is greater than the corresponding index of J, the result is also O; for example, what is D1p(O) when I = (2, 3), p = aJXJ, aJ = 3, and J = (2, 2)? 10 6.
Taylor polynomials in higher dimensions Although the polynomial in equation 3.3.26 is called the Taylor polynomial of f at a, it is evaluated at a+ h, and its value there depends on h, the increment to a. Recall (Definition 1.9.7) that a function is k times continuously differentiable: all of its partial derivatives up to order k exist and are continuous on U.
ck
Now we are ready to define Taylor polynomials in higher dimensions and to see in what sense they can be used to approximate functions in n variables. Definition 3.3.13 {Taylor polynomial in higher dimensions). Let U c Rn be an open subset and let f : U  t R be a Ck function. Then the polynomial of degree k k k
...
def """"
P1,a(a+h) =
""""
L.,, L.,,
1 ...I J!Dd(a)h
3.3.26
m=OIEI::'
is called the Taylor polynomial of degree k of f at a. If f: U  t Rn is a Ck function, its Taylor polynomial is the polynomial map U  t Rn whose coordinate functions are the Taylor polynomials of the coordinate functions of f. 10 This
corresponds to D~D~(3x~x~); already, D~(3x~x~)
= 0.
3.3
Taylor polynomials in several variables
321
In equation 3.3.26, remember that I is a multiexponent; if you want to write the polynomial out in particular cases, it can get complicated, especially if k or n is big. Example 3.3.14 below illustrates notation; it has no mathematical content. Note that the first term (the term of degree m = O) corresponds to the 0th derivative, i.e., the function f itself. Example 3.3.14 (Multiexponent notation for a Taylor polynomial of a function in two variables). Let f: "JR 2 t "JR be a function. The formula for the Taylor polynomial of degree 2 of f at a is then Example 3.3.14: Remember (Definition 3.3.6) that
= x~1 x~"; ii1 = h~1 ••• h~".
x1
•••
similarly, For instance, if I= (1, 1) we have
ii1 = iif(a)h1h2 =
which we can write more simply as P12,a(a + h)
1
The term 1/I! in the formula for the Taylor polynomial weights the various terms to take into account" the existence of crossed partials. This advantage of multiexponent notation is increasingly useful as n gets big.
3.3.28
1 2 + D(1,1)!(a)h1h2 + 2D(o,2)! 1 ()2 + 2D(2,o)!(a)h a h 2. 1
1
2D1D2f(a)h1h2
+ 2D2Dif(a)h1h
= f(a) + D(l,o)i(a)h1 + D(o, 1)!(a)h2
Remember that D(l,o)!
= Dif, while D(o,l)f = D2f, and so on.
6.
What are the terms of degree 2 (second derivatives) of the Taylor polynomial at a, of degree 2, of a function with three variables? 11
Example 3.3.15 {Computing a Taylor polynomial). Taylor polynomial of degree 2 of the function
What is the
f ( ~) = sin(x + y 2 ), at
0 = (8)? The first term, of degree 0, is f(O) = sinO = 0. The terms of
degree 1 are D( 1,0 )! ( ~)
11 The
third term of
= cos(x + y 2) and D(o,1)! ( ~) = 2y cos(x + y 2),
322
Chapter 3.
so D(i,o)! (
Manifolds, Taylor polynomials, quadratic forms, curvature
g) = 1 and D(o,l)f ( g) = 0. For the terms of degree 2, D(2,o)! ( ~) =  sin(x + y 2 ) D(l,i)! ( ~) = 2y sin(x + y 2 ) D(o,2)! (
~)
2 cos(x + y 2 )
=

3.3.29 4y 2 sin(x + y 2 );
at 0, these give 0, 0, and 2. So the Taylor polynomial of degree 2 is 3.3.30 For the Taylor polynomial of degree 3, we compute In Example 3.4.5 we will see how to reduce this computation to two lines, using rules we will give for computing Taylor polynomials.
D(3,o)! ( ~) =Di ( sin(x + y 2 )) =  cos(x + y 2 ) D(o,3)! (
~)
= 4y sin(x
D(2,i)! (
~)
= Di (2y sin(x + y 2 ))
D(i,2)! (
~)
= 2 sin(x
+ y2 )
8y sin(x + y 2 )

+ y2 ) 

8y 3 cos(x + y 2 )
= 2y cos(x + y2 )
3.3.31
4y 2 cos(x + y 2 ).
At 0 all are 0 except D( 3 ,o), which is 1. So the term of degree 3 is )h~ =  ~ h~, and the Taylor polynomial of degree 3 of f at 0 is
(ti
6.
In Theorem 3.3.16, note that since we are dividing by a high power of lhl and ii is small, the limit being 0 means that the numerator is very small. We must require that the partial derivatives be continuous; if they aren't, the statement isn't true even when k = 1, as you will see if you look at equation 1.9.10, where f is the function of Example 1.9.5, a function whose partial derivatives are not continuous. Taylor's theorem with remainder is discussed in Appendix Al2.
3.3.32
Theorem 3.3.16 (Taylor's theorem without remainder in higher dimensions). Let U C Rn be open, a E U a point, and f : U + R a
Ck function. 1. The polynomial PJ,a (a + h) is the unique polynomial of degree k with the same partial derivatives up to order k at a as f. 2. It best approximates f near a: it is the unique polynomial of degree at most k such that
.
hm ii+o
f(a+h)PJa(a+h) ....
JhJk
'
= 0.
3.3.33
To prove Theorem 3.3.16 we need the following proposition, which says that if all the partial derivatives of a function g up to some order k equal 0 at a point a, then g is small compared to lhlk in a neighborhood of a. It is proved in Appendix AlO.
3.3 Taylor polynomials in several variables
323
Proposition 3.3.17 (Size of a function with many vanishing partial derivatives). Let U be an open subset of Rn and let g: U+ Ill be a Ck function. If at a E U all partial derivatives of g up to order k vanish (including g(a), the 0th partial derivative), then lim g(a_+ ii) = O.
3.3.34
lhlk
iio
Proof of Theorem 3.3.16. 1. Let Q},a be the polynomial that, evaluated ii, gives the same result as the Taylor polynomial PJ,a evaluated at a+ii:
at Equation 3.3.35: The right side follows the general multiexponent notation for a polynomial given by formula 3.3.8:
multiexponent notation for polynomial Q,,a k
L L
L L
a1x1.
m=OIEI;:'
~!DJf(a) iiJ .
3.3.35
m=O J EI;:'..__......., '..." coefficient variable
k
By Proposition 3.3.11,
Here, h corresponds to x, and JrDJf(a) corresponds to aJ.
D1p(O)
........
Proposition 3.3.11: for a polynomial p with coefficient a1, we have
1
D1Q},a(O) =I!
=
I!D1f(a)
___....
D1f(a).
3.3.36
coeff. of h 1 from eq. 3.3.35
Since D1Q},a(O) = D1PJ,a(a), we have D1PJ,a(a) = Dif(a); the partial derivatives of PJ,a, up to order k, are the same as the partial derivatives of f, up to order k. Part 2 then follows from Proposition 3.3.17. To lighten notation, denote by g(a +ii) the difference between f(a +ii) and the Taylor polynomial of f at a. Since all partials of g up to order k vanish, by Proposition 3.3.17, lim g(a_+ ii) h+O lhlk
Compute D1 (D2f), D2(D3f), D3(D1f), and
t;on f definOO by
f
G) ~
3.3.37
3.3
EXERCISES FOR SECTION 3.3.1
= 0. D
D1 ( D2(D3f))
for the func
x'y + xy' + yz'. 5
3.3.2
a. Write out the polynomial
L L
a1x1 ,
where
m=OIEI'f{'
aco.o,o)
= 4,
ac2,2,oi
= 1,
U(0,1,0) ac3,o,2)
= 3, = 2,
U(l,0,2)
= 4,
a(5,o,o)
= 3,
U(l,1,2)
= 2,
324
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
and all other ar = 0, for I E I;' for m $ 5. b. Write the polynomial 2x2 + x1 x2  X1 X2X3 + x~ + 5x~x3 using multiexponent notation. c. Write 3x1x2 
X2X3X4
+ 2x~x3 + x~x~ + x~
using multiexponent notation.
3.3.3 Following the format of Example 3.3.14, write the terms of the Taylor polynomial of degree 2 of a function f of three variables x, y, z at a point a. 3.3.4
a. Redo Example 3.3.14, finding the Taylor polynomial of degree 3.
b. Repeat, for degree 4.
3.3.5
Find the cardinality (number of elements) of I;:'.
3.3.6
a.
Let
f be a realvalued Ck function defined in a neighborhood of
= f(x). Consider the Taylor polynomial of f at O; show that all the coefficients of even powers (a 0 , a2, ... ) vanish. 0 E R. Suppose f(x)
b. Let f be a realvalued Ck function defined in a neighborhood of 0 E Rn. Suppose f(x) =  f(x). Consider the Taylor polynomial of f at the origin; show that the coefficients of all terms of even total degree vanish.
3.3. 7
Show that if IE T.:', then (xh) 1
= xmhI.
3.3.8 For the function f of Example 3.3.9, show that all first and second partial derivatives exist everywhere, that the first partial derivatives are continuous, and that the second partial derivatives are not. 3.3.9
Consider the function
x 2 y(x y)
f
(
xy )
x'f
={
+ y2
0
The object of Exercise 3.3.10 is to illustrate how long successive derivatives become.
a. Compute Dif and D2f. Is
b. Show that all second partial derivatives of f exist everywhere. c. Show that D1 ( D2f (
L'Hopital's rule (used in Exercise 3.3.11): If f, g: U > R are k times differentiable in a neighborhood U C R of 0, and
= · · · = lk 1>(0) = 0, g(O) = · · · = g 3" in the second line includes terms of degree > 3 from (x
Manifolds, Taylor polynomials, quadratic forms, curvature
Remark. In the proof of Theorem 5.4.6, we will see a striking example of how Proposition 3.4.3 can simplify computations even when the final goal is not to compute a Taylor polynomial. In that proof we compute the derivative of a function by breaking it up into its component parts, computing the Taylor polynomials to degree 1 of those parts, combining them to get the Taylor polynomial to degree 1 of the function, and then discarding the constant terms. Using Taylor polynomials allows us to discard irrelevant !::::,. higherdegree terms before doing our computations.
So it is reasonable to substitute that "something small" for the increment u when evaluating the polynomial PJ,b(b + u). Exercise 3.26 asks you to use Proposition 3.4.4 to derive Leibniz's rule. Note that the composition of two polynomials is a polynomial. For example, if f (x) = x 3 + x + 1 and g(y) = y 2  y, then
Chapter 3.
sin(x + y 2 ) = (x
+ y2)a
+ y2 ) x + y2
6
(x + y2)3 6
+
x3 

6
terms of degree > 3
+ terms of degree> 3.
3.4.13
~
Taylor polynomial degree 3
in the first line.
Presto: Half a page becomes two lines.
6
Example 3.4.6 (A harder example). Let U C R be open, and let + R be of class C 2 • Let V c U x U be the subset of R 2 where f(x) + f(y) =f. 0. Compute the Taylor polynomial of degree 2, at a point
f :U Equation 3.4.15, line 2: When computing the Taylor polynomial of a quotient, a good tactic is to factor out the constant terms and apply equation 3.4.9 to what remains. Here, the constant term is
f(a)
+ f(b).
F
(b) E V, of the function F : V R given by F (x) _ 1 Y  f(x) + f(y) · +
bt
Set ( ~) = ( ~). First we write f(a + u) and f(b the Taylor polynomial off evaluated at a+ u and b + v:
( a+u) b+v
3.4.14
+ v)
in terms of
1
= (f(a)+f'(a)u+f"(a)u2/2+o(u 2))+(f(b)+f'(b)v+f"(b)v2/2+o(v 2)) (l+:z:) 1 , where :z: is the fraction in the denominator
f(a)
+ f(b)
.,,,.....,.,...,.,,.,1==.,.,...,..::::::n::+o(u2 +v2) f'(a)u + f"(a)u 2/2 + f'(b)v + f"(b)v 2/2 · 1+ f(a) + f(b)
3.4.15
3.4
Rules for computing Taylor polynomials
The second factor is of the form 12 (1 + x) 1 = 1 x
F(a+u) b+ v
+(
1
=
f(a)
+ f(b)
(l
+ x2 
329
···,leading to
f'(a)u+f"(a)u 2 /2+f'(b)v+J"(b)v 2 /2 f(a) + f(b)
f'(a)u + f"(a)u 2 /2 + f'(b)v f(a) + f(b)
+ f"(b)v 2 /2) 2 _
... )
.
3.4.16
6.
3.4.17
We discard the terms of degree> 2, to find
p2
F.(:)
(a+u) b+v
=
1
f'(a)u+ f'(b)v
f(a)+f(b)
(!(a)+ f(b)) 2
f"(a)u 2 + J"(b)v 2 2{f(a) + f(b)) 2
(f'(a)u+ J'(b)v) 2
~'~,.+':'
(!(a)+ f(b)) 3
Taylor polynomials of implicit functions
It follows from Theorem 3.4. 7 that if you write the Taylor polynomial of the implicit function with undetermined coefficients, insert it into the equation specifying the implicit function, and identify like terms, you will be able to determine the coefficients. We do this in the proof of Theorem 3.9.10, in Appendix Al5.
We are particularly interested in Taylor polynomials of functions given by the inverse and implicit function theorems. Although these functions are only known via some limit process like Newton's method, their Taylor polynomials can be computed algebraically. Assume we are in the setting of the implicit function theorem, where F is a function from a neighborhood of c = ( E Rn+m to Rn, and g : V + Rn is an implicit function defined on an open subset V C Rm
b)
containing b, with g(b) =a, and such that F ( g~)) = 0 for ally EV.
Theorem 3.4.7 (Taylor polynomial of implicit function). Let F be a function of class Ck for some k ~ 1, sucli that F ( = 0. Then
b)
the implicit function g is also of class Ck, and its Taylor polynomial of degree k, P;,b : Rm + Rn, satisfies In Example 3.4.8 we use equation 2.10.29 to compute
g'(l)
pk
F,(:)
= a1 = 1,
but this was not necessary; we could have used undetermined coefficients. If we had written 1 + a1h+ a2h 2
for x rather than 1  h + a2h 2 in equation 3.4.22, the vanishing of the linear terms would have imposed a1 = 1.
(
pk
g,b
(b+h)) 
E

o(lhlk).
3.4.18
b+h
It is the unique polynomial map of degree at most k that does so. Proof. By Proposition 3.4.4, it is enough to show that g is of class Ck. By equation 2.10.29, if F is of class Ck, then [Dg] is of class ck 1 , hence g is of class Ck. D Example 3.4.8 (Taylor polynomial of an implicit function). The equation F (:) 12 The
equation (1
+ x) 1 =
= x 3 + xy + y 3 1 x
+ x2 
We saw this in Example 0.5.6, where we had
· ••
f
n=O
3=0
3.4.19
is equation 3.4.9 with m = 1.
arn = 1 ~ r.
330
Of course, equation 3.4.19 also implicitly expresses y in terms of x near the same point.
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
implicitly expresses x in terms of y near ( ~), since at that point the derivative [3x 2 + y, 3y 2 + x] is [4, 4], and 4 is invertible. So there exists
g such that g(l)
= 1, i.e., (
g~))
=
0).
By equation 2.10.29, we have
g'(l) = 1. We will compute Pff, 1 , the Taylor polynomial at 1 of this implicit function g to degree 2: g(l + h)
= g(l) + g'(l)h + ~g"(l)h 2 + · · · = ~ + · · ·.
3.4.20
Thus we want to compute a2 in equation 3.4.20. We will write F ( Pff' 1 (1+h)) Eo (h 2) ,
substituting F for Equation 3.4.22: It is handy to know that
P;·U)
3.4.21
l+h
in equation 3.4.18; this is allowed since they
coincide through quadratic terms. This leads to

(1  h + a2h 2)3 + (1  h + a2h2)(h + 1) + (1 + h) 3 3 E o(h2).
(1+x) 3 =1+3x + 3x 2 + x 3 •
xy
:z:3
3.4.22
y3
If we multiply out, the constant and the linear terms on the left vanish; the quadratic terms are
3.4.23 so to satisfy equation 3.4.22 we need (4a 2+5)h 2 = 0 (i.e., a2 = 5/4). Thus g"(l) = 5/2, and the Taylor polynomial at 1 of the implicit function g is 1 h its graph is shown in Figure 3.4.2.
45 h 2 + ...
;
3.4.24
6
Example 3.4.9 (Taylor polynomial of implicit function: a harder
FIGURE 3.4.2. The darker curve is the curve of equation F(x) = 0, where
F(x)
= x 3 + xy + y 3 
~) ~ x
2
+ y3 + xyz3 
3.
The other curve, which is a parabola, is the graph of the Taylor polynomial of degree 2 of the implicit function, which represents x as a function of y. Near (
example. The equation F (
~), the
two graphs are almost identical.
"" a function of x, y in a neighborhood of (
3
~ 0 determines z
t) ,
since D,F ( : )
Thus there exis"' a function g such that F (
9
(~) ) ~
~ 3 # O.
O. Let us
compute the Taylor polynomial of the implicit function g to degree 2. We set
3.4 Rules for computing Taylor polynomials Inserting this expression for z into x 2
(1 + u) 2 +(1 + v) 3 + ( (l+u)(l+v) ( 1 + al,ou + ao,1v +
+ y 3 + xyz 3 
331
3 = O leads to
a~,o u 2 + a 1,1uv + a~, 2 v 2) 3 )3 E o(u2+v2).
Now we multiply out and identify like terms. We get
Constant terms: The linear terms could have been derived from equation 2.10.29, which gives in this case
[nu(}) J
33 = 0.
Linear terms: 2u + 3v + u + v + 3a1,ou + 3ao,1 v = 0,
i.e., Quadratic terms:
= [3] 1 [3,4]
al,O
= 1, ao,1 = 4/3.
u 2 ( 1+3a1,o + 3(a1,0) 2 +
= [1/3][3, 4] = [1, 4/3].
~a2,o) + v 2 ( 3 + 3ao,1 + 3(ao,1) 2 + ~ao,2)
+ uv(l + 3a1,o + 3ao,1 + 6a1,oao,1 + 3a1,1) = 0.
3.4.26
Identifying the coefficients of u 2, v 2, and uv to 0, and using a 1 ,0 ao,1 = 4/3 gives
=
1 and
a2,0 = 2/3,
3.4.27
ao,2 = 26/9,
al,1 = 2/3. Finally, this gives the Taylor polynomial of g:
1(x1)~(y1)~(x1) 2  193 (y1) 2 ~(xl)(y1).
P 2 () = g,
~
Therefore, P2 g,
g (x)
y
(
1) : Taylor polynomial of degree 2 of g at (
~)
1
= 1 (x 1)  ~(y 1)  ~(x 1) 2 3
+
3.4.28
3

13 (y 1) 2 9
o ((x  1) 2 + (y1) 2 ). length squared of [ :::::
~]
, small near

~(x  l)(y  1) 3 3.4.29
U)
EXERCISES FOR SECTION 3.4 Exercise 3.4.2, part a: The techniques of this section make this easy; see Example 3.4.5. Hint for part b: Use the Taylor polynomial of 1/(1 + u).
3.4.1
Write, to degree 2, the Taylor polynomial off (
~) = Jl + sin(x + y)
at the origin. 3.4.2
a. What is the Taylor polynomial of degree 5 of sin(x+y 2 ) at the origin?
b. What is the Taylor polynomial of degree 4of1/(1 +x2 +y2 ) at the origin?
332 Exercise 3.4.3: Notice how much easier this exercise is with the techniques of Section 3.4 than it was without, in Exercise 3.3.13.
Chapter 3.
3.4.3
Find the Taylor polynomial of degree 2 of the function
f ( ~) = v'x + y + xy 3.4.4
1h
f
is C 3 ,
h(af(O) + bf(h) + cf(2h)) 1h f(t) dt
af(O) + bf(h) + cf(2h) 3.4.5
Find numbers a, b, c such that when
f
f(t) dt
as Taylor polynomials of degree 3 in h, and equate coefficients.
=~).
at the point (
Find numbers a, b, c such that when
Exercise 3.4.4: Develop both
and
Manifolds, Taylor polynomials, quadratic forms, curvature
h ( af(O) + bf(h) + cf(2h)) 
o(h 3 ).
E
is C 3 ,
fo
2
h
f(t) dt E o(h3 )
(same hint as for Exercise 3.4.4.) 3.4.6 Find a Taylor polynomial at the point x = 0 for the Fresnel integral cp(x) = sin t2 dt that approximates cp(l) with an error smaller than 0.001.
J;
3.4. 7
Show that the equation z
neru the point ( • Exercise 3.4.8 generalizes Exercise 3.3.14. The equations
Dih1  D2h2 D2h1
=0
+ Dih2 =
0
are known as the CauchyRiemann equations. Hint for Exercise 3.4.9: Use the error estimate for alternating series.
2
as a function g of x, y
) . Find the Tuylm polynomial of degree 2 of g at ( • { 2 ) .
(We owe this exercise, and a number of others in Chapters 3 and 4, to Robert Piche.) 3.4.8 Prove that if f : IR2 > JR satisfies Laplace's equation DU+ DU = 0 and h : IR2 > IR2 satisfies the equations Dih1  D2h2 = 0 and D2h1 + Dih2 = 0, then f oh satisfies Laplace's equation. 3.4.9
Find a Taylor polynomial at x = 0 for the error function erf(x) =
~
1"'
et 2 dt
that approximates erf(0.5) with an error smaller than 0.001. 3.4.10
Exercise 3.4.11 is much easier with the techniques of Section 3.4 than it was without, in Exercise 3.3.12.
f
= sin(xyz) determines z
3.4.11
Prove the formulas of Proposition 3.4.2. a. Write, to degree 3, the Taylor polynomial of
J+ 1
+ y at the 1x+xz
origin.
h. What;,, 3.4.12
3.5
v,•.•.•11
(g)?
Prove the general case of Proposition 3.2.12 (substituting GP for C 1 ).
QUADRATIC FORMS
A quadratic form is a polynomial all of whose terms are of degree 2. For instance, x~ + x~ and 4x~ + x1x2  x~ are quadratic forms, as is X1X3. But x1x2x3 is not a quadratic form; it is a cubic form.
3.5 Quadratic forms
The margin note next to Exercise 3.5.13 defines a quadratic form on an abstract vector space. A quadratic form in one dimension is simply an expression of the form ax 2 , with a E JR. Example 3.5.2: The map Q is a function of the coefficients of p. If p = x 2 + 2x + 1, then
Q(a)~Q =
m
1 1
(x 2
+ 2x +
Definition 3.5.1 (Quadratic form). A quadratic form Q: Rn+ JR is a polynomial function in the variables xi, ... , Xn, all of whose terms are of degree 2. Although we will spend much of this section working on quadratic forms that look like x 2 + y 2 or 4x 2 + xy  y 2, the following is a more realistic example. Most often, the quadratic forms one encounters in practice are integrals of functions, often functions in higher dimensions.
Example 3.5.2 (An integral as a quadratic form). Let p be the polynomial p(t) = ao + a 1 t + a 2 t 2 , and let Q: JR 3 +JR be the function Q(a)
1) 2 dx.
333
=
1 1
(p(t)) 2 dt.
3.5.1
Then Q is a quadratic form, as we can confirm by computing the integral:
1
1 The quadratic form of Example Q(a) = (ao +alt+ a2t 2) 2 dt 3.5.2 is fundamental in physics. If a tight string of length 1 (think 1 2 + 4 + 2aoa1t + 2aoa2t 2 + 2a1a2t3) dt of a guitar string) is pulled away = from its equilibrium position, then its new position is the graph of a function f, and the potential = + [ 3 + [ 5 + [2ao;1t2 + [2a0; 2t 3 energy of the string is
1(a~+ a~t [a~t]~
Q(f)
=
11
(f(x))2 dx.
Similarly, the energy of an electromagnetic field is the integral of the square of the field, so if p is the electromagnetic field, Q(p) gives the amount of energy between 0 and 1. In contrast to quadratic forms,
no one knows anything about cubic forms. This has ramifications for the understanding of manifolds. The abstract, algebraic view of a fourdimensional manifold is that it is a quadratic form over the integers; because integral quadratic forms are so well understood, a great deal of progress has been made in understanding 4manifolds. But even the foremost researchers don't know how to approach sixdimensional manifolds; that would require knowing something about cubic forms.
2
= ao
a~
a~t
a~3t J: a~
J:
a~5t J:
2aoa2
3.5.2
J: +
4 [2a1: 2t
J:
al a2
+ 3 + 5 + aoa1 + 3 + 2.
Above, p is polynomial of degree 2, but the function given by equation 3.5.1 is a quadratic form if p is a polynomial of any degree, not just quadratic. This is obvious if p is linear; if a 2 = 0, equation 3.5.2 becomes Q(a) =a~+ aU3 + aoa1. Exercise 3.5.14, part casks you to show that Q is a quadratic form if p is a cubic polynomial. /::,,. In various guises, quadratic forms have been an important part of mathematics since the ancient Greeks. The quadratic formula, always the centerpiece of high school math, is one aspect. A much deeper problem is the question, what whole numbers can be written in the form x 2 + y 2 ? Of course, any number a can be written ...[0,2 + 02 , but suppose you impose that x and y be whole numbers. For instance, 5 can be written as a sum of two squares, 22 + 12 , but 3 and 7 cannot. The classification of quadratic forms over the integers is thus a deep and difficult problem, though now reasonably well understood. But the classification over the reals, where we are allowed to extract square roots of positive numbers, is relatively easy. We will be discussing quadratic forms over the reals. In particular, we will be interested in classifying such quadratic forms by associating to each quadratic form two integers, together called its signature. In Section 3.6 we will see that signatures of quadratic
334
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
forms can be used to analyze the behavior of a function at a critical point: a point where the derivative of the function is O.
Quadratic forms as sums of squares Essentially everything there is to say about real quadratic forms is summed up by Theorem 3.5.3, which says that a quadratic form can be represented as a sum of squares of linearly independent linear functions of the variables. The term "sum of squares" is traditional; it might be more accurate to call it a combination of squares, since some squares may be subtracted rather than added.
Theorem 3.5.3 (Quadratic forms as sums of squares). 1. For any quadratic form Q: !Rn+ JR, there exist m = k+l linearly
independent linear functions a 1 , ... , am : !Rn
+
JR such that
Q(x) = (a1(x)) 2+· · ·+ (ak(x)) 2  (ak+1(x)) 2  · · · (ak+1(x))2. 3.5.3 Theorem 3.5.3: The Cti are row matrices: elements of the vector space Mat (1, n). Thus the notion of linear independence makes sense. Moreover, Mat (1, n) has dimension n, so m :::; n, since there can't be more than n linearly independent elements of a vector space of dimension n. More than one quadratic form can have the same signature. The quadratic forms in Examples 3.5.6 and 3.5.7 have signature (2, 1). The signature of a quadratic form is sometimes called its type. A famous theorem due to Fermat asserts that a prime number p =f. 2 can be written as a sum of two squares if and only if the remainder after dividing p by 4 is 1. The proof of this and a world of analogous results (due to Fermat, Euler, Lagrange, Legendre, Gauss, Dirichlet, Kronecker, ... ) led to algebraic number theory and the development of abstract algebra.
2. The number k of plus signs and the number l of minus signs depends only on Q and not on the specific linear functions chosen.
Definition S.5.4 (Signature). The signature of a quadratic form is the pair of integers (k, l). The word "signature" suggests, correctly, that the signature remains unchanged regardless of how the quadratic form is decomposed into a sum of linearly independent linear functions; it suggests, incorrectly, that the signature identifies a quadratic form. Before giving a proof, or even a precise definition of the terms involved, we want to give some examples of the main technique used in the proof; a careful look at these examples should make the proof almost redundant.
Completing squares to prove the quadratic formula The proof is provided by an algorithm for finding the linearly independent functions ai: completing squares. This technique is used in high school to prove the quadratic formula. Indeed, to solve ax 2 + bx + c = 0, write
ax 2
+ bx + c = ax 2 + bx + ( 2 ~) 2 
(
2 ~)
2
+ c = 0,
3.5.4
which gives
( Jax
b )
+ 2J(i
2
b2
= 4a 
c.
3.5.5
3.5
Quadratic forms
335
Taking square roots gives r;:
b
v ax + 2y'a =
±Vb
2 
4ac 4a '
leading to the famous formula
b± ..../b2 4ac 2a
x=
3.5.6
3.5.7
Example 3.5.5 (Quadratic form as a sum of squares). Since
x 2 + xy = x 2 + xy + ~y 2 Equations 3.5.9: functions 01 (
~) =
x+
~'
Clearly the
02 (
1 [0
1/2] 1/2 .
It is not necessary to row reduce
this matrix to see that the rows are linearly independent.
(x
+ ~) 2  (~)2,
the quadratic form Q(x) = x 2 + xy can be written as (a 1 (x)) 2 where 0:1 and 0:2 are the linear functions
~) = ~
are linearly independent: no multiple of y/2 can give x + y/2. If we like, we can be systematic and write these functions as rows of a matrix:
~y 2 =

0:1
(~) = x + ~
and
0:2
(~) = ~·

3.5.8
(a 2 (x)) 2 , 3.5.9
Express the quadratic form x 2 + xy  y 2 as a sum of squares, checking your answer in the footnote. 13 Example 3.5.6 (Completing squares: a more complicated example). Consider the quadratic form Q(x) = x 2
+ 2xy 4xz + 2yz 
3.5.10
4z 2 •
We take all the terms in which x appears, which gives us x 2 + (2y  4z)x. Since x 2 + (2y  4z)x + (y  2z) 2 = (x + y  2z) 2 , adding and subtracting (y  2z) 2 yields Q(x) = (x + y  2z) 2 = (x
+y 

(y 2

4yz + 4z 2 )
+ 2yz 
4z 2
2z) 2 y2 + 6yz  8z2 .
3.5.11
Collecting all remaining terms in which y appears and completing the square gives Q(x) = (x + y  2z) 2
In this case, the linear functions are
13

(y  3z) 2
+ (z) 2 •
3.5.12
336
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
This decomposition of Q(x) is not the only possible one. Exercise 3.5.1 6 asks you to derive an alternative decomposition. The algorithm for completing squares should be pretty clear: as long as the square of some coordinate function actually figures in the expression, every appearance of that variable can be incorporated into a perfect square; by subtracting off that perfect square, you are left with a quadratic form in precisely one fewer variable. (The "precisely one fewer variable" guarantees linear independence.) This works when there is at least one square, but what should you do with something like the following?
Example 3.5.7 (Quadratic form with no squares). quadratic form
There wasn't anything magical about the choice of u, as Exercise 3.5.2 asks you to show; almost anything would have done.
Q(x) = xy  xz + yz. One possibility is to introduce the new variable u can trade x for u + y, getting
(u + y)y (u + y)z + yz
= y 2 + uy uz = = (y = (
+ ~) 2 
(
(y
=
Consider the
3.5.14 x  y, so that we
2
u)2  u +2 4
uz  z 2 + z 2
~ + z) 2 + z2
~ + ~ )2  (~  ~ + z) 2 + z2 •
3.5.15
Again, to check that the functions 3.5.16 are linearly independent, we can write them as rows of a matrix:
1/2 [ 1/2
1/2 OJ 1
1/2
0
0
3.5.17
row reduces to
1
Theorem 3.5.3 says that a quadratic form can be expressed as a sum of linearly independent functions of its variables, but it does not say that whenever a quadratic form is expressed as a sum of squares, those squares are necessarily linearly independent.
Example 3.5.8 (Squares that are not linearly independent). We can write 2x 2 + 2y 2 + 2xy as
x' +y' + (x + y)'
°' os
(
v"ix +
}.) '+ (
f. y)'
3.5.18
Only the second decomposition reflects Theorem 3.5.3. In the first, the three functions [ dependent.
6
~]
f+
x, [ ~]
f+
y, and [
~]
f+
x
+y
are linearly
3.5
Quadratic forms
337
Proof of Theorem 3.5.3 All the essential ideas for the proof of part 1 are contained in the examples; a formal proof is in Appendix A13. To prove part 2, which says that the signature (k, l) of a quadratic form does not depend on the specific linear functions chosen for its decomposition, we need to introduce some new vocabulary and a proposition. Definition 3.5.9 (Positive and negative definite). A quadratic form Q is positive definite if Q(x) > 0 when x "# 0. It is negative definite if Q(x) < 0 when x "# 0. Definition 3.5.9 is equivalent to saying that a quadratic form on Jr is positive definite if its signature is (n,O) (as Exercise 3.5.7 asks you to show) and negative definite if its signature is (0, n). The fact that the quadratic form of Example 3.5.10 is negative definite means that the Laplacian in one dimension (i.e., the transformation that takes p to p") is negative. This has important ramifications; for example, it leads to stable equilibria in elasticity.
An example of a positive definite quadratic form is the quadratic form Q(x)
= lxl 2 . Another is
Q(p)
=
fo (p(t)) 1
2
dt, which we saw in Example
3.5.2. Example 3.5.10 gives an important example of a negative definite quadratic form. Example 3.5.10 (Negative definite quadratic form). Let Pk be the space of polynomials of degree ::; k, and let Va,b c Pk be the space of polynomials p that vanish at a and b for some a < b. Consider the quadratic form Q : Va,b + JR given by
Q(p) =
1b
p(t)p"(t) dt.
3.5.19
Using integration by parts,
Q(p)
=
1b
=0 by def.
p(t)p"(t) dt
= p(b)p'(b)  p(a)p'(a)
1b
(p'(t)) 2 dt < 0.
3.5.20
Since p E Va,b, by definition p(a) = p(b) = O; the integral is negative unless p' = 0 (i.e., unless pis constant); the only constant in Va,b is 0. D:. Recall from Theorem 3.5.3 that when a quadratic form is written as a "sum" of squares of linearly independent functions, k is the number of squares preceded by a plus sign, and l is the number of squares preceded by a minus sign.
Proposition 3.5.11. The number k is the largest dimension of a subspace oflR.n on which Q is positive definite and the number l is the largest dimension of a subspace on which Q is negative definite. Proof. First let us show that Q cannot be positive definite on any subspace of dimension > k. Suppose 3.5.21 k terms
l terms
is a decomposition of Q into squares of linearly independent linear functions, and that W C JR.n is a subspace of dimension k1 > k.
338
Chapter 3. Manifolds, Taylor polynomials, quadratic forms, curvature
Consider the linear transformation W ~ JR.k given by w...
[0:1~w)l
1t
.
3.5.22
•
o:k(w)
"Nontrivial" kernel means the kernel is not {O}.
Since the domain has dimension kl, which is greater than the dimension k of the codomain, this mapping has a nontrivial kernel. Let w "! 0 be an element of this kernel. Then, since the terms (o: 1(w)) 2 + ... + (o:k(w)) 2 vanish, we have 3.5.23 So Q cannot be positive definite on any subspace of dimension > k. Now we need to exhibit a subspace of dimension k on which Q is positive definite. We have k + l linearly independent linear functions o: 1 , ... , O:k+l · Add to this set linear functions O:k+l+I, ... , O:n such that 0:1, ... , O:n form a maximal family of linearly independent linear functions; that is, they form a basis of the space of 1 x n row matrices (see Exercise 2.6.9). Consider the linear transformation T : !Rn ~ !Rnk given by
...
T:xtt
[ o:k+.1 (x) l .
.
3.5.24
o:n(x) The rows of the matrix corresponding to T are the linearly independent row matrices O:k+l> ••• , o:n; like Q, they are defined on !Rn, so the matrix T is n wide. It is nk tall. Let us see that ker T has dimension k, and is thus a subspace of dimension k on which Q is positive definite. By Proposition 2.5.11, the rank ofT is equal to nk (the number of its linearly independent rows). So by the dimension formula, dimker T + dimimgT = n,
i.e.,
dimker T
= k.
3.5.25
For any v E ker T, the terms o:k+ 1(v), ... ,o:k+1(v) of Q(v) vanish, so
Q(v) = (0:1(v)) 2
+ ... + (o:k(v)) 2 ::::: o.
3.5.26
If Q(v) = 0, this means that every term is zero, so
0:1(v) = · · · = o:n(v) = o, which implies that v = 0. So if argument for l is identical. D Corollary 3.5.12 says that the signature of a quadratic form is independent of coordinates.
v "! 0,
3.5.27
then Q is strictly positive. The
Corollary 3.5.12. If Q: !Rn  t JR is a quadratic form and A: !Rn  t !Rn is invertible, Q o A is a quadratic form with the same signature as Q. Proof of Theorem 3.5.3, part 2. Since the proof of Proposition 3.5.11 says nothing about any particular choice of decomposition, we see that k and l depend only on the quadratic form, not on the particular linearly independent functions we use to represent it as a sum of squares. D
3.5
Quadratic forms
339
Classification of quadratic forms
3.5.5 has rank 2; the quadratic form of Example 3.5.6 has rank 3.
Definition 3.5.13 {Rank of a quadratic form). The rank of a quadratic form is the number of linearly independent squares that appear when the quadratic form is represented as a sum of linearly independent squares.
It follows from Exercise 3.5. 7 that only nondegenerate forms can be positive definite or negative definite.
Definition 3.5.14 {Degenerate and nondegenerate quadratic forms). A quadratic form on !Rn with rank m is nondegenerate if m = n. It is degenerate if m < n.
The quadratic form of Example
The examples we have seen so far in this section are all nondegenerate; a degenerate quadratic form is shown in Example 3.5.16. The following proposition is important; we will use it to prove Theorem 3.6.8 about using quadratic forms to classify critical points of functions. Proposition 3.5.15 applies just as well to negative definite quadratic forms; just use C and :::;. Another proof (shorter and less constructive) is sketched in Exercise 3.5.16.
Proposition 3.5.15. IEQ: !Rn t JR is a positive definite quadratic form, then there exists a constant C > 0 such that
Q(x) ~ Clxl 2 for all x E !Rn.
3.5.28
Proof. Since Q has rank n, we can write Q(x) as a sum of squares of n linearly independent functions:
Q(x) = (a1(x)) 2 + · · · + (an(x)) 2.
3.5.29
The linear transformation T : !Rn t !Rn whose rows are the ai is invertible. Since Q is positive definite, all the squares in equation 3.5.29 are preceded by plus signs, and we can consider Q(x) as the length squared of the vector Tx. Since !xi= 1r 1Txl:::; 1r111rx1,
1x12 Q( x) = 1r12 x > 1r112, so we can take C =
1/IT1 2 . 1
3.. 5 30
D
Example 3.5.16 (Degenerate quadratic form). The quadratic form
Q(p)
=
11
(p'(t))2 dt
3.5.31
on the space Pk of polynomials of degree at most k is a degenerate quadratic form, because Q vanishes on the constant polynomials. 6.
Quadratic forms and symmetric matrices Recall (Definition 1.2.18) that a symmetric matrix is a matrix that is equal to its transpose.
In many treatments of quadratic forms, quadratic polynomials are seldom mentioned; instead quadratic forms are viewed in terms of symmetric matrices. This leads to a treatment of the signature in terms of eigenvalues
340
Example of equation 3.5.32:
x
Ax
§ G~ ~l [::J quadratic form
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
and eigenvectors rather than in terms of completing squares (we discuss this in Section 3.7; see Theorem 3.7.16). We think this is a mistake: completing squares involves only arithmetic; finding eigenvalues and eigenvectors is much harder. But we will need Proposition 3.5.17 in Section 3.7. If A is an n x n matrix, then the function QA defined by
QA (x)
def 
=X·
AX
3.5.32
is a quadratic form on Rn. A particular quadratic form can be given by many different square matrices, but by only one symmetric matrix. Thus we can identity the set of quadratic forms with the set of symmetric matrices.
Equation 3.5.32 is identical to QA(x) =
x:T Ax.
Note that
QTT AT(x)
= xTrT ATx
= (Tx) T A(Tx) = QA(Tx) = (QA 0 T)x
Proposition 3.5.17 (Quadratic forms and symmetric matrices). The mapping A 1+ QA is a bijective map from the space of symmetric n x n matrices to the space of quadratic forms on Rn. The symmetric matrix associated to a quadratic form is constructed as follows: each diagonal entry ai,i is the coefficient of the corresponding variable squared in the quadratic form (i.e., the coefficient of while each entry ai,j is onehalf the coefficient of the term XiXj· For the quadratic form in the margin, the corresponding matrix has entry a 1 , 1 = 1 because the coefficient of is 1, while a2,l = a 1 ,2 = 0 because the coefficient of x 2x 1 = x 1x2 is 0. Exercise 3.16 asks you to turn this into a formal proof.
xn
xi
EXERCISES FOR SECTION
3. 5
3.5.1 For the quadratic form Q(x) = x 2 +2xy4xz+2yz4z 2 of Example 3.5.6, what decomposition into a sum of squares do you find if you start by eliminating the terms in z, then the terms in y, and finally the terms in x? 3.5.2 Consider the quadratic form of Example 3.5. 7: Q(x) = xy  xz + yz. Decompose Q(x) with a different choice of u, to support the statement that u = x  y was not a magical choice. 3.5.3 Decompose each of the following quadratic forms by completing squares, and determine its signature. a. x 2
+ xy 
3y2
b. x 2
+ 2xy 
y2
c. x 2
+ xy + yz
3.5.4 Are the following quadratic forms degenerate or nondegenerate? Are they positive definite, negative definite, or neither? a. x 2 + 4xy + 4y 2 on JR.2 b. x 2 + 2xy + 2y2 + 2yz + z 2 on R 3 c. 2x2 + 2y 2 + z 2 + w 2 + 4xy + 2xz  2xw  2yw on R4 3.5.5 What is the signature of the following quadratic forms? a. x 2 + xy on R2 b. xy + yz on R3 3.5.6
Confirm that the symmetric matrix A= [
~
1/2
0
0 1/2
1/2] 1/2 1
3.5 represents the quadratic form Q = x 2
Exercise 3.5. 7: The main point is to prove that if the quadratic form Q has signature (k, O} with k < n, there is a vector v =f. 0 such that Q(v) = 0. You can find such a vector using the transformation T of formula 3.5.24. Exercise 3.5.8: Recall from Example 2.10.18 that the trace of a square matrix A, denoted tr A, is the sum of its diagonal elements. Exercise 3.5.9: In special relativity, spacetime is a 4dimensional vector space with a quadratic form of signature (1, 3}. This quadratic form assigns length 0 to any two events connected by a light ray. Thus the space of 2 x 2 Hermitian matrices is a useful model for spacetime.
Exercise 3.5.11: See the note for Exercise 3.5.8.
Q(v)
3.5.8
yz  z 2 •
ldentil'y
(!)
Ell' with the upper triangulruc matrix M
~ [ ~ !l·
a. What is the signature of the quadratic form Q(M) = tr(M 2 }? What kind of surface in IR3 do you get by setting tr(M 2 ) = 1? b. What is the signature of the quadratic form Q(M) kind of surface in IR3 do you get by setting tr(MT M) = 1? 3.5.9
= tr( MT M)?
What
Consider the vector space of 2 x 2 matrices such that
H=[b~ic
b1;/]'
where a, b, c, d are real numbers. (Such a matrix, whose complex conjugate is equal to its transpose, is called a Hermitian matrix.) What is the signature of the quadratic form Q(H) = det H? 3.5.10 For each of the following equations, determine the signature of the quadratic form represented by the left side. Where possible, sketch the curve or surface represented by the equation.
a. x 2 + xy  y 2 c. x 2 + xy + yz
=1 =1
+ 2xy  y 2 = xy+yz = 1
b. x 2
d.
1
3.5.11 a. Let A be a 2 x 2 matrix. Compute tr A 2 and tr AT A, and show that both are quadratic forms on Mat (2, 2).
b. What are their signatures? What is the signature of X1X2
+ x2xa + · · · + Xn1Xn
on !Rn?
3.5.13 Let V be a vector space. A symmetric bilinear function on V is a mapping B : V x V + JR such that 1. B(av1 + bv2, w) = aB(v1, w) a, b E JR;
+ bB(v2, w)
for all v1, v2, w E V and
2. B(v, w) = B(w, v) for all v, w E V.
= B(v, v)
for some symmetric bilinear function B : V x V + R
341
3.5. 7 Show that a quadratic form on !Rn is positive definite if and only if its signature is (n, 0).
*3.5.12 Exercise 3.5.13 tells us what a quadratic form on an abstract vector space should be: it is a map Q : V + JR of the form
+ xz 
Quadratic forms
a. Show that a function B : !Rn x Rn + JR is symmetric and bilinear if and only if there exists a symmetric matrix A such that B(v, w) = vT Aw. b. Show that every quadratic form on Rn is of the form vi+ B(v, v) for some symmetric bilinear function B. c. Let Pk be the space of polynomials of degree at most k. Show that the function B : Pk x Pk + JR given by B(p, q) = f01 p(t)q(t) dt is a symmetric bilinear function. by
d. Denote by p 1 (t) = 1, p2(t) = t, ... , Pk+i(t) = tk the usual basis of Pk, and ~P the corresponding "concrete to abstract" linear transformation. Show that ~p(a), ~p(b)) is a symmetric bilinear function on Rn, and find its matrix.
B(
3.5.14 a. If p( t) = a 0 + a 1 t, show that Q( a) and write it explicitly.
= f 01 (p( t)) 2 dt is a quadratic form,
342
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
b. Repeat for p(t) = ao +alt+ a 2t 2. c. Repeat for p(t) = ao +alt+ a2t 2 + a3t 3. 3.5.15 a. Let Pk be the space of polynomials of degree at most k. Show that the function
Q(p) =
fo (p(t)) 1
2 
(p'(t)) 2 dt
b. What is the signature of Q when k
is a quadratic form on Pk.
= 2?
Jr
3.5.16 Here is an alternative proof of Proposition 3.5.15. Let Q : + R be a positive definite quadratic form. Show that there exists a constant C > 0 such that Q(x) ~ Cl:Xl 2 for all E Rn, as follows: a. Let sn 1 ~r {x E Rnl l:XI = l}. Show that snl is compact, so there exists Xo E snl with Q(:Xo) :::; Q(x) for all E sn 1 .
x
x
b. Show that Q(:Xo) > 0. c. Use the formula Q(x)
= l:Xl 2 Q ( l:I)
to prove Proposition 3.5.15.
3.5.17 Identify (for instance, as an ellipse, hyperbola, ... ) and sketch the conic sections and quadratic surfaces of equation :XT Q(x) = 1, where Q is one of the following matrices:
.. [~ i l rn ~ ~ b
3.5.18
l ' rn
~ _fl
d.
u
! ~fl
Let A be a real n x m matrix, and define M
=
e [
~ ~l
AT A. Show that m.
x . . . . :XT Mx is a positive definite quadratic form if and only if rankA =
3.6 CLASSIFYING CRITICAL POINTS OF FUNCTIONS Minima and maxima of functions (Definitions 1.6.6 and 1.6.8) are both known as extrema; the singular is extremum. Part 2: "Strict local minimum" means that there exists a neighborhood V C U of xo such that f(xo) < f(x) for all x EV  {xo}. Part 3: "Strict local maximum" means that there exists a neighborhood V C U of xo such that f(xo) > f(x) for all x EV  {xo}. Theorem 3.6.1 elaborates on Proposition 1.6.12.
In onevariable calculus we find the maximum or minimum of a function by looking for places where the derivative vanishes. Theorem 3.6.1 (Extrema of functions of one variable). Let U c JR be an open interval and f : U  JR a differentiable function.
1. If xo E U is a local minimum or maximum off, then f'(xo) = 0. 2. If f is twice differentiable, and f'(xo) = 0 and f"(xo) x 0 is a strict local minimum off.
> 0, then
3. If f is twice differentiable, and f'(xo) = 0 and f"(xo) x 0 is a strict local maximum off.
< 0, then
Note that Theorem 3.6.1 says nothing about the degenerate case, where f'(xo) = 0 and f"(xo) = 0.
3.6
Classifying critical points
343
For a function of several variables, classifying critical points  points where the derivative vanishes  is more complicated: a nondegenerate critical point may be a local maximum, a local minimum, or a saddle (see Figure 3.6.1). In this section we see how to classify nondegenerate critical points of such a function, using the quadratic terms of its Taylor polynomial. Definition 3.6.2 (Critical point, critical value). Let Uc Rn be an open subset and let f : U + R be a differentiable function. A critical point of f is a point where the derivative vanishes. The value of f at a critical point is a critical value. Part 1 of Theorem 3.6.1 generalizes to functions of several variables in the most obvious way:
The derivative [Df(xo)] in Theorem 3.6.3 is a row matrix n wide.
Theorem 3.6.3 (Derivative zero at extremum). Let U c Rn be an open subset and let f : U + R be a differentiable function. IE :xo E U is a local minimum or maximum off, then [Df(xo)) =[OJ. Proof. Since the derivative is given by the Jacobian matrix, it is enough to show that if xo is a local extremum of f, then Dd(xo) = 0 for all i = 1, ... , n. But Dd(xo) = g' (0), where g is the function of one variable g(t) = /(xo+tei), and our hypothesis also implies that t = 0 is an extremum of g, so g'(O) = 0 by Theorem 3.6.1. D
Finding critical points
FIGURE
3.6.1.
The graph of x 2  y 2 , a typical saddle. Near a saddle point, a surface is like a mountain pass, with mountains to either side, a valley one has left behind, and a different valley in front.
Theorem 3.6.3 says that when looking for maxima or minima, the first step is to look for critical points, places where the derivative vanishes. For the derivative of a function f : U C Rn + R to vanish means that all the partial derivatives vanish. Thus finding critical points means solving [D f (x)) = [OJ, a system of n equations inn unknowns. Usually there is no better way to find critical points than applying Newton's method; finding critical points is an important application of Newton's method. Example 3.6.4 (Finding critical points). What are the critical points of the function f: R 2 + R given by
f ( ~)
= x + x 2 + xy + y 3 ?
3.6.1
The partial derivatives are Dif ( ~)
= 1+2x +
3.6.2
y,
In this case we don't need Newton's method, since the system can be solved explicitly. Substitute x = 3y2 from the second equation into the first, to find 1+y6y2 =0,
which gives
y =
1±v'l+24 1 1 =  or  . 12 2 3
3.6.3
344
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
Substituting this into x critical points
=
(1
+ y)/2
(or into x
=
3y2 ) gives the two
 (3/4) 1/2
3.6.4
ai 
In Section 3.7, we will show how to analyze the behavior of a function restricted to the boundary.
Remark 3.6.5 (Critical points on closed sets). A major problem in using Theorem 3.6.3 is the hypothesis that U is open. Seeking a minimum or maximum by finding places where the derivative vanishes will usually not work if the critical point is on the boundary. For instance, the maximum value of x 2 on [O, 2] is 4, which occurs at x = 2, not a point where the derivative of x 2 vanishes. Especially when we have used Theorem 1.6.9 to assert that a maximum or minimum exists in a compact subset U, we need to check that the extremum occurs in the interior U before we can say that it is a critical point. 6
The second derivative criterion
Equation 3.6.5: In evaluating the second derivative, remember that DU(a) is the second partial derivative DiDif, evaluated at a. In this case we have
DU
=2
and
DiD2f
= 1.
These are constants, so where we evaluate the derivative doesn't matter. But DU = 6y; evaluated at a1 this gives 3.
Is either critical point in equation 3.6.4 an extremum? In one variable, we would answer this question by looking at the sign of the second derivative (Theorem 3.6.1). The right generalization of "the second derivative" to higher dimensions is "the quadratic form given by the terms of degree 2 of the Taylor polynomial", and the right generalization of "sign of the second derivative" is "signature of the quadratic form", which we will also call the signature of the critical point. Since a sufficiently differentiable function f is well approximated near a critical point by its Taylor polynomial, it seems reasonable to hope that f should behave like its Taylor polynomial. Evaluating the Taylor polynomial of the function f ( ~) = x+x 2 +xy+y3 at the critical point a 1 =
(~~~),
we get 3.6.5 terms of degree 2
The terms of degree 2 form a positive definite quadratic form: with signature (2, 0).
3.6.6
How should we interpret this? The quadratic form is positive definite, so (by Proposition 3.5.15) it has a local minimum at ii = 0, and if P},a 1 approximates f sufficiently well near a 1 , then f should have a minimum at a 1 . (A neighborhood of ii= 0 for the Taylor polynomial corresponds to a neighborhood of a 1 for f .) Similarly, if the signature of a critical point is (0, n), we would expect the critical point to be a maximum.
3.6 Classifying critical points What happens at the critical point a 2 = (
=~~D?
345
Check below. 14
Definition 3.6.6 (Signature of critical point). Let U c Rn be an open set, let f : U + R be of class C 2 , and let a E U be a critical point of f. The signature of the critical point a is the signature of the quadratic Definition 3.6.6: In order to define the signature of a critical point, the function f must be at least C 2 ; we know of no theory of classification of critical points of functions that are only differentiable or of class C 1 . The matrix H of second partials is the Hessian matrix off at x. One can think of it as the "second derivative" of f; it is really only meaningful when x is a critical point off. Note that a Hessian matrix is always square and (because cross partials are equal) symmetric.
form 3.6.7
The quadratic form in equation 3.6. 7 is the seconddegree term of the Taylor polynomial off at a (see Definition 3.3.13). Recall (Proposition 3.5.17) that every quadratic form is associated to a unique symmetric matrix. The quadratic form Qf,a (h) is associated to the matrix composed of second partial derivatives of f: if we define H by 3.6.8 A critical point a is called degenerate or nondegenerate precisely when the quadratic form Qf,a is degenerate or nondegenerate. Proposition 3.6. 7 says that the signature of a critical point is independent of coordinates.
Proposition 3.6. 7. Let U, V c Rn be open, let
O
for
t2
/(a+ th) < /(a)
we have
3.6.16
'
Jtl > 0 sufficiently small.
Thus a is not a local minimum.
D
Proof of Theorem 3.6.10 (Behavior of functions near saddles). As in equations 3.6.11 and 3.6.12, write
A similar argument about W shows that there are also points c where f(c) < /(a). Exercise 3.6.3 asks you to spell out this argument.
/(a+ h) =/(a)+ QJ,a(h)
+ r(h)
lim
with
h>0
By DefiJition 3.6.9, there exist vectors and QJ,a(k) < 0. Then
/(a+ th)  /(a) t2
=
r~h) lhl 2
= 0.
3.6.17
Ji and k such that Q1,a(h) >
t 2QJ,a(h) + r(th) t2
=
Q
f,a
(h)
+
r(th) t2
0
3.6.18
is strictly positive for t > 0 sufficiently small, and
/(a+ tk)  /(a) = t 2QJ,a(k) + r(th) = Q (k) r(tk) t2 t2 f,a + t2 is negative for t "I 0 sufficiently small. D
3.6.19
Theorems 3.6.8 and 3.6.10 say that near a critical point a, the function "behaves like" the quadratic form Qf,a (although Theorem 3.6.10 is a weak sense of "behaves like"). The statement can be improved: Morse's lemma15 says that when a is nondegenerate, there is a change of variables cp as in Proposition 3.6.7 such that f o cp = Qf,a·
f
Degenerate critical points Degenerate critical points are "exceptional", just as zeros of the first and second derivative of functions of one variable do not usually coincide. We 15 See lemma 2.2 on page 6 of J. Milnor, Morse Theory, Princeton University Press, 1963.
348
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
will not attempt to classify them (it is a big job, perhaps impossible), but simply give some examples. Example 3.6.11 (Degenerate critical points). The three functions x 2 + y 3 , x 2 + y 4 , and x 2  y 4 all have the same degenerate quadratic form for the Taylor polynomial of degree 2: x 2 . But they behave very differently, as shown in Figure 3.6.2 and the left surface of Figure 3.6.3. The function shown in Figure 3.6.2, bottom, has a minimum; the other three do not. 6.
FIGURE 3.6.2.
TOP: The surface of equation z = x2 + y3 . BOTTOM: The surface of equation z = x 2 + y 4 •
FIGURE 3.6.3. LEFT: The surface of equation z
= x2

y 4 • Although its graph
look very different from those shown in Figure 3.6.2, all three functions have the same degenerate quadratic form for the Taylor polynomial of degree 2. RIGHT: The monkey saddle; it is the graph of z = x 3  2xy 2 , whose quadratic form is 0. The graph goes up in three directions and down in three also (two for the legs, one for the tail).
EXERCISES FOR SECTION
3.6.1
a.
Sh~ that f
G) ~
3.6
x' + xy + z'
 = y hM a critkal point atth•
origin. b. What kind of critical point does it have? 3.6.2
a. Find the critical points of the function
f ( ~)
= x3

12xy + 8y3 .
b. Determine the nature of each of the critical points. 3.6.3 Complete the proof of Theorem 3.6.10, showing that if f has a saddle at a E U, then in every neighborhood of a there are points c with f(c) < f(a). 3.6.4 Use Newton's method (preferably by computer) to find the critical points of x3 + y 3 + xy + 4x  5y. Classify them, still using the computer.
3. 7 Constrained critical points and Lagrange multipliers
3.6.5
a. Find the eritical points of the funetion f (
349
~) ~ xy + yz ~ xz + xyz.
b. Determine the nature of each critical point. 3.6.6
Find all the critical points of the following functions: 8
b. xy + x
a. sinxcosy
+ 1 y
*c. sinx + siny + sin(x + y) 3.6.7
a. Find the critical points of the function f (
~) = (x2 + y2 )e"' 2 Y2 .
b. Determine the nature of each critical point. 3.6.8
a. Write the Taylor polynomial of f (
~)
=
Jl  x + y2 to degree 3 at
the origin. b. Show that
g (
~)
=
Jl  x + y2 + x/2 has a critical point at the origin.
What kind of critical point is it?
3. 7
Finding shortest paths goes under the name of the calculus of variations. The set of paths from New York to Paris is an infinitedimensional manifold. We will be restricting ourselves to finitedimensional problems. But the tools we develop apply quite well to the infinitedimensional setting. Another example occurs in Section 2.9: the norm sup
lxl=l
IAXI
of a matrix A answers the question, what is sup IAxl when we require that x have length 1?
CONSTRAINED CRITICAL POINTS AND LAGRANGE MULTIPLIERS
The shortest path between two points is a straight line. But what is. the shortest path if you are restricted to paths that lie on a sphere (for example, because you are flying from New York to Paris)? This example is intuitively clear but quite difficult to address. Here we will look at easier problems in the same spirit. We will be interested in extrema of a function f when f is either defined on a manifold X C Rn or restricted to it. In the case of the set X C R 8 describing the position of four linked rods in the plane (Example 3.1.8), we might imagine that each of the four joints connecting the rods at the vertices Xi is connected to the origin by a rubber band, and that the vertex Xi has 2 • Then what is the equilibrium position, where the link a "potential" Ix\ 1 realizes the minimum of the potential energy? Of course, all four vertices try to be at the origin, but they can't. Where will they go? In this case the function "sum of the lxil 2 " is defined on the ambient space, but there are important functions that are not, such as the curvature of a surface. In this section we provide tools to answer this sort of question.
Finding constrained critical points using derivatives Recall that in Section 3.2 we defined the derivative of a function defined on a manifold. Thus we can make the obvious generalization of Definition 3.6.2.
350
The derivative [Df(x)] off is only defined on the tangent space TxX, so saying that it is 0 is saying that it vanishes on tangent vectors toX.
Analyzing critical points of functions f : X > R isn't quite the focus of this section; we are really concerned with functions g defined on a neighborhood of a manifold X and studying critical points of the restriction of g to X. Traditionally a critical point of g restricted to X is called a constrained critical point.
Chapter 3. Manifolds, Taylor polynomials, quadratic forms, curvature
Definition 3.7.1 (Critical point of function defined on manifold). Let X c Rn be a manifold, and let f : X  t R be a C 1 function. Then a critical point off is a point x EX where [D/(x)] = 0. An important special case is when f is defined not just on X but on an open neighborhood U c Rn of X; in that case we are looking for critical points of the restriction fix off to X.
Theorem 3.7.2. Let X
f:
Ra C 1 function, and c E X a local extremum off. Then c is a critical point off. C Rn be a manifold,
X
t
Proof. Let "I : V  t X be a parametrization of a neighborhood of c E X, with 1(xo) = c. Then c is an extremum off precisely if xo is an extremum off o "I· By Theorem 3.6.3, [D(f o "t)(x0 )] = [O], so, by Proposition 3.2.11, [D(f o "f)(xo)]
= [Df("t(xo))][D1(xo)] = [Df(c)][D1(Xo)] = [OJ.
By Proposition 3.2.7, the image of [D1(xo)] is TeX, so [D/(c)] vanishes on TeX. The proof is illustrated by Figure 3.7.1.
r TT TTT I
I I I I I
i
I

' I
elx1 I I I
1
f
_"I_.
The parametrization
X; f takes it to R. An extremum of the composition f o 'Y corresponds to an extremum off. D
x
~
FIGURE 3.7.1
'Y takes a point in R2 to the manifold
I
We already used the idea of the proof in Example 2.9.9, where we found the maximum of
IAxl
restricted to the unit circle, for A = [ ~
~] .
In Examples 3.7.3 and 3.7.4
we know a critical point to begin with, and we show that equation 3.7.1 is satisfied: TeX
C
ker [Df(c)].
Theorem 3.7.5 will show how to find critical points of a function restricted to a manifold (rather than defined on the manifold, as in Definition 3.7.1), when the manifold is known by an equation F(z) = 0.
Examples 3.7.3 and 3.7.4 illustrate constrained critical points. They show how to check that a maximum or minimum is indeed a critical point satisfying Definition 3. 7.1. Suppose a manifold X is defined by the equation F(z) = 0, where F : U c Rn  t Rnk has onto derivative [DF(x)] for all x E Un X, and suppose f: U  t Risa C 1 function. Then Definition 3.7.1 says that c is a critical point of f restricted to X if
v Thm. 3.2.4
ker [DF(c)]
.$.
ker [D/(c)].
3.7.1
Def. 3.7.1
Note that both derivatives in formula 3.7.1 have the same width, as they must for that equation to make sense; [DF(c)] is a (n  k) x n matrix, and [D/(c)] is a 1 x n matrix, so both can be evaluated on a vector in Rn. It
3.7 Constrained critical points and Lagrange multipliers
A space with more constraints is smaller than a space with fewer constraints: more people belong to the set of musicians than belong to the set of redheaded, lefthanded cello players with last name beginning with W.
351
also makes sense that the kernel of the taller matrix should be a subset of the kernel of the shorter matrix. Saying that E ker [DF(c)] means that
l[v l [0] v
D1F1(c)
[
DnF~k(c) v~
D1Fn:k(c)
3.7.2
1
DnF1(c)
n  k equations need to be satisfied. Saying that
=
b;
v E ker [Df(c)]
means
that only one equation needs to be satisfied:
3.7.3
[Dd(c) ··· Dnf(c)] [ ] = O.
Equation 3.7.1 says that at a critical point c, any vector that satisfies equation 3.7.2 also satisfies equation 3.7.3.
Example 3.7.3 (Constrained critical point: a simple example). Suppose we wish to maximize the function
f ( ~ ) = xy on the first quad
rant of the circle x 2 + y 2 = 1, which we will denote by X. As shown in Figure 3. 7.2, some level sets of that function do not intersect the circle, and some intersect it in two points, but one, xy = 1/2, intersects it at the point FIGURE 3.7.2. The unit circle and several level curves of the function xy. The level curve xy = 1/2, which realizes the maximum value of xy restricted to the circle, is tangent to the circle at the point (~j~), where the maximum value is realized.
c
= G~ ~).
To show that c is the critical point of f constrained to X, we
need to show that TeX C ker [Df(c)]. Since F ( ~) = x 2
+ y2 
1 is the function defining the circle, we have
TeX= ker [DF(c)]
= ker [2c1, 2c2] = ker
ker [Df(c)]
[22J v'2' v'2
= ker [c2, c1] = ker [~ v'2' ~] v'2 .
3.7.4 3.7.5
Clearly the kernels of both [D F (c)] and [D f (c)] consist of vectors [ such that v1 = v2 • In particular, TeX critical point of f restricted to X. 6.
Equation 3.7.6: See Example 3.1.14, which discusses a curve in space as the intersection of two surfaces.
~~ ]
c ker [Df(c)], showing that c is a
Example 3. 7 .4 (Constrained critical point in higher dimensions). Let us find the minimum of the function f(x) = x~ + x~ + x~, when it is constrained to the ellipse X that is the intersection of the cylinder x~ + x~ = 1 and the plane of equation x 1 = x 3 , as shown in Figure 3.7.3. Since f measures the square of the distance from the origin, we are looking for the points on the ellipse that are closest to the origin. Clearly they a..1, ... , >..m. So together we have n + m equations in n + m unknowns.
while [D f
( ~ ) J = [1, l] .
So at a critical point, there will exist
[1, 1] = ,\[2x, 4y];
i.e.,
x = 2\;
>. such that
y = 4\ .
3.7.11
Inserting these values into the equation for the ellipse gives
4 ~ 2 + 2 161,\
2
= 1,
>. =
which gives
±/[;
3.7.12
inserting these values for,\ in equation 3.7.11 gives 1
f8
1
f8
x = ± 2V3 and y = ± 4V3· So the maximum of the function on the ellipse is
~~+~~=~~= 2V3 4V3 4V3 .._..,,....,, .._..,,....,, x
The value,\=
fF. v2·
3.7.13
y
j'i gives the minimum, x + y = ..,/372.
6.
Example 3. 7. 7 (Critical points of function constrained to ellipse). Let us follow this procedure for the function f(x) = x 2 +y 2 +z 2 of Example 3.7.4, constrained as before to the ellipse given by FIGURE
3. 7.4.
JosephLouis Lagrange was born in Italy in 1736, one of 11 children, nine of whom died before reaching adulthood. He was in Paris during the French Revolution; in 1793, the great chemist AntoineLaurent Lavoisier (who was guillotined the following year) intervened to prevent his arrest as a foreigner . In 1808 Napoleon named him Count of the Empire. Lagrange's book Mecanique analytique set the standard in the subject for a hundred years.
F(x) = F
(x) ;
= (x2 + x ~2z 1) = 0.
3.7.14
We have
[Df(x)] = [2x,2y,2z], [DF1(x)] = [2x,2y,O], [DF2(x)] = [1,0,1], so Theorem 3.7.5 says that at a critical point there exist numbers ,\ 1 and ,\ 2 such that [2x, 2y, 2z] = ,\ 1[2x, 2y, O] + A2 [1, 0, 1], which gives
2x = ,\ 12x + A2,
2y = A12Y + 0,
rr
2z = A1 · 0  A2.
3.7.15
an:;,:: ::g:~:·.~.: ~ ::::,±:d~ (f) ::: F:,:..: cal points. Butll y = 0, then F 1 = 0 and F, = 0 giW> x = z = ±I. So
G)
354
Chapter 3.
and (
=D
Manifolds, Taylor polynomials, quadratic forms, curvature
are also critical points. F°' the first, )" = 2 and ,\ 2 = 2; f°'
the second, A1 = 2 and A2 = 2. Since the ellipse is compact, the maximum and minimum values of f restricted to F = 0 are attained at constrained critical points of f. Since
f
m (0 =f
=
I and
f
m CD =f
a.  y2'
~] z2 '
3.7.45
leading to the four equations
x
2
=
>.
~,
y
2
>. = ;;·
>.
2
1 1 1 ++=1. x y z
z =, c
3.7.46
In order for solutions to this system of equations to exist, we must have either >. > 0 and a < 0, b < 0, c < 0, or >. < 0 and a > 0, b > 0, c > 0. These two possibilities aren't really different: they correspond to changing f to /, which has the same critical points, with signature {p, q) changed to (q,p). We will assume>.< 0 and a> 0, b > 0, c > 0. Without loss of generality, we can uniquely write the solutions
x
R
= 81 Va '
y
R
R
= 82 Vb ' z = 83 Vc '
with all square roots > 0, and where each constraint becomes
Si
3.7.47
is either +1 or 1. The 3.7.48
so a solution with signs s 1, s2, s3 exists if and only if s1 fo+s2Vb+s3JC > 0. If none of the three numbers JO,, Vb, JC is larger than the sum of the other two, there are four solutions: all Si are +1, or two are +1 and the other is 1. The corresponding solutions have either all coordinates positive, or two positive and one negative. If one of the three numbers JO,, Vb, JC is larger than the sum of the other two, say JO, > Vb+ JC, then we must have s1 > 0, and s2 and 83 can be anything. Thus there are still four solutions, one with all coordinates positive, two with two coordinates positive and one negative, and one with one coordinate positive and two negative.
362
Equation 3. 7.49: Here m and F is the function x1 + y1 + z1 1,
=
1
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
In the "degenerate" case where one number equals the sum of the other two, say ya = Vb + VC,, there are only three solutions: the combination s1 = +1, s2 = s3 = 1 is impossible. Now let us analyze the nature of these critical points. Here n = 2 and m = 1; the augmented Hessian matrix is the 4 x 4 matrix
[ ~
so the entry hi,i of H is
2.X DiDif(a)  .XiDiDiF = 0  3, x where a is a critical point of f restricted to F = 0.
H =
*]
~
O
O
O 0 ~
1r ~
~
~
~
0
3.7.49
where x, y, z, .A are the coordinates of one of the critical points found above. This matrix corresponds to the quadratic form  2..x A 2
We discussed how to construct the symmetric matrix associated to a quadratic form immediately after Proposition 3.5.17. Here we go in the other direction. The A, B, C, D in equation 3.7.50 are the coordinates of a vector corresponding to the vector x in equation 3.5.32:
~>.
0
x3
=

2..x B 2 y3
2..xc2

z3
+~AD+ 2.BD + 2.cD x2 y2 z2
2 )  2_ (~B 2 ~ (~A 2 AD+ .!:_n x2 x 4.A y2 y
 2_ (~c 2 z2
z

CD+ .!....D2 ) 4.A
+ _!_ 2.A
BD + .JLD2 ) . 4.A
(.!. + _!.y + .!.) D x
3.7.50
2
z
The four summands on the right side are all plus or minus perfect squares. The first three terms have the same sign as the corresponding coordinate of the critical point since .A < 0. 16 Because ~ + + ~ = 1 and .A < 0, the last square comes with a negative sign. Thus the signature (p1, Q1) of the quadratic form represented by the augmented Hessian matrix can be
fi
1. (3, 1) in the case where all coordinates of the critical point are pos
itive. In that case the signature of the constrained critical point is (2, 0), so the critical point is a minimum 2. (2, 2) in the case where two coordinates of the critical point are positive and one is negative. In that case the signature of the constrained critical point is (1, 1), so the critical point is a saddle 3. (1, 3). In the case where one coordinate of the critical point is positive and two are negative; in that case the signature of the constrained critical point is (0, 2), so the critical point is a maximum. Note that this occurs only if one of the numbers ya, Vb, VC, is bigger than the sum of the other two 6.
By Theorem 3.7.13, if the signature of the quadratic form associated to the augmented Hessian matrix is (pi, qi), then the signature of the quadratic form associated to a constrained critical point is (pi  m, qi  m). Here, m = 1.
The spectral theorem for symmetric matrices Now we will prove what is probably the most important theorem of linear algebra. It goes under many names: the spectral theorem, the principal axis theorem, Sylvester's principle of inertia. i 6 For
2
x2 (
instance, if x
J=I. A
A+
> 0, the first summand can be written
2~D ) 2 ; if x < 0, it can be written 1
x
.
2
x2
(I§A A  21 ~x D )2.
3. 7 Constrained critical points and Lagrange multipliers
We use ,\ to denote both eigenvalues and the Lagrange multipliers of Theorem 3.7.5. When working on a problem involving constrained critical points, Lagrange used ,\ to denote what we now call Lagrange multipliers. At that time, linear algebra did not exist, but later (after Hilbert proved the spectral theorem in the harder infinitedimensional setting!) people realized that Lagrange had proved the finitedimensional version, Theorem 3.7.15. Generalizing the spectral theorem to infinitely many dimensions is one of the central topics of functional analysis. Our proof of the spectral theorem uses Theorem 1.6.9, which guarantees that a continuous function on a compact set has a maximum. That theorem is nonconstructive: it proves existence of a maximum without showing how to find it. Thus Theorem 3. 7.15 is also nonconstructive. There is a practical way to find an eigenbasis for a symmetric matrix, called Jacobi's method; see Exercise 3.29.
363
Theorem 3.7.15 {Spectral theorem). Let A be a symmetric n x n matrix with real entries. Then there exists an orthonormal basis vi, ... , Vn ofll~n and numbers Ai, ... , An E JR such that 3.7.51 By Proposition 2.7.5, Theorem 3.7.15 is equivalent to the statement that B 1 AB = BT AB is diagonal, where B is the orthogonal matrix whose columns are vi, ... , Vn. (Recall that a matrix Bis orthogonal if and only if BT B =I.)
Remark. Recall from Definition 2.7.2 that a vector vi satisfying equation 3.7.51 is an eigenvector, with eigenvalue Ai· So the orthonormal basis of Theorem 3.7.15 is an eigenbasis. In Section 2.7 we showed that every square matrix has at least one eigenvector, but since our procedure for finding it used the fundamental theorem of algebra, it had to allow for the possibility that the corresponding eigenvalue might be complex. Square real matrices exist that have no real eigenvectors or eigenvalues: for instance, the matrix
A
= [_~
~]
rotates vectors clockwise by 7r /2, so clearly there is no A E JR
and no nonzero vector v E JR 2 such that Av = Av; its eigenvalues are ±i. Symmetric real matrices are a very special class of square real matrices, whose eigenvectors are guaranteed not only to exist but also to form an orthonormal basis. !:::,,
Proof. Our strategy will be to consider the function QA : !Rn + JR given by QA(x) = ·Ax, subject to various constraints. The first constraint ensures that the first basis vector we find has length 1. Adding a second constraint ensures that the second basis vector has length 1 and is orthogonal to the first. Adding a third constraint ensures that the third basis vector has length 1 and is orthogonal to the first two, and so on. We will see that these vectors are eigenvectors. First we restrict QA to the (n1 )dimensional sphere S C !Rn of equation F 1 (x) = lxl 2 = l; this is the first constraint. Since Sis a compact subset of !Rn, QA restricted to S has a maximum, and the constraint F 1 (x) = lxl 2 = 1 ensures that this maximum has length 1. Exercise 3.7.9 asks you to justify that the derivative of QA is
x
[DQA(a)]h =a. (Ah)+ h. (Aa) = a:T Ah+ hT Aa = 2a:T Ah.
3.7.52
The derivative of the constraint function is Remember that the maximum of QA is an element of its domain; the function achieves a maximal value at a maximum. Exercise 3.2.11 explores other results concerning orthogonal and symmetric matrices.
[DF,(li)]h
~ {2a, ... 2a,,]
[l] ~
2aTh.
3.7.53
Theorem 3.7.5 tells us that if v 1 is a maximum of QA restricted to S, then there exists Ai such that
2vi A= Ai2v{, so (vi A)T = (Ai"v{)T, i.e., ATv\ = AiV1. (remember that (AB)T =BT AT.)
3.7.54
364
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
Since A is symmetric, we can rewrite equation 3.7.54 as Av\ = >. 1v\. Thus v1, the maximum of QA restricted to S, satisfies equation 3.7.51 and has length 1; it is the first vector in our orthonormal basis of eigenvectors.
Remark. We can also prove the existence of the first eigenvector by a more geometric argument. By Theorem 3. 7.2, at a critical point v 1 , the derivative [DQA(v1)] vanishes on T;; 1 S: [DQA(v1)]w = 0 for all w E T;; 1 S. Since T;;1 S consists of vectors tangent to the sphere at v1 , hence orthogonal to v1, this means that for all w such that w · V1 = 0,
[DQA(vi)Jw = 2(vi A)w =
o.
3.7.55
Taking transposes of both sides gives w T Av1 = w · Av 1 = 0. Since w ..l v 1, this says that Av1 is orthogonal to any vector orthogonal to v1. Thus it points in the same direction as v1: Av1 = >.1v1 for some number >.1. !::::.
The space (v1).l is called the orthogonal complement to the subspace spanned by v1. More generally, the set of vectors satisfying
x
x · v1 = 0,
x · v2 = 0
(see equation 3.7.62) is called the orthogonal complement of the subspace spanned by V1 and V2. Equation 3.7.57: The subscript 2, 1 for µ indicates that we are finding our second basis vector; the 1 indicates that we are choosing it orthogonal to the first basis vector.
The first equality of equation 3. 7.60 uses the symmetry of A: if A is symmetric,
v. (Aw)= vT (Aw)= (vT A)w = (vT AT)w = (Av)T w =(Av) ·w. The third the fact that
For the second eigenvalue, denote by is subject to the two constraints
v2 the maximum of QA
when QA
3.7.56 In other words, we are considering the maximum of QA restricted to the space Sn (v1)..L, where (v1)..L is the space of vectors orthogonal to v1. Since [DF2(v2)] = v"[, equations 3.7.52 and 3.7.53 and Theorem 3.7.5 tell us that there exist numbers >.2 and µ2,1 such that
2vJA ...__.,,
3.7.57
=
(DQA(v2)]
(Note that here >. 2 corresponds to >. 1 in equation 3.7.9, and µ2,1 corresponds to >. 2 in that equation.) Take transposes of both sides (remembering that, since A is symmetric, A = AT) to get
3.7.58 Now take the dot product of each side with v1, to find  ) \  µ2,1  µ2,1 (A V2 ·V1=A2~+2~=2 . 0
3.7.59
1
(We have v 1 · v 1 = 1 because v 1 has length 1; we have v2 · v1 = 0 because the two vectors are orthogonal to each other.) Using
V2 E Sn (v1).l.
3.7.60 equation 3.7.59 becomes 0 = µ 2,1, so equation 3.7.58 becomes
3.7.61
3. 7 Constrained critical points and Lagrange multipliers
365
Thus we have found a second eigenvector: it is the maximum of QA constrained to the sets given by F 1 = 0 and F 2 = 0. It should be clear how to continue, but let us take one further step. Suppose v3 is a maximum of QA restricted to Snvfnv:f, that is, maximize QA subject to the three constraints
You should be impressed by how easily the existence of eigenvectors drops out of Lagrange multipliers. Of course, we could not have done this without Theorem 1.6.9 guaranteeing that the function QA has a maximum and minimum. In addition, we've only proved existence: there is no obvious way to find these constrained maxima of QA.
F1(x) = 1,
F3(x)=x·v2=0.
3.7.62
The argument above says that there exist numbers ..\3, µ3,li µ3,2 such that 3.7.63
Dot this entire equation with v1 (respectively, with µ3,1 = µ3,2 = o. Thus Av3 = ..\3v3. D
v2 );
you will find
The spectral theorem gives another approach to quadratic forms, geometrically more appealing than completing squares.
Theorem 3.7.16. Let A be a real n x n symmetric matrix. The quadratic form QA has signature (k, l) if and only if there exists an orthonormal basis v,, ... ,vn ofRn with Avi = ..\ivi, where Ai, ... ,>.k > 0, Ak+l, ... , Ak+l < 0, and all Ak+l+li ... , An are 0 (if k + l < n). Proof. By the spectral theorem, there exists an orthonormal eigenbasis
Equation 3. 7.65 uses the fact the v1, ... , Vn form an orthonormal basis, so Vi · Vj is 0 if i =/= j and 1 if i = j.
vi, ... ,vn of Rn for A; denote by A1 , ... , An the corresponding eigenvalues. Then (Proposition 2.4.18) any vector
x E Rn can be written n
n
:X = ~)x · vi)vi,
so
Ax= L Ai(x. vi)vi.
i=l
3.7.64
i=l
The linear functions ai(x) ~ x ·vi are linearly independent, and
~
eq.
3.5.32 n
n
n
=LL ai(x)ai(x) >.i vi· vi = L Ai ai(:X'.) i=l j=l
2
3.7.65
i=l
This proves Theorem 3.7.16: if QA has signature (k, l), then k of the Ai are positive, l are negative, and the rest are O; conversely, if k of the eigenvalues are positive and l are negative, then QA has signature (k, l). D
EXERCISES FOR SECTION
3.7
3.7.1 What is the maximum volume of a box of surface area 10, for which one side is exactly twice as long as another?
366
Chapter 3.
3. 7.2
Manifolds, Taylor polynomials, quadratic forms, curvature
Confild.. the 'i==•"imtion" g ' (
~) ~ ( :::~) of the •ndace X
of equation x = sinz (shown in the figure in the margin). Set f(x) = x + y + z. a. Show that for all x EX, the derivative [Df(x)] does not vanish on TxX. b. Show that
f
o g does have critical points.
c. By part a, these critical points are not critical points of X. What went wrong? FIGURE FOR EXERCISE 3.7.2. The("su)rface'(' ~n of g :
~
........
u
:v
,! v
t)he image
; it is a
subset of the surface X of equation x = sin z, which resembles a curved bench. The tangent plane to X at any point is always parallel to the yaxis.
3. 7 .3 a. Find the critical points of the function x 3 + y 3 + z 3 on the intersection of the planes of equation x + y + z = 2 and x + y  z = 3. b. Are the critical points maxima, minima, or neither? 3. 7.4 a. Show that the set X C Mat (2, 2) of 2 x 2 matrices with determinant 1 is a smooth submanifold. What is its dimension? b. Find a matrix in X that is closest to the matrix [ ~
Exercise 3.7.8: Think about when the maximum is inside the triangle and when it is on the boundary.
~] .
3. 7.5 What is the volume of the largest rectangular parallelepiped aligned with the axes and contained in the ellipsoid x 2 + 4y 2 + 9z 2 ~ 9? 3. 7.6
Exercise 3.7.7: Use the equation of the plane to write z in terms of x and y (i.e., parametrize the plane by x and y).
f constrained to
Find all the critical points of the function
f 3. 7. 7
G) ~
2xy + 2yz  2x'  2y'  2z'
on the unit '!>here of R'.
a. Generalize Example 3. 7.9: show that the function xyz has four critical
points on the plane of
eq~tion ~
G) ~
= + by + cr 
1
~ 0 when a, b, o> O.
b. Show that three of these critical points are saddles and one is a maximum. 3. 7.8 Find the maximum of the function xae"'ybey on the triangle x ~ 0, y ~ 0, x + y ~ 1, in terms of a and b, for a, b > 0. 3. 7.9 Justify equation 3.7.51, using the definition of the derivative and the fact that A is symmetric. *3.7.10 Let D be the closed domain described by the two inequalities x+y and x 2 + y 2 ~ 1, as shown in the margin. a. Find the maximum and minimum of the function b. Try it again with
f (
~
0
f ( ~) = xy on D.
~) = x + 5xy.
3. 7.11 The cone of equation z 2 = x 2 + y 2 is cut by the plane z = 1 + x +yin a curve C. Find the points of C closest to and furthest from the origin.
3. 7.12 FIGURE FOR EXERCISE 3.7.12.
Find local critical points of the function
f (
n
~ x + y + z on the
=f~ p~etrized by g , (:) ~ ( •inu: :~ u} "'°"" in the figure in the margin. (Exercise 3.1.25 asked you to show that g really is a parametrization.)
3.8
3.7.13 Hint for Exercise 3.7.13, part c: The maximum and minimum values k occur either at a critical point in the interior By12(xo), or at a constrained critical point on the boundary 8By12(xo)
JR'~ R by f
Define f
a. Show that
f
Probability and the singular value decomposition
m
~ xyx+y+z'
Let xu
~
367
n)
has only one critical point, xo. What kind of critical point is
it? b. Find all constrained critical points off on the manifold Sy12(xo) (the sphere of radius v'2 centered at xo), which is given by the equation F
= Sy12(xo).
(n ~
(x+ 1i' + (•1)' +z'2
~o.
c. What are the maximum and minimum values of By12(xo)?
f on the closed ball
3. 7.14 Analyze the critical points found in Example 3.7.9, using the augmented Hessian matrix. 3. 7 .15
Let X C li2 be defined by the equation y 2 = ( x  1) 3 .
a. Show that
f ( ~) = x 2 + y 2 has a minimum on X.
b. Show that the Lagrange multiplier equation has no solutions. c. What went wrong? 3.7.16 x1
3.8
[20 01 00 0OJ 0
0
0
0
This rectangular diagonal matrix is nonnegative. The "diagonal" entries are 2, 1, and 0. In applications of the SVD, it is essential that A be allowed to be nonsquare. The matrices involved are often drastically "lopsided", like 200 x 1000000 in Example 3.8.10. The matrix D stretches in the direction of the axes; we will see in Exercise 4.8.23 that the orthogonal matrices P and Q are compositions of reflections and rotations.
Find the critical points of f(x,y,z)
+ y1 + z1 = 1.
= x 3 + y3 + z3
on the surface
PROBABILITY AND THE SINGULAR VALUE DECOMPOSITION
If A is not symmetric, the spectral theorem does not apply. But AAT and AT A are symmetric (see Exercise 1.2.16), and the spectral theorem applied to these matrices leads to the singular value decomposition (SVD). This result has immense importance in statistics, where it is also known as principal component analysis (PCA). We call an n x m matrix D rectangular diagonal if di,j = 0 for i =I j; such a matrix is nonnegative if the diagonal entries di,i are ~ 0 for all 1::::; i::::; min{n,m}.
Theorem 3.8.1 (Singular value decomposition). Let A be a real n x m matrix. Then there exist an orthogonal n x n matrix P, an orthogonal m x m matrix Q, and a nonnegative rectangular diagonal n x m matrix D such that A= PDQT. The nonzero diagonal entries di,i of D are the square roots of the eigenvalues of AT A. The diagonal entries
di,i
of D are called the singular values of A.
368
Lemma 3.8.2: We say that Lemma 3.8.2 is rather surprising because the matrices AT A and AA T may have drastically different sizes. Note that although they have the same nonzero eigenvalues with the same multiplicities, they may have different numbers of zero eigenvalues.
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
To prove Theorem 3.8.1 we will require the following rather surprising statement. Lemma 3.8.2. The symmetric matrices AT A and AAT have the same nonzero eigenvalues with the same multiplicities: for all A =f. 0, the map v
I+
is an isomorphism ker(AT A>.!)
1 AJ>. v
t
3.8.1
ker(AAT  >.!).
Proof of Lemma 3.8.2. If vis in ker(AT A>.!), i.e., AT Av= Av, then Av/J>. is in ker(AAT  >.!):
AAT ( JxAv) = JxA(AT Av)= A ( JxAv).
3.8.2
The map is an isomorphism, since w 1+ JxAT w is its inverse. Therefore, AT A and AAT have the same nonzero eigenvalues, and by Definition 2.7.2, these eigenvalues have the same multiplicities. D Proof of Theorem 3.8.1. Exercise 3.8.5 asks you to use Lemma 3.8.2 and the spectral theorem to show that AT A has an orthonormal eigenbasis v1, ... , Vm, with eigenvalues Ai ~ · · · ~ Ak > 0, Ak+i = · · · = Am = 0, and that AAT has an orthonormal eigenbasis w 1, ... , wn, with eigenvalues µi = Ai, ... , µk = Ak and µk+l = · · · = µn = 0, such that To find the singular value decomposition of a square matrix A using MATLAB, type [P,D,Q] = svd(A) You will get matrices P, D, Q such that A= PDQ, where Dis diagonal and P and Q are orthogonal. (This Q will be our QT.)
 = Wi
l A~ Vi, i. v Ai
~
k
and
 = 1 AT Wi,  i. < Vi _ k.
./iii
3.8.3
(Note that if A is n x m, then AT A ism x m and AAT is n x n; we have Vi E ~m, Av E ~n, w E ~n, and AT w E ~m, so equation 3.8.3 makes sense.) Now set P = [wi, ... , wn]
and
Q = [v1, ... , vm]
3.8.4
Set D = pT AQ. Both P and Q are orthogonal matrices, and the (i,j)th entry of Dis ifi>korj>k
ifi,j
~
k
3.8.5
since if i > k then w"[ A= 0 and if j > k then Avi = 0. Moreover,
=f. j if i = j if i
Recall that a matrix B is orthogonal if and only if BT B = I. So DDT= pT AQQT AT p =PTAATP
= p 1 AAT P.
or i = j and i > k and i,j
~
k.
3.8.6
Thus D is rectangular diagonal with diagonal entries ~, ... , y');k, and since P and Qare orthogonal, D = pT AQ implies A= PDQT. The diagonal entries of the diagonal matrix DDT = PT AA T P are the squares of the "diagonal" entries of D. Thus the singular values of A are square roots of the diagonal entries (eigenvalues) of DDT. Since, by Proposition 2.7.5, the eigenvalues of DDT = p 1AAT Pare also the eigenvalues of AA T, the singular values of A depend only on A. D
3.8 Definition 2.9.6 The norm of A is
llAll
llAll =sup IAxl, when x E Ir and lxl = 1.
Probability and the singular value decomposition
369
One interesting consequence is a formula for the norm of a matrix. Proposition 3.8.3. For any real matrix A, the norm of A is
llAll =
max
>. eigenvalue of AT A
v'A.
Proof. By Proposition and Definition 2.4.19, an orthogonal matrix preserves lengths. Thus if P and Q are orthogonal, and B any matrix (not necessarily square) llQBll = suplvl=l IQBvl = suplvl=l IBvl = llBll and llBQll = sup IBQvl = sup IBQvj = sup jBwj = llBll , lvl=l IQvl=l lwl=l
jjAjj = jjPDQTjj = jjDjj =
max
't/'A.
D
>.eigenvalue of AT A
The language of finite probability
FIGURE 3.8 .1. Andrei Kolmogorov (19031987) , one of the greatest mathematicians of the twentieth century, made probability a rigorous branch of mathematics. Part of his contribution was to say that for any serious treatment of probability, you have to be careful: you need to specify which subsets have a probability (these subsets are called the measurable subsets), and you need to replace requirement 3 of Definition 3.8.4 by countable additivity: if i ,_. A ; is a sequence of disjoint measurable subsets of X, then
P(QA;)= ~P(A;); see Theorem A21.6. Kolmogorov solved the 13th of the 23 problems Hilbert posed in 1900, and made major contributions to dynamical systems and to the theory of complexity and information. (Photograph by Jurgen Moser)
To understand how the singular value decomposition is used in statistics, we need some terminology from probability. In probability we have a sample space S, which is the set of all possible outcomes of some experiment, together with a probability measure P that gives the probability of the outcome being in some subset Ac S (called an event) . If the experiment consists of throwing a sixsided die, the sample space is {l, 2, 3, 4, 5, 6}; the event A = {3, 4} means "land on either 3 or 4" , and P (A) says how likely it is that you will land on either 3 or 4. Definition 3.8.4 (Probability measure). The measure Pis required to obey the following rules: 1. P(A) E
[O, l]
2. P(S) = 1 3. If P(A n B) =for A, B c S, then P(A U B) = P(A)
+ P(B).
In this section we limit ourselves to finite sample spaces; the probability of any subset is then the sum of the probabilities of the outcomes in that subset. In the case of a fair sixsided die, P( {3}) = 1/6 and P( {4}) = 1/6, so P( {3, 4}) = 1/3. In Exercise 3.8.3 we explore some of the difficulties that can arise when a sample space is countably infinite; in Section 4.2 we discuss continuous sample spaces, where individual outcomes have probability 0. A random variable is some feature of an experiment that you can measure. If the experiment consists of throwing two dice, we might choose as our random variable the function that gives the total obtained. Definition 3.8.5 (Random variable). A random variable is a function S > JR. We denote by RV (S) the vector space of random variables.
370
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
When the sample space is finite: S = {s 1, ... , Sm}, the space of random variables on S is simply !Rm, with f E RV(S) given by the list of numbers /(s1), ... , f(sm)· But there is a smarter way of identifying RV(S) to !Rm: the map The map provides a bridge from probability and statistics on one side and geometry of ]Rm on the other.
Equation 3.8.8: (!, g) is standard notation for inner products, of which the dot product is the archetype. The geometry of ]Rm comes from the dot product; defining a kind of dot product for RV(S) gives that space a geometry just like the geometry of ]Rm. Note that the inner product (!, g)(S,P) has the same distributive property as the dot product; see equation 1.4.3.
l
/(s1h/P({si}) iP: RV(S)
t
!Rm : f
tt
[
:
,
3.8.7
/(sm)\/P({sm}) which scales the values off by the square root of the corresponding probability. This is smarter because with this identification, the main constructions of probability and statistics all have natural interpretations in terms of the geometry of !Rm. To bring out these relations, we write (/,g)(S,P) ~ iP(/) · iP(g)
3.8.8
11/llrs,P) ~ liP(/)12 = (!, /)(S,P)·
3.8.9
Thus
11/llZs,P)
= (/, /}(s,P) = / 2 (s1) P( {si}) + · · · + / 2 (sm) P( {sm} ).
3.8.10
Let ls be the random variable that is the constant function 1; note that lllsll(S,P) Definitions 3.8.6: The definition of the expected value says nothing about repeating the experiment, but it follows from the law of large numbers that E(J) is the value one would expect to get if one did the experiment a great many times and took the average of the results. The expected value can be misleading. Suppose the random variable f assigns income to an element of the sample space S, and S consists of 1000 supermarket cashiers and Bill Gates (or, indeed, 1000 school teachers or university professors and Bill Gates); if all you knew was the average income, you might draw very erroneous conclusions.
=
L°:P({s})
=
3.8.11
1.
sES
Definitions 3.8.6. Let f,g E RV(S). Then 1. The expected value of f is
E(f) ~ L°:f(s)P({s}) = (f,ls)(S,P)>
3.8.12
sES
which by equations 3.8.10 gives (f,g)(S,P) = E(fg) 2. The random variable
and
1~ f 
llfllrs,P) = E(/ 2 ).
3.8.13
E(f) is called "/ centered".
3. The variance of f is Var(/)
~
lllllrs,P)
= E( (! E(/)) 2 ) = E(/2 ) 
(E(/)) 2 •
3.8.14
4. The standard deviation a(f) of f is
a(f) ~ JVar(f)
=
llJll(s,P)·
3.8.15
5. The covariance of f and g is cov(f,g) ~ (J,g)(S,P) = E(fg)  E(f)E(g).
3.8.16
3.8
Probability and the singular value decomposition
371
6. The correlation of f and g is corr
(
f, g)
def
cov(J,g)
= a(J)a(g) .
3.8.17
Each of these words needs an explanation. 1. Expected value: The expected value is the average of the values of each value weighted by the probability of the corresponding outcome. The words "expected value", "expectation", "mean", and "average" are all synonymous. That E(J) = (!, ls}cs,P) is a simple computation; for S = {s1, ... , sm}, we have
f,
By Corollary 1.4.4, it follows that the expected value is the signed length of the projection off onto ls, i.e., on the space of constant random variables.
1
2. f centered: The random variable tells how far off from average one is. If a random variable f gives weights of 6monthold babies, then tells how much lighter or heavier than average a particular baby is. Clearly E(f) = 0.
Mean absolute deviation: See the margin note next to Example 3.8.9.
J
3. Variance: The variance measures how spread out a random variable is from its expected value. We can't use the average of f  E(J) because f is just as often larger than f than it is smaller, so E(J  E(J)) = O; squaring f  E(f) makes the variance positive. It might seem (it might be) more natural to consider the mean absolute deviation E(lf  E(f)I), but the absolute deviation is less geometric and therefore much harder to handle. The last equality in the definition of variance is justified in the note below for the covariance. 4. Standard deviation: The variance is the length squared of f; the standard deviation turns that into a length. 5. Covariance: Since (the second equality below uses equation 3.8.13)
Geometrically, the covariance has the same relation to the variance as the dot product has to length squared: cov(f, f)
= Var(!)
x·x= IX.12
cov(f,g) = (f,g}(s,P) = E(fg) = E((fE(f))(g E(g))
3.8.18
we see that the covariance measures to what extent f and g vary "together" or "opposite": (f(s)E(f))(g(s)E(g) ispositivewhen/(s) andg(s) are both more than or both less than their averages, and negative when one is more than and one less than. The following computation, which uses the fact that Eis a linear function on the vector space of random variables, justifies the second equality in equation 3.8.16 defining the covariance (hence the last equality in equation
372
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
3.8.14 defining the variance): cov(f,g)
=
= E((f  E(f))(g  E(g))
(f,g)(S,P)
+ E(f)E(g)) = E(fg)  E(f E(g))  E(gE(f)) + E(f)E(g) =E(fg) 2E(f)E(g) + E(f)E(g) = E(!g JE(g)  gE(f)
Equation 3.8.19, going from the third to the fourth line: Since E(g) is a constant, E(f E(g))
= E(f)E(g);
= E(fg)  E(f)E(g)
similarly, since E(f)E(g) is a constant, E(E(f)E(g))
= E(f)E(g).
3.8.19
= (!, 9)(8,P)  E(f)E(g).
6. Correlation: Proposition 1.4.3 says that v·w 3.8.17 says that
=
lvl lwl cos 0. Equation
= a(f)a(g) corr(!, g).
cov(f, g)
3.8.20
So the correlation is the cosine of the angle between two centered random variables. Recall that the geometric interpretation of the dot product required Schwarz's inequality Iv · Vil ~ !VI lwl. To interpret the correlation as a cosine we need to check that the analogous result is true here. Proposition 3.8.7, part 2: We leave it to the reader to check that the correlation is +1 if a > 0 and 1 if a < 0. In particular, when
f
Proposition 3.8. 7. Let Then
f, g
: (S, P) + R be two random variables.
L lcorr(f,g)l~l
=g,
corr (f, g) = 1 and when f
= g,
2. Icorr(!, g) I = 1 if and only if there exist a, b E R such that
f
= ag+b.
corr(f,g) = 1. Saying that the correlation of two random variables is ±1 means that one is a multiple of the other. Saying that corr (f, g) = 0 means that f and g are orthogonal; if you know f, you only know g E f.L. But f .L C IRn is a subspace of dimension n  1. Inequality 3.8.21: The equality marked 1 is equation 3.8.8; the inequality marked 2 is Schwarz; equality 3 is equation 3.8.9.
Proof. 1. By Schwarz's inequality, I cov(f, g)I = I(f, Y)(s P) I = l(f) · (g)I '
(1)
~ l(f)ll(g)I
(2)
3.8.21 (=) 3
llfll(s,PJll:9ll(s,P)
=
a(f)a(g).
So we have lcorr(f,g)I =
cov(f,g) I Ia(f)a(g) ~ 1.
3.8.22
2. Again by Schwarz, we have I corr(!, g)I = 1 if and only if (f) is a multiple of (9)1 or (9) is a multiple of (f)I (likely both). Suppose the former, which occurs if and only if j is a multiple of g, i.e.,
f  E(f)
=
a(g  E(g)),
i.e.,
f
=
ag + (E(f)  aE(g))
=
ag + b
D
Thus we know what corr(!, g) = ±1 means: f and g are affinely related. What if corr(!, g) = 0, i.e., f and g are uncorrelated? This turns out not to have a clear answer.
3.8
Probability and the singular value decomposition
373
Example 3.8.8. Let our sample space be the subset of the plane consisting of the set of points (
~~i)
for some collection of m numbers
Xi :f 0, all
given the same probability 1/(2m). Let f and g be the two random variables
f ( ±Xi) x~ =±xi,
3.8.23
see Figure 3.8.2.
FIGURE 3.8.2 Let S be a collection of points on the parabola y = x 2 , symmetric with respect to the yaxis. The random variables x and y are uncorrelated, even though y = x 2 •
In example 3.8.9, the mean absolute deviation for the "total obtained" random variable g is "" 1.94, significantly less than the standard deviation "" 2.41. Because of the square in the formula for the variance, the standard deviation weights more heavily values that are far from the expectation than those that are close, whereas the mean absolute deviation treats all deviations equally. Equation 3.8.26: If you throw two dice 500 times and the average total is not close to 7, you would be justified in suspecting that the dice are loaded. The central limit theorem (Theorem 4.2. 7) would allow you to quantify your suspicions.
1 1 2
JR=~4 5 6
1 1 1 1 1 1
2 2 2 2 2 2 2
3 3 3 3 3 3 3
4 4 4 4 4 4 4
Then, since E(f)
= 0 and E(/3) = 0, we have
cov(f, / 2 )
= E(f / 2 ) 
E(f)E(f 2 )
= 0,
3.8.24
hence corr(!, g) = 0. This emphatically does not coincide with our intuitive notion of "uncorrelated": the value off determines the value of g, and even by a very simple (but nonlinear) function. The moral is that correlation should really be called linear correlation (or affine correlation); it is not a useful notion when applied to nonlinear functions. This is the main weakness of statistics. /::,,.
Example 3.8.9. (Expected value, variance, standard deviation, covariance). Consider throwing two dice, one red and one black. The sample space S is the set consisting of the 36 possible tosses, so RV(S) is JR36 ; an element of this space is a list of 36 numbers, best written as a 6 x 6 matrix. Each outcome is equally likely: P ({s}) = 6 for any s E S. Below we see three random variables; fR is "the value of the red die", fB is "the value of the black die", and g is their sum. The horizontal labels correspond to the red die, vertical labels to the black die. For example, for fR, if the red die comes up 5 (underlined) and the black comes up 3, the entry (!Rh,s is 5.
l
Q
5 5 Q
5 5 5
1
6 6 6 6 6 6 6
1 2 3 fB = 4
5 6
1 2 3 4 5 6
2 1 2 3 4 5 6
3 1 2 3 4 5 6
4 1 2 3 4 5 6
5 1 2 3 4 5 6
6
1
1 2 3 4 5 6
2 3 4 5 6 7
1 2
g=fR+!B=3
4 5 6
2
3
4
5
6
3 4 5 6 7 8
4 5 6 7 8 9
5 6 7 8 9 10
6 7 8 9 10
8 9 10
11
12
7
11
The expected value is simply the sum of all 36 entries, divided by 36. Thus 4 + 5 + 6) = 3.5 36 E(g) =EUR+ fB) = E(/R) + E(/B) = 7.
E(/R)
= E(/B) = 6(1+2 + 3 +
3.8.25 3.8.26
374
1 52 42 32 22 12 0
1 2 3 4 5 6
2 42 32 22 12 0 12
3 32 22 12 0 12 22
4 22 12 0 12 22 32
5 12 0 12 22 32 42
6 0 12 22 32 42 52
The matrix (g  E(g)) 2
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
We use the formula Var(!) = E(J2)  (E(f)) 2 (the second equality in equation 3.8.14) to compute the variance and standard deviation of fR and fB: Var(!R)
= Var(fB) = 6(1+4+9+16 + 25 + 36) _ (~) 2 = 35 ~ 2 92 36
a(fR) = a(!B) =
2
12
.
~ ~ 1.71.
The variance of g can be computed from Var(g) = E( (g  E(g)) 2 )
=
g2(s1) P( {si})
+ · · · + g2(sm) P( { 835} ).
Adding all the entries of the matrix (g  E(g) ) 2 in the margin and dividing 1 1 2 3 4 5 6
2
3
1 2 2 4 3 6 4 8 5 10 6 12
3 6 9
12 15 18
4
5
6
4 5 6 8 10 12 12 15 18 16 20 24 20 25 30 24 30 36
The matrix fRfB The second row is the first row times 2, the third row is the first row times 3, and so on. So
by 36 gives Var(g)
= 3i ~ 5.83, so the standard deviation is a(g) =
To compute the covariance we will use cov(f, g) = E(f g)  E(f)E(g) (see equation 3.8.19). The computation in the margin shows that
= 0, The random variables f R and f B cov(/R, fB)
hence
corr (/R, fB)
= 0.
3.8.27
are uncorrelated. What about the corr (JR, g)? If we know the throw of the red die, what does it tell us about the throw of two dice? We begin by computing the covariance, this time using the formula cov(f, g) = (J, g) (S,P):
cov(fR,g) = (/R,Y)(S,P) =(JR, fR
E(!RfB)
+ fB)(S,P)
= 316 (1 + 2 + 3 + 4 + 5 + 6) 2
llfnllts,P)=Var(fR)
= E(!R)E(!B),
= (/R, /R)cs,P) + (/R, /B)(S,P) = Var(/R) = 12 ·
giving
E(!RfB)  E(!R)E(!B)
= 0.
cov(fn,Js) by eq. 3.8.16
j¥.

The correlation is then cov(JR,g) a(JR)a(g)
Oby eq. 3.8.27
~
35/12 J35/12J35/6
the angle between the two random variables
fR
35
1
v'2; and g is 45 degrees.
6.
The covariance matrix Note that since
If / 1 , ... , f k are random variables, their covariance matrix is the k x k matrix [Cov] whose (i,j)thentry is
[Cov]i,j ~ cov(/i, fj) the covariance matrix is symmetric. Indeed, for any matrix A, the matrix AT A is symmetric; see Exercise 1.2.16. So we can apply the spectral theorem.
= (h, h)(S,P)
'P(h) · iP(J;).
3.8.28
where C = [iP(fi), ... , 'P(h)],
3.8.29
=
We could also write [Cov] =CT C,
since the (i,j)th entry of CT C is the dot product iP(h) · iP(J;). Recall that in the proof of the spectral theorem for a symmetric n x n matrix A, we first found the maximum of the function QA (x) = x ·Ax on the sphere S = { x E !Rn I !xi = 1 }; the point where the maximum is realized
3.8
In equation 3.8.30, Xi/1
+ · ·· + Xkfk
is a linear combination of the k random variables. If w is the unit eigenvector of [Cov) with largest eigenvalue, then the variance is maximal at Wi/1
375
is our first eigenvector V1 of the matrix A, and the maximum value >. 1 is the first eigenvalue. Our second eigenvector is then the point v 2 E S n v:f where QA achieves its maximum value >. 2 , etc. In this setting we are interested in Q[cov] (x) = x · [Cov] x. This function is a quadratic form on JRk and represents the variance in the sense that k
Q[covJ(x) = x · [Cov] x =
k
L ~.::::Xixi(h, J.i)cs,P) i=l j=l
= (x1li + ... + Xkh, x1h + ... + Xkh)(S,P) 2 = llxi/1 + · · · + xkfkllcs,P) = Var(xi/1 + · · · + xkfk)·
+ · ·· + Wkfk
among unit vectors. You may find it useful to review the proof of the spectral theorem.
Probability and the singular value decomposition
3.8.30
Therefore x 1+ Q[cov]X achieves its maximum among unit vectors at the leading eigenvector of [Cov]. The eigenvectors of [Cov] are called the principal components; the subject is called principal component analysis (PCA).
PCA and statistics
A few minutes on the Internet reveal that the uses of principal component analysis are myriad: latent semantic indexing for querying databases, determining the origin of a gasoline sample taken from a suspected arsonist, analyzing paint in a 16th century altarpiece in Slovakia, determining whether Italian mozzarella comes from Campania or from Apulia.
Statistics is the art of teasing patterns from data. What is data? Usually, it is a cloud of N points in some JRk. We may have many points in a fairly lowdimensional space, if we are measuring a few features of a large population, or few points in a highdimensional space. Or both dimensions may be large, as in all pictures in the State Department's file of passport pictures and pictures on visa applications: perhaps 100 000 000 pictures, each perhaps a megabyte. Such a cloud of points can be written as a k x N matrix A: it has as many rows as there are features measured and one column for each data point. 17 In these days of "big data" the matrices may be huge: 1 000 x 1 000 000 is nothing unusual, sizes in the billions (Walmart transactions, census data) are common, and even trillions (astronomical objects, weather prediction, geological exploration) are sometimes used. The object of PCA is to extract useful information from such a huge database A. The idea is to compute an appropriate average A to center the data, and apply the SVD to the covariance matrix [AA]T[AA]. The unit eigenvector of this covariance matrix with largest eigenvalue points in the direction in which the data has the most variance; thus it corresponds to the direction that carries the most information. Of the eigenvectors orthogonal to the first eigenvector, we choose the eigenvector with largest eigenvalue, which carries the next greatest amount of information, and so on. The hope is that a relatively small number of these eigenvectors will carry most of the information. If this "most" is actually justified, projecting the data into the space spanned by those directions will lose little information. Example 3.8.10 (Eigenfaces). Suppose you have a large database of images of faces. Perhaps you are the State Department, with all the faces of visa applicants and passport holders. Or Facebook, with many more 17 The
decision that rows represent features and columns data is arbitrary.
376
FIGURE
3.8.3.
Kevin Huang used these training faces for a study of eigenfaces when he was an undergraduate at Trinity College (photos reprinted with permission).
FIGURE
3.8.4.
The average of the ten training faces above.
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
photographs, not so carefully framed. These agencies and companies want to do different things with their databases, but clearly identifying a new photograph as that of someone in a database is one of the basic tasks they will want to accomplish. To use eigenfaces, you first choose a collection of m "training faces". The seminal paper by Matthew Pentland and Alex Turk 18 had just m=16, and to our eyes they all look suspiciously like computer science graduate students, but the paper dates back to 1991, the dinosaur age for technology. The training faces should be chosen randomly within the database for maximum effectiveness. These images might look like Figure 3.8.3. From them training faces, form an m x n 2 matrix P, where n might be 128 (for very low resolution pictures), or 1024 (for lK x lK images, still pretty low resolution). The entries of the matrix might be integers from 0 to 255 in the low resolution world, or integers between 0 and 224 for color images. Denote by p 1 , . . . , Pm the rows of this matrix; each is a line matrix n 2 wide, representing a picture with n x n pixels. We think of the shade of each pixel as a random variable. In any case, n 2 is much larger than m . The space of images is immense: n 2 will be about 10 12 if the pictures are lK by lK. The idea is to locate face space: a subspace of this immense space of much lower dimension such that images of faces will lie close to this subspace. First find the "average picture" 1
P = (p1 + · · · +Pm); m
FIGURE
3.8.5.
Five centered faces : test faces with the average p subtracted.
where the Pi are the training faces. This average p is another image but not the picture of anyone; see Figure 3.8.4. Now (see Figure 3.8.5) subtract p from each training face Pi to get the "centered" picture Pi  p, and let CT be the n 2 X m matrix CT= [pl p, ···,pm p];
FIGURE
3.8.6.
The four eigenfaces with largest eignvalues created from the faces of Figure 3.8.3. The face marked 1 has the largest egenvalue.
3.8.31
3.8.32
i.e., the ith column of cT is Pi  p. Now apply the spectral theorem to the n 2 X n 2 covariance matrix CT C. This is a terrifyingly large matrix, but the task is actually quite feasible, because by Lemma 3.8.2 the nonzero eigenvalues of CT C are the same as the nonzero eigenvalues of the matrix CCT, and CCT is only m X m, perhaps 500 x 500 or 1000 x 1000 (and symmetric, of course). Finding the eigenvalues and eigenvectors such a matrix is routine. Lemma 3.8.2 then tells us how to find the eigenvectors of CT C. These eigenvectors are called eigenfaces: they are vectors n 2 high that encode (usually ghostly) images. See Figure 3.8.6. The next step involves an element of judgment: which are the "significant" eigenfaces? They should of course be the ones with largest eigenvalues, because they are the directions in which the test pictures show maximal 18 M.
Turk and A. Pentland, Face recognition using eigenfaces.Computer Vision and Pattern Recognition, 1991. Proceedings CVPR '91., IEEE Computer Society Conference on 1991
3.8
Note that what linear algebra tells us are the principal "features" are not features that we can describe. If in Figure 3.8.6 we had not labeled the eigenfaces with largest eigenvalues, you probably could not have even have figured out which had larger eigenvalues. When we recognize the face of a friend or acquaintance, how do we do it? How do we store and retrieve that data? In the absence of knowledge, we can speculate: perhaps our brains run some form of "eigenfaces" . This might explain comments by some whites that "all blacks look the same" or by some Chinese that "all whites look the same": the features we best discern are those of the "training faces" we see as young children. Something similar might explain why we can discern with precision the sounds of our native language, but often we hear the sounds of a foreign language as the same, when to native speakers they sound completely different. Rather than speak of the largest eigenvalues of CT C, we could speak of the largest singular values of C, since they are the square roots of the eigenvalues of cT c by Theorem 3.8.1 and the definition of singular values.
Probability and the singular value decomposition
377
variance. But how many should you take? There isn't a cleancut answer to the question: perhaps there is a large gap in the eigenvalues, and the ones beneath the gap can be neglected; perhaps one chooses to cut off at some threshold. In the final analysis, the question is whether the chosen eigenfaces give the user the discrimination needed to accomplish the desired task. Facebook is said to use the largest 125150 singular values. Suppose you have chosen k eigenfaces qi, ... , qk as significant; face space :F c R.n 2 is the subspace spanned by these eigenfaces. The eigenfaces form an orthonormal basis for :F, and any image p can first be centered by setting p = p  p, then be projected to face space. The projection rr,r(fi) can be written as a linear combination 3.8.33 These coefficients should give a lot of information about p, since the chosen eigenfaces are the directions of maximal variance of the sample of faces used. Apply this to all the faces in the database, making a list of elements of R.k which is supposed to encode the main features of each face of the database. When a new picture Pnew comes in, project Pnew = Pnew  P onto face space to find the corresponding coefficients a1, ... , ak. There are two questions to ask: how large is IPnew  rr.r(Pnew)I, i.e., how far is Pnew from face space. If this distance is too large, the picture is probably not a face at all. Suppose that Pnew does pass the "face test"; then how close is 7r:FWnew) top for one of the faces p in the database? If this distance is sufficently small, Pnew is recognized as p, otherwise it is a new face, perhaps to be added to the database. The final judgment of whether the cutoffs were well chosen is whether the algorithm works: does it correctly recognize faces? !:::.
EXERCISES FOR SECTION
3.8
3.8.1 Suppose an experiment consists of throwing two 6sided dice, each loaded so that it lands on 4 half the time, while the other outcomes are equally likely. The random variable f gives the total obtained on each throw. What are the probability weights for each outcome? 3.8.2 Repeat Exercise 3.8.1, but this time one die is loaded as before, and the other falls on 3 half the time, with the other outcomes equally likely. 3.8.3 Toss a coin until it comes up heads. The sample space S of this game is the positive integers S = {1, 2, ... }; n corresponds to first coming up heads on the nth toss. Suppose the coin is fair, i.e., P({n}) = 1/2n. a. Let f be the random variable f(n) = n. Show that E(f) = 2. b. Let g be the random variable g(n) = 2n. Show that E(g) = oo. c. Think of these random variables as payoffs if you play the game: if heads first comes up on throw n, then for the payoff f you collect n dollars; for g you collect 2n dollars. How much would you be willing to pay to enter the game with payoff f? How about g?
378
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
3.8.4 Suppose a probability space X consists of n outcomes, {1, 2, ... , n}, each with probability 1/n. Then a random function f on X can be identified with an element f E .!Rn. a. Show that E(f) = ~(f · i), where i is the vector with all entries 1. b. Show that Var(/) c. Show that
= ~If E(f)il 2 cov(f,g) =
and
a(/)
= ~If E(f)il. v ••
n1(fE(f)l) (gE(g)l) ;
corr(f,g) = cosO, where 0 is the angle between the vectors 3.8.5 Let A be an n x m matrix. a. Show that if v E ker AT A, then w E kerAT.
f  E(f)f and§ E(g)f.
v E ker A and that if w E ker AAT, then
b. Using Lemma 3.8.2 and the spectral theorem, show that AT A has an orthonormal eigenbasis v1, ... , Vm with eigenvalues A1 ;::: · · · ;::: Ak > 0, Ak+l =···=Am= 0, and that AAT has an orthonormal eigenbasis w1, ... , Wn, with eigenvalues µ1 = A1, ... , µk = Ak and µk+l = · · · = µn = 0, such that Wi
3.9
= . f1\ AVi, •• ~ k v Ai
and
Vi
=
1 AT Wi,  Z. < r,;: _ k. yµi
GEOMETRY OF CURVES AND SURFACES
In which we apply what we have learned about Taylor polynomials, quadratic forms, and critical points to the geometry of curves and surfaces: in particular, their curvature. Curvature in geometry manifests itself as gravitation.C. Misner, K. S. Thorne, J. Wheeler,
Gravitation
A curve acquires its geometry from the space in which it is embedded. Without that embedding, a curve is boring: geometrically it is a straight line. A onedimensional worm living inside a smooth curve cannot tell whether the curve is straight or curvy; at most (if allowed to leave a trace behind it) it can tell whether the curve is closed or not. This is not true of surfaces and higherdimensional manifolds. Given a longenough tape measure you could prove that the earth is spherical without recourse to ambient space; Exercise 3.9. 7 asks you to compute how long a tape measure you would need. The central notion used to explore these issues is curvature, which comes in several flavors. Its importance cannot be overstated: gravitation is the curvature of spacetime; the electromagnetic field is the curvature of the electromagnetic potential. The geometry of curves and surfaces is an immense field; our treatment cannot be more than the barest overview. 19 19 For further reading, we recommend Riemannian Geometry, A Beginner's Guide, by Frank Morgan (A K Peters, Ltd., Natick, MA, second edition 1998) and Differential Geometry of Curves and Surfaces, by Manfredo P. do Carmo
(Prentice Hall, Inc., 1976).
3.9
Recall the fuzzy definition of "smooth" as meaning "as many times differentiable as is relevant to the problem at hand" . In Sections 3.1 and 3.2, once continuously differentiable was sufficient; here it is not.
Geometry of curves and surfaces
379
We begin with curvature as it applies to curves in the plane and in space. In both cases we write our curve as the graph of a function in the coordinates best adapted to the situation, and read the curvature (and other quantities of interest) from the quadratic terms of the function's Taylor polynomial. Differential geometry only exists for functions that are twice continuously differentiable; without that hypothesis, everything becomes a million times harder. Thus the functions we discuss all have Taylor polynomials of degree at least 2. For curves in space, we will need our functions to be three times continuously differentiable, with Taylor polynomials of degree 3.
The geometry and curvature of plane curves
y
y
b
x
For a smooth curve in the plane, the "best coordinate system" X, Y at a point a = ( is the system centered at a, with the Xaxis in the direction of the tangent line, and the Yaxis orthogonal to the tangent at that point, as shown in Figure 3.9.1. In these (X, Y)coordinates, the curve is locally the graph of a function Y = g(X), which can be approximated by its Taylor polynomial. This Taylor polynomial contains only quadratic and higher terms 20 :
b)
A2
Y=g(X)=TX FIGURE
3.9.1.
To study a smooth curve at a = (
b) , we make a the origin
of our new coordinates and place the Xaxis in the direction of the tangent to the curve at a. Within the shaded region, the curve is the graph of a function Y = g(X) that starts with quadratic terms. K
+ 6A3 x 3 +···,
3.9.1
where A 2 is the second derivative of g (see equation 3.3.1). All the coefficients of this polynomial are invariants of the curve: numbers associated to a point of the curve that do not change if you translate or rotate the curve. (Of course, they do depend on the point where you are on the curve.) In defining the curvature of a plane curve, the coefficient that will interest us is A2 , the second derivative of g. Definition 3.9.1 (Curvature of a curve in JR2 ). Let a curve in JR2 be locally the graph of a function g, with Taylor polynomial g(X) =
The Greek letter
2
~2 x 2 + ~3 X 3 + . . . .
3.9.2
is "kappa".
We could avoid the absolute value in Definition 3.9.1 by defining the signed curvature of an oriented curve. The dots in _!x 2 + · · · rep2 resent higherdegree terms. We avoided computing the derivatives for g(X) by using the formula for the Taylor series of a binomial (equation 3.4.9). In this case, m is 1/2 and a= X 2 .
Then the curvature coordinates) is
K.
of the curve at 0 (i.e., at a in the original (x,y)3.9.3
Remark. The unit circle has curvature 1: near ( ~) the "best coordinates" for the unit circle are X
= x, Y = y 1, soy= Jf=X2 becomes
g(X) = Y = y  1 = ~  1 =
with the Taylor polynomial g(X) = 
Vl  X 2 
1,
3.9.4
1
2x 2 + · · ·.
20 The point a has coordinates X = 0, Y = 0, so the constant term is O; the linear term is 0 because the curve is tangent to the Xaxis at a.
380
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
(When X = 0, both g(X) and g'(O) vanish, while g"(O) = 1; the quadratic term for the Taylor polynomial is ~g".) So the unit circle has curvature I  1I = l. 6 Proposition 3.9.2 tells how to compute the curvature of a smooth plane curve that is locally the graph of a function f (i.e., when we are working in any coordinate system x, yin which the curve is locally the graph of a function, not in the "best" coordinates X, Y). We prove it after giving a couple of examples.
Proposition 3.9.2 (Computing the curvature of a plane curve known as a graph). The curvature r;, of the curve y = f(x) at (
is
1
(
part of a circle of radius approx. 5 .6
FIGURE
3.9.2.
At (~), which corresponds to
a = 1, the parabola y = x 2 looks much flatter than the unit circle. Instead, it resembles a circle of radius 5../5/2 ~ 5.59. (A portion of such a circle is shown. Note that it crosses the parabola. This is the usual case, occurring when, in adapted coordinates, the cubic terms of the Taylor polynomial of the difference between the circle and the parabola are nonzero.) At the origin, which corresponds to a = 0, the parabola has curvature 2 and resembles the circle ofradius 1/2 centered at (i~ 2 ), which also has curvature 2. "Resembles" is an understatement. At the origin, the Taylor polynomial of the difference between the circle and the parabola starts with fourthdegree terms.
!~))
r;,
a ) f(a) =
lf"(a)I
(1 + (f'(a))2)3/2.
3.9.5
Example 3.9.3 (Curvature of a parabola). The curvature of the curve y
= x 2 at the point ( : 2 ) is 2
r;, =
{1
3.9.6
+ 4a2)3/2 .
Note that when a = 0, the curvature is 2, twice that of the unit circle. As a increases, the curvature rapidly decreases. Already at ( ~ ) , corresponding to a = 1, the curvature is 5 2 5 ~ 0.179. Figure 3.9.2 shows that at this point, the parabola is indeefmuch flatter than the unit circle. At the origin, it resembles the circle with radius 1/2, centered at ( 1 ~ 2 ).
6
Example 3.9.4 (Curvature of a curve that is the graph of an implicit function). We saw in Example 3.4.8 that
x 3 + xy + y 3 = 3
3.9. 7
implicitly expresses x in terms of y near ( ~ ) and that the first and second derivatives at 1 of the implicit function g are g'{l)
= 1, and g"{l) = 5/2.
Thus near ( ~), the curve described by equation 3.9. 7 has curvature r;,
=
lf"(a)I
1 5/21
(1 + (f'(a))2)3/2
23/2
''''~
3.9.8
Proof of Proposition 3.9.2. We express f as its Taylor polynomial, ignoring the constant term, since we can eliminate it by translating the coordinates, without changing any of the derivatives. This gives
f(x) = J'(a)x + f"(a) x 2 + · · · . 2
3.9.9
3.9
Geometry of curves and surfaces
381
Now rotate the coordinates x, y by () to get coordinates X, Y shown in Figure 3.9.3: use the rotation matrix
R=[
c?s()
sm()
sin()] to et cos() g
[X] =R[x] Y y
= [
xc?sO+ysin()]. xsm() + ycos()
3 ·9 ·10
Then express x,y in terms of X, Y by multiplying by R 1 :
[
cos() sin()
sin()] [ cos() cos ()  sin ()
sin()] [xy] cos ()
[xy].
I
to get x = X cos()  Y sin() and y into equation 3.9.9 leads to
=X
sin() + Y cos(). Substituting these
X sin()+ Y cos()= J'(a)(X cos()  Y sinO) +f"~a) (X cos()  Y sin0) 2 +... . y
x
3.9.11
x2
z
FIGURE 3.9.3 In the coordinates X, Y centered at ( the curve
JCa)),
a x
Here we are treating Y as the pivotal unknown and X as the nonpivotal unknown, so D2F corresponds to the invertible matrix [D1F(c), ... , DnkF(c)] in Theorem 2.10.14. Since D2F is a number, being nonzero and being invertible are the same. (Alternatively, we could say that X is a function of Y if D 1 F is invertible.)
of equation y = f(x) is the curve of equation Y = F(X),where the Taylor polynomial of F starts with quadratic terms.
Recall (Definition 3.9.1) that curvature is defined for a curve that is locally the graph of a function g whose Taylor polynomial starts with quadratic terms. So we want to choose() so that equation 3.9.11 expresses Y as a function of X, with derivative 0, so that its Taylor polynomial starts with the quadratic term:
y
= g(X) = A2 x2 + ....
3.9.12
2 If we subtract X sin()+ Y cos() from both sides of equation 3.9.11, we can write the equation for the curve in terms of the (X, Y)coordinates:
F (;)
= 0 = X sin()  Y cos()+ J'(a)(X cosO Y
sin())+··· , 3.9.13
with derivative [D F (
8)) =
[  sin () +
f' (a) cos (), 
cos () 
!' (a) sin OJ .
3.9.14
382
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
The implicit function theorem says that there is a function g expressing Y in terms of X if D2F is invertible (that is, if f'(a) sin()  cos() "I 0). In that case, equation 2.10.29 for the derivative of an implicit function tells us that in order to have g'(O) = 0, so that g(X) starts with quadratic terms, we must have DiF = f'(a) cos()  sin()= 0:

g'(O) = 0 = [D2F(O)t 1 (f'(a) cos()  sin 0) ,eo
Saying that tan 8 = J' (a) simply says that f' (a) is the slope of the curve.
Thus we must have tan() =
f' (a).
3.9.15
[D 1 F(O)], must be 0
Indeed, if () satisfies tan() = f' (a), then
1 D2F(O) = J'(a) sin()  cos()=  cos 0
"I 0,
3.9.16
so the implicit function theorem applies. Now we compute the Taylor polynomial of the implicit function, as we did in Example 3.4.8. We replace Y in equation 3.9.13 by g(X):
F
(g~)) = 0 = X sin() 
g(X) cos()+ J'(a)(X cos8  g(X) sinO)
f"(a)
2 +2 (Xcos8g(X)sin0) +···,
3.9.17
additional term; see eq.3.9.11
Now we rewrite equation 3.9.17, putting the linear terms in g(X) on the left, and everything else on the right. This gives =0
Equation 3.9.19: g'(O) = 0, so g(X) starts with quadratic terms. Moreover, by theorem 3.4.7, the function g is as differentiable as F, hence as f. So the term Xg(X) is of degree 3, and the term g(X) 2 is of degree 4.
g(X)=
f'
(). 1
(!'(a) sin()+ cos O)g(X) = (!'(a) cos()  sinO)X f"(a)
2 +  2 (cosOX  sinOg(X)) + · · · 3.9.18
f"(a)
=
2
2 (cosOX  sinOg(X)) + · · ·.
We divide by f'(a)sinO+cosO to obtain
a smO+cos 0
f"(a)(cos 2 8X 2 2cos0sin0Xg(X)+sin2 0(g(X)) 2)+···. 2
3.9.19
these are of degree 3 or higher
Now we express the coefficient of X 2 as A2/2 (see Definition 3.9.1), getting f"(a) cos 2 () A2 = 3.9.20 f'(a) sin 8 +cos 8 f'(a)
sin()=
1 FIGURE
Since f'(a) = tanO, we have the right triangle of Figure 3.9.4, so
3.9.4.
This justifies equation 3.9.21.
f'(a)
J1 + (f'(a))
and 2
1
cos (J = ;:=====
J1 + (f'(a))
Substituting these values in equation 3.9.20, we have
2
3.9.21
3.9
Geometry of curves and surfaces
"'= IA2I =
so
lf"(a)I
(1 + (/'(a))2)3/2
383
. D 3.9.22
Geometry of curves parametrized by arc length There is an alternative approach to the geometry of curves, both in the plane and in space: parametrization by arc length. This method reflects the fact that curves have no interesting intrinsic geometry: if you were a onedimensional bug living on a curve, you could not make any measurements that would tell whether your universe was a straight line, or all tangled up. Recall (Definition 3.1.19) that a parametrized curve is a map'/': I  JR.n, where I is an interval in R You can think of I as an interval of time; if you are traveling along the curve, the parametrization tells you where you are on the curve at a given time. The vector {' is the velocity vector of the parametrization 1'·
Definition 3.9.5 (Arc length). The arc length of the segment '"Y([a, b]) of a curve parametrized by ')' is given by the integral
FIGURE
lb
3.9.5.
A curve approximated by an inscribed polygon. You may be more familiar with closed polygons, but a polygon need not be closed.
1i'(t)I dt.
A more intuitive definition is to consider the lengths of straight line segments ("inscribed polygonal curves") joining points ')'(to),')'(t1), ... ,')'(tm), where t 0 = a and tm = b, as shown in Figure 3.9.5. Then take the limit as the line segments become shorter and shorter. In formulas, this means to consider m1
L
m1
b(ti+i)  '"Y(ti)I, which is almost
Parametrization by arc length is more attractive in theory than in practice: computing the integral in equation 3.9.25 is painful, and computing the inverse function t(s) is even more so. Later we will see how to compute the curvature of curves known by arbitrary parametrizations, which is much easier (see equation 3.9.61).
I: 1i'(ti)l(ti+l 
ti).
3.9.24
i=O
i=O
Equation 3.9.25: If your odometer says you have traveled 50 miles, then you have traveled 50 miles on your curve.
3.9.23
(If you have any doubts about the "which is almost", Exercise 3.24 should remove them when ')' is twice continuously differentiable.) This last expres1i'(t)I dt. sion is a Riemann sum for If you select an origin ')'(to), then you can define s(t) by the formula
J:
s(t) ..__., odometer reading at time t
=
1:
1i'(u)I
du;
3.9.25
'.,'
speedometer reading at time u
s(t) gives the odometer reading as a function of time: "how far have you gone since time to". It is a monotonically increasing function, so (Theorem 2.10.2) it has an inverse function t(s) (at what time had you gone distance s on the curve?) Composing this function with '/' : I  JR. 2 or '/' : I  JR.3 now says where you are in the plane, or in space, when you have gone a
384
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
distance s along the curve (or, if 'Y : I curve
8(s)
t
JR.n, where you are in JR.n). The
= 7(t(s))
3.9.26
is now parametrized by arc length: distances along the curve are exactly the same as they are in the parameter domain where s lives.
Proposition 3.9.6 (Curvature of a plane curve parametrized by arc length). The curvature K, of a plane curve 8(s) parametrized by arc length is given by /'i,(8(s)) = FIGURE
3.9.6.
In an adapted coordinate system, a surface is represented as the graph of a function from the tangent plane to the normal line that starts with quadratic terms. See equation 3.9.28.
Since v\, v2, va are an orthonormal basis, the coordinate functions X, Y, Z: li3 > Ii are given by
X(x) = (x  a)· v1
3.9.27
Exercise 3.9.12 asks you to prove Proposition 3.9.6.
The best coordinates for surfaces Since surfaces have intrinsic geometry, we cannot generalize the parametrization approach we used for curves. Instead we will return to the "best coordinates" approach, as in equation 3.9.1. Let S be a surface in JR.3 , and let a be a point in S. Coordinates X, Y, Z with respect to an orthonormal basis v\, v2 , v3 anchored at a are adapted to S at a if v 1 and v2 span TaS, so that v3 is a unit vector orthogonal to S at a, often called a unit normal and denoted ii. In such a coordinate system, the surface S is locally the graph of a function quadratic terms of Taylor polynomial
Y(x) = (x  a) · v2
Z(x) = (x  a) · V3.
l87'(s)I.
Z
= f ( ;)
=
~(A2,0X 2 + 2A1,1XY + Ao,2Y2) +higherdegree terms;
(See Proposition 2.4.18, part 1.)
3.9.28 see Figure 3.9.6. The quadratic form
Since the second fundamental form of S at a is a quadratic form, it is given by a symmetric matrix, the matrix M
=
[A2,o Al,1
Al,1] . Ao,2
The mean curvature is half its trace; the Gaussian curvature (Definition 3.9.8) is its determinant: 1
H(a)
= 2 trM
K(a)
= detM
The eigenvalues of M are the principal curvatures of S at a.
(x) y
tt
A2,oX 2 + 2A1,1XY + Ao,2Y2
3.9.29
is known as the second fundamental form of the surface S at the point a; it is a quadratic form on the tangent space TaS. Many interesting things can be read off from the numbers A2,o, Ai,i, and Ao,2, in particular, the mean curvature and the Gaussian curvature, both generalizations of the single curvature of smooth curves.
Definition 3.9.7 (Mean curvature, mean curvature vector). The mean curvature H of a surface at a point a is H(a)
~ ~(A2,o + Ao,2).
3.9.30
The mean curvature vector is H(a) ~ H(a)v3, where v3 is the chosen unit normal.
3.9 Geometry of curves and surfaces
385
Changing the adapted coordinate system by choosing the normal to be
v3 instead of V3 (i.e., changing the sign of Z) changes the sign of the mean curvature H(a), but does not change the mean curvature vector. The fiat disc is the minimal surface bounded by a circle; the hemisphere bounded by the same circle is not. Very nice minimal surfaces can be formed by twisting a wire to form a closed curve and dipping it into a soapy solution. The mean curvature of the soap film is zero. You may imagine that your wire loop is fl.at, but it can have many other shapes, and the soap films that span the loop will still be "fl.at" as far as mean curvature is concerned.
The mean curvature vector is fairly easy to understand intuitively. If a surface evolves so as to minimize its area with boundary fixed, like a soap film spanning a loop of wire, the mean curvature points in the direction in which the surface moves (this statement will be justified in Section 5.4; see Theorem 5.4.4 and Corollary 5.4.5). A minimal surface is one that locally minimizes surface area among surfaces with the same boundary; according to this description a surface is minimal precisely when the mean curvature vector vanishes. The Gaussian curvature does not depend on the choice of the normal V3. It is the prototype of all the really interesting things in differential geometry. It measures to what extent pieces of a surface can be made flat, without stretching or deformation  as is possible for a cone or cylinder, but not for a sphere.
Definition 3.9.8 (Gaussian curvature of a surface). The Gaussian curvature K of a surface at a point a is Exercise 3.9.3 asks you to show that the Gaussian curvature of the unit sphere is 1.
K(a) ~ A2,0Ao,2
At1·
3.9.31
It follows (see Exercise 3.28) that if at some point the quadratic terms of the function f representing S in adapted coordinates form a positive definite or negative definite quadratic form, the Gaussian curvature at that point is positive; otherwise, the Gaussian curvature is nonpositive.
Theorem 3.9.9, proved in Section 5.4, says that the Gaussian curvature measures how big or small a surface is compared to a flat surface. It is a version of Gauss's Theorema Egregium (Latin for remarkable theorem).
Theorem 3.9.9. Let Dr(x) be the set of all points q in a surface S C R 3 such that there exists a curve of length :5 r in S joining x to q. Then Area(Dr(x)) = ....____area of curved disc
~

K~~)7r r 4 + o(r4 ).
3.9.32
area of flat disc
If the curvature is positive, the curved disc Dr(x) is smaller than the flat disc, and if the curvature is negative, it is larger. The discs have to be measured with a tape measure contained in the surface; in other words, Dr(x) is the set of points that can be connected to x by a curve contained in the surface and of length at most r. An obvious example of a surface with positive Gaussian curvature is the surface of a ball. Wrap a napkin around a basketball; you will have extra fabric that won't lie smooth. This is why maps of the earth always distort areas: the extra "fabric" won't lie smooth otherwise. An example of a
386
If you have ever sewed a setin sleeve on a shirt or dress, you know that when you pin the under part of the sleeve to the main part of the garment, you have extra fabric that doesn't lie flat; sewing the two parts together without puckers or gathers is tricky, and involves distorting the fabric. Sewing is something of a dying art, but the mathematician Bill Thurston, whose geometric vision was legendary, maintained that it is an excellent way to acquire some feeling for the geometry of surfaces.
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
surface with negative Gaussian curvature is a mountain pass. Another is an armpit. Figure 3.9. 7 gives another example of Gaussian curvature.
FIGURE 3. 9. 7. The Billy Goat Gruff in the foreground gets just the right amount of grass to eat; he lives on a flat surface, with Gaussian curvature zero. The goat at far left is thin; he lives on the top of a hill, with positive Gaussian curvature; he can reach less grass. The third is fat. His surface has negative Gaussian curvature; with the same length chain, he can get at more grass. This would be true even if the chain were so heavy that it lay on the ground.
Computing curvature of surfaces Proposition 3.9.11 tells how to compute curvature for a surface in any coordinate system. First we use Proposition 3.9.10 to put the surface into "best" coordinates, if it is not in them already. It is proved in Appendix A15.
Equation 3.9.33: There is no constant term because we translate the surface so that the point under consideration is the origin.
Proposition 3.9.10 (Putting a surface into "best" coordinates). Let U c JR.2 be open, f : U > JR. a C 2 function, and S the graph of f. Let the Taylor polynomial off at the origin be z
= f ( ~) = aix + a2y + ~ (a2,ox 2 + 2a1,1XY + ao,2Y2) + · · · . 3.9.33 Set c=
Ja~ +a~.
3.9
The number c = ../ai + a~ can be thought of as the "length" of the linear term. If c = 0, then ai = a2 = 0 and there is no linear term; we are already in best coordinates. The vectors in equation 3.9.34 are the vectors denoted v1, v2, v3 at the beginning of the subsection. The first vector is a horizontal unit vector in the tangent plane. The second is a unit vector orthogonal to the first, in the tangent plane. The third is a unit vector orthogonal to the previous two. (It is a downwardpointing normal.) It takes a bit of geometry to find them, but the proof of Proposition 3.9.10 will show that these coordinates are indeed adapted to the surface.
Geometry of curves and surfaces
387
If c # 0, then with respect to the orthonormal basis
v2=
[ck] [kl c~
va=
v'l +c2
~
3.9.34
v'l +c2
S is the graph of Z as a function
F
(9) =
~ ( A2,oX 2 + 2A1,1XY + Ao,2Y 2 )
+ ···
3.9.35
which starts with quadratic terms, where 1 ( A 2,0 = 2 ~ a2,oa22  2a1,1a1a2 + ao,2a 12) c v.i+c2
A1,1 = c2(l
~ c2) (a1a2(a2,o 
ao,2)
+ a1,1(a~  a~))
1 ( 2 Ao,2  c2(l + c2) 312 a2,oa1 + 2a1,1a1a2
3.9.36
+ ao,2a22) .
Proposition 3.9.11 (Computing curvature of surfaces). and ff. be as in Proposition 3.9.10.
Let S
1. The Gaussian curvature of S at the origin is Note the similarities between the equations of Proposition 3.9.11 and equation 3.9.5 for the curvature of a plane curve. In each case the numerator contains second derivatives (a2,o, ao,2, etc., are coefficients for the seconddegree terms of the Taylor polynomial) and the denominator contains something like 1 + IDJl 2 (the ai and a2 of c2 = ai + a~ are coefficients of the firstdegree term). A more precise relation can be seen if you consider the surface of equation z = f(x), y arbitrary, and the plane curve z = f(x}; see Exercise 3.9.5.
K(O) =
a a
a2
2,(1~c~)21,1
3.9.37
2. The mean curvature of S at the origin with respect to ff. is
H(O) = 2 (1 +1c2) 3/ 2 (a2,o(l + a 22 )  2a1a2a1,1 + ao,2(1 + a 12 ) ) . 3.9.38
Proposition 3.9.11 gives a reasonably straightforward way to compute the curvature and mean curvature of any surface at any point. It is stated for a surface representing z as a function
f (~)
such that
f (
8)
= 0 (a
surface that passes through the origin); it gives the curvatures at the origin in terms of coefficients of the Taylor polynomial of f at (
g) . For any point
a of a surface S, we can translate the coordinates to put a at the origin and perhaps rename our coordinates, to bring our surface to this situation. Note that even if we cannot find explicitly a function of which Sis locally the graph, we can find the Taylor polynomial of that function to degree 2, using Theorem 3.4. 7.
388
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
Proof of Proposition 3.9.11. For the Gaussian curvature, plug into Definition 3.9.8 the values for A2,o, Al,1, Ao,2 given in Proposition 3.9.10: K(O) = A2,0Ao,2  At1
= c4(l: c2) 2 ( (a2,oa~ 2 a2,oao,2  al,1 (1 + c2)2
(a2,oa~ + 2a1,1a1a2 + ao,2a~)
2a1,1a1a2 + ao,2aD
 (a2,oa1a2 +
al,1a~  al,1a~ 
3.9.39 ao,2a1a2) 2)
This involves some quite miraculous cancellations, as does the computation for the mean curvature, which is left as Exercise A15.1. D Example 3.9.12 {Computing the Gaussian and mean curvature of a surface given as the graph of a function). Suppose we want to measure the Gaussian curvature at a point a
~ ( a2 b2 ~ )
of the surface
given by the equation z = x 2  y 2 and shown in Figure 3.9.8. To determine the Gaussian curvature at the point a, we make a our new origin; i.e., we use new translated coordinates, u, v, w, where FIGURE
The graph of saddle.
3.9.8.
x2 
x=a+u
y 2 , a typical
3.9.40
y=b+v z
=
a2
b2
+w.
(The uaxis replaces the original xaxis, the vaxis replaces the yaxis, and thewaxis replaces the zaxis.) Now we rewrite z = x 2  y 2 as a 2  b2 + w =(a+ u) 2  (b + v) 2 ...... ......_,_...... "'...'
z
3.9.41
x2
=
a 2 + 2au + u 2  b2  2bv  v 2,
which gives w
= 2au 
1 2bv + u 2  v 2 = 2a u + 2b v + 2 ( 2 u 2 + 2 v 2). 3.9.42 ~ ~ ~ ~ aa,o
ao,2
Now we have an equation of the form of equation 3.9.33, and (remembering that c = y'a~ +a~) we can read off the Gaussian curvature, using Proposition 3.9.11: The Gaussian curvature at a is
aa,oao,2a~. 1
(2. 2)  0 4 K(a) = (1+4a2 + 4b2)2 = (1+4a2 + 4b2)2.
3.9.43
3.9 Geometry of curves and surfaces
389
Looking at this formula for K, what can you say about the Gaussian curvature of the surface the further one goes from the origin? 21 Similarly, we can compute the mean curvature: 4(a 2  b2)
3.9.44
H(a) = (1+4a2 + 4b2)3/2.
Example 3.9.13 (Computing the Gaussian and mean curvature of the helicoid). The helicoid is the surface of equation ycosz = xsinz. You can imagine it as swept out by a horizontal line going through the zaxis, and which turns steadily as the zcoordinate changes, making an angle z with the parallel to the xaxis through the same point, as shown in Figure 3.9.9. A first thing to observe is that the mapping FIGURE 3.9.9. The helicoid is swept out by a horizontal line, which rotates as it is lifted. "Helicoid" comes from the Greek for "spiral".
x) ( ~
( f+
3.9.45
is a rigid motion of R 3 that sends the helicoid to itself. (Can you justify that statement? 22 ) In particular, setting a= z, this rigid motion sends any point to a point of the form (
The mapping in Example 3.9.13 simultaneously rotates by a clockwise in the (x, y)plane and lowers by a along the zaxis. The first two rows of the right side of equation 3.9.45 are the result of multiplying [ : ] by the rotation matrix
xcosa + ysina) xsinza_+;;cos a
~)
(do you see why? 23 ), and it is enough to
compute the Gaussian curvature K(r) at such a point. We don't know the helicoid as a graph, but, as Exercise 3.11 asks you to show, by the implicit function theorem, the equation of the helicoid determines z as a function g, (
~) ne.. ( ~) when r f
O. What we need
then is the Taylor polynomial of Yr· Introduce the new coordinate u such that r + u = x, and write
we already saw in formula 3.9.10. Yr (
~) = z = a2y + ai,1 uy + ~ao,2Y 2 +
··· .
3.9.46
Part b of Exercise 3.11 asks you to justify our omitting the terms aiu and a2,ou 2 Now write ycosz and (r + u) sinz using the Taylor polynomials of cosz and sin z, keeping only quadratic terms, to get 21 The Gaussian curvature of this surface is always negative, but the further you go from the origin, the smaller it is, so the flatter the surface. 22 1£ you substitute xcosa + ysina for x, xsina + ycosa for y, and z  a for z in the equation y cos z = x sin z, and do some computations, remembering that cos(z  a) = coszcosa + sinzsina and sin(z  a)= sinzcosa  coszsina, you will land back on the equation ycosz = xsinz. 23 1£ z =a, plugging the values of equation 3.9.45 into ycosz = xsinz gives y cos 0 = 0, so y = 0.
390 To get equation 3.9.47, we replace cos z by its Taylor polynomial z2
l  2!
+
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature ycosz~y·l
(r
3.9.47
+ u) sinz ~ (r + u) (a2y + ai,1uy + ~ao,2Y 2 + · · ·)
3.9.48
z4
z as in equation 3.9.46
4!  ...
keeping only the first term. {The term z2 /2! is quadratic but it isn't kept since yz 2 /2! is cubic.)
~
r
ra2y + a2uy + ra1,1uy + 2ao,2Y
(In equation 3.9.48, we replace sin z by the first term of its Taylor polynomial, z; in the second line we keep only linear and quadratic terms.) So ifin ycosz = (r+u)sinz, we replace the functions by their Taylor polynomials of degree 2, we get r
When identifying linear and quadratic terms, note that the only linear term on the right of equation 3.9.49 is ra2y, giving a2 = 1/r. The coefficient of uy is a2 + ra1,1, which must be 0. Equation 3.9.50 says that the coefficients a2 and ai,1 blow up as r + 0, which is what you should expect, since at the origin the helicoid does not represent z as a function of x and y. But the curvature does not blow up, since the helicoid is a smooth surface at the origin. When we first made this computation, we landed on a Gaussian curvature that did blow up at the origin. Since this did not make sense, we checked and found that we had a mistake in our statement. We recommend making this kind of internal check {like making sure units in physics come out right), especially if, like us, you are not infallible in your computations. Knot theory is an active field of research today, with remarkable connections to physics (especially the latest darling of theoretical physicists: string theory).
2
2
y = ra2y + a2uy + ra1,1 uy + 2ao,2Y .
3.9.49
Identifying linear and quadratic terms gives (see the margin) ai = 0, a2
1
= r ,
ai 1 =
'
1
r2 , ao '2 = 0,
a2 o = 0.
'
We can now read off the Gaussian and mean curvatures: 1 1 K(r) and H(r)
.._,
.._,
Gaussian
mean curvature
curvature
=0.
3.9.50
3.9.51
The first equation shows that the Gaussian curvature is always negative and does not blow up as r + 0: as r + 0, we have K(r) + 1. This is what we should expect, since the helicoid is a smooth surface. The second equation is more interesting yet. It says that the helicoid is a minimal surface: every patch of the helicoid minimizes area among surfaces with the same boundary. /::;.
Coordinates adapted to space curves Curves in JR3 have considerably simpler local geometry than do surfaces: essentially everything about them is in Propositions 3.9.15 and 3.9.16 below. Their global geometry is quite a different matter: they can tangle, knot, link, etc. in the most fantastic ways. Suppose C c JR3 is a smooth curve, and a E C is a point. What new coordinate system X, Y, Z is well adapted to C at a? Of course, we will take a as the origin of the new system. If we require that the Xaxis be tangent to C at a, and call the other two coordinates U and V, then near a the curve C will have an equation of the form
U= V
2 + ~a3X 3 + · · · f (X) = ~a2X 2 6
= g(X) = ~b2X 2 + ~b3 X 3 + · · ·, 2
3.9.52
6
where both coordinates start with quadratic terms. But we can do better, at least when C is at least three times differentiable, and a~ + b~ "! 0.
3.9
Geometry of curves and surfaces
391
Suppose we rotate the coordinate system around the Xaxis by an angle 0, and denote by X, Y, Z the new (final) coordinates. Let c = cos(} and s =sin(} and (as in the proof of Proposition 3.9.2) write
U = cY + sZ
V = sY + cZ.
and
3.9.53
Substituting these expressions into equation 3.9.52 leads to cY + sZ
1 2 a2X
1
3
+ a3X + ··· 2 6 1 2 1 3 sY + cZ = 2°b2X + 6b3X + · · · . =
3.9.54
We solve these equations for Y and Z to get Remember that s = sin(} and c =cos(}, so c 2 + s2 = 1.
1 3 ( 2 +s 2) =Y= 1 (ca2sb2)X 2 +6(ca3sb3)X Ye 2 1 Z = 2(sa2
1 + cb2)X + 6(sa3 2
+ cb3)X
3
3.9.55
.
The point of all this is that we want to choose (} (the angle by which we rotate the coordinate system around the Xaxis) so that the Zcomponent of the curve begins with cubic terms. We achieve this by setting If a2 = b2 = 0 these values of (} do not make sense. At such a point we define the curvature to be 0 (this definition agrees with the general Definition 3.9.14, as the case where A2 = 0).
We choose ,,, = A2 to be the positive square root, +y'a~ + b~; thus our definition of curvature of a space curve is compatible with Definition 3.9.1 of the curvature of a plane curve.
The word "osculating" comes from the Latin "osculari", meaning "to kiss" . The osculating plane is the plane that the curve is most nearly in, and the torsion measures how fast the curve pulls away from it. It measures the "nonplanarity" of the curve.
cos(}=
a2
y'a~
+ b~
and
sin(}=
b2 b2 , so that tan(}=; y'a~ + b~ a2
this gives
The Zcomponent measures the distance of the curve from the (X, Y)plane; since Z is small, the curve stays mainly in that plane (more precisely, it leaves the plane very slowly). The (X, Y)plane is called the osculating plane to C at a. This is our best coordinate system for the curve at a, which exists and is unique unless a 2 = b2 = 0. The number A 2 ~ 0 is called the curvature "' of Cat a, and the number B3/A2 is called its torsion T.
Definition 3.9.14 {Curvature and torsion of a space curve). Let C be a space curve. The curvature of Cat a is The torsion of Cat a is
K.(a) ~ A2 T(a) ~ B3/A2.
Note that torsion is defined only when the curvature is not zero.
392
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
Parametrizing space curves by arc length: the Frenet frame
A curve in !Rn can be parametrized by arc length because curves have no intrinsic geometry; you could represent the Amazon River as a straight line without distorting its length. Surfaces and other manifolds of higher dimension cannot be parametrized by anything analogous to arc length; any attempt to represent the surface of the globe as a flat map necessarily distorts sizes and shapes of the continents. Gaussian curvature is the obstruction. Imagine that you are riding a motorcycle in the dark and that the first unit vector is the shaft of light produced by your headlight.
Usually the geometry of space curves is developed using parametrization by arc length rather than adapted coordinates. Above, we emphasized adapted coordinates because they generalize to manifolds of higher dimension, while parametrizations by arc length do not. When we use parametrization by arc length to study the geometry of curves, the main ingredient is the Prenet frame. Imagine driving at unit speed along the curve, perhaps by turning on cruise control. Then (at least if the curve really is curvy, not straight) at each instant you have a distinguished orthonormal basis of R 3 , called the Frenet frame. The first basis vector is the velocity vector f, pointing in the direction of the curve; it is a unit vector since we are traveling at unit speed. The second vector ii is the unit vector in the direction of the acceleration vector. It is orthogonal to the curve and points in the direction in which the force is being applied  i.e., in the opposite direction of the centrifugal force you feel. (We know the acceleration must be orthogonal to the curve because the speed is constant; there is no component of acceleration in the direction of motion. 24 ) The third is the binormal b, orthogonal to the other two. So, if 8 : R + R 3 is the parametrization by arc length of a curve C, and l 11 (s) =f=. 0, the three vectors are
t(s) = Ji(s),
.___,,__..., velocity vector
Proposition 3.9.15: Requiring =f; 0 im_plies that K =/; 0, since t'(O) = 8"(0) (equation 3.9.57) and t(O) = Kii(O) (Proposition 3.9.16).
6
11
In particular, formula 3.9.59 says that the point 8(0) in the
old roonlinatoo ;,, tho point (
~)
in the new coordinates, which is what we want.
_ n(s)
=
f'(s) lf'(s)I
=
811 (s) l87'(s)I'
b(s) = t(s) x ii(s).
normalized acceleration vector
3.9.57
bi normal
Propositions 3.9.15 and 3.9.16 relate the Frenet frame to adapted coordinates; they provide another description of curvature and torsion, and show that the two approaches coincide. They are proved in Appendix A15.
Proposition 3.9.15 {Frenet frame). Let 8: R+ R 3 parametrize a curve C by arc length, with 8'1 ( s) =f=. 0. Then the coordinates with respect to the F'renet frame at 0,
t(O), ii(O), b(O),
3.9.58
are adapted to the curve Cat 8(0). Thus the point with coordinates X, Y, Zin the new, adapted coordinates is the point 8(0)
+ Xt(O) + Yii(O) + Zb(O)
in the old (x, y, z)coordinates. 24 Alternatively,
you can derive 2°6' · 611 = 0 from lcYl 2 = 1.
3.9.59
3.9
Proposition 3.9.16: The derivatives are easiest to compute at 0, but the equations t'(s) = Kii(s) and so on are true at any point s (any odometer reading). Equation 3.9.60 corresponds to the antisymmetric matrix
0 A= [
K
0]
K
0
T
0
7
0
,
in the sense that
[t', ii', b'] = [f, ii, b]A. Exercise 3.9.11 asks you to explain where this antisymmetry comes from.
Propositions 3.9.17 and 3.9.18: Many texts refer to K(t) and r(t), not K('Y(t)) and r('Y(t)). However, curvature and torsion are invariants of a curve: they depend only on the point at which they are evaluated, not on the parametrization used or on time (except in the sense that time can be used to specify a point on the curve). You might think of a curvy mountain road: each point on the road has its intrinsic curvature and torsion. If two drivers both start at time t = 0, one driving 30 mph (parametrization 'Yl) and another driving 50 mph (parametrization 'Y2), they will arrive at a point at different times, but will be confronted by the same curvature and torsion.
Geometry of curves and surfaces
393
Proposition 3.9.16 (Frenet frame related to curvature and torsion). Let 8: JR+ JR 3 parametrize a curve C by arc length. Then the F'renet frame satisfies the following equations, where "" is the curvature of the curve at 8(0) and r is its torsion:
f' (0) =
Kn(O)
+ rb(O)
n'(O) = Kt(O) b'(O) =
3.9.60
 n1(0).
The derivatives i', n', and b' are computed with respect to arc length. Think of the point 8(0) as the place where you set the odometer reading to 0, accounting for i'(o), etc.
Computing curvature and torsion of parametrized curves We can now give formulas (in Propositions 3.9.17 and 3.9.18) that make the computation of curvature and torsion straightforward for any parametrized curve in Jl3 . We already had two equations for computing curvature and torsion of a space curve: equations 3.9.56 and 3.9.60, but they are hard to use. Equation 3.9.56 requires knowing an adapted coordinate system, which leads to cumbersome formulas. Equation 3.9.60 requires a parametrization by arc length; such a parametrization is only known as the inverse of a function which is itself an indefinite integral that can rarely be computed in closed form. The Frenet formulas can be adapted to any parametrized curve, and give the following propositions.
Proposition 3.9.17 (Curvature of a parametrized curve). The curvature "" of a curve parametrized by 'Y : JR + JR3 is
K("'(t)) = 11'(t) X 1"(t)1 • I 17'(t)l 3
3.9.61
Proposition 3.9.18 (Torsion of a parametrized curve). The torsion r of a curve parametrized by 'Y : JR + JR3 is
r('Y(t)) =
Example 3.9.19. Let 1(t)
f(t)
~ c~).
(1'(t) x 1"(t)) . 7111 (t) · 17'(t) x 7"(t)l 2
~ ( ~) . Then
f'(t)
~OJ
'"'") ~ m
3.9.62
3.9.63
Chapter 3.
394
Manifolds, Taylor polynomials, quadratic forms, curvature
So we find
1
K(y(t)) = (1+4t2 + 9t4)3/2
(1) (0):t ~t2 X
3.9.64
(1+9t2+9t4)1/2 = 2''~ (1 + 4t2 + 9t4 )3/2 and
( ~:) (~)6
2
Sll>?(t)
~
m, 
ha~
y = X 2 and z = X 3 ' so equation 3.9.56 says that y = X 2 = ~X 2 , so A2 = 2 ....
 _ 1 + 9t23 + 9t4 .
3.9.65
At the origin, the standard coordinates are adapted to the curve, so from equation 3.9.56 we find A2 = 2, B3 = 6, giving K = A2 = 2 and T = B3/A2 = 3. This agrees with equations 3.9.64 and 3.9.65 when t = 0. .6.
Proof of Proposition 3.9.17 {Curvature of a parametrized curve). We will assume that we have a parametrized curve 1': JR~ JR3 ; you should imagine that you are driving along some winding mountain road, and that y(t) is your position at time t. Since our computation will use equation 3.9.60, we will also use parametrization by arc length; we will denote by 8(s) the position of the car when the odometer is s, while 1' denotes an arbitrary parametrization. These are related by the formula
y(t) = 8(s(t)),
where
s(t) =
1.t 1f(u)I du,
3.9.66
to
Equation 3.9.68, second line: Note that we are adding vectors to get a vector: 1t(s(t))(s'(t)) 2 ii(s(t)) ~..._,_,__,
number
vector
+ s"(t) t(s(t)) ~'v'
no.
vec.
Equation 3.9.69: The derivative K: 1 is the derivative of K: with respect to arc length. We use equation 3.9.69 in the proof of Proposition 3.9.18, but state it here because it fits nicely.
and t 0 is the time when the odometer was set to 0. The function t 1+ s(t) gives you the odometer reading as a function of time. The unit vectors t, ii and b will be considered as functions of s, as will the curvature K and the torsion T. 25 We now use the chain rule to compute three successive derivatives of 1'. In equation 3.9.67, recall (equation 3.9.57) that 81 = t; in the second line of equation 3.9.68, recall (equation 3.9.60) that t'(O) = Kii(O). This gives
1'(t) = (8'(s(t))s'(t) = s'(t)t(s(t)),
3.9.67
1"(t) = t'(s(t)) (s'(t)) 2 + t(s(t))s"(t) =
K(s(t)) (s'(t)) 2ii(s(t)) + s"(t)t(s(t)),
3.9.68
25 Strictly speaking, t, ii, b, K:, and r should be considered as functions of c5(s(t)): the point where the car is at a particular odometer reading, which itself depends on time. However, we hesitate to make the notation any more fearsome than it is.
3.9
Geometry of curves and surfaces
395
1"'(t) = 11:'(s(t))ii(s(t)) (s'(t)) 3 + 11:(s(t))ii'(s(t)) (s'(t)) 3 + 211:( s(t) )ii( s(t)) (s' (t)) (s" (t)) + f' (s(t)) (s' (t)) (s" (t)) + t(s(t)) (s"'(t)) = ((11:(s(t))) 2 (s'(t)) 3
+s"'(t))t(s(t)~
3.9.69
+ ( 11:'(s(t)(s'(t)) 3 + 311:(s(t))(s'(t))(s"(t)) )ii(s(t)) + ( 11:(s(t))r(s(t)) (s'(t)) 3 )b(s(t)). Equation 3.9.71: By equations 3.9.67 and 3.9.68,
?Ct) x ?'Ct) = s' (t)t(s(t))
Since
f
has length 1, equation 3.9.67 gives us
which we already knew from the definition of s. Equations 3.9.67 and 3.9.68 give
x [11:(s(t))(s1 (t)) 2 ii(s(t))
+ s" (t)t(s(t)) J. Since for any vector
tE
since the cross product is distributive, and since (equation 3.9.57)
b,
1'(t) x ?"(t) = 11:(s(t))(s'(t)) 3 b(s(t)), since f x ii =
b.
Since
R3 ,
ix t = 0,
t x ii =
3.9.70
s'(t) = 1?'(t)I,
this gives
1 (t) x 7" (t) = s'(t)t(s(t))
3.9.71
b has length 1, 3.9.72
1?'(t) x 1"(t)1=11:(s(t))(s'(t))3.
Using equation 3.9.70, this gives the formula for the curvature of Proposition 3.9.17. D
Proof of Proposition 3.9.18 (Torsion of a parametrized curve). Since x 1'' points in the direction of b, dotting it with 1"' picks out the coefficient of b for 1"'. This leads to
7'
(?'(t) x ?"(t)). ?"'(t) = r(s(t)) ( 11:(s(t))
x [11:(s(t))(s'(t)) 2 ii(s(t))J
r
(s'(t)) 6
D
.
3.9.73
square of equation 3.9.72
= 11:(s(t))(s (t)) 3 b(s(t)). 1
EXERCISES FOR SECTION 3.9.1
3.9
a. What is the curvature of a circle of radius r?
b. What is the curvature of the ellipse of equation :: 3.9.2
+ ~:
What is the curvature of the hyperbola of equation :: 
= 1 at (
~:
~) ?
= 1 at (
~)?
3.9.3 Show that the absolute value of the mean curvature of the unit sphere is 1 and that the Gaussian curvature of the unit sphere is 1. 3.9.4 a. What is the Gaussian curvature of the sphere of radius r? b. What are the Gaussian and mean curvature of the ellipsoid of equation
x2 + b
a2
y2 2
z2
+ c2
= 1,
•
at a pomt
(
u) ?
~
396
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
3.9.5 Check that if you consider the surface of equation z = f(x), y arbitrary, and the plane curve z = f(x), the absolute value of the mean curvature of the surface is half the curvature of the plane curve.
Useful fact for Exercise 3.9.7: The arctic circle is those points that are 2 607.5 kilometers south of the north pole. "That radius" means the radius as measured on the surface of the earth from the pole, i.e., 2607.5 kilometers.
3.9.6
What are the Gaussian and mean curvature of the surface of equation
~a2  ~ + ~ b2
c2
= 1,
at
(u) :
?
3.9. 7 a. How long is the arctic circle? How long would a circle of that radius be if the earth were fiat?
b. How big a circle around the pole would you need to measure in order for the difference of its length and the corresponding length in a plane to be one kilometer? 3.9.8
a. Draw the cycloid given parametrically by (
x) = ( alcost a(( t  sin t )) ) .
Y
b. Can you relate the name "cycloid" to "bicycle"? c. Find the length of one arc of the cycloid. 3.9.9
3.9.10. Part d: The catenoid of equation y 2 + z 2 = (coshx) 2 . FIGURE FOR EXERCISE
Repeat Exercise 3.9.8 for the hypocycloid (
J"(x)
b.ShowthatK(x)=
c. Show that
is a mapping I >+ S0(3), the space of orthogonal 3 x 3 matrices with determinant + 1. So
t
>+
2.
f(x) ( 1+(t'(x))
[t(t), ii(t), b(t)] = T(t)
>+
1 H(x) = (1 + (f'(x))2)3/2
2
)
(IIf (x) 
d. Show that the catenoid of equation y 2 + margin) has mean curvature 0.
r 1 (to)T(t)
is a curve in 80(3) that passes through the identity at to.
= ( : ~~:::).
3.9.10 a. Let f : [a,b] . IR be a smooth function satisfying f(x) > 0, and consider the surface obtained by rotating its graph around the xaxis. Show that the Gaussian curvature K and the mean curvature H of this surface depend only on the xcoordinate.
Exercise 3.9.11: The curve F: t
~)
l+(f'(x)) 2 ) f(x) . z2
= (coshx) 2 (shown in the
*3.9.11 Use Exercise 3.2.11 to explain why the Frenet formulas give an antisymmetric matrix. 3.9.12
Prove Proposition 3.9.6, using Proposition 3.9.16.
3.10 REVIEW EXERCISES FOR CHAPTER 3 3.1 a. Show that the set X C IR3 of equation x 3 + xy 2 + yz 2 + z 3 = 4 is a smooth surface.
b . Gire the "'luaUons of the
tang~t plane and tangent •PO£• to X at ( : ) .
3.10
Review exercises for Chapter 3
397
3.2 a. For what values of c is the set of equation Yc = x 2 +y 3 + z 4 =ca smooth surface?
Exercise 3.2: We strongly advocate using MATLAB or similar software.
b. Sketch this surface for a representative sample of values of c (for instance, the values 2, 1, 0, 1, 2). c. Give the equations of the tangent plane and tangent space at a point of the surface Ye. 3.3 Consider the space X of positions of a rod of length 2 in JR3 , where one endpoint is constrained to be on the sphere of equation (x  1)2 + y 2 + z 2 = 1, and the other on the sphere of equation (x + 1) 2 + y2 + z 2 = 1.
1 1
0 1 1
0 Point for Exercise 3.3, parts b and c.
a. Give equations for X as a subset of R 6 , where the coordinates in JR6 are the roo«linatos (
~: )
of the end of the rod on the fust •phere, and the roonlinatos
(;,) of the othfil end of the rod. b. Show that near the point in R 6 shown in the margin, the set X is a manifold. What is the dimension of X near this point? c. Give the equation of the tangent space to the set X, at the same point as in part b.
Exercise 3.4: The notation p, q means the segment going from p to q.
3.4
Consider the space X of triples
such that y =I 0 and the segments p, q and q, r form an angle of 7r / 4.
a. Wdte an equation f (
~ ) ~ 0 whicl> all points of X will •atisfy.
b. Show that X is a smooth surface.
'· Ture o' fame? Let • O,! (
D.
Then • E X, and
n"'' • the •urlace X
is
locally the graph of a function expressing z as a furnction of x and y. d. What is the tangent plane to X at a? What is the tangent space TaX ? 3.5
Find the Taylor polynomial of degree 3 of the function
f
3.6
G) ~
Show that if f
+ z)
at the point
7r/6) ( 7r/4 . 7r/3
( ~) = r.p (x  y) for some twice continuously differentiable
function r.p : R+ IR, then 3. 7
•in(x + y
DU  DU =
0.
Write, to degree 3, the Taylor polynomial PJ,o of
f ( ~) =cos( 1 + sin(x2 + y)) at the origin.
398
Chapter 3.
Manifolds, Taylor polynomials, quadratic forms, curvature
*3.8 a. Let Mi (m, n) C Mat (m, n) be the subset of matrices of rank 1. Show that the mapping cp1 : (Rm  {O} )x IR.nl >Mat (m, n) given by
~{~ [~]) ~ [~A,~ ... ,>.a] is a parametrization of the open subset U1 C M 1(m,n) of those matrices whose first column is not 0. b. Show that Mi(m,n)  U1 is a manifold embedded in M 1 (m,n). What is its dimension? c. How many parametrizations like cp1 do you need to cover every point of Mi(m,n)? *3.9 A homogeneous polynomial in two variables of degree four is an expression of the form p(x, y) = ax 4 + bx 3 y + cx 2 y 2 + dxy 3 + ey4 • Consider the function A homogeneous polynomial is a polynomial in which all terms have the same degree.
if(:)#(~) if(:)=(~), where p is a homogeneous polynomial of degree 4. What condition must the coefficients of p satisfy in order for the crossed partials D1 ( D2 (!)) and D2 ( D1 (!)) to be equal at the origin? a. Show that yeY = x implicitly defines y as a function of x, for x b. Find a Taylor polynomial of the implicit function to degree 4.
3.10
Exercise 3.11 is relevant to Example 3.9.13. Hint for part b: The xaxis is contained in the surface.
3.11
~ g,. ( ; ) neadhe point
b. Show that D1gr (
q,
m~ n~ ;l dot
3.12
Q(M)
O) =
Dfor (
=
xsinz expresses z implicitly as a
0) = 0.
On IR.4 as described by M =
= det
0.
m.
a. Show that the equation ycosz
funoUon z
~
[~
~] ,
consider the quadratic form
M. What is its signature?
a. Are the functions Q 1 and Q2 in the margin quadratic forms on IR.3? b. For any that is a quadratic form, what is its signature? Is it degenerate or nondegenerate?
Q,m~""[: ~ ~l
3.13
Functions for Exercise 3.13
Let Pk be the space of polynomials p of degree at most k. a. Show that the function 8a : Pk > JR. given by 8a (p) = p( a) is a linear function. b. Show that 80 , .•. , 8k are linearly independent. First say what it means, being careful with the quantifiers. It may help to think of the polynomial
Exercise 3.14 part c: There is the clever way, and then there is the plodding way.
3.14
x(x  1) · · · ( x  (j  1)) (x  (j
+ 1)) · · · (x 
k),
which vanishes at O, 1, ... ,j  1,j + 1, ... , k but not at j.
r r
c. Show that the function Q(p) = (p(O)
(p(l)
+ ... + (l)k (p(k)
r
3.10
Review exercises for Chapter 3
399
is a quadratic form on Pk. When k = 3, write it in terms of the coefficients of p(x) = ax 3
+ bx 2 +ex+ d.
d. What is the signature of Q when k
= 3?
= [ ~ ~] definite quadratic form if and only if det G > 0, a + d > 0. 3.15
Show that a 2 x 2 symmetric matrix G
represents a positive
3.16 Let Q be a quadratic form. Construct a symmetric matrix A as follows: each entry Ai,i on the diagonal is the coefficient of x~, while each entry A,; is half the coefficient of the term XiXj.
a. Show that Q(x) = x.
Exercise 3.16, part b: Consider
Q(ei) and Q(ei + e;).
Ax.
b. Show that A is the unique symmetric matrix with this property. 3.1 7
a. Find the critical points of the function
f ( ~) = 3x2

6xy + 2y 3 •
b. What kind of critical points are they? Exercise 3.18, part a: This is easier if you use
3.18
a. What is the Taylor polynomial of degree 2 of the function
f ( ~) = sin(2x + y) at the point (
sin( a+ /3) = sinacos/3 + cosasin/3.
b. Show that (
Fm~dot[:: ~] Function of Exercise 3.19
f ( ~) +
i(
2x + y  2; )  ( x 
~~~) ?
i r has a critical point at
~~~) . What kind of critical point is it? The function in the margin has exactly five critical points.
3.19
a. Find them. b. For each critical point, what are the quadratic terms of the Taylor polynomial at that point? c. Say everything you can about the type of critical point each is. 3.20 a. Find the critical points of xyz, if x, y, z belong to the surface S of equation x + y + z 2 = 16.
b. Is there a maximum on the whole surface; if so, which critical point is it? c. Is there a maximum on the part of S where x, y, z are all positive? 3.21 Let A, B, C, D be a convex quadrilateral in the plane, with the vertices free to move but with a the length of AB, b the length of BC, c the length of CD, and d the length of DA, all assigned. Let
0 there exists N such that
L
voln(C) $
4.1.62
€.
CE VN(llln)
cnx ¥4> Proposition 4.1.24 (Scaling volume). If A C llln has volume and t E lll, then tA has volume, and 4.1.63
Proof. By Proposition 4.1.20, this is true if A is a parallelogram, in particular, a cube CE VN. Assume A is a subset of llln whose volume is well defined. This means that lA is integrable, or, equivalently, that lim
N+oo
~
L....,,
CEVN,CCA
voln (C) =
lim
voln(C),
N+oo CEVN,CnA'i'U)
4.1.64
Chapter 4.
414
Integration
and that the common limit is voln(A). Since
LJ
u
tC c tA c
CE"DN,CCA
tC,
4.1.65
CE"DN ,CnAi(J)
and since voln(tC) = ltlnvoln(C) for every cube, this gives Equation 4.1.66: The integral in the first line equals the corresponding lower sum, giving the inequality:
=L
(
L
CEVN,CCA
L
ltln
voln(C) =
CE"DN,CCA
J
L
lte::; L(ltA)
CE"DN,CCA
voln(C).
lte)
4.1.66
CE"DN,CnAf. ~
The outer terms have a common limit as N and
:::; L(ltA).
voln(tA) =
+
oo, so L(ltA) = U(ltA),
J
ltA(x)lcflxl =!tin voln(A).
EXERCISES FOR SECTION
D
4.1.67
4.1
4.1.1 a. What is the twodimensional volume (i.e., area) of a dyadic cube CE V3(1R.2)? of CE V4(1R.2)? of CE V5(1R.2)?
b. What is the volume of a dyadic cube C E V 3(R3)? of C E V 4(R 3)? of CE Vs(R3 )? Dyadic cubes for Exercise 4.1.3:
b. C(
a.
c.
C(
0) 1 3 1 ' 1
d. C(
0) 1 ,2 3
0) 1 ,3 4
4.1.2 In each group of dyadic cubes below, which has the smallest volume? the largest? a. C (;) 04 ; C (;) ,2; C (;) 06 b. CE V2(R 3); CE V 1 (R3); CE V 8 (R3) 4.1.3 What is the volume of each of the dyadic cubes in the margin at left? What dimension is the volume? (For example, are the cubes twodimensional? threedimensional?) What information is given that you don't need to answer those two questions? 4.1.4
Prove Proposition 4.1.23.
4.1.5
a. Calculate I:~=O i.
b. Calculate directly from the definition the integrals Exercises 4.1.5 and 4.1.6: In parts c and d, you need to distinguish between the cases where a and bare "dyadic" (i.e., endpoints of dyadic intervals), and the case where they are not.
l
xl[o,1)(x)jdxj,
l
xlro,11(x)jdxj,
l
xlco,11(x)ldxl,
l
xlco,1)(x)jdxj.
In particular, show that they all exist and that they are equal. c. Choose a
l
> 0, and calculate directly from the definition the integrals
xl[o,a)(x)jdxj,
l
xl[o,aJ(x)jdxj,
l
xlco,aJ(x)jdxj,
l
xl(o,a)(x)jdxj.
Defining the integral
4.1
415
d. If 0 xl(a,b], xl(a,b) are all integrable and compute their integrals. (The first is shown in the figure in the margin.) In particular, show that they all exist and that they are equal. 4.1.6
a. Calculate L~o i 2 .
b. Choose a
k.
> 0,
and calculate directly from the definition the integrals
l
x 2 l[o,a)(x) ldxl,
x 2 l[o,aJ(x) ldxl,
l
x 2 l(o,aJ(x) ldxl,
l
x 2 l(o,a)(x) ldxl.
In particular, show that they all exist and that they are equal.
a 4.1.5. The dark line is the graph of the function xl[o,a)(x). FIGURE FOR EXERCISE
c. IfO x 2 1(a,b)• x 2 1(a,b) are all integrable and compute their integrals, which are all equal. 4.1. 7
Prove the general case of Proposition 4.1.16.
4.1.8
Prove that the distance Ix yl between two points x, yin the same cube
c E VN(llr) is ~ 4.1.9
::: .
Let QC IR 2 be the unit square 0 ~ x, y ~ 1. Show that the function
f (
y
~)
= sin(x  y)lq (
~)
is integrable by providing an explicit bound for UN(!)  LN(f) that tends to 0 as N> oo.
I
4.1.10
1/2
a. What are the upper and lower sums U1(f) and L1(f) for the function if 0 < x, y < 1
1/2
I
FIGURE FOR EXERCISE
x
4.1.10.
otherwise, i.e., the upper and lower sums for the partition left?
1J1 (IR 2 ),
shown in the figure at
b. Compute the integral off and show that it is between the upper and lower sum. 4.1.11
Define the dilation by a of a function
Daf(x) = f
f : !Rn >IR by
(~)
·
Show that if f is integrable, then so is D 2 N f, and
r DzN f(x) Wxl =
}Rn
2nN
r f(x) Wxl.
}Rn
4.1.12 Recall that the dyadic cubes are half open, half closed. Show that the closed cubes also have the same volume. (This result is obvious, but remarkably harder to pin down than you might expect.) 4.1.13
Complete the proof of Lemma 4.1.19.
4.1.14
Consider the function
f(x)
={~
if x
fl. [O, 1], or x
is rational
if x E [O, 1], and xis irrational.
a. What value do you get for the "left Riemann sum", where for the interval k+ Ck,N = x 2kN ~ x < ~ you choose the left endpomt k/2 N ? For the sum
{ I
1}
.
416
Chapter 4.
Integration
you get when you choose the right endpoint ( k + 1) /2N? The midpoint Riemann sum? b. What value do you get for the "geometric mean" Riemann sum, where the point you choose in each Ck,N is the geometric mean of the _two endpoints,
4.1.15 a. Show that if X and Y have volume 0, then XnY, Xx Y, and XUY have volume 0.
Exercise 4.1.16: The unit cube has the lower left corner anchored at the origin. Exercise 4.1.16 shows that the behavior of an integrable function f : Rn + R on the boundaries of the cubes ofVN does not affect the integral.
b. Show that { (
~i)
e. Show that { (
~)
c
t
CJ= {x
< ki2N + E Rn I ~ 2N <  x,_
1}
I;~
0, then choose Ni to make
uNi {f) then choose N2
> Ni
LN1 (f) < €/2,
to make vol{XavN2 ) 0
0
U N{f)  LN{f)
< E/2.
< f.
Now show that for N
> N2,
and
TJN(f)  LN(f) < f. d.
Suppose = 0.
f : Rn
+
R is integrable, and that
f(x) = f(x).
Show that
fJRn f Wxl
4.1.17 An integrand should take a piece of the domain and return a number, in such a way that if we decompose a domain into little pieces, evaluate the integrand on the pieces, and add, the sums should have a limit as the decomposition becomes infinitely fine (and the limit should not depend on how the domain is decomposed). What will happen if we break up (0,1] into intervals [xi,Xi+i],
4.2
Probability and centers of gravity
for i = 0, 1, ... , n  1, with 0 = xo < X1 < · · · < following numbers to each of the [xi, Xi+i]? a.
lxi+i  xil 2
d.
l(xi+1) 2 
(xi)21
b.
sin lxi 
e.
l(xH1) 3 
Xi+i
I
Xn
= 1, c.
417
and assign one of the
v'lxi  Xi+1 I
(xi) 3 1
4.1.18 As in Exercise 4.1.17, what will happen if we break up [O, 1] into intervals [xi,XH1], for i = 0, 1, ... ,n  1, with 0 = Xo < x1 < · · · < Xn = 1, and assign one of the following numbers to each of the [xi, XH1]? In parts a, b, and c, the function f is a 0 1 function on a neighborhood of [O, l]. a.
f(xH1)  f(xi)
4.1.19
b.
f ( (XH1)
r
f (x~)
Repeat Exercise 4.1.18 but in R 2 , the integrand to be integrated over b, c < y < d and returns the
[O, 1] 2 • The integrand takes a rectangle a < x < number The integral of Exercise 4.1.20 is much easier to compute using Fubini's theorem (Section 4.5). This exercise should convince you that you never want to use upper and lower sums to compute integrals. To evaluate the outer sum, it is useful to remember that
fk= m(~+l) k=O
f
k2
=
m(m + 1~(2m + 1).
lb al 2 JiC='di.
4.1.20 Let a, b > 0 be real numbers, and let T be the triangle defined by x 2:: 0, y 2:: 0, x/a + y/b::; 1. Using upper and lower sums, compute the integral
l
xdxdy
= a;b.
Hint: For all c E R, the floor LcJ is the largest integer ::; c. Show that
and find the analogous formula for UN(!). In evaluating the double sum, note that the term k/2N does not depend on j, so the inner sum is just a product. 4.1.21
k=O
Let
f, g: Rn
+
R be bounded with bounded support.
a. Show that U(f + g) ::; U(f) b.
4.2
+ U(g). Find f and g such that U(f + g) < U(f) + U(g).
PROBABILITY AND CENTERS OF GRAVITY
Computing areas and volumes is one important application of multiple integrals. There are many others, in a wide range of fields: geometry, mechanics, probability, etc. Here we touch on two: computing centers of gravity and computing probabilities when they cannot be computed in terms of individual outcomes. The formulas are so similar that we think each helps in understanding the other. The center of gravity x of a body A is the point on which the body A could be balanced.
418
Chapter 4.
Integration
Definition 4.2.1 (Center of gravity of a body). Equation 4.2.1 gives the average value of the ith coordinate over the entire body. Equation 4.2.2: Integrating density gives mass. In physical situations µ ("mu") is nonnegative. In many problems in probability there is a similar function µ, giving the "density of probability".
1. If a body A C !Rn (i.e., a pavable set) is made of some homogeneous material, then the center of gravity of A is the point x whose ith coordinate is
4.2.1
2. If A has variable density given by the function µ : A the mass M(A) of such a body is
M(A)
~
t
L
µ(x)lcrxJ,
JR, then
4.2.2
and the center of gravity of A is the point X: with ith coordinate
b
_
def
FIGURE
4.2.1.
Each median cuts the triangle into two parts with equal area; the triangle, if made of wood, could be balanced on the median. The intersection of the medians is the center of gravity. The xcoordinate of the center of gravity gives the average value of x over the triangle; the ycoordinate gives the average value of y.
IA Xiµ(x)jdnxl M(A)
Xi=
4.2.3
.
Example 4.2.2 {Center of gravity). What is the center of gravity of the right triangle T shown in Figure 4.2.1? The area of T, i.e., dx dy, is ab/2. Using the techniques of Section 4.5, we can easily compute
Ir
2
Jr[ xdxdy = 6a b
and
Jr[ ydxdy = 6ab
2
4.2.4
(Exercise 4.1.20 asks you to compute these integrals with upper and lower sums, which is more painful), so by equation 4.2.1, the center of gravity of T is the point x with coordinates a2b
x=
;b 2
a
= 3 and y =
ab2
;b 2
b
=3
4.2.5
Note that this agrees with what you probably learned in high school: that the center of gravity of the triangle is the point where the medians (lines from vertex to midpoint of opposite side) intersect, and that the center of gravity cuts every median in the ratio 2:1, as shown in Figure 4.2.1. /::;.
Probability and integration Recall from Section 3.8 that in probability, a sample space S is the set of all possible outcomes of some experiment, together with a "probability measure" P assigning numbers to events (i.e., to appropriate subsets of the sample space S). This measure is required to satisfy the conditions of Definition 3.8.4. When S is finite or countably infinite, we can think of P in terms of the probability of individual outcomes. But when S = JR or S = JRk, usually the individual outcomes have probability 0. How then can we evaluate P?
4.2
Probability and centers of gravity
419
In many interesting cases, one can describe the probability measure in terms of a probability density . Definition 4.2.3 (Probability density). Let S, P be a probability space, with S C !Rn. If there exists a nonnegative integrable function µ : !Rn + JR such that for any event A
P(A) =
i
µ(x)
l~xl,
4.2.6
then µ is called the probability density of S, P.
It follows that a probability density must satisfy
µ(x)
FIGURE 4.2 .2. The French mathematician and natural historian Georges Leclerc Comte de Buffon (17071788) hit on this surprising method for computing 1r. Scientists at the University of Bath report that a species of ants appears to use a Buffon's needle algorithm to measure the area of potential nest sites.
~0
and
r µ(x) ldkxl
}Rk
= 1.
4.2.7
Example 4.2.4 (Computing 7r using Buffon's needle). Toss a needle of length 1 on a piece of lined paper, with lines parallel to the xaxis spaced 1 apart. The sample space should be the space of positions of the needle. We will be interested in the probability that the needle intersects a line, so we will disregard which line is intersected and where, and will encode a position of needle by its polar angle() E [O, 7r) and the distances E [O, 1/2] of the center of the needle to the nearest line; see Figure 4.2.3, left. (You can find simulations of the Buffon needle experiment on the Internet, including one at vis.supstat.com/2013/04/buffonsneedle/)
FIGURE 4.2.3.
LEFT: The position of a needle is encoded by the distance
s E (0, 1/2] from the center of a needle to the nearest line and the angle() E (0, 7r)
that the needle makes with the horizontal. RIGHT: The needle will intersect a line ifs< ! sinfJ. The probability density FIGURE 4.2.4. The shaded rectangle is S; the darker region A c S is the subset where a needle intersects a line.
µ(
~)
jd()dsl =
~jd()dsj,
4.2.8
models an experiment where we throw our needle "randomly", not favoring any position of the needle with respect to the lines of the paper; the 2/n is
(n
ld()dsl = 1. needed so that lio·l/2]x[0,7r] µ As shown in Figure 4.2.3, right and Figure 4.2.4, the event "the needle intersects a line" corresponds to the set A
= {s 
~ sin() < 0} ,
4.2.9
420
Chapter 4.
Integration
So the probability that the needle will intersect a line is
P(A) =
~ 'Tr
Equation 4.2.10: It may seem peculiar that an infinite number of outcomes each with probability 0 can add up to something positive (here, 2/7r), but it is the same as the more familiar notion that a line has length, while the points that compose it have length 0.
Example 4.2.5: It is perhaps surprising that this probabilistic experiment ends up giving something so simple as length. It wouldn't have come out this way if the coin had been biased. Suppose that instead of a coin you toss a die, writing a 1 if it comes up 1, and a 0 otherwise, i.e., if it comes up 2,3,4,5 or 6. This is also a "way of choosing a number at random". It also gives a probability measure on [O, 1], but clearly it does not correspond to a length. For instance P([O, 1/2])
= 5/6,
P([l/2, 1])
= 1/6,
and understanding the function to+ P([O, x]) for x E [0, 1] would be quite difficult. x
{ ldOdsl =
}A
~ 7r
(' 12 sinOdO =
lo
~7r
4.2.10
Thus the probability of intersecting the line is 2/7r. This provides a (cumbersome) method for computing 7r: if as above you toss a needle onto lined paper N times, and for n of the tosses the needle intersects a line, in the long run 2N/n should be approximately 'Tr. .6.
Example 4.2.5 {Heads, tails, and length). In parlor games, you may be asked to "choose a number, any number". A priori, this is meaningless, but there is a way (there are many ways) to choose x E [O, 1] "at random". Write the number in base 2, and choose the successive binary digits after the binary point by tossing a coin, writing 0 for heads and 1 for tails. The probability of any individual outcome (number in [O, 1]) is 0: the probability of any string of length m is 1/2m, since there are 2m such strings and each is equally likely. But it is reasonable to ask about the probability of events i.e., subsets of the sample space [O, 1]. For instance, it is reasonable to ask "what is the probability that the first digit after the binary point is 1", i.e., that the number is between 1/2 and 1. It should be clear that that probability is 1/2: it is precisely the probability of the first coin toss coming up tails, and is not affected by further tosses. The same argument shows that, for 0 :::; k < 2N, every dyadic interval [kj2N, (k + 1)/2N] has probability 1/2N, since the numbers in such an interval are precisely those whose first k binary digits are the digits of k, written in base 2. Thus the probability of any dyadic interval is exactly its length. Since our definition of length is in terms of dyadic intervals, this shows that for any Ac [O, 1] that has a length,
P(A)
=
l
lAidxi.
.6.
4.2.11
Definition 4.2.6 (Expected value). Let SC Rn be a sample space with probability of density µ. If f be a random variable such that f(x)µ(x) is integrable, the expected value off is E(f)
~
ls f
(x)µ(x)
l~xl
4.2.12
The other definitions are identical to those in Definitions 3.8.6, when they are expressed in terms of the expectation. Let f, g E RV(S). Then Var(/) ~ E(/2)  (E(f)) 2 ,
a(!) ~ ./Var(/)
cov(f,g) ~ E(f E(f)) (g  E(g)) = E(fg)  E(f)E(g) def cov(f, g) corr(f,g) = a(f)a(g)"
4.2.13
4.2
Probability and centers of gravity
421
Central limit theorem One probability density is ubiquitous in probability theory: the normal distribution given by The function µ is called the Gaussian; its graph is a bell curve.
µ(x) = _1_ ex2/2.
J27r
4.2.14
The theorem that makes the normal distribution important is the central limit theorem. Suppose you have an experiment and a random variable, with expected value E and standard deviation u. Suppose that you repeat the experiment n times, with results xi, ... , Xn· Then the central limit theorem asserts that the average As n grows, all the detail of the original experiment gets ironed out, leaving only the normal distribution. The standard deviation of the new experiment (the "repeat the experiment n times and take the average" experiment) is the standard deviation of the initial experiment divided by fo. The exponent for e in formula 4.2.16 is so small it may be hard to read; it is
1 n
x =(xi+···+ Xn)
4.2.15
is approximately distributed according to the normal distribution with mean E and standard deviation u / .;:;;,, the approximation getting better and better as n+ oo. Whatever experiment you perform, if you repeat it and average, the normal distribution will describe the results. Below we will justify this statement in the case of coin tosses. First let us see how to translate the statement into formulas. There are two ways of doing it. One is to say that the probability that x is between A and B is approximately
1B J21ru Vn

e
_.n("'E)2 2 " dx.
4.2.16
A
We will use the other in our formal statement of the theorem. For this we make the change of variables A = E + ua/ .;:;;,, B = E + ub/ Vn· Formula 4.2.16 puts the complication in the exponent; formula 4.2.17 puts it in the domain of integration. There are a great many improvements on and extensions of the central limit theorem; we cannot hope to touch upon them here.
Theorem 4.2. 7 (The central limit theorem). If an experiment and a random variable have expectation E and standard deviation u, then if the experiment is repeated n times, with average result x, the probability that xis between E + and E + is approximately
Jna
1 t a= .5*erf(20/sqrt(2000)) EDU> a = 0.236455371567231 EDU> b= .5*erf(40/sqrt(2000)) EDU> b = 0.397048394633966 EDU> ba ans= 0.160593023066735 The "error function" erf is related to the "standard normal distribution function" as follows:
1 {a a ) ../2if Jo e2t dt = 21 erf ( v'2 . 2
Computations like this are used everywhere: for instance, when drug companies figure out how large a population to try out a new drug on, or when industries figure out how long a product can be expected to last.
4.2.18
k=510
JlOOO
1
..j2ir 2
1·
52
e
_
1000 { x.5 )2 2 ,5
dx.
4.2.19
.51
Now we set 1000 ( x
~ ·5 ) 2 = t
2,
so that
2J1000 dx
= dt.
4.2.20
Substituting t 2 and dt in formula 4.2.19, we get 1
140/VlOOO
..j2ir
20/y'lOOO

et 2 12 dt ~ 0.1606.
4.2.21
Does this seem large to you? It does to most people.
Example 4.2.9 (Political poll). How many people need to be polled to call an election, with a probability of 953 of being within 13 of the "true value"? A mathematical model of this is tossing a biased coin, which falls heads with unknown probability p and tails with probability 1  p. If we toss this coin n times (i.e., sample n people) and return 1 for heads and 0 for tails (1 for candidate A and 0 for candidate B), the question is: how large does n need to be to achieve 953 probability that the average we get is within 13 of p? We need to know something about the bell curve to answer this, namely that 953 of the mass is within two standard deviations of the mean, as shown in Figure 4.2.5. That means that we want ~ 3 to be the standard deviation of the experiment of asking n people. The experiment of asking one person has standard deviation a = p 1  p). Of course, pis what we don't know, but the maximum of p(l  p) is 1/2 (which occurs for p = 1/2). So we will be safe if we choose n so that the standard deviation a/ y'ri, is 1 1 4.2.22 200 ; i.e., n = 10000. 2.,fii, How many would you need to ask if you wanted to be 953 sure to be within 23 of the true value? Check in the footnote below. 2 !::. 2
The number is 2500. Note that gaining 1% quadrupled the price of the poll.
4.2
Probability and centers of gravity
423
50
Figure 4.2.5 gives three typical values for the area under the bell curve. For other values, you need to use a table or software, as described in Example 4.2.8.
It is a good idea to memorize Figure 4.2.5; it allows you to distinguish significant results from those that can be explained by chance, so that you can assess results of experiments of all sorts: effectiveness of drugs, election results, etc.
993
683 953
4
3
2
1
1
2
'
3
4
FIGURE 4.2.5. For the normal distribution, 683 of the probability is within one standard deviation; 953 is within two standard deviations; 993 is within 2.5 standard deviations.
Example 4.2.10 (Real data that don't follow normal distribution). The BlackScholes formula is used to price derivatives; it revolutionized financial mathematics in the 1970s. It is based on the idea that prices oscillate around an expected price, roughly following the normal distribution3 , with a standard deviation called volatility. In this view, price fluctuations are due to a myriad of unknowable causes, which roughly cancel, and should lead to behavior like a random walk. Wild swings are assumed to be virtually impossible. This assumption was proved false on Oct. 19, 1987, when the stock markets lost more than 20 percent of their value. !:::,
EXERCISES FOR SECTION 4.2.1
4.2
Assume you are given the following 15 integers:
8, 2, 9, 4, 7, 7, 1, 12, 6, 6, 5, 10, 9, 9, 1. You are told that each number is the number of heads that came up after tossing a coin 14 times. Is this believable? Do the numbers follow a normal distribution?
4.2.2 What are the expectation, variance, and standard deviation of the function (random variable) f(x) = x for the probability density
µ(x)
1
= 2a 1[a,aj(x),
where the sample space is all of JR?
4.2.3 a. Repeat Exercise 4.2.2 for f(x) = x 2 . b. Repeat for f(x) = x 3 . 4.2.4 Let A= [a 1 , b1 ] x · · · x [an, bn] be a box in Rn, of constant densityµ= 1. Show that the center of gravity x of the box is the center of the box  i.e., the point c with coordinates Ci= (a;+ bi)/2.
3 More precisely, the lognormal distribution, in which prices are equally likely to go up or down by a given percentage, rather than a given number of dollars.
424
4.3 Theorem 4.3.9 is adequate for most functions you will meet. However, it is not the strongest In Section possible statement. 4.4 we prove a harder result: a function f : nr > JR, bounded and with bounded support, is integrable if and only if it is continuous except on a set of measure 0. The notion of measure 0 is rather subtle and surprising; with this notion, we see that some very strange functions are integrable. Such functions actually arise in statistical mechanics. In Section 4.11 we discuss Lebesgue integration, which does not require that functions be bounded with bounded support. Inequality 4.3.1:
and that oscc (f) denotes the oscillation of f over C: the difference between its supremum and infimum over C. Epsilon has the units of vol n. If n = 2, epsilon is measured in centimeters (or meters . . . ) squared; if n = 3 it is measured in centimeters (or whatever) cubed.
x
FIGURE
Integration
WHAT FUNCTIONS CAN BE INTEGRATED?
What functions are integrable? It would be fairly easy to build up a fair collection by ad hoc arguments, but instead we prove in this section three theorems answering that question. In particular, they will guarantee that all usual functions are integrable. The first, Theorem 4.3.1, is based on our notion of dyadic pavings. The second, Theorem 4.3.6, states that any continuous function on Rn with bounded support is integrable. The third, Theorem 4.3.9, is stronger than the second; it tells us that a function with bounded support does not have to be continuous everywhere to be integrable; it is enough that it be continuous except on a set of volume 0. First, we will discuss Theorem 4.3.1, based on dyadic pavings. Although the index under the sum sign may look unfriendly, the proof is reasonably easy, which doesn't mean that the criterion for integrability that it gives is easy to verify in practice. We don't want to suggest that this theorem is not useful; on the contrary, it is the foundation of the whole subject. But proving that your function satisfies the hypotheses is usually a difficult theorem in its own right. The other theorems state that entire classes of functions satisfy the hypotheses, so that verifying integrability becomes a matter of seeing whether a function belongs to a particular class.
Recall that
VN denotes the collection of all dyadic cubes at a single level N,
1
Chapter 4.
1
y
4.3.1.
Graph of the indicator function of the unit disc, lv.
Theorem 4.3.1 (Criterion for integrability). A function f: Rn t R is integrable if and only if it is bounded with bounded support, and for all E > 0, there exists N such that volume of all cubes for which the oscillation of f over the cube is >e
< €. { CE'DN
4.3.1
I osca(f)>e }
In inequality 4.3.1 we sum the volume of only those cubes for which the oscillation of the function is more than epsilon. If, by making the cubes very small (choosing N sufficiently large) the sum of their volumes is less than epsilon, then the function is integrable: we can make the difference between the upper sum and the lower sum arbitrarily small; the two have a common limit. (The other cubes, with small oscillation, contribute arbitrarily little to the difference between the upper and the lower sum.) Examples 4.3.2 (Integrable functions). Consider the indicator function lv that is 1 on a disc and 0 outside, as shown in Figure 4.3.1. Cubes C that are completely inside or completely outside the disc have oscc(lv) = 0. Cubes straddling the border have oscillation equal to 1. (Actually, these cubes are squares, since n = 2.) By making the squares small enough (choosing N sufficiently large), you can make the area of those that straddle the boundary arbitrarily small. Therefore, lv is integrable.
4.3
What functions can be integrated?
425
You may object that there will be a whole lot of squares, so how can their volume be less than epsilon? Of course, when we make the squares small, we need more of them to cover the border. But as we divide the original border squares into smaller ones, some of them no longer straddle the border. (Note that this is not quite a proof; it is intended to help you understand the meaning of the statement of Theorem 4.3.1.) Figure 4.3.2 shows another integrable function, sin~ Near 0, we see that a small change in x produces a big change in f(x), leading to a large oscillation. But we can still make the difference between upper and lower sums arbitrarily small by choosing N sufficiently large, and thus the intervals sufficiently small. Theorem 4.3.9 justifies our statement that this function is integrable. 6.
FIGURE 4.3.2. The function sin ~ is integrable over any bounded interval. Dyadic intervals sufficiently near 0 always have oscillation 2, but they have small length when the paving is fine. The center region is black because there are infinitely many oscillations in that region.
Example 4.3.3 (A nonintegrable function). The function that is 1 at rational numbers in [O, 1] and 0 elsewhere is not integrable. No matter how small you make the cubes (intervals in this case), each cube will still contain both rational and irrational numbers and will have osc = 1. 6. Proof of Theorem 4.3.1. It follows from Definition 4.1.12 that to be integrable, a function must be bounded with bounded support, so we are only concerned with inequality 4.3.1. First we will prove that the existence of an N satisfying inequality 4.3.1 implies integrability. Choose any € > 0, and let N satisfy inequality 4.3.1. Then contribution from cubes with osc>e
{ CE'DN
2 sup If Ivoln C + I oscc(f)>e }
contribution from cubes with osc:5e
€VOlnC {CE'DNIOScc(/):5< and C n Supp(!)# €. Each such cube contributes at most 2 sup lfl voln C to the maximum difference between upper and lower sums. (It is 2 sup If I rather than sup If I because the value of f over a single cube might swing from a positive number to a negative one. We could also express this difference as sup f  inf f.) The second sum concerns the cubes for which osc ::; €. We must specify that we count only those cubes for which f has, at least somewhere in the cube, a nonzero value; that is why we say { C I C n Supp(!) =f. r:b }. Since by definition the oscillation for each of those cubes is at most €, each contributes at most €voln C to the difference between upper and lower sums. We have assumed that it is possible to choose N such that the cubes for which osc > € have total volume less than €, so in the second line of inequality 4.3.2 we replace the first sum by 2€sup lfl. Factoring out €,
426
Note in inequality 4.3.2 the surprising but absolutely standard way we prove that something is zero: we prove that it is smaller than an arbitrarily small E > 0. Or we prove that it is smaller than u(E), when u is a function such that u(E) + 0 as E+ 0. Theorem 1.5.14 states that these conditions are equivalent. You might object that in inequality 4.3.4 we argue that the c:~ in the last line means the sums don't converge; yet the square of a small number is smaller yet. The difference is that inequality 4.3.4 concerns one particular c:o > 0, which is fixed, while inequality 4.3.2 concerns any c: > 0, which we can choose arbitrarily small. To review how to negate statements, see Section 0.2.
In Proposition 4.3.4, it would be simpler to write voln+l ( r(f)) = 0. But our definition for integrability requires that an integrable function have bounded support. Although f is integrable, hence has bounded support, it is defined on all of Jr. So although it has value 0 outside some fixed big cube, its graph still exists outside the fixed cube, and the indicator function of its graph does not have bounded support. We fix this problem by speaking of the volume of the intersection of the graph with the (n + 1)dimensional bounded region Cox IR. You should imagine that Co is big enough to contain the support off, though the proof works in any case. Propositions 4.3.4 and 4.3.5 are proved in Appendix A20.
Chapter 4.
Integration
we see that by choosing N sufficiently large, the difference between upper and lower sums can be made arbitrarily small. Therefore, the function is integrable. This takes care of the "if" part of Theorem 4.3.1. For the "only if" part we must prove that if f is integrable, then there exists an appropriate N. Suppose not. Then there exists Eo > 0, such that for all N, 4.3.3 { CE'DN \ oscc(f)>•o }
In this case, for any N we have
UN(f)LN(f)=
2:
oscc(f)volnC
CE'DN
>•o
>
oscc(f)
{ CE'DN \ oscc(f)>•o }
sum of these is ~fO ,...
4.3.4
volnC
The sum ofvoln C is at least Eo, by inequality 4.3.3, so the upper and the lower integrals will differ by at least E~, and will not tend to a common limit. But we started with the assumption that the function is integrable. D Proposition 4.3.4 tells us why the indicator function of the disc discussed in Example 4.3.2 is integrable. We argued in that example that we can make the area of cubes straddling the boundary arbitrarily small. Now we justify that argument. The boundary of the disc is the union of two graphs of integrable functions; Proposition 4.3.4 says that any bounded part of the graph of an integrable function has volume 0.
Proposition 4.3.4 (Bounded part of graph of integrable function has volume O}. Let f: Rn t R be an integrable function with graph r(f), and let Co c Rn be any dyadic cube. Then voln+1( r(f) n (Cox R) ) = 0.
4.3.5
bounded part of graph
As you would expect, a curve in the plane has area 0, a surface in R 3 has volume 0, and so on. Below we must stipulate that such manifolds be compact, since we have defined volume only for bounded subsets of Rn.
Proposition 4.3.5 (Volume of compact submanifold). If M C Rn is a manifold of dimension k < n, then any compact subset X C M satisfies voln(X) = 0. In particular, any bounded part of a subspace of dimension k < n has ndimensional volume 0.
Integrability of continuous functions with bounded support What functions satisfy the hypothesis of Theorem 4.3.1? One important class consists of continuous functions with bounded support.
4.3
What functions can be integrated?
427
Theorem 4.3.6. Any continuous function IR.n + JR. with bounded support is integrable. The support of a function is closed by definition, so if it is bounded, it is compact.
Proof. Theorem 4.3.6 follows almost immediately from Theorem 1.6.11. Let f be continuous with bounded (hence compact) support. Since sup Ill is realized on the support, f is bounded. By Theorem 1.6.11, f is uniformly continuous: choose E and find 8 > 0 such that
Ix  YI < 8
==}
lf(x)  f(y)I < E.
4.3.6
For all N such that ./ii/2N < 8, any two points of a cube of VN(:JRn) are at most distance 8 apart. Thus if CE VN(:JRn), then
lf(x)  J(y)I
is at most A2nN. Choose E > 0, and use Theorem 1.6.11 to find 8 > 0 such that if Xi, X2 EX, 4.3.8 Further choose N such that ./ii/2N < 8, so that for any CE VN(Rn) and any xi,x2 EC, we have lx1  x2I < 8. For any C E 'DN(:JRn) such that C n X :f. cf>, at most 2N E + 2 cubes of VN(:JRn+l) that project to intersect r ,, hence r f is covered by at most A2nN (2N E + 2) cubes with total volume
c
1 A nN( N ) 2{n+l)N 2 2 € +2 ' which can be made arbitrarily small by taking sufficiently large. D Corollary 4.3.8: You might expect that any graph of a continuous function f : U + JR would have (n + 1)dimensional volume 0, but this is not true, even if U and f are bounded; see Exercise 4.33.
E
4.3.9 sufficiently small and N
Corollary 4.3.8. Let U c :!Rn be open and let f : U + :JR be a continuous function. Then any compact part Y of the graph of f has (n + 1)dimensional volume 0. Proof. The projection of Y into IR.n is compact, and the restriction of to that projection satisfies the hypotheses of Corollary 4.3.7. D
f
428
Chapter 4.
Integration
Integrability of function continuous except on set of volume 0 Our third theorem shows that a function need not be continuous everywhere to be integrable. This theorem is much harder to prove than the first two, but the criterion for integrability is much more useful.
Theorem 4.3.9. A function f : JR.n + JR, bounded with bounded support, is integrable if it is continuous except on a set of volume 0.
I I I
~
"~ I
~
z~
I I
4.3.3. Proof of Theorem 4.3.9: The black curve is Li, the set of points where f is discontinuous. The dark cubes intersect ti.. The light cubes are cubes at the same depth that border at least one of the dark cubes. We denote by L the union of all the shaded cubes. FIGURE
Note that Theorem 4.3.9 is not an "if and only if" statement. As will be seen in Section 4.4, it is possible to find functions that are discontinuous at all the rationals yet still are integrable.
Proof. Denote by D. ("delta") the set of points where f is discontinuous. Since f is continuous except on a set of volume 0, we have vol n D. = 0. So (by Proposition 4.1.23) for every E > 0, there exist N and some finite union of cubes C 1, ... ,Ck E 'DN(IR.n) such that k
D.
c C1 u ... u ck and
4.3.4. LEFT: It takes 32  1 = 8 cubes to surround a cube C; in JR 2 . RIGHT : In JR 3 ' it takes 33  1 = 26 cubes. If we include C;, then 32 cubes are enough in JR 2 , and 33 are enough in JR 3 .
4.3.10
i=l
Now we create a "buffer zone" around the discontinuities: let L be the union of the Ci and all the bordering cubes at level N, as shown in Figure 4.3.3. As illustrated by Figure 4.3.4, we can completely surround each Ci, using 3n  1 cubes. Since the total volume of all the Ci is less than E/3n, voln(L) :::;
FIGURE
L voln Ci :::; 3:.
E.
4.3.11
All that remains is to show that there exists M ~ N such that if CE 'DM(IR.n) and C E is less than E, which is the criterion for integrability given by Theorem 4.3.1. Suppose no such M exists. Then for every M ~ N, there is a cube CE 'DM and points XM,YM EC with lf(xM)  f(YM)I > E. The sequence M r; XM is bounded in !Rn, so we can extract a subsequence i r; xM, that converges to some point a . Since, by the triangle inequality, 4.3.12 we see that at least one of lf(xM,)  f(a)I and lf(YMJ  f(a)J does not converge to 0, so f is not continuous at a, i.e., a is in .::l. But this contradicts the fact that a is a limit of points outside of L. Since the length of a side of a cube is 1/2N, all xM, are at least 1/2N away from points of .::l, so a is also at least 1/2N away from points of D.. D
4.3
What functions can be integrated?
429
Corollary 4.3.10. Let f: R.n + R. be integrable, and let g: R.n + R. be a bounded function. If f = g except on a set of volume 0, then g is integrable, and 4.3.13
Proof. The support of g is bounded, since the support of f is bounded and so is the support of g  f. For any N, a cube C E VN(R.n) where oscc(g) > t: is either one where oscc(J) > t: or one that intersects the support off  g (or both). Choose f > 0. The total volume of cubes of first kind is < f for N sufficiently large by Theorem 4.3.1, and those of the second kind have total volume< f by Proposition 4.1.23. D Corollary 4.3.11: For example, the indicator function of the disc is integrable, since the disc is bounded by the union of the graphs of
y
= +J 1 
x2,
y
= J 1 
x2 .
Exercise 4.3.3 asks you to give an explicit bound for the number of cubes of 'DN(JR.2) needed to cover the unit circle.
Corollary 4.3.11 says that virtually all examples that occur in vector calculus are integrable. It follows from Theorem 1.6.9, Corollary 4.3.8, and Theorem 4.3.9. Corollary 4.3.11. Let A C ]Rn be a compact region bounded by a finite union of graphs of continuous functions, and let f : A + JR be continuous. Then the function ]Rn+ R. that is f(x) for x EA and 0 outside A is integrable.
f:
Polynomials are of course not integrable, since they do not have bounded support, but they can be integrated over sets of finite volume: Corollary 4.3.12. Any polynomial function p can be integrated over any set A of finite volume; that is, p · lA is integrable. Proof. The function p · lA meets the conditions of Theorem 4.3.9: it is bounded with bounded support and is continuous except on the boundary of A, which has volume 0. 0
EXERCISES FOR SECTION Hint for Exercise 4.3.3: Imitate the proof of Proposition 4.3.4, writing the unit circle as the union of four graphs: the graph of y=
Jlx 2
for lxl : : ; ../2/2 and the three other graphs obtained by rotating that graph around the origin by multiples of 7r /2.
4.3.1
a. Prove that if
f: "JR.n
+
4.3
"JR. is integrable, then
I/I
is integrable and
If JI::::; f lfl. b. Find an example where 4.3.2 4.3.3
I/I
is integrable and
f is not integrable.
Show that a subset of a set of volume zero has volume zero. a. Give an explicit upper bound (in terms of N) for the number of squares
CE 'DN("JR. 2) needed to cover the unit circle in "JR. 2, so that the area of the squares goes to 0 as N goes to infinity.
b. Use that bound to show that the area of the unit circle is 0.
430
Chapter 4.
4.3.4
Integration
< b, let
For any real numbers a
b
Q:,b
= {x
E Rn
I a :S Xi :S b for
all 1 :S i :S n } ,
and let P:,b C Q:,b be the subset where a :S x1 :S x2 :S · · · :S Xn :S b. (The case where n = 2 is shown in the figure in the margin.) a. Let f : Rn + JR be an integrable function that is symmetric in the sense that a
a
f
b
Figure for Exercise 4.3.4 Exercise 4.3.4, part a: Permutations are discussed in Section 4.8. Exercise 4.5.17 gives further applications of Exercise 4.3.4.
Exercise 4.3.5: You may either apply theorems or prove the result directly. If you use theorems, you must show that they apply.
xi ) (
'.
=f
( Xu(l) )
:
Xn
Show that
r
}Qna,b b. Let
for any permutation u of the symbols 1, 2, ... , n.
Xu(n)
f : [a, b]
in
+
j(x)
l E. The sequence j 1+ Xj is bounded, since the support of f is bounded and Xj is in the support off. So (by Theorem 1.6.3) it has a convergent subsequence Xjk converging to some point p. Since lxj y j I . 0 as j . oo, the subsequence YJk also converges to p. The function f is certainly not continuous at p, so p E D. and by formula 4.4.4, p has to be in a particular box; we will call it Bp . (Since the boxes can overlap, it could be in more than one, but we just need one.) Since Xjk and YJk converge top, and since the Bij get small as j gets big (their total volume being less than E), then all Biik after a certain point are contained in Bp. But this contradicts our assumption that we had pruned our list of Bi so that no one box was contained in any other. Therefore, Lemma 4.4.9 is correct: there are only finitely many Bi on which oscsi f > € . 0 Lemma 4.4.9 Lemma 4.4.10. There exists N such that ifC E VN(!Rn) and oscc f then CCL.
> E,
Ifwe prove this, we will be finished, because by Theorem 4.3.1, a bounded function with bounded support is integrable if there exists N at which the total volume of cubes with osc > E is less than €. We know that L is a finite set of Bi, and that the Bi have total volume < E.
Proof of Lemma 4.4.10. We will again argue by contradiction. Suppose the lemma is false. Then for every N, there exists a C N not a subset of L such that osccN f > E. In other words, 3 points XN,YN,ZN in CN, with ZN
E.
4.4.5
436 You may ask, how do we know they converge to the same point? Because XNi, y Ni, and ZNi are all in the same cube, which is shrinking to a point as N + oo. Recall that ~ is the set of points where f is not continuous, and L is the finite union of the boxes Bi on which OSCBi (!) > e. The boxes Bi making up L are in ne, so the complement of L is Ir  L. Recall (Definition 1.5.4) that a closed set C C !Rn is a set whose complement !Rn C is open.
Definition 4.4.1 specifies that the boxes Bi are open. Equation 4.1.12 defining dyadic cubes shows that they are halfopen: Xi is greater than or equal to one amount, but strictly less than another: ki
0 sufficiently small, U is not pavable. What would happen if the 3 were replaced by 2? (This is really hard.) 4.4.5
Show that Corollary 4.4.11 is false if you replace "measure" by "volume".
4.4.6 Show that if the boundary of A C Rn has a welldefined ndimensional volume, that volume is 0: voln &A = 0. Hint : Use Theorem 4.4.8. 4.4. 7 Show that the set of numbers which in base 10 can be written using only the digits 16 has measure 0.
4.5
FUBINI'S THEOREM AND ITERATED INTEGRALS
We now know  in principle, at least  how to determine whether a function is integrable. Assuming it is, how do we integrate it? Fubini's theorem allows us to compute multiple integrals by hand, or at least reduce them to the computation of onedimensional integrals. It asserts that if f : :!Rn > JR is integrable, then
FIGURE
4.5.1.
The Italian mathematician Guido Fubini (18791943) was forced to retire when the Manifesto of Fascist Racism made antiSemitism official policy of Italy in 1938. He joined the Institute for Advanced Study in Princeton the following year.
4.5.1 That is, first we hold the variables x2, .. . , Xn constant and integrate with respect to x 1 ; then we integrate the resulting (no doubt complicated) function with respect to x 2 , and so on. Note that the expression on the left side of equation 4.5.1 doesn't specify the order in which the variables are taken, so the iterated integral on the right can be written in any order: we could integrate first with respect to Xn, or any other variable. This is important for both theoretical and computational uses of Fubini 's theorem. Note also that in one dimension, equation 4.5.1 becomes equation 4.1.30.
Remark. Equation 4.5.1 is not quite correct, because some of the functions in parentheses on the right side of equation 4.5.1 may not be integrable; this problem is discussed in Appendix Al 7. We state Fubini's theorem correctly at the end of this section. For now, assume that we are in the (common) situation where equation 4.5.1 works. !::::,. In practice, the main difficulty in setting up a multiple integral as an iterated onedimensional integral is dealing with the "boundary" of the region over which we wish to integrate the function. We tried to sweep difficulties like the fractal coastline of Britain under the rug by choosing to integrate over all of Rn, but of course those difficulties are still there. This
4.5
Fubini's theorem and iterated integrals
439
is where we have to come to terms with them: we have to figure out the upper and lower limits of the integrals. If the domain of integration looks like the coastline of Britain, it is not obvious how to go about this. For domains of integration bounded by smooth curves and surfaces, formulas exist in many cases that are of interest (particularly during calculus exams), but this is still the part that gives students the most trouble. So before computing any multiple integrals, let's see how to set them up. While a multiple integral is computed from inside out  first with respect to the variable in the inner parentheses  we recommend setting up the problem from outside in, as shown in Examples 4.5.1 and 4.5.2.
By "integrate over the triangle" we mean that we imagine that the function f is defined by some formula inside the triangle, and by f = 0 outside the triangle.
Example 4.5.1 (Setting up multiple integrals: an easy example). Suppose we want to integrate a function f: JR 2  JR over the triangle
T = {(
~)
E
JR 2 I 0 '.5 2x '.5 y '.5 2}
4.5.2
shown in Figure 4.5.2. This triangle is the intersection of the three regions (here, halfplanes) defined by the inequalities 0:::; x, 2x :::; y, and y:::; 2. Say we want to integrate first with respect toy. We set up the integral as follows, temporarily omitting the limits of integration: 4.5.3 (We just write f for the function, as we don't want to complicate issues by specifying a particular function.) Starting with the outer integral thinking first about x  we hold a pencil parallel to the yaxis and roll it over the triangle from left to right. We see that the triangle (the domain of integration) starts at x = 0 and ends at x = 1, so we write in those limits:
FIGURE 4.5.2. The triangle defined by equation 4.5.2.
4.5.4 Now think about what happened as you rolled the pencil from x = 0 to x = 1: for each value of x, what values of y were in the triangle? In this simple case, for every value of x, the pencil intersects the triangle in an interval going from the hypotenuse to y = 2. So the upper value is y = 2, and the lower value is y = 2x, the line on which the hypotenuse lies. Thus we have 4.5.5
If we want to start by integrating
l
2
f
(~)
f with respect to x, we write
ldxdyl
=
j (j f
dx )dy;
4.5.6
starting with the outer integral, we hold our pencil parallel to the xaxis and roll it from the bottom of the triangle to the top, from y = 0 to y = 2. As we roll the pencil, we ask what are the lower and upper values of x for
440
Chapter 4.
Integration
each value of y. The lower value is always x = 0, and the upper value is set by the hypotenuse, but we express it as x = y/2. This gives
4.5.7 Now suppose we are integrating over only part of the triangle, as shown in Figure 4.5.3. What limits do we put in the expression J(f f dy) dx? Try it yourself before checking the answer in the footnote. 5 !:::. a 1 FIGURE
4.5.3.
The shaded area represents a truncated part of the triangle of Figure 4.5.2. Exercise 4.5.3 asks you to set up the multiple integral for Example 4.5.2 when the outer integral is with respect toy. The answer will be a sum of integrals.
Example 4 .5.2 (Setting up multiple integrals: a somewhat harder example). Now let's integrate an unspecified function f: JR 2 ~ JR over the area bordered on the top by the parabolas y = x 2 and y = (x  2) 2 and on the bottom by the straight lines y = x and y = x  2, as shown in Figure 4.5.4. Let's start again by sweeping our pencil from left to right, which corresponds to the outer integral being with respect to x. The limits for the outer integral are clearly x = 0 and x = 2, giving
4.5.8 As we sweep our pencil from left to right, we see that the lower limit for y is set by the straight line y = x, and the upper limit by the parabola
\
y = x 2 , so we are tempted to write
12(J_x x
2
f
dy ) dx, but at x = 1, we
have a problem. The lower limit is now set by the straight line y = x  2, and the upper limit by the parabola y = (x  2) 2 . How can we express !:::. this? Try it yourself before looking at the answer in the footnote. 6 2
I
FIGURE
4.5.4.
The region of integration for Example 4.5.2.
Example 4.5.3 (Setting up a multiple integral in JR 3 ). Already in JR 3 this kind of visualization becomes much harder. Suppose we want to integrate a function over the pyramid P shown in Figure 4.5.5 and given by the formula P
~{
G)
E R3
I 0 ,; x;
0 ,; y; 0 ,; z; x
+ y + z ,; I
} .
4.5.9
5 When the domain of integration is the truncated triangle in Figure 4.5.3, the integral is written
The other direction is harder; we will return to it in Exercise 4.5.2. 6 We need to break up this integral into a sum of integrals:
1 1xx2 1 (
f
dy
) dx +Ji{2 (1(x x
2
2)
2
f
dy
)
dx.
Exercise 4.5.l asks you to justify ignoring that we have counted the line x twice.
=1
4.5
Fubini's theorem and iterated integrals
441
We want to figure out the limits of integration for the multiple integral 4.5.10
y
x
FIGURE
4.5.5.
Pyramid for Example 4.5.3. Note that when setting up a multiple integral, the upper and lower limits for the outer integral are just numbers. Then (going from outer to inner) the limits for the next integral may depend on the variable of the outer integral. See, for instance the upper limit y/2 in integral 4.5.7, and the upper limit 1  z in integral 4.5.11. The limits for the next integral may depend on the variables of the two outer integrals; see the upper limit 1  y  z in formula 4.5.12.
Formula 4.5.12: When we integrate with regard to x, why is the upper value 1yz, when the upper value with regard to y is 1  z, not 1 z  x? Since the inner integral, first to be computed, is with regard to x, we must consider all values of y and z. But when we integrate with regard to y, we have already integrated with regard to x. Clearly the value of x for which y = 1  z  x is largest is x = 0, giving upper value y = 1  z.
There are six ways to apply Fubini's theorem, which in this case because of symmetries will result in the same expressions with the variables permuted. Let us vary z first. For instance, we can lift a piece of paper and see how it intersects the pyramid at various heights. Clearly the paper will only intersect the pyramid when its height is between 0 and 1. This gives
fo
1
(
)dz , where the space needs be filled in by the double integral off
over the part of the pyramid Pat height z, pictured at left in Figure 4.5.6, and again at right, this time drawn flat . y
x
1
y 1
/
x+y=lz
x
4 .5.6 . LEFT: The pyramid of Figure 4.5.5, truncated at height z. The plane at height z , put flat.
FIGURE RIGHT:
This time we are integrating over a triangle (which depends on z); see the right side of Figure 4.5.6. So we are in the situation of Example 4.5.1. Let's vary y first. Holding a pencil parallel to the xaxis and rolling it over the triangle from bottom to top, we see that the relevant yvalues are between 0 and 1  z (the upper value is the largest value of y on the part of the line y = 1  z  x that is in the triangle) . So we write
)dy) dz,
4.5.11
where now the blank represents the integral over part of the horizontal line segment at height z and "depth" y. These xvalues are between 0 and 1  z  y , so finally the integral is 4.5.12
Example 4.5.4 (A less visual example) . In the preceding examples, we could deduce the upper and lower limits from pictures of the region over
442
Chapter 4.
Integration
which we want to integrate. Here is a case where the picture doesn't give the answer. Let us set up a multiple integral over the ellipse E defined by
E = { ( ~)
I x 2 + xy + y2 :::; 1 } .
4.5.13
The boundary of E is defined by y 2 + xy + x 2  1 = 0, so on the boundary, y
x±y'x2 4(x 2 1) x±v'43x2 2  2 
Such values exist only if 4  3x 2 ;::: 0, i.e., integral of a function f over E we write Here, x and y play symmetric roles, so to integrate first with respect to x, we would just replace every y by x and vice versa.
1
2v'3/3 (
2v'3/3
Jx+~ f ( %~
lxl :::; x) dy y
4.5.14
2v'3/3. So to set up the
)
dx.
4.5.15
Now let's actually compute a few multiple integrals.
Example 4.5.5 (Computing a multiple integral). Let f: JR 2 the function f ( ~) = xy ls ( Figure 4.5.7. Then
~2 f ( ~)
1
0
x FIGURE 4.5 .7.
The integral in equation 4.5.16 is 1/4: the volume under the surface defined by
f ( ~) =
xy and
above the unit square is 1/4.
IR be
~), where S is the unit square, as shown in
1(1 1=1
ldx dyl =
+
1
xy dx) dy
[x2y] x=l dy = 2 x=O
11 0
4.5.16
y 1 dy = . 2 4
In Example 4.5.5 it is clear that we could have taken the integral in the opposite order and found the same result, since our function is f ( ~) = xy. Fubini's theorem says that this is always true as long as the functions involved are integrable. This fact is useful; sometimes a multiple integral can be computed in elementary terms when written in one direction, but not in the other, as you will see in Example 4.5.6. It may also be easier to determine the limits of integration if the problem is set up in one direction rather than another, as we saw in the case of the truncated triangle shown in Figure 4.5.3.
Example 4.5.6 (Choose the easy direction). Let us integrate the function eY 2 over the triangle shown in Figure 4.5.8: 4.5.17 Fubini's theorem gives us two ways to write this integral as an iterated onedimensional integral: and
4.5.18
Fubini's theorem and iterated integrals
4.5
443
The first cannot be computed in elementary terms, since eY 2 does not have an elementary antiderivative. But the second can:
1 1
(1Y eY 2 dx) dy
=
1 1
yeY 2 dy
= ~
[eY2 ]~ = ~ ( 1  ~).
(They in yeY 2 makes it possible to integrate in elementary terms.)
4.5.19 6.
Older textbooks contain many examples of such computational miracles. We believe it is sounder to take a serious interest in the numerical theory and go lightly over computational tricks, which do not work in any great generality in any case. 1 FIGURE
x
4.5.8.
The triangle { (
~)
E lR2
I O~ x
~y ~1}
Example 4.5.7 (Volume of a ball in JRn). Let BR(O) be the ball of radius R in Rn, centered at 0, and let bn(R) be its volume. By Proposition 4.1.24, bn(R) = Rnbn(l). We will denote by f3n = bn(l) the volume of the unit ball. By Fubini's theorem, (n1)dimensional vol. of one slice of Bf (0)
of Example 4.5.6.
Example 4.5.7: The ball BI'{O) is the ball of radius 1 in Rn, centered at the origin; the ball Bn1
v'ix~
(0)
xa
is the ball of radius v'l in Rn 1 , still centered at the origin. In the first line of equation 4.5.20 we imagine slicing the ndimensional ball horizontally and computing the (n1 )dimensional volume of each slice.
1 1
f3n1 .._,,_,.
(1 
1
x~) n2l dxn.
4.5.20
vol. of unit ball in IRnl
This reduces the computation of f3n to computing the integral Cn
=
ll
(1  t 2 ) n2l dt.
4.5.21
1
This is a standard tricky problem from onevariable calculus: Exercise 4.5.4 asks you to show that c0 = 7r and c1 = 2 and that for n 2: 2, nl Cn = Cn2· 4.5.22 n This allows us to make Table 4.5.9. It is easy to continue the table. (What is (36? Check below. 7 ) 6. If you enjoy inductive proofs, you might try Exercise 4.5.5, which asks you to find a formula for f3i for all i. Exercise 4.5.6 asks you to show that
as n increases, the volume of the ndimensional unit ball becomes a smaller and smaller proportion of the smallest ndimensional cube that contains it. 7

C5 
5

5C4 
157'
48'
f.I

SO tJ6 
f.I

C5,_,5 
,.s
lf·
444
Chapter 4.
Integration
n
Cn  nlc n n2
0
7r
1
2
2
71" 2 4 3 371"
3 4
8
5
15
Volume of ball
f3n = Cnf3n l
2 7r
471" 3 71"2 2 871"2
16
15
TABLE 4.5.9. Computing the volume of the unit ball in JR 1 through JR 5 .
Computing probabilities using integrals We saw in Section 4.2 that integrals are useful in computing probabilities.
Exercise 4.5.10 asks you to show that if two points x, y are chosen in the unit square, without privileging any part of the square, then
E(lx  Yl 2 ) = 1/3. Note that the expected value of Ix  YI is more complicated to compute than the expected value of (x  y)2. Asked to give the expected distance between x and y, a probabilist would most likely compute E((x y) 2 ) and take the square root, getting 1/../6, which is not the actual expected distance of 1/3. Computing E((x  y)2) and taking the square root is analogous to computing the standard deviation; computing E(lx yl) is analogous to computing the mean absolute deviation.
Example 4.5.8 (Using Fubini to compute probabilities). Recall (equation 4.2.12) that the expectation (expected value) of a random variable f is given by E(f) = fs f(s)µ(s) ldsl, where the function µ weights outcomes according to how likely they are, and S is the sample space. If we choose two points x, yin the interval [O, 1], without privileging any part of that interval, what is the expected value of the function f ( ~) = (x y) 2? "Without privileging any part of that interval" means that µ = 1. Thus
E((x  y)2) =fol fol (x y)2dxdy =fol (fol x2  2xy + y2 dx) dy = fol [
[ ¥. 3
~3 
x2y + y2x
J:
y2+y3]1
1
2
6
3
0
dy = fo1
~
y + y2 dy 4.5.23
Given x, yin the same interval, what is the expected value of the function
f ( ~)
= Ix  yl? Now we want to include positive values of x  y and
positive values of y x. To do this, let's assume that x > y (so the integral for y goes from 0 to x) and double the integral to account for the cases where y > x:
E(lx yl) = 2 fo
1
(fox x ydy) dx = 2 fo 1 [xy ~2 [ dx 4.5.24
Fubini's theorem and iterated integrals
4.5
445
Example 4.5.9 {Computing a probability: A harder example). Choose at random two pairs of positive numbers between 0 and 1 and use those numbers as the coordinates (x1, Y1), (x2, Y2) of two vectors anchored at the origin, as shown in Figure 4.5.10. (You might imagine throwing darts at the unit square.) What is the expected (average) area of the parallelogram spanned by those vectors? In other words (recall Proposition 1.4.14), what is the expected value of I ~•+O+~~~!\/"v~~~ 1 implies n ~ 3. We assume that n ~ 3, and that the function Anl is antisymmetric, so that An1(A[i,lj) = An1(A[i,lJ) for each i. Then
= An (A). m
The case where either m or p equals 1 is trickier. = 1 and p = 2. 9 It will be convenient to denote by
4.8.12
Let's assume
(a[i,jJ)k,l
the k, lentry of A[i,j]
A[i,j][k,!J
the matrix A[i,j] with the kth row and lth column omitted.
The numbers (a[i,j])k,1 and entries of A[i,j][k,!] are all entries of A, but sometimes with different indices; Figure 4.8.1 should help you figure out what corresponds to what. Using this notation, we can go one level deeper into our recursive formula, writing
Of course there is a similar formula for A. We need to show that each term in the double sum for A occurs also in the double sum for A, with opposite sign. 9 We can restrict ourselves to p = 2 because if p > 2, we can switch the pth column with the second, then the second with the first, then the second with the pth again. By the inductive hypothesis, the first and third exchanges would each change the sign of the determinant, resulting in no net change; the only exchange that "counts" is the change of the first and second positions.
4.8 Determinants
465
The unshaded part C on left and right in Figure 4.8.1 are the same (n  2) x (n  2) matrix, though it is called C = A[i,l][jl,l] on the left and C = A[j,l][i,l] on the right. Moreover the coefficient 4.8.13
The key to the proof is the fact (see the caption to Figure 4.8.1) that A [i, l)[j1,1)
= A(j,l)[i,l).
of ~n2(C) is called ai,1(a[i,1J)j1,1 on the left and aj, 1 (a[j,lJ)i,l on the right. Thus the term ab~n2(C) comes with the sign (l)i+l+jl+l
on the left and
(l)Hl+i+l
on the right,
4.8.14
i.e., with opposite signs.
This is because, after removing the ith row of A, the jth row becomes the (j l)st row of A[i,l)· This leads to the opposite signs in formula 4.8.14.
12 a
j
a
b
b
I A
FIGURE 4.8.1. In both matrices, C is the unshaded part. LEFT: The n x n matrix A, the ith and jth row and the first two columns shaded. RIGHT: The matrix A, identical to A except that the first two columns are exchanged. Let a, b be the shaded entries a = a ;, 1 = a;,2 and b = aj ,2 = ai ,l · Note that the unshaded parts coincide: A1;,1)[j1,11 = A[j, l)[i,1) . 3. Normalization This is much simpler. If A= [e1 , ... ,en], then in the first column, only the first entry al,l = 1 is nonzero, and A1,1 is the identity matrix one size smaller, so that ~n1 of it is 1 by induction. So 4.8.15 and we have also proved property 3. This completes the proof of existence. In Figure 4.8.1 we have assumed that i < j. For the case i > j we can simply reverse the roles of A and A.
Proving uniqueness We have shown that~ satisfies conditions 13 of Theorem and Definition 4.8.1. To prove uniqueness of the determinant, suppose~' is another function satisfying those conditions; we must show that ~(A)= ~'(A) for any square matrix A. First, note that if a matrix A 2 is obtained from A1 by a column operation, then 4.8.16
466
Chapter 4.
Integration
for a number µ depending on the type of column operation: 1. A2 is obtained by multiplying some column of Ai by m =I 0. By multilinearity, 6'(A2) = m6'(Ai), soµ= m. 2. A2 is obtained by adding a multiple of the ith column of Ai to the j th column, with i =I j. By property 1, this does not change the determinant, because
Equation 4.8.17: The second term on the right is zero, since two columns are equal (Exercise 4.8. 7).
6' [ai. ... , ai, ... , (aj +,Bai), ... , a..] =
4.8.17
6'[ai, ... ,~, ... ,aj,··· ,an]+ ,B6'[ai, ... ,~, ... ,~, ... ,a,.]. = 0 because 2 identical columns
a;
Thus in this case, µ = 1. 3. A2 is obtained by exchanging two columns of Ai. By antisymmetry, this changes the sign of the determinant, so µ = 1.
The factors 1/µ are imposed by multilinearity and antisymmetry, not by any specifics of how a function is defined, so if 11' is any function satisfying those conditions, we must have 11' (Ai1) = ..!..111 (Ai)· µi Uniqueness also requires normalization, which allows us to assert that if Ap is in column echelon form, then 11'(Ap) = 11(Ap)·
Any square matrix can be column reduced until at the end, you either get the identity, or a matrix with a column of zeros. Column reducing a matrix A to column echelon form Ap can be expressed as follows, where the µi corresponding to each column operation is on top of an arrow denoting the operation: µ1 A i + µ2 A 2 + µa µ.,,1 A µ.,, Ap . A + • . • + p i + 4.8.18 Now let us see that 6(A) = 6'(A). If Ap =I I, then by property 1, 6(Ap) = 6'(Ap) = 0 (see Exercise 4.8.7). If Ap =I, then by property 3, 6(Ap) = 6'(Ap) = 1. Then, working backward,
6(Api) = 2_6(Ap) = 2_6'(Ap) = 6'(Api), µp µp
6(Ap_ 2 ) =
6(A) =
l
µpµpi
6(Ap) =
l
µpµpi
6'(Ap) = 6 1 (Ap2), 4.8.19
1 l 6(Ap) = 6'(Ap) = 6'(A). µpµpi ... µi µpµpi ... µi
This proves uniqueness.
D
Remark. Equation 4.8.19 also provides an effective way to compute the determinant: ifAp=l 1 det A = det Ap = { µ.,,µ.,,~ 1 ···µ 1 4.8.20 µpµpi ... µi 0 if Ap =/: I. This is a much better algorithm for computing determinants than is development according to the first column, which is very slow. How slow? It takes time T(k) to compute the determinant of a k x k matrix. Then to compute the determinant of a (k + 1) x (k + 1) matrix
4.8
Determinants
467
we will need to compute k + 1 determinants of k x k matrices, and k multiplications and k additions: T(k
+ 1) = (k + l)T(k) + (k + 1) + k.
+1
4.8.21
In particular, T(k) > k!. For a 15 x 15 matrix, this means 15! ~ 1.3 x 1012 calls or operations; even the biggest computers of 2015 could barely do this in a second. And 15 x 15 is not a big matrix; engineers modeling bridges or airplanes and economists modeling a large company routinely use matrices that are more than 1000 x 1000. For a 40 x 40 matrix, the number of operations that would be needed to compute the determinant using development by the first column is bigger than the number of seconds that have elapsed since the beginning of the universe. In fact, bigger than the number of billionths of seconds that have elapsed. If you had set a computer computing the determinant back in the days of the dinosaurs, it would have barely begun. But column reduction of an n x n matrix, using equation 4.8.20, takes about n 3 operations (see Exercise 2.2.11). Using column reduction, one can compute the determinant of a 40 x 40 matrix in 64 000 operations, which would take a gardenvariety laptop only a fraction of a second. 6
Theorems relating matrices and determinants Theorem 4.8.3. A matrix A is invertible if and only if det A Theorem 4.8.3: Equivalently, : Qa c. What is
A is one to one.
+
IA y ldxdyl?
4.10.18 What is the volume of the part of the ball of equation x 2 + y 2 + z 2 where z 2 ;::::: x 2 + y 2 , z > O? 4.10.19 Let Q = [O, 1] x [O, 1] be the unit square in JR2 , and let CI> : JR 2 be defined by
+
:::;
4
JR2
a. Sketch A, by computing the image of each of the sides of Q (they are all arcs of parabolas). b. Show that CI> : Q + A is 11. c. What is Change of variables for Exercise 4.10.20:
(u) ( ((w ~ l)v)u) . w3
CI>
:
=
3

?
4.10.20
Solve Exercise 4.5.19 again, using the change of variables in the margin.
4.10.21
The moment of inertia of a body X
l
1
IA x ldxdyl
(r(x)) 2
ld
3 xl,
c JR3 around an axis is the integral
where r(x) is the distance from x to the axis.
a. Let f be a nonnegative continuous function of x E [a, b], and let B be the body obtained by rotating the region 0:::; y:::; f(x), a:::; x:::; b around the xaxis. What is the moment of inertia of B around the xaxis? b. What number does this give when f(x)
= cosx,
7r
a=
2'
4.11 LEBESGUE INTEGRALS This new integral of Lebesgue is proving itself a wonderful tool. I might compare it with a modern Krupp gun, so easily does it penetrate barriers which were impregnable.Edward Van Vleck, Bulletin of the American Mathematical Society, vol. 23, 1916.
There are many reasons to study Lebesgue integrals. An essential one is the Fourier transform, the fundamental tool of engineering and signal processing, not to mention harmonic analysis. (The Fourier transform is discussed at the end of this section.) Lebesgue integrals are also ubiquitous in probability theory.
So far we have restricted ourselves to integrals of bounded functions with bounded support, whose upper and lower sums are equal. But often we will want to integrate functions that are not bounded or do not have bounded support. Lebesgue integration makes this possible. It also has two other advantages: 1. Lebesgue integrals exist for functions plagued with the kind of "local nonsense" that we saw in the function that is 1 at rational numbers in [O, 1] and 0 elsewhere (Example 4.3.3). The Lebesgue integral ignores local nonsense on sets of measure 0. 2. Lebesgue integrals are better behaved with respect to limits.
4.11 Lebesgue integrals
Our approach to Lebesgue integration is very different from the standard one. The usual way of defining the Lebesgue integral
is to cut up the codomain JR into small intervals Ji = [xi, Xi+1], and to approximate the integral by
L
Xiµ(fl(Ii)),
499
Remark. If Lebesgue integration is superior to Riemann integration, why did we put so much emphasis on Riemann integration earlier in this chapter? Riemann integrals have one great advantage over Lebesgue integrals: they can be computed using Riemann sums. Lebesgue integrals can only be computed via Riemann integrals (or perhaps by using Monte Carlo methods). Thus our approach is in keeping with our emphasis on computationally effective algorithms. 6 Before defining the Lebesgue integral, we will discuss the behavior of Riemann integrals with respect to limits.
Integrals and limits
i
where µ(A) is the measure of A, then letting the decomposition of the codomain become arbitrarily fine. Of course, this requires saying what subsets are measurable, and defining their measure. This is the main task with the standard approach, and for this reason the theory of Lebesgue integration is often called measure theory. It is surprising how much more powerful the theory is when one decomposes the codomain rather than the domain. But one pays a price: it isn't at all clear how one would approximate a Lebesgue integral: figuring out what the sets 1 1 (/i) are, never mind finding their measure, is difficult or impossible even for the simplest functions. We take a different tack, building on the theory of Riemann integrals, and defining the integral directly by taking limits of functions that are Riemann integrable. We get measure theory at the end as a byproduct: just as Riemann integrals are used to define volume, Lebesgue integrals can be used to define measure. This is discussed in Appendix A21.
The behavior of integrals under limits is often important. Here we give the best general statements about Riemann integrals and limits. We would like to be able to say that if k tt fk is a convergent sequence of functions, then, ask~ oo,
j
limfk = lim
j
fk·
4.11.1
In one setting this is true and straightforward: when k tt f k is a uniformly convergent sequence of integrable functions, all with support in the same bounded set. The key condition in Definition 4.11.1 is that given f, the same K works for all x.
Definition 4.11.1 (Uniform convergence). A sequence k tt fk of functions f k : !Rn ~ JR converges uniformly to a function f if for every f > 0, there exists K such that when k ~ K, then, for all x E !Rn, lfk(x)  f(x)I < f. The three sequences of functions in Example 4.11.3 do not converge uniformly, although they do converge. Uniform convergence on all of !Rn isn't a very common phenomenon, unless something is done to cut down the domain. For instance, suppose that k tt Pk is a sequence of polynomials 4.11.2 all of degree :::; m, and that this sequence "converges" in the "obvious" sense that for each degree i (i.e., each xi), the sequence of coefficients ai,o, ai,1' ai,2, ... converges. Then k tt Pk does not converge uniformly on JR. But for any bounded set A, the sequence k tt PklA does converge uniformly. 15 15 Instead
of writing PklA we could write "Pk restricted to A".
500
Chapter 4.
Integration
Let k 1+ fk be a sequence of integrable functions Rn + JR, all with support in a fi.xed ball B C Rn, and converging uniformly to a function f. Then f is integrable, and
Theorem 4.11.2 (Convergence for Riemann integrals).
lim k+oo
f fk(x) ltrxl = lJRn f f(x) ltrxl. lJRn
4.11.3
Proof. Choose€> 0 and K so large that supxEJRn l/(x) fk(x)I < k > K. Then when k > K, we have, for any N,
LN(f) > LN(fk)  €VOln(B)
UN(!)< UN(fk) + fVOln(B).
and
E
when
4.11.4
Now choose N so large that UN(/k)  LN(fk) < E; we get Inequality 4.11.4: If you picture this as Riemann sums in one variable, f is the difference between the height of the lower rectangles for f, and the height of the lower rectangles for fk· The total width of all the rectangles is voh(B), since Bis the support for
UN(/)  LN(f) < UN(/k)  LN(/k) +2€VOln(B),
4.11.5
0 and choose M such that
k=~+l kn lhk(x)lldnxl < so that for l
kn
>M
4.11.24
€,
we have l
IH1(x)  HM(x)lld"xl:::;
:::;
k=~+l knihk(x)jjd"xj
k=~+l
ln
4.11.25
lhk(x)lld"xl
M, we have
f
H1(x)  [Ht]R(x) jcrxj =
}Rn
f
H1(x)  HM(x) jcrxj 
}Rn
[ [Hi]R(x)  HM(x) ldnxl. }Rn
n, this geometric series converges, since the sum I:i 2 n. Exercise 4.11.21 asks you to prove that if m ~ n, then I is not Lintegrable on Rn. 6
Example 4.11.12 (An unbounded, integrable function). The function l(x) ~ l[o,11(x) lnx is Lintegrable, even though it isn't bounded. As shown in Figure 4.11.4, it can be written as a sum of bounded functions: 00
l(x) ~ Lli(x), i=O
where
Ii= (1c 2 , 2 i1(x)) lnx.
4.11.49
4.11
Lebesgue integrals
507
The functions fi are Rintegrable, and the series
8
()() fl/2i Ji/ 2i+ 1 llnxl dx
4.11.50
is convergent, as Exercise 4.11.15 asks you to check.
Example 4.11.13 shows that Lebesgue integrals are not a strict generalization of improper integrals: some improper integrals, those which depend on cancellations, do not exist as Lebesgue integrals. Since there is no version of Fubini's theorem or the change of variables formula for integrals that depend on cancellations, Lebesgue integration wisely forbids them.
Example 4.11.13 (A function that is not Lebesgue integrable). Some improper onedimensional integrals do not correspond to Lebesgueintegrable functions: integrals whose existence depends on cancellations, like
1
sinx dx. 4.11.51 x As you may recall from onevariable calculus, this improper integral is defined to be 00
0
. 1lm A+oo
1
A.
o
smx dx,
4.11.52
X
and we can show that the limit exists, for instance, by saying that the series
L 1(k+l)7r sinx dx 00
k=O
Exercise 4.11.16 asks you to justify the argument in Example 4.11.13. Note that we would not want this integral to exist, since the change of variables formula fails for this sort of improper integral (see Exercise 4.11.17).
6
k7r
4.11.53
X
is a decreasing alternating series whose terms go to 0 as k + oo. But this works only because positive and negative terms cancel: the area between the graph of sin x / x and the x axis is infinite, and there is no limit lim
fA Isinx I dx.
4.11.54
x
A+oo}0
The integral in 4.11.51 does not exist as a Lebesgue integral. By our definition, "/ Lintegrable" implies that I/I is Lintegrable, which is not the case here. 6
Some elementary properties of the Lebesgue integral Proposition 4.11.14 (The Lebesgue integral is linear). If f and g are Lintegrable and a, b are constants, then af + bg is Lintegrable and { (af
J.B.n
+ bg)(x)ldnxl =a
{
}ll{n
/(x)l~xl + b
{
}ll{n
g(x)l~xl.
Proof. If f = E fk and g = E gk, then af + bg = E(afk + bgk)· Indeed, L L L the series E(afk + bgk) will converge except on the union of the sets where the series Ek lf(x)I and Ek lg(x)I diverge, and the union of two sets of measure 0 is still of measure 0 (see Theorem 4.4.4). For x not in this set, the result follows from the corresponding statement for series of numbers. D
508
Chapter 4.
Integration
It is not true that the product of two Lintegrable functions is necessarily
Lintegrable; for instance, the function
l[~X)
is Lintegrable, but its lxl square is not. However, we have the following result.
Proposition 4.11.15. If f is £integrable on :!Rn, and g is Rintegrable on ]Rn, then f g is £integrable. Proof. Since f is Lintegrable, we can set f = Lk fk, where the functions L fk are Rintegrable and 4.11.55 We have f g = Lk fkg, where fkg is Rintegrable; since g is bounded, L
L }JRn r l/k(x)g(x)l lldnxl ~ k
sup lg(x)I
L }JRn r l/k(x)l ll~xl
+ Cn is a sequence of matrices converging to C, and tr(CnD) = tr(DCn) for all n, then tr( CD)= tr(DC).
00
2
ex cosxdx
and
1""
2
e"' dx. _ 00 l+x 2
Compare the approximations with the exact values. 4.17
Check part 3 of Theorem 4.8.15 when A = [;
~]
and B = [ ~
: ];
that is, show that [D det(A))B = Cn(t) for part d. The functions cn(t) are positive monotone increasing as functions of t, and decreasing as functions FIGURE FOR EXERCISE
ofn.
a. Show that for all 0 lb
(1
00
sinx d X= 1Im . la sinx dx . X a+oo 0 X

< a < b < oo,
epx sin x dx) dp
=
1
00
(lb epx sin x dp) dx.
b. Use part a to show
1
00
arctanb  arctana =
(eax  ebx) sinx
x
0
dx.
c. Why does Theorem 4.11.4 not imply that lim lim roo (eax  ebx) sinx dx = roo sinx dx? (1) b+oo lo X lo X d. Prove that equation 1 in part c is true anyway. The following lemma is the key: if n 1> cn(t) is a sequence of positive monotone increasing functions of t, with limt+oo Cn(t) = Cn, and decreasing as a function of n for each fixed t, tending to 0 (see the figure in the margin), then a+O
Exercise 4.26, part d: Remember that the next omitted term is a bound for the error for each partial sum.
00
00
e. Write
1
00
0
(eax  ebx) sinx '' dx
oo l(k+l),..
= I:
X
n=O •
and use part d to prove the equation 00
f(x) =
1
~ 2k
1
Jlx  aki.
Function f for Exercise 4.27 Exercise 4.27, part c: This depends on the order chosen.
4.27
(1)
k
(eax  ebx)I sin xi
k,.. { 00
l0
dx,
X
sinx
;
dx =
7r
2.
Let a 1 , a 2 , .•. be a list of the rationals in [O, 1].
a. Show that the function f given in the margin is Lintegrable on [O, l]. b. Show that the series converges for all x except x on a set of measure O. *c. Find an x for which the series converges. 4.28
a. Show that :
2 1[1 , 001(x)
is a probability density.
4.12
Review exercises for Chapter 4
523
b. Show that for the probability density found in part a, the random variable = x does not have an expectation (i.e., the expectation is infinite). 2 c. Show that x 3 111 , 001(x) is a probability density.
f(x)
d. Show that the random variable f(x) = x has an expectation with respect to the probability density of part c; compute it. Show that it does not have a variance (i.e., the variance is infinite). 4.29 Let T : R 2 + R 2 be the linear transformation given by T(u) =au+ bU, where we identify R 2 with C in the standard way. Show that n, then volk(P(v1, ... , vk))
5.1.1
What about a kparallelogram in Rn? Clearly if we draw a parallelogram on a rigid piece of cardboard, cut it out, and move it about in space, its area will not change. This area should depend only on the lengths of the vectors spanning the parallelogram and the angle between them, not on where they are placed in R 3 . It isn't obvious how to compute this volume: equation 5.1.1 can't be applied, since the determinant only exists for square matrices. A formula involving the cross product (see Proposition 1.4.20) exists for a 2parallelogram in R 3 . How will we compute the area of a 2parallelogram in R 4 , never mind a 3parallelogram in R 5 ? The following proposition is the key. It concerns kparallelograms in Rk, but we will be able to apply it to kparallelograms in Rn.
= 0. Proposition 5.1.1 (Volume of a kparallelogram in Rk). Let vi, ... , vk be k vectors in Rk, so that T = [Vi, ... , vk] is a square k x k matrix. Then 5.1.2
Proof of Proposition 5.1.1: Recall that if A and B are n x n matrices, then
Proof. Jdet(TTT) = J(detTT)(detT) = J(detT) 2 = ldetTI
D
det A det B = det(AB) det A= det AT
Example 5.1.2 (Volume of twodimensional and threedimensional parallelograms). When k = 2, we have
(Theorems 4.8.4 and 4.8.8).
det(TTT) = I  2Nl.
5.2.5
In fact, (as we saw in Example 4.4.3) au does not have Idimensional volume, but it does have Idimensional measure and its measure satisfies inequality 5.2.5. 6. Our entire theory of integrals over manifolds will be based on parametrizations. Fortunately, with our relaxed definition, all manifolds can be parametrized.
Theorem 5.2.6 (Existence of parametrizations). All manifolds can be parametrized. Remark. Theorem 5.2.6 doesn't mean that parametrizations can be written explicitly: in the proof (the object of Exercise 5.2.8, not tremendously difficult), we will need to know that they can be locally parametrized, which requires the implicit function theorem: the parametrizing mappings only exist in the sense that the implicit function theorem guarantees their existence. 6.
A small catalog of parametrizations Often Theorem 5.2.6 is cold comfort. Constructing a parametrization usually requires Newton's method and the implicit function theorem; such parametrizations can't be written explicitly. Fortunately, some can be written explicitly, and they turn up, disproportionately, in exam problems and in applications, especially applications where there is an underlying symmetry. Below we give examples. There are two classes: graphs and surfaces of revolution.
• Graphs. If U c JR.k is open with boundary of kdimensional volume 0, and f : U > JR.nk is a 0 1 map, the graph of f is a manifold in Rn, and x
1+ (
f~))
is a parametrization.
5.2.6
532
Chapter 5.
Volumes of manifolds
Example 5.2.7 (Parametrizing as a graph). The surface of equation
z = x2 ~~Ji~~i"'"Y
+ y2
is parametrized by
(x)
tt (
:
given in equation 3.1.24.
)
•
Another example was
x2+y2
Y
!::::.
There are many cases where parametrizing as a graph still works, even though the conditions above are not satisfied: those where you can "solve" the defining equation for n  k of the variables in terms of the other k. Example 5.2.8 (Parametrizing as a graph: a more complicated case). Consider the surface in JR3 of equation x 2 + y 3 + z 5 = 1. In this case you can "solve" for x as a function of y and z:
x = ±JI  y 3

z5.
5.2.7
You could also solve for y or for z, as a function of the other variables, and the three approaches give different views of the surface, as shown in Figure 5.2.2. Of course, before you can call any of these a parametrization, you have to specify the domain of the function. When the equation is solved for x, the domain is the subset of the (y, z)plane where 1  y 3  z 5 ~ 0. When solving for y, remember that every real number has a unique real cube root, so the function y = (1  x 2  z 5 ) 113 is defined at every point, but it is not differentiable when x 2 + z 5 = 1, so this curve must be included !::::. in the set X of trouble points that can be ignored. • Surfaces of revolution. A surface of revolution can be obtained by rotating a curve, whether the curve is the graph of a function or is known as a parametrization. FIGURE
5.2.2.
The surface of equation x2
+ y3 + zs = 1
discussed in Example 5.2.8. TOP: The surface seen as a graph of x as a function of y and z (i.e., parametrized by y and z). The graph consists of two pieces: the positive square root and the negative square root. MIDDLE: Surface parametrized by x and z. BOTTOM: Surface parametrized by x and y. Note that the lines in the middle and bottom graphs are drawn differently: different parametrizations give different resolutions to different areas.
1. The case where the curve is the graph of a function. Suppose the curve C is the graph of a function f(x) = y. Let us suppose that f takes only positive values, and rotate C around the xaxis, to get the surface of revolution of equation y2 + z2
=
(f(x))2.
5.2.8
This surface can be parametrized by "f:
(~) (f(x)xc~s()) , f (x) tt
5.2.9
Slll ()
where the angle() measures the rotation. Why this parametrization? Since the curve is rotated around the xaxis, the xcoordinate remains unchanged. Certainly equation 5.2.8 is satisfied: (f(x) cos0) 2 + (f(x) sin0) 2 = (f(x))2. Again, to be precise one must specify the domain of y. Suppose that f: (a, b) ~JR is defined and continuously differentiable on (a, b). Then we can choose the domain of y to be (a, b) x [O, 27r], and y is one to one, with derivative also one to one on (a, b) x (0, 27r). 2. The case where the curve to be rotated is a parametrized curve.
5.2 Parametrizations
If C is a curve in the (x, y)plane, parametrized by t
r+
(
533
:~!~),
the
surface obtained by rotating C around the xaxis can be parametrized by
(t) (v(t) ()
u(t)
v(t)c~sO
r+
)
.
5.2.10
sm()
Spherical coordinates on the sphere of radius R are a special case of this construction. If C is the semicircle of radius R in the (x, z)plane, centered at the origin, and parametrized by O; ii. x > 0, y > O; iii. z > x + y. curve bounding the parametrizing region, you may need a computer *5.2.5 Consider the open subset of JR. constructed in Example 4.4.3: list the to visualize it. rationals between 0 and 1, say a1, a2, a3, ... , and take the union
LJ 00
U=
(
1 ' ai ai  2i+k
1 ) + 2i+k
i=l
for some integer k > 2. Show that U is a onedimensional manifold and that it can be parametrized according to Definition 5.2.3. 5.2.6 Use Definition 5.2.1 to show that a single point in any Rn never has 0dimensional volume 0. 3 5.2. 7 a. Show that if X C Rn is a bounded subset of kdimensional volume 0, then its projection onto the subspace spanned by any k standard basis vectors also has kdimensional volume 0. b. Show that this is not true if X is unbounded. For instance, produce an unbounded subset of IR. 2 of length 0, whose projection onto the xaxis does not have length O. *5.2.8 Prove Theorem 5.2.6. There are many approaches, all quite fiddly. One possibility is to cover M by a sequence of cubes Ci such that Mn Ci is the graph ri of a map fi expressing n  k variables in terms of the other k. These ri will probably not have disjoint interiors: doctor up Ci to remove the part of r i that is in the interior of some rj, j < i. Spread out the domains of these fi so they are well separated. The first condition of part 5 of the theorem is then easy, and the second follows from the HeineBorel theorem (Theorem A3.3 in Appendix A3). 3 We did not ask what the 0dimensional volume of a single point is. In fact, 0dimensional volume of a set simply counts its points. For this to be a special case of Definition 5.1.3, you need to see that the determinant of the empty matrix (not the 0matrix) is 1, and that is what the normalization condition of the determinant says: the determinant of the identity Rn + Rn is 1 for all n, including n = 0, when the matrix of the identity is the empty matrix. If you feel that this is a stretch, we sympathize.
538
5.3
Chapter 5.
Volumes of manifolds
COMPUTING VOLUMES OF MANIFOLDS
The kdimensional volume of a kmanifold M embedded in Rn is given by volk M
We feel that dl, ds, dS, dA, and dV are unfortunate, because this notation does not generalize to higher dimensions.
=JM ldkxl,
5.3.1
where ldkxl is the integrand that takes a kparallelogram and returns its kdimensional volume. Thus the length of a curve C can be written fc ld1xl, and the area of a surface S can be written f 8 ld2 xl. Often, ld1xl is written dl or ds, and called the element of length; ld2 xl is written dS or dA, and called the element of area; and ld3 xl is often written dV, and called the element of volume. Heuristically, the integral in equation 5.3.1 is defined by cutting up the manifold into little anchored kparallelograms, adding their kdimensional volumes and taking the limits of the sums as the decomposition becomes infinitely fine. We know how to compute ldkxl of a kparallelogram: if T = [v\, ... , vk], ldkx1Px(v1, ... , vk) = volk Px(vi, ... , vk) = vdet (TTT).
In equation 5.3.3, Py(u) ( .0i;(u),
... , 5,;,(u))
is the kparallelogram anchored at ')'(u) and spanned by the partial derivatives +
Dn(u), ... , Dk')'(u);
To compute the volume of a kmanifold M, we parametrize M by a mapping 'Y and then compute the volume of the kparallelograms spanned by the partial derivatives of "f, sum them, and take the limit as the decomposition becomes infinitely fine. This gives the following definition.
Definition 5.3.1 (Volume of manifold). Let M C Rn be a smooth kdimensional manifold, U a pavable subset of Rk, and 'Y : U  M a parametrization according to Definition 5.2.3. Let X be as in that definition. Then
~
1
Note that the volume of a manifold may be infinite.
=
fu_x (idkxl (P
In Definitions 5.3.1 and 5.3.2 we integrate over U  X, not over U, because ')' may not be difBut X has ferentiable on X. kdimensional volume 0, so this doesn't affect the integral.
=
ldkxl of this parallelogram is the volume of the parallelogram (see Proposition 5.1.1).
5.3.2
volk M
ldkxl
y(UX) 7 (u)
Lx
(.Di;(u), ... , D;;;(u)) )) ldkul
)det([D'Y(u)JT[D'Y(u)]) ldkul.
5.3.3
Remark. When the manifold is a curve parametrized by 'Y : [a, b]  C, equation 5.3.3 can be written
1
1d1x1 =
C
r
J[a,~
vdet({'(t) ·i(t)) 1dt1 =
rb 1i(t)ldt,
~
which is compatible with Definition 3.9.5 of arc length.
5.3.4
/::;.
Definition 5.3.1 is a special case of the following. In Definition 5.3.2 recall that 1 denotes the indicator function (Definition 4.1.1); note that ly('Y(u)) = ly1(Y)(u).
5.3
Definition 5.3.2: In Chapter 6 we will study integrals of differential forms over manifolds. The integral of Definition 5.3.2 is sometimes referred to as the integral of a density. Differential forms are better, but less intuitive. The reason is that for forms there is a notion of derivative, which makes possible the generalization of the fundamental theorem of calculus. There is no such thing for densities.
539
Computing volumes of manifolds
Definition 5.3.2 (Integrals over manifolds, with respect to volume). Let MC Rn be a smooth kdimensional manifold, U a pavable subset of Rk, and 'Y : U  t M a parametrization. Let X be as in Definition 5.2.3. Then f : M  t R is integrable over M with respect to volume if the integral on the right of equation 5.3.5 exists, and then
[ f(x)ldkxl
JM
~
[
J('Y(u))Vdet([D'Y(u)jT[D'Y(u)]) ldkul.
lux
5.3.5
In particular, if Y CM is a subset such that ly1(Y) is integrable, then volk Y
~
1
UX
ly('Y(u))Vdet((D'Y(u)JT[D'Y(u)]) ldkul.
Let us see why Definition 5.3.2 corresponds to our notion of volume (or area). To simplify the discussion, let us consider the area of a surface S parametrized by 'Y : U  t R 3 . This area should be
'°"
lim
N+oo
~
Area of 'Y(C n U).
5.3.6
CEVN(IR 2 )
That is, we make a dyadic decomposition of R 2 and see how 'Y maps to S the dyadic squares C that are in U or straddle it. We then sum the areas of the resulting regions 'Y( C n U). For C C U, this is the same as 'Y( C); for C straddling U, we add to the sum the area of the part of C that is in U. The sidelength of a square CE VN(JR. 2 ) is 1/2N, so at least when Cc U, the set 'Y(C n U) is, as shown in Figure 5.3.1, approximately the parallelogram FIGURE
5.3.1.
A surface approximated by parallelograms. The point xo corresponds to 1(u), and the vectors v1 and v2 correspond to the vectors 1 1 2N Dn(u) and 2 N D21(u).
Py(u)
1 2ND2'Y(u) 1  )' (2NDn(u),
5.3.7
where u is the lower left corner of C. That parallelogram has area
;N v'det(D'Y(u)jT (D'Y(u)j.
5.3.8
2
So it seems reasonable to expect that the error we make by replacing Area of 'Y(C n U)
by
vol2 (C)Vdet[D'Y(u)JT[D'Y(u)]
in formula 5.3.6 will disappear in the limit as N Riemann sum for the integral giving surface area:
t
oo, and we have a
1
J~ CEVN(JR2) L voh cv'det[D'Y(u)jT[D'Y(u)] = _u__________ v'det[D'Y(u)]T[D'Y(u)] ld2 ul. _ formula 5.3.6 with substitution given by 5.3.9
5.3.9
5.3.10
area of surface by eq. 5.3.3
This argument isn't entirely convincing. We can imagine the parallelograms as tiling the surface: we glue small fl.at tiles at the corners of a grid drawn on the surface, like using ceramic tiles to cover a curved counter. We get a better and better fit by choosing smaller and smaller tiles, but is it good enough? To be sure that Definition 5.3.2 is correct, we need to show that the integral is independent of the choice of parametrization.
540
Chapter 5.
Volumes of manifolds
Proposition 5.3.3 (Integral independent of parametrization). Let M be a kdimensional manifold in an and f : M t JR a function. If U and V are subsets of JRk and 'Yl : U t M, "(2 : V  M are two parametrizations of M, then
fu J('Y1(u)) Jdet([D'Y1(u)JT[D'Y1 (u)]) ldkul
5.3.11
exists if and only if [
5.3.12
J('Y2(v)) y'det([D'Y2(v)JT[D'Y2(v)]) ldkvl
exists, and in that case, the integrals are equal. In the onedimensional case, this gives
Equation 5.3.16: The equality marked 1 uses the change of variables formula
i
i !(
f(v}ldnvl
=
'f>(u)) I det[D'f>(u)]I Wul.
Note that we replace each v in the integrand in the first line by 'f>(u). This application of the change of variables formula is justified by Theorem 5.2.11. Equation 5.3.16, equality 2: [Dif>(u)] is a square matrix, and for a square matrix A,
5.3.13
1f
Integrating the length of the velocity vector, (t)I, gives us distance along the curve. Thus Proposition 5.3.3 says that if you take the same route from New York to Boston two times, making good time the first and caught in a traffic jam the second, giving two different parametrizations by time of your route, you will go the same distance both times.
Proof. Define
nr+ 1 given by FIGURE 5.5.1. Felix Hausdorff ( 18681942) Hausdorff worked in Bonn until 1935, when he was forced to retire because he was Jewish. In 1942 he and his wife committed suicide rather than be sent to a concentration camp.
'Pt(x)
= x + tw(x).
Show that voln(Mt) has the Taylor polynomial voln(Mt)
= voln M
 2t
ls
H(x). w(x)ldnxl
+ o(t) .
sn
be a unit normal vector 5 .4 .8 Let M be as in Exercise 5.4.5. Let fi : M > field on M. a . Show that the domain and image of [Dfi(x)J are both TxM, so det[Dfi(x)) makes sense (see Remark 4.8.7.) b . Show that det[Dfi(x)]
5.5
= K(x), where K
is as in Exercise 5.4.5.
FRACTALS AND FRACTIONAL DIMENSION
In 1919, Felix Hausdorff showed that dimensions are not limited to length, area, volume, and so on: we can also speak of fractional dimension . This discovery acquired much greater significance with the work of Benoit Mandelbrot, who showed that many objects in nature (the lining of the lungs, the patterns of frost on windows, the patterns formed by a film of gasoline on water) are fractals, with fractional dimension.
FIGURE 5.5.2. The first five steps in constructing the Koch snowflake. Its length is infinite, but length is the wrong way to measure this fractal object. The Koch snowflake was first described by the Swedish mathematician Helge von Koch in 1904.
Example 5.5.1 (Koch snowflake) . To construct the Koch snowflake curve K, start with a line segment, say 0:::; x:::; 1, y = 0 in IR 2 • Replace its middle third by the top of an equilateral triangle, as shown in Figure 5.5.2. This gives four segments, each onethird the length of the original segment. Replace the middle third of each by the top of an equilateral triangle, and so on. What is the length of this "curve"? At resolution N = 0, we get length 1. At N = 1, when the curve consists of four segments, we get length 4 · 1/3. At N = 2, the length is 16 · 1/9; at the Nth, it is (4/3)N. As the resolution becomes infinitely fine, the length becomes infinite! "Length" is the wrong word to apply to the Koch snowflake, which is neither a curve nor a surface. It is a fractal, with fractional dimension: the Koch snowflake has dimension ln 4/ ln 3 ~ 1.26. Let us see why this might
5.5 Fractals and fractional dimension
561
FIGURE 5.5.3. The curve B consists of four copies of A. It is also A scaled up by a factor of 3. It is the difference between these factors 3 and 4 that results in the fractional dimension of the curve. be the case. Call A the part of the curve constructed on [O, 1/3], and B the whole curve, as in Figure 5.5.3. Then B consists of four copies of A. (This is true at any level, but it is easiest to see at the first level, the top graph in Figure 5.5.2.). Therefore, in any dimension d, it should be true that vold(B) = 4vold(A). However, if you expand A by a factor of 3, you get B. (This is true in the limit, after the construction has been carried out infinitely many times.) According to the principle that area goes as the square of the length, volume goes as the cube of the length, etc., we would expect ddimensional volume to go as the dth power of the length, which leads to vold(B)
= 3dvold(A).
5.5.1
If you put this equation together with vold(B) = 4vold(A), you will see that the only dimension in which the volume of the Koch curve can be different from 0 or oo is the one for which 4 = 3d (i.e., d = ln4/ln3). If we break up the Koch curve into the pieces built on the sides constructed at the nth level (of which there are 4n, each of length 1/3n) and raise their sidelengths to the dth power, we find 4n
(
l)nln4/ln3
4n
3
4n
_
= 4nenl~~(ln !l = 4nenl~~(ln3) = 4nenln4 = _ = 1.
5.5.2
(In equation 5.5.2 we use the fact that ax = ex In a.) Although the terms have not been defined precisely, you might expect this computation to mean
L
ldxln 4/ In 31
FIGURE 5.5.4. The Sierpinski gasket: second, fourth, fifth, and sixth steps.
= 1.
!:::,,
5.5.3
Example 5.5.2 (Sierpinski gasket). While the Koch snowflake looks like a thick curve, the Sierpinski gasket looks more like a thin surface. This is the subset of the plane obtained by taking a filled equilateral triangle of sidelength l, removing the central inscribed subtriangle, then removing the central subtriangles from the three triangles that are left, and so on, as sketched in Figure 5.5.4. We claim that this is a set of dimension ln3/ln2. At the nth stage of the construction, sum, over all the little pieces, the sidelength to the power p: 5.5.4
562
Chapter 5.
Volumes of manifolds
(If measuring length, p = 1; if measuring area, p = 2.) If the set had a length, the sum would converge when p = 1, as n+ oo; in fact, the sum is infinite. If it really had an area, then the power p = 2 would lead to a finite limit; in fact, the sum is 0. But when p = ln 3/ ln 2, the sum converges to 5s. This is the only dimension in which the Sierpinski gasket zln 3 / In 2 ~ has finite, nonzero volume; in larger dimensions, the volume is O; in smaller dimensions, it is infinite. 6
zi.
EXERCISES FOR SECTION
5.5
5.5.1 Consider the triadic Cantor set C obtained by removing from (0, 1) first the open middle third (1/3, 2/3), then the open middle third of each of the segments left, then the open middle third of each of the segments left, etc.:
a. Show that an alternative description of C is that it is the set of points that can be written in base 3 without using the digit 1. Use this to show that C is an uncountable set. b. Show that C is a pavable set, with onedimensional volume 0. c. Show that the only dimension in which C can have volume different from 0 or infinity is In 2/ In 3.
Hint for Exercise 5.5.1, part a: the number written as .02220000022202002222 ... in base 3 is in C.
5.5.2 This time let the set C be obtained from the unit interval by omitting the open middle 1/nth, then the open middle 1/nth of each of the remaining intervals, then the open middle 1/nth of the remaining intervals, etc. (When n is even, this means omitting an open interval equivalent to 1/nth of the unit interval, leaving equal amounts on both sides, and so on.) a. Show that C is a pavable set, with onedimensional volume 0 . b. What is the only dimension in which C can have volume different from 0 or infinity? What is this dimension when n = 2?
5.6
REVIEW EXERCISES FOR CHAPTER
5
5.1 Verify that equation 5.3.30 parametrizes the torus obtained by rotating around the zaxis the circle of radius r in the (x,z)plane that is centered at x = R, z = 0.
f : [a, b)
5.2
Let
R be a smooth positive function. Find a parametrization x2 y2 ( )2 for the surface of equation A2 + B 2 = f(z) .
5.3
For what values of a does the spiral ( r(t))
t
+
= 1 and t = oo have finite length?
B(t)
(l/t"), a > 0 between
5.6 5.4
Review exercises for Chapter 5
Compute the area of the graph of the function
f ( ~)
=
j (x 312 +
563
y 3 12 )
above the region 0 $ x $ 1, 0 $ y $ 1. 5.5
Exercise 5.5, part b: The answer should be in the form of a onedimensional integral.
Let
f be a positive C 1 function of x E [a, b).
a. Find a parametrization of the surface in R 3 obtained by rotating the graph of f around the xaxis. b. What is the area of this surface? *5.6
Let Wn+i(r)
= voln+1(B;.'+l(O))
be the (n+l)dimensional volume of the ball of radius r in Rn+l, and let voln(S:.') be the ndimensional volume of the sphere of radius r in Rn+l. a. Show that w~+ 1 (r) = voln(S;.'). b. Show that voln(S;.')
= rn voln(Sf ).
c. Derive equation 5.3.50, using Wn+1(l) The total curvature of a surface is defined in Exercise 5.3.11.
= fo 1 w~+i(r)dr.
5.7 Let H be the helicoid of equation ycosz = xsinz (see Example 3.9.13). What is the total curvature of the part of H with 0 $ z $a?
5.8
For z E C, the function cos z is by definition cos z
a. If z
=
eiz
+ eiz
= x + iy, write the real and imaginary parts of cos z
b. What is the area of the part of the graph of cos z where 1$ y$1?
2 in terms of x, y. 71"
$ x $
7r
and
5.9 Let the set C be obtained from the unit interval [O, 1) by omitting the open middle 1/5th, then the open middle fifth of each remaining interval, then the open middle fifth of each remaining interval, etc.
Exercise 5.10 is inspired by a proposition in Functional Analysis, Volume 1: A Gentle Introduction, by Dzung Minh Ha (Matrix Editions, 2006).
a. Show that an alternative description of C is that it is the set of points that can be written in base 5 without using the digit 2. Use this to show that C is an uncountable set. b. Show that C is a pavable set, with onedimensional volume 0. c. What is the only dimension in which C can have volume different from 0 or infinity? 5.10 Let :Xo,x1, ... ,Xk be vectors in Rn, with :X1, ... ,xk linearly independent, and let M C Rn be the subspace spanned by xi, ... ,xk. Let G be the k x k matrix G = [x 1 ... xk)T[:X1 ... xk) and let a+ be the (k + 1) x (k + 1) matrix
a+= [:Xo:X1 ... xk)T[:Xo:X1 ... :Xk)·
Vectors for Exercise 5.10, part b
The distance d(x, M) is by definition d(x, M) = infyeM(x  y). a. Show that 2 ( d(:Xo, M) ) = detG+ det G . b. What is the distance between :Xo and the plane M spanned by :X1 and :X2 (as defined in the margin)?
6 Forms and vector calculus Gradient a 1form? How so? Hasn't one always known the gradient as a vector? Yes, indeed, but only because one was not familiar with the more appropriate 1form concept.C. Misner, K. S. Thorne, J. Wheeler, Gravitation
6.0 In onevariable calculus, the standard integrand f(x) dx takes a piece [xi, Xi+1] of the domain and returns the number
f(xi)(Xi+l  Xi) :
the area of a rectangle with height f(xi) and width Xi+i  Xi. Note that dx returns Xi+l  Xi, not lxi+l xii; this accounts for equation 6.0.1. In Chapter 4 we studied the integrand ldnxl, which takes a (flat) subset A C Rn and returns its ndimensional volume. In Chapter 5 we showed how to integrate ldkxl over a (curvy) kdimensional manifold in Rn to determine its kdimensional volume. Such integrands require no mention of the orientation of the piece.
Differential forms are a special case of tensors. A tensor on a manifold is "anything you can build out of tangent vectors and duals of tangent vectors": a vector field is a tensor, as is a quadratic form on tangent vectors. Although tensor calculus is a powerful tool, especially in computations, we find that speaking of tensors tends to obscure the nature of the objects under consideration.
INTRODUCTION
What really makes calculus work is the fundamental theorem of calculus: that differentiation, having to do with speeds, and integration, having to do with areas, are somehow inverse operations. We want to generalize the fundamental theorem of calculus to higher dimensions. Unfortunately, we cannot do so with the techniques of Chapters 4 and 5, where we integrated using ldnxj. The reason is that ldnxl always returns a positive number; it does not concern itself with the orientation of the subset over which it is integrating, unlike the dx of onedimensional calculus, which does:
lb
f(x) dx = 
la
f(x) dx.
6.0.1
The cancellations due to opposite orientations make possible the fundamental theorem of calculus. To get a fundamental theorem of calculus in higher dimensions, we need to define orientation in higher dimensions, and we need an integrand that gives one number when integrating over a domain with one orientation, and the opposite number when integrating over a domain with the opposite orientation. It follows that orientation in higher dimensions must be defined in such a way that choosing an orientation is always a choice between one orientation and its opposite. It is fairly clear that you can orient a curve by drawing an arrow on it; orientation then means, what direction are you going along the curve, with the arrow or against it? For a surface in JR3 , an orientation is a specification of a direction in which to go through the surface, such as crossing a sphere "from the inside to the outside" or "from the outside to the inside" . These two notions of orientation, for a curve and for a surface, are actually two instances of a single notion: we will provide a single definition of orientation that covers these cases and all others as well (including 0manifolds, or points, which in other approaches to orientation are sometimes left out). Once we have determined how to orient our objects, we must choose our integrands: the mathematical creature that assigns a little number to 564
6.1
y
FIGURE
6.0.1.
The radial vector field
The important difference between determinants and kforms is that a kform on Rn is a function of k vectors, while the determinant on nr is a function of n vectors; determinants are defined only for square matrices.
Section 6.12 is an ambitious treatment of electromagnetism using forms; we will see that Maxwell's laws can be written in the elegant form dlF=O,
dM=4d.
Forms on Rn
565
a little piece of the domain. If we were willing to restrict ourselves to IR.2 and IR.3 , we could use the techniques of vector calculus. Instead we will use forms, also known as differential forms. Forms make possible a unified treatment of differentiation and of the fundamental theorem of calculus: one operator (the exterior derivative) works in all dimensions, and one short, elegant statement (the generalized Stokes's theorem) generalizes the fundamental theorem of calculus to all dimensions. In contrast, vector calculus requires special formulas, operators, and theorems for each dimension where it works. But the language of vector calculus is used in many science courses, particularly at the undergraduate level. In addition, the functions and vector fields of vector calculus are more intuitive than forms. A vector field is an object that one can picture, as in Figure 6.0.1. Coming to terms with forms requires more effort: we can't draw you a picture of a form. A kform is, as we shall see, something like the determinant: it takes k vectors, fiddles with them until it has a square matrix, and then takes its determinant. For these two reasons we have devoted three sections to translating between forms and vector calculus: Section 6.5 relates forms on IR.3 to functions and vector fields, Section 6.8 shows that the exterior derivative we define using forms has three separate incarnations in the language of vector calculus: grad, curl, and div. Section 6.11 shows how Stokes's theorem, a single statement in the language of forms, becomes four more complicated statements in the language of vector calculus. In Section 6.9 we discuss the pullback of form fields, which describes how integrands transform under changes of variables. Because forms work in any dimension, they are the natural way to approach two towering subjects that are inherently fourdimensional: electromagnetism and the theory of relativity. Electromagnetism is the subject of Section 6.12. Section 6.13 introduces the cone operator to deal with potentials in sufficient generality to apply to electromagnetism. We begin by introducing forms; we will then see (Section 6.2) how to integrate forms over parametrized domains (domains that come with an inherent orientation), before tackling the issue of orientation in Sections 6.3 and 6.4.
6 .1 FORMS ON ]Rn
Our treatment of forms, especially the exterior derivative, was influenced by Vladimir Arnold's book Mathematiool Methods of Classiool Mechanics.
In Section 4.8 we saw that the determinant is the unique antisymmetric and multilinear function of n vectors in !Rn that gives 1 if evaluated on the standard basis vectors. Because of the connection between the determinant and volume described in Section 4.9, the determinant is fundamental to changes of variables in multiple integrals, as we saw in Section 4.10. Here we will study the multilinear antisymmetric functions of k vectors in !Rn, where k ;:::: 0 may be any integer, though we will see that the only interesting case is when k :::; n. Again there is a close relation to volumes; these objects, called forms or kforms, are the right integrands for integrating over oriented kdimensional domains.
566
Definition 6.1.1 is actually the definition of a constant kform. In this section we mainly discuss the algebra of constant forms, which we will refer to as forms. Later in this chapter we will use kform fields, which have a kform at every point; see Definition 6.1.16. There is the same relationship between constant forms and form fields as between numbers and functions.
Chapter 6.
Forms and vector calculus
Definition 6.1.1 (kform on Rn). A kform on Rn is a function cp that takes k vectors in Rn and returns a number cp(vi, ... , vk), such that cp is multilinear and antisymmetric as a function of the vectors. The number k is called the degree of the form. The next example is the fundamental example. Example 6.1.2 (kform). Let ii, ... ,ik be any k integers between 1 and n. Then dxi 1 /\ • • • /\ dxik is that function of k vectors 1 , ... , Vk in Rn that puts these vectors side by side, making the n x k matrix
v
6.1.1
Antisymmetry If you exchange any two of the arguments of cp, you change the sign of cp:
and selects k rows: first row ii, then row i2, etc., and finally row ik, making a square k x k matrix, and finally takes its determinant. For instance,
cp (v1, ... , vi, ... , vi, ... , vk) =  cp(V1, ... , vj,
~([~] .[l])
... , vi, ... , vk).
Multilinearity If cp is a kform and
=det
2
[; ~J .._,,_......
6.1.2
=8.
1st and 2nd rows of original matrix
vi= aii+ bW,
then cp( v1, ... , (aii + bw), ... ,vk)
1
2form
=
dx1 /\ dx2 /\ dx4 ( 3form
[~] 1
acp(v1, ... ,Vi1, ii, VH1, ... ,vk)+ bcp(v1, ... , Vi1, w, VH1, ... , vk)· dx2/\dx1/\dx4([j] 3form 1
Equation 6.1.3: Note that to give an example of a 3form we had to add a third vector. You cannot evaluate a 3form on two vectors (or on four); a kform is a function of k vectors. But you can evaluate a 2form on two vectors in !i4 (as we did in equation 6.1.2) or in R 16 . This is not the case for the determinant, which is a function of n vectors in Rn.
[l]
[~])
[~
3 2 2
n~7
[l] [~])=det [:
2 3 2
~]
2
2
1
1
=det
=7
6.1.3
!:::,,.
Example 6.1.3 (0form). Definition 6.1.1 makes sense even if k = 0: a 0form on Rn takes no vectors and returns a number. In other word, it is that number. !:::,,. Remarks. 1. For now think of a form like dx1 /\ dx2 or dx1 /\ dx2 /\ dx4 as a single item, without worrying about the component parts. The reason for the wedge /\ will be explained at the end of this section, where we discuss the wedge product, we will see that the use of /\ in the wedge product is consistent with its use here. In Section 6.8 we will see that the use of d in our notation here is consistent with its use to denote the exterior derivative. 2. The integrand ldkxl of Chapter 5 also takes k vectors in Rn and gives a number:
ldkxl(v1, ... , v\) =
det ([Vi. ... , vk]T [vi. ... , vkl).
But these integrands are neither multilinear nor antisymmetric.
6.1.4 !:::,,.
6.1
Forms on
~n
567
Note there are no nonzero kforms on !Rn when k > n. If v\, ... , vk are vectors in !Rn and k > n, then the vectors are not linearly independent, and at least one of them is a linear combination of the others, say k1 6.1.5
v\, ... , Vk gives
Then if cp is a kform on !Rn, evaluation on cp(v1, ... , vk)
= cp (vi, ... , Eaivi) = Eaicp(vi. ... , vk1, vi)· i=l
6.1.6
i=l
The first term of the sum at right is a 1cp(vi, ... , vk1' v\), the second is a2cp(v1, v2, ... , vk1' v2), and so on; each term evaluates cp on k vectors, two of which are equal, and so (by antisymmetry) the kform returns 0.
Geometric meaning of kforms Evaluating the 2form dx1 /\ dx2 on vectors Rather than imagining projecting a and 6 onto the plane to get the vectors of equation 6.1.8, we could imagine projecting the parallelogram spanned by a and 6 onto the plane to get the parallelogram spanned by the vectors of formula 6.1.8.
If we project
a, b E JR
3 ,
we have
a and b onto the (x1, X2)plane, we get the vectors [
~~ ]
and
[
:~ ] ;
6.1.8
the determinant in equation 6.1. 7 gives the signed area of the pamllelogmm spanned by the vectors in equation 6.1. 8. Thus dx 1 /\ dx 2 deserves to be called the (x 1 , x 2 )component of signed area. Similarly, dx2 /\ dx3 and dx1 /\ dx3 deserve to be called the (x2, X3) and (xi, X3)components of signed area.
We can now interpret equations 6.1.2 and 6.1.3 geometrically. The 2form dx 1 /\ dx 2 tells us that the (x 1, x 2)component of signed area of the parallelogram spanned by the two vectors in equation 6.1.2 is 8. The 3form dx1 /\ dx2 /\ dx4 tells us that the (x1, x2, X4)component of signed volume of the parallelepiped spanned by the three vectors in equation 6.1.3 is 7. Similarly, the 1form dx gives the xcomponent of signed length of a vector, while dy gives its ycomponent: dx
([fl) ~
det2
~2
and dy (
[m ~
det(3)
~ 3.
6.1.9
More generally (and an advantage of kforms is that they generalize so easily to higher dimensions), we see that 6.1.10
568
Chapter 6.
Forms and vector calculus
is the ith component of the signed length of v, and that 6.1.11 gives the (xii> ... , Xik )component of signed kdimensional volume of the kparallelogram spanned by vi, ... , vk. Figure 6.1.1 shows a parallelogram P spanned by two vectors, vi and v 2 ,
p
x FIGURE
6.1.1.
The parallelogram P spanned by
v1
=[
~ ] , = [0~5] 0.5 1.5 v2
and its projections onto the (x, y)plane (P3), the (x, z)plane (P2), and the (y, z)plane (P1). Evaluating appropriate 2forms on v1, v2 gives the area of those projections:
= dy A dz(v1, v2) = 5/4 voh P2 = dx A dz(v1, v2) = 2 voh P3 = dx A dy(v1, v2) = 3/2.
voh Pi
~
) , and the projections of Ponto the (x, y)plane 1.5 (P3), the (x, z)plane (P2), and the (y, z)plane (Pi). Evaluating dx A. dy on v\, v2 gives the signed area of P3, while dx A. dz gives the signed area of P2 and dy A. dz gives the signed area of Pi. Note that although we defined a kform as a function of k vectors, we can also think of it as a function of the kparallelogram spanned by those k vectors. In terms of this geometric description, the statement that there are no nonzero kforms when k > n should come as no surprise: you would expect any kind of threedimensional volume in JR2 to be zero, and more generally any kdimensional volume in ]Rn to be 0 when k > n. both anchored at (
z
Elementary forms There is a great deal of redundancy in the expressions dxi 1 A. · • · A. dxik. Consider dxi A. dx3 A. dxi. This 3form stacks three vectors in JRn side by side to make an n x 3 matrix, selects the first row, then the third, then the first again, to make a 3 x 3 matrix and takes its determinant. This determinant is always 0. (Do you see why?i) But dxi A. dx3 A. dxi is not the only way to write the form that takes three vectors and returns O; both dxi A. dxi A. dx3 and dx2 A. dx3 A. dx3 do so as well. Indeed, if any two of the indices ii, ... , ik are equal, then dxi 1 A. · · · A. dxik = 0. Next, consider dxi A.dx3 and dx3 A.dxi. When these forms are evaluated
on a~ [2] and b~ []find dxi A. dx3(a, b)
= det
[ai a3
dx3 A. dxi(il, b)
= det
[a 3 ai
6.1.12
Clearly dxi A. dx3 = dx3 A. dx 1 ; these two 2forms, evaluated on the same two vectors, always return opposite numbers. 1 The determinant of a square matrix containing two identical columns is always O, since exchanging them reverses the sign of the determinant, while keeping it the same. Since (Theorem 4.8.8), det A = det AT, the determinant of a matrix is also 0 if two of its rows are identical.
6.1
Recall that the signature of a permutation a is denoted sgn(a); see Theorem and Definition 4.8.11. Theorem 4.8.12 gives a formula for det using the signature. Definition 6.1.4: Putting the indices in increasing order selects one particular permutation for any set of distinct integers }1, ... ,jk. For example, if n = 4 and k = 3, then 1 ~ ii < · · · < ik ~ n is the set {(i1=l,i2=2,ia = 3),
Forms on 1r
569
More generally, if ii, ... , ik and ji, ... , jk are the same integers, just taken in a different order, so that ji = iu(i),j2 = iu(2), ... ,jk = iu(k) for some permutation a of {1, ... , k }, then dx1·1
Indeed, dx1 1 dxi 1
/\ • • •
/\ • · • /\
dx Jk · = sgn(a)dx·•1 /\ · · · /\ dx·•k.
6.1.13
dx1k computes the determinant of the same matrix as /\ dxik, only with the rows permuted by a. For instance, /\ • • • /\
6.1.14
We will find it useful to define a special class of kforms, elementary kforms, which avoid this redundancy: Definition 6.1.4 (Elementary kforms on Rn). kform on Rn is an expression of the form
An elementary
(i1=l,i2=2,ia = 4), (i1=l,i2=3,ia = 4), (i1=2,i2=3,ia = 4)},
which corresponds to dx1 /\ dx2 /\ dxa, dx1 /\ dx2 /\ dx4,
and so on. The kform dx2 /\ dx1 /\ dxa
exists, but it is not an elementary kform.
6.1.15
vi, ... ,
where 1 ~ ii < · · · < ik :$ n. Evaluated on the vectors Vk, it gives the determinant of the k x k matrix obtained by selecting rows ii, ... , ik of the matrix whose columns are the vectors Vi, ... , vk. The only elementary 0form is the form, denoted 1, which evaluated on zero vectors returns 1. We saw that there are no nonzero kforms on Rn when k > n; not surprisingly, there are no elementary kforms on Rn when k > n. (That there are no elementary kforms when k > n is an example of the "pigeonhole" principle: if more than n pigeons fit in n holes, one hole must contain at least two pigeons. Here we would need to select more than n distinct integers between 1 and n.)
All forms are linear combinations of elementary forms
Adding kforms 3dx/\dy+2dx/\dy = 5dx/\dy dx /\ dy + dy /\ dx = 0
Multiplying kforms by scalars 5 (dx /\ dy + 2dx /\dz)
= 5 dx /\ dy + 10 dx /\ dz
We said that dxi 1 /\ • • • / \ dxik is the fundamental example of a kform. We will now justify this statement by showing that any kform is a linear combination of elementary kforms. The following definitions say that speaking of such linear combinations makes sense: we can add kforms and multiply them by scalars in the obvious way. Definition 6.1.5 (Addition of kforms). Let cp and t/J be two kforms. Then 6.1.16
Definition 6.1.6 (Multiplication of kforms by scalars). If cp is a kform and a is a scalar, then 6.1.17
570
Chapter 6.
Forms and vector calculus
Using these definitions, the space of kforms on Rn is a vector space. The elementary kforms form a basis of this space. Saying that the space of kforms on Rn is a vector space (Definition 2.6.1) means that kforms can be manipulated in familiar ways, for example, for scalars a, /3, a(/3cp)
= (af3)cp
(associativity for multiplication),
Definition 6.1.7 (A~(Rn)).
The space of kforms on Rn is denoted
A~(Rn).
We use the notation A~ because we want to save the notation Ak for the space of form fields (Definition 6.1.16). The c in A~ stands for "constant": elements of A~ (Rn) can be thought of as constant form fields.
(a+ f3)cp = acp + f3cp (distributive law for scalar addition), and
Theorem 6.1.8 (Elementary kforms form a basis for A~(Rn)). The elementary kforms form a basis for A~(Rn): every kform can be
uniquely written
a(cp + ,,P) = acp + a,,P
L
... , Vu(k+l)).
shuffles uEPerm(k,!)
l vectors
k vectors
where the sum is over all permutations a of the numbers 1, 2, ... , k such that a(l) < a(2) < · · · < a(k) and a(k + 1) < · · · < a(k + l). 2 The
following computation also justifies this equality:
(
n ) nk
=
n! (nk)!(n(nk))!
n!
= k!(n k)! ·
+l
6.1
Forms on
Jr 573
On the left we have a {k + l)form evaluated on k + l vectors. On the right we have a complicated expression involving a kform
R 2 . 6 Example 6.3.7 is a bit misleading: it seems to suggest that in order to orient a surface you need to guess a vector field. In reality, when a surface S C JR3 is given by an equation f(x) = 0, and moreover [Df(x)] =f. [OJ at all points of S, then the gradient (j f (x) = [D f (x)] T is a transverse vector field (in fact, normal) on S, and hence always defines an orientation.
\
Make a big Mobius strip out of paper. Give one child a yellow crayon, another a blue crayon, and start them coloring on opposite sides of the strip. Compare this figure with Figure 6.3.2.
Example 6 .3 .8. Consider the surface S defined by f(x) We saw in Example 3.1.13 that the vector field '(jj(x)
~
[ l x Df(y) z
T
=
[
= sin(x +yz) = 0.
cos(x + yz) zcos(x+yz) ycos(x+yz)
l
never vanishes at points of S . Better yet, it is orthogonal to S since, by Theorem 3.2.4, v E ker[Df(x)] implies v E TxS. Thus, for v E TxS,
6.3.14
O = [Df(x)]v = (j f(x) · v . Example 6.3.8: The gradient V f(x) was defined in equation 1.7.22 as
6.3.13
So the vector
x=
(j f (x) defines a n orientation of S .
6
This construction can be considerably generalized.
Vf(x) ~f [Df(xW Usually we prefer to work with the line matrix [Df(x)] rather than the vector Vf(x), but this is an exception. If a manifold M is defined by an equation f(x) = 0 (i.e., M has dimension one less than the ambient space), then at every x E M the vector Vf(x) is orthogonal to TxM.
Proposition 6.3.9 (Orienting manifolds given by equations). Let u c !Rn be open, and let f: u + ]Rnk be a map of class C 1 such that [Df(x)] is surjective at all x E M ~ r 1 (0). Then the map f!x: B(TxM)+ {+1, 1} given by f!(vi, ... , vk) ~ sgn det['(j Ji (x), ... , (j f nk(x), vi, . .. , vk]
6.3.15
is an orientation of M. Proposition 6.3.9 is perhaps a bit misleading, tempting the reader to think that all surfaces are orientable. Remember that manifolds can always
6.3
Orientation of manifolds
587
be given locally by equations (one equation for a surface in JR3 ), but it may be impossible to define a kdimensional manifold in ]Rn by n  k equations with linearly independent derivatives. In particular, it will be impossible if the manifold is nonorientable. See Example 6.4.9.
Orientation of connected and unconnected manifolds Recall from the discussion in Example 3.1.8 (margin note below Figure 3.1.9) that a manifold Mis connected if given any two points u 1 , u 2 EM, there exists a path in M connecting them: a continuous map o: [a, b] + M such that O(a) = U1 and O(b) = U2. FIGURE
6.3.5.
Johann Listing (18081882) Listing was the first to describe the Mobius strip; he also coined the words "topology" and "micron". At age 13, he helped support his family with money he earned drawing and doing calligraphy. He became a physicist as well as mathematician, and worked with Gauss on experiments concerning terrestrial magnetism:
(a)
Proposition 6.3.10 (Orientation of connected, orientable manifold). If Mis a connected manifold, then either Mis not orientable, or it has two orientations. If M is orientable, then specifying an orientation ofTxM at one point defines the orientation at every point. Note that if an orientable manifold is not connected, knowing its orientation at one point does not tell you its orientation everywhere.
Proof. If Mis orientable and !1: l3(M)+ {+1, 1} is an orientation of M, then !1 is also an orientation, so an orientable manifold has at least two orientations. We must show that if M is connected and !1', !1" are two orientations of M such that n~o = n~o for one point Xo E M, then n~ = n~ for all x EM. Choose a path 'Y: [a,b]+ M with 'Y(a) = xo and "f(b) = x. Then for all t there exists s(t) E { +1, 1} such that n~(t) = s(t)D~(t)" Moreover, the function s is continuous, and a continuous function on [a, b] that takes only the values 1 and +1 must be constant, by the intermediate value theorem. Since !1~ 0 = !1~0 , we have s(a) = +1, so s(b) = +l. D
(b)
EXERCISES FOR SECTION (c)
6.3.1
Is the constant vector field [ ~] a tangent vector field defining an ori
entation of the line of equation x x y = O?
6.3.2 (d)
6.3.3
6.3
+y =
O? How about the line of equation
Which of the surfaces in the margin are orientable? Does any constant vector field define an orientation of the unit sphere in
IR.3?
Surfaces for Exercise 6.3.2
6.3.4
Find a vector field that orients the curve given by x
6.3.5
Which of the vector fields
m. [:i. [=:J. [=:i
+ x 2 + y 2 = 2.
588
Chapter 6.
Forms and vector calculus
define an orientation of the plane P C R 3 of equation x these, which pairs define the same orientation?
+y +z =
0, and among
6.3.6
Find a vector field that orients the surface Sc R 3 given by x 2 +y3 +z
6.3. 7
Let V be the plane of equation x
+ 2y 
z
= 0.
= 1.
Show that the bases
give the same orientation. 6.3.8 Let P be the plane of equation x a. Of the three bases
+ y + z = 0.
which gives a different orientation than the other two? b. Find a normal vector to P that gives the same orientation as that basis. 6.3.9 a. Let C c R 2 be the circle of equation (x  1) 2 + y 2 = 4. Find the unit tangent vector field f describing the orientation "increasing polar angle". (A "unit vector field" is a vector field in which all the vectors have length 1.) b. Explain carefully why the phrase "increasing polar angle" does not describe an orientation of the circle of equation (x  2)2 + y 2 = 1. 6.3.10
Exercise 6.3.13: Part a shows that there is no way to choose an orientation for all planes in R 3 that depends continuously on the plane. If there were such a standard orientation, then given any two linearly independent vectors v, w in R 3 , it would be possible to choose one: the plane that they span would have an orientation, and either (v, w) in that order, or (w, v) in that order would be a direct basis. We could then choose the first vector of the direct basis. Part b shows that in R 2 , this does work. Either det[v, w] > 0 or det[w, v] > 0, and we can choose the first column of whichever matrix has positive determinant. Part c shows that it does not work in Rn, n > 2. As in R 3 , we can exchange the vectors by a continuous motion, keeping them linearly independent at all times.
Let M
centered at (
c
R 2 be a manifold consisting of two circles of radius 1/2, one
i),
the other at
(A).
How many orientations does M have?
Describe them. 6.3.11 Let S C R 4 be the locus given by the equations x~  x~ 2x1x2 = X4.
X3
and
a. Show that S is a surface. b. Find a basis for the tangent space to S at the origin that is direct for the orientation given by Proposition 6.3.9. 6.3.12
Consider the manifold M C R 4 of equation x~
Fmd a"""' fo, the tangent'"""' to Mat the poffit (
~)
+ x~ + x~

X4
=
0.
that ls direct £o, the
orientation given by Proposition 6.3.9. 6.3.13
a. Find two continuous maps [O, 1]+ R 3 , denoted v and w, such that
v(O)
= e1,
v(l)
= 02,
w(O)
= 02,
w(l)
= 01,
and v(t) and w(t) are linearly independent for all t. b. Show that there are no such mappings [O, 1]+ R 2 • c. Show that given any two linearly independent vectors iii, li2 in Rn, n there exist maps v, w: [O, 1] +Rn such that
v(O)
= li1,
v(l)
= li2,
w(O)
= li2,
and for each t, v(t) and w(t) are linearly independent.
w(l)
= li1,
> 2,
6.4 Integrating forms over oriented manifolds
589
6.3.14 Where is the vector field ei transverse to the unit sphere in R 3 ? Where does the orientation given by this vector field to the unit sphere coincide with the orientation given by the outwardpointing normal? Exercise 6.3.15 shows how to orient an ndimensional complex vector space E: given a complex we can orient E basis by declaring the real basis
6.3.15 Let E be a complex vector space of dimension n. A basis of E is a set of vectors v1, ... , Vn such that every vector of E can be uniquely written c1 V1 + · · · + Cn v n, where the coefficients c1, ... , Cn are complex numbers. a. Show that the 2n vectors v1, iv1, ... , v n, ivn are a basis of E viewed as a real vector space, i.e., every vector x EE can be written uniquely as
to be direct. We will say that this orientation is standard. The exercise shows that this orientation does not depend on the choice of complex basis of E.
with ai, ... , an, bi, ... , bn E R. Denote this basis by {v}. b. Let w1, ... , Wn be another basis of E, with complex change of basis matrix C = [Pwv]i see Proposition and Definition 2.6.17. Denote the basis w1,iw1, ... ,wn,iWn by {w}. Write the change of basis matrix C = [Pw.vl in terms of the real and imaginary parts of the Cj,i· Hint: Try n = 1 first, then n = 2; the pattern should become clear. **c. Show that det = Idet c1 2 • Again, do n = 1 and n = 2 first. d. Let n be the orientation of E such that V1' iv1' ... 'v n, iv n is a direct basis. Show that then W1, iw1, ... , Wn, iwn is also a direct basis.
vi, ... ,vn,
X
= aiv1 + bi(iv1) + · · · + anVn + bn(ivn),
c
6.4 INTEGRATING FORMS OVER ORIENTED MANIFOLDS
Definition 6.4.1: For a linear transformation to preserve orientation, the domain and codomain must be oriented, and must have the same dimension. Of course,
The object of this section is to define the integral of a kform over an oriented kdimensional manifold. We saw in Chapter 5 that when we integrate ldkxl over a manifold, the integral does not depend on the choice of parametrization; this enabled us to define the integral of ldkxl over a manifold. In Example 6.2.5 we saw that the choice of parametrization does matter when we integrate a kform over an oriented manifold. To define the integral of a kform over an oriented manifold, we must stipulate that the parametrization preserve the orientation of the manifold. Are we going down the down staircase (the parametrization preserves the orientation) or up the down staircase (the parametrization reverses the orientation)? Definition 6.4.1 (Orientationpreserving linear transformation). Let V beakdimensional vector space oriented by n: B(V)+ {+1,1}. A linear transformation T: JRk+ Vis orientation preserving if
6.4.1
are the columns of T.
It is orientation reversing if 6.4.2 Note that if T in Definition 6.4.1 is not invertible, then it neither preserves nor reverses orientation, since in that case T(e1), ... , T(ek) is not a basis of V.
590
Once more, faced with a nonlinear problem, we linearize it: a parametrization of a manifold preserves orientation if its derivative preserves orientation. Definition 6.4.2: Since
Chapter 6.
Forms and vector calculus
Definition 6.4.2 (Orientationpreserving parametrization of a manifold). Let Mc Rm beakdimensional manifold oriented by n, and let U C Rk be a subset with boundary of kdimensional volume 0. Let 'Y : U + Rm parametrize M as described in Definition 5.2.3, so that the set X of "trouble spots" satisfies all conditions of that definition; in particular, X has kdimensional volume 0. Then 'Y is orientation preserving if for all u E (U  X), we have +
(UX) cRk is open, it is necessarily kdimensional. (We haven't defined dimension of a set; what we mean is that U  X is a kdimensional manifold.) Think of a disc in R 3 and a ball in R 3 ; the ball can be open but the disc cannot. The "fence" of a disc in R 3 is the entire disc, and it is impossible (Definition 1.5.2) to surround each point of the disc with a ball contained entirely in the disc. We could write the left side of equation 6.4.3 as
n([n'Y(u)]e1, ... , [D'Y(u)]ek ).
+
6.4.3
!l(Dn(u), ... , Dk'Y(u)) = +l.
Note that the vectors in equation 6.4.3 are linearly independent by part 4 of Definition 5.2.3, so, by Proposition 3.2.7, they form a basis of T"'f(u)M. Example 6.4.3 (Parametrizations of unit circle). Let C be the circle of equation x 2+y 2 = R 2, R > 0 oriented by the vector field t ( ~) = [
~] ,
as in Example 6.3.6. This is the counterclockwise orientation. . . 'Y (t) = (Rcost) . . smce . The parametrization R sin t preserves t h at orientation t('Y(t)) .;y'(t) = [RRsint]. [RRsint] = R 2 > O. cost cost
6.4.4
Ast increases, 'Y(t) travels around the circle counterclockwise. The parametrization 'Yi (t) t(yi(t)) ·
= ( ~ ~~~ ~ ) reverses the orientation since
7~(t) = [~~~::]
· [
~~~::]
= R2
As t increases, 'Yi (t) travels around the circle clockwise.
< 0.
6.4.5
6.
Recall from Proposition 6.3.4 that a transverse vector field ii orients a surface S c R 3 by the formula 6.4.6 +
If 'Y is a parametrization of S, and if we set ii = Dn x D2'Y, then 'Y is orientation preserving, since +
det[Di'Y x D2y,Dn,D2'Y] > 0.
6.4.7
This is a restatement of the fact (Proposition 1.4.20) that if v and linearly independent vectors in R 3 , then det[v x w, v, w] > 0. 6.
w are
Example 6.4.4 (Orientationpreserving parametrization of surface in C 3 ). Consid& the surface S
c
C 3 prucametdzed by z
~ ( ~: )
,
lzl < 1.
6.4 Integrating forms over oriented manifolds
The tangent space to S at (
~)
591
is the complex line consisting of the
1
complex multiples of ( 2z ) ; in particular, it carries a standard orientation 3z 2
r=(~)~
=
X2
x y x2 y2
Y2 X3
x 3  3xy 2
Y3
3x2y y3
Xl
Yi
2xy
Parametrization for Example 6.4.4: The first two entries correspond to the real and imaginary parts of z, the third entry to the real part of z 2 , the fourth entry to the imaginary part of z 2 , and so on.
1
= ( 2 z ) , v2 = (
2~z
) is direct; see Exercise 3z2 3iz 2 6.3.15. Think of S as a surface in JR6 , parametrized by the function 'Y in the margin. Does this parametrization preserve the standard orientation? Using Definition 6.3.1, we must compute the determinant of the change of matrix basis between the basis for which the real basis
V1
1 0
Di;(~)=
0 1
2x 2y 3x 2 
D;;(~)
=
2y 2x
6.4.8
6xy
3y2
6xy
3x2

3y2
and the basis (vi, v2). If we write v 1 and v 2 in real and imaginary parts, we get the vectors in equation 6.4.8, so the change of basis matrix is the identity and 'Y is orientation preserving. 6. Example 6.4.5: It is important to realize that a parametrization may be neither orientation preserving nor orientation reversing: it may preserve orientation on one part of the manifold and reverse it on another.
Example 6.4.5 (A parametrization that neither preserves nor reverses orientation). Consider the unit sphere parametrized by
7 (~)~(~~!). fo,o,;o,;~,[71'~lr.p~71', and oriented by the outwardpointing normal, ii =
det [
~
6.4.9
. We have
n(1 (:))'Di;(:)' n;; (:)] =cosr.p.
6.4.10
For 71' ~ r.p < 71' /2, we have  cos r.p > 0; for 71' /2 < r.p < 71' /2, we have  cos r.p < O; for 71' /2 < r.p ~ 71', we have  cos r.p > O; the sign of the determinant changes, so the parametrization keeps switching between reversing and preserving orientation. What goes wrong here? The mapping 'Y takes the entire line r.p = 71' /2 in the ('{>, 8)plane to the single point ( _
the point (
~) ; it takes the line ~ ~/2 to
~) ; fo• those values, 7 is not one to one.
r > 0 and taking the circle of radius r in the (x, z)plane that is centered at x = R, z = 0, and rotating it around the zaxis, as shown in Figure 6.4.1. (This is a twodimensional object, not the solid torus.) Assume it is oriented by the outwardpointing normal. Does the parametrization 'Y
(~)
(R + rcosu) cosv)
= ( (R+rcosu)sinv
6.4.11
rsinu preserve that orientation? Since +
Dn=
[r sin ucosvl + rsinusinv ,D27= rcosu
[(R +
r cos u) sin v (R+rcosu)cosv 0
we can take as our normal vector field ii the vector field +
Proposition 6.4.8: The mapping 'Y2 1 o 'Yl is the "change of parameters" mapping. Recall from Section 5.2 that the superscript "OK" (okay) denotes the domain or codomain of a change of variables mapping from which we have removed any trouble spots, assumed to be of volume 0.
+
Dn x D2'Y = r(R + rcosu)
[
cos u cos v cos~sinv
l
6.4.12
l
6.4.13
smu Does this normal vector field point in or out? At 'Y (
nH8)) ~ r(r+Rl
m
8), we have 6.4.14
which points in, so 'Y does not preserve the orientation given by the outward!:::,. pointing vector.
Proposition 6.4.8 (Orientationpreserving parametrizations). Let M C Rn be a kdimensional oriented manifold. Let U1 , U2 be subsets of Rk, and let 7 1 : U1 ~ Rn and 'Y2 : U2 ~ Rn be two parametrizations of M. Then 7 1 and 'Y2 are either both orientation preserving or both orientation reversing if and only if for all U1 E u~K and all U2 E u~K with 'Y1(u1) = 'Y2(u2) we have
det[D('Y2 1 o 'Y1)(u1)] > 0.
6.4.15
6.4 Integrating forms over oriented manifolds
593
Proof. Set x = 'Y1 (u1) = 'Y2 (u2). Then (by Proposition 3.2. 7), the columns of then x k matrix [D11 (u1)] form a basis for TxM, as do the columns of [D12(u2)]. Write [D11 (u1)]
= [D('Y2 o 'Y2 1 o 'Y1)(u1)]

= [D12('Y2 1 o 'Y1)(u1)][D(12 1 o 'Y1)(u1)]
6.4.16
1'2(u2)
6.4.1. Example 6.4.7: The torus with the u and v coordinates drawn. You should imagine the straight lines as curved. By "torus" we mean the surface of the object. The solid object is called a solid torus.
= [D12(u2)][D('Y2 1 o 'Y1)(u1)]
FIGURE
This is equation 2.6.22, as rewritten in matrix multiplication form in the margin note next to that equation: [D11(u1)] and [D12(u2)] are two bases, and [D('Y2 1 o 'Y1){u1)] is the change of basis matrix. By Definition 6.3.1, +
O(D111(u), ... , Dk'Y1(u)) = sgn(det[D('Y2 1 o 'Y1)(u1)1)
O(D~2(u), ... , D;;;2(u)).
6.4.17
It then follows from Definition 6.4.2 that if det[D(12 1 o y1)(u1)] > 0, then if 'Yl is orientation preserving, y2 will also preserve orientation, and if y1 is orientation reversing, 'Y2 will also reverse orientation. Conversely, if y1 and 'Y2 both preserve orientation, or both reverse orientation, then we must have det[D('Y2 1 o 'Y1)(u1)] > 0. 0
A nonorientable manifold We saw that the Mobius strip can't be oriented, but there is something unsatisfactory about a proof that requires the evidence of one's eyes. Mathematics should not be limited to objects (however curious) brought to class for show and tell. Now we can give an example of a manifold that we can prove is not orientable, without benefit of a drawing or scissors and glue.
Example 6.4.9 (A nonorientable manifold). Consider the subset X c Mat (2, 3) made of all 2 x 3 matrices of rank 1. This is a manifold of dimension 4 in Mat (2, 3), as Exercise 6.4.2 asks you to show; we will see that it is nonorientable. Indeed, suppose that X is orientable. Let 'Y1, 'Y2 : ( R 2  { ( Exercise 3.8 spells out some of the details used without proof in
g) }) x R 2 ~ X
6.4.18
be given by
Example 6.4.9.
( 'Yl :
~~ ) C1
f+
[ ai b
1
6.4.19
di Both maps are strict parametrizations of their images, and since the domain of each is connected, by Proposition 6.4.6 they are both either orientation preserving on the whole domain or orientation reversing on the whole domain. In particular, (u1))](D~(u1)), ... , [D72(cI>(u1))](D~(u1))))
= [D"Y2((u 1)], with no absolute value signs. Except for the absolute values, the integrands are the same. Therefore, the integral we get using 'Yl and the integral we get using 'Y2 will be the same if det[DcI>] = det[D('Y2 1 o y1)] > 0 for all u 1 E U°K, which (by Proposition 6.4.8) will be true if both parametrizations are orientation preserving. D Now we can define the integral of a form field over an oriented manifold. Definition 6.4.12 (Integral of a form field over an oriented manifold). Let MC Rn beakdimensional oriented manifold, cp a kform field on a neighborhood of M, and 'Y : U + M any orientationpreserving parametrization of M. Then
= 1 defl cp
M
b(U)]
cp =
1( cp
U
Py(u) (Dn(u),
 )Id
... , Dky(u))
k uJ.
6.4.30
Example 6.4.13 (Integrating a 2form over an oriented surface). What is the integral of the 2form w = y dy /\ dz + x dx /\ dz + z dx /\ dy
6.4 Integrating forms over oriented manifolds
597
through the piece P of the plane defined by x + y + z = 1 where x, y, z ;:::: 0, Example 6.4.13: This is an example of the first class of parametrizations listed in Section 5.2, parametrizations as graphs; see formula 5.2.6.
and oriented by ii
~[
t]
? Tills surfa 0, since dx1 "dy1 (
iM, ~)
= det [c?sO rsinO]
smO
rcosO
(Do you see why we set r
= d t[cos ()
e sin ()
= r
In equations 6.4.39, both integrals are over the same oriented point, x = +2. We use curly brackets to avoid confusion between integrating over the point +2 with negative orientation, and integrating over the point 2.
6.4.35
Then
(+
+
P·(;) Di8 ((Jr) ,D28 ( r{))
+ dx2 /\ dy2 + dx3 /\ dy3)
)
)
u
6.4.36
r sin()]+ d t[2r cos 20 2r 2 sin 20 ] + d t[ 3r2 cos 3() 3r3 sin 30] r cos {)
e 2r sin 20
2r 2 cos 20
e
3r2 sin 3()
3r 3 cos3{)
+ 4r3 + 9r5 .
So we find the following for our integral:
=r. (We know from equation 6.4.8 that dxi /\ dy1 (Dn, D2'Y) = 1, so a basis of the tangent space will be direct if dx1 /\ dy1 evaluated on the basis vectors returns a positive number.)
< 1?6 ) (
(dx1 /\ dy1
< 1.
for 0::; {) ::; 271", 0::; r
271"
fo (r + 4r 1
3
+ 9r 5 ) dr =
611".
6.4.37
!::::.
Example 6.4.15 (Integrating a 0form over an oriented point). Let x be an oriented point and f a function (i.e., a 0form field) defined in some neighborhood of x. Then
ix f =
+ f (x)
1x f =  f
and
(x).
6.4.38
Thus
f
x 2 = 4 and
1+{+2}
1
x2
= 4.
!::::.
6.4.39
{+2}
EXERCISES FOR SECTION 6.4
6.4.1
H the o=e M of equation
;, o ' (;)
~ ( ~;:)
P"""""'
orientation? 6.4.2 Confirm (Example 6.4.9) that the subset X C Mat (2, 3) made of all 2 x 3 matrices of rank 1 is a manifold of dimension 4 in Mat {2, 3). would be considerably harder to compute the integral using the parametrization of Example 6.4.4. 6 Recall that r = lzl, which was stipulated in Example 6.4.4 to be less than 1. 5 1t
6.5
Exercises 6.4.4, 6.4.5, 6.4.6: Recall (discussion after Definition 6.3.3) that a kdimensional manifold M can oriented by choosing a kform w and defining
Forms in the language of vector calculus
599
6.4.3 In Example 6.4.5 we saw that the map 'Y (equation 6.4.9) parametrizing the unit sphere oriented by the outward normal was neither orientation preserving nor orientation reversing. Can you make that mapping either orientation preserving for the entire sphere, or orientation reversing for the entire sphere, by choosing a different range of values of 8 and cp? 6.4.4
What is the integral
ls
X3 dx1 /\ dx2 /\ dx4, where S is the part of the
threedimensional manifold of equation
n = sgnw, so long as w(Px(v1, ... , vk) =f. 0 for all x E M and for all bases v1, ... , vk of TxM.
X4
= X1X2X3
where 0:::; X1, X2, X3 :::; 1,
oriented by n = sgn dx1 /\ dx2 /\ dx3? Hint: This surface is a graph, so it is easy to parametrize.
6.4.5 Let Z1 = X1 + iy1' Z2 = X2 + iy2 be coordinates in IC 2 • Compute the integral of dx1 /\ dy1 + dy1 /\ dx2 over the part of the locus of equation z 2 = z~ where lz1I < 1, oriented by n = sgndx1 A dy1.
Part b of Exercise 6.4.7: Use the same method as in Example 6.4.9; this time you can find an orientation of Mi (3, 3) such that parametrizations cp1, cp2, and cp3 of parts of Mi (3, 3) are all orientation preserving.
6.4.6 Let z1 = X1 + iy1' z2 = X2 + iy2 be coordinates in IC 2 = IR 4 . Integrate the 2form dx1 /\ dy1 + dx2 /\ dy2 over the part of the surface X of equation z2 = ez 1 where IRe Z1 I :=:; a, I Im Z1 I :=:; b, oriented by n = sgn dx1 /\ dy1. Hint : Use Euler's formula (equation 1.5.55). 6.4. 7
a. Show that Mi (2, 2) is orientable. Hint: Mi (2, 2) is three dimensional in IR 4 , so it can be oriented by choosing a normal vector field. *b. Show that Mi (3, 3) is orientable.
6.4.8
6.5
FIGURE
6.5.l.
One of t