4.9 Newton's Method

The iteration derived from the tangent line, its rapid convergence with correct digits roughly doubling each step, the ways it fails through a vanishing derivative, a cycle or divergence, the importance of the initial guess, and the comparison with bisection.

Subject: Calculus I · 66 slides · symbolic lesson

Open the interactive version of this deck

What this lesson covers

The lesson, slide by slide

1. Newton's Method

Title

Calculus I · Section 4.9

Tangent lines that home in on a root

2. What this lesson gives you

Objectives

Most equations cannot be solved by algebra. This lesson turns the tangent line from Section 4.2 into a machine that produces better and better approximations to a root, and teaches you when to trust it.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 409 — learning objectives 4.9.1 to 4.9.4

Up to now, solving an equation has meant doing algebra until x stands alone. That works for linear and quadratic equations and for a handful of special forms. For almost everything else it fails, and yet calculators and computers give you roots of complicated equations instantly. This lesson shows you how they do it.

The idea is one you already own: the tangent line from Section 4.2. A line is easy to solve, so solve the tangent line instead of the curve, and use the answer as a better guess. Repeat, and the guesses close in on the root astonishingly fast.

Half of the lesson is about using the method and half is about distrusting it. You will see it fail in four different ways, learn to recognise each, and meet the slower but safer alternative, bisection. The last part widens the view to any repeated rule and the numbers such rules settle on.

3. Before anything new: where does a tangent line cross the axis?

Warm-up

Discussion prompt

From Section 4.2: write the tangent line to f(x) = x³ at x = 1, then find where that line crosses the x-axis.

Write your answer before revealing it. You need the value of the function at 1, which is 1, and the slope there, which is 3. The tangent line is the point-slope line through that point with that slope.

Setting the line's height to zero and solving takes one short line of algebra and gives two thirds. Notice that two thirds is not a root of x cubed; the only root of x cubed is zero. But it is closer to zero than 1 was.

Keep that in mind. The tangent's root is not the curve's root, but it is usually a better guess than the one you started with. Newton's method is nothing more than doing this again from the new point, and again, until the guesses stop changing.

4. Why we need an approximation method

Section

Part 1

5. Most equations have no formula for their roots

Concept

A linear equation has its root by formula, and so does a quadratic. Beyond that the formulas run out.

\[ ax + b = 0 \;\Longrightarrow\; x = -\frac{b}{a}, \qquad ax^2 + bx + c = 0 \;\Longrightarrow\; x = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a} \]

\[ x^5 + 8x^4 + 4x^3 - 2x - 7 = 0 \qquad \tan x - x = 0 \]

Cubics and quartics have formulas too complicated to use; for degree five and higher it is proved that no formula exists; and an equation mixing tangent and x has no algebraic solution at all.

Figure (svg): The curves y equals tan x and y equals x for x from 0 to 5.5. They meet at the origin and again near x equals 4.49, where the right-hand branch of tan x climbs through the line.

The second crossing is a perfectly good number, about 4.4934, but no algebra will produce it. Newton's method will, to as many digits as you like.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 409 — introduction

You know formulas that solve linear and quadratic equations exactly. Formulas for cubics and quartics exist too, but they are so long that nobody uses them by hand. For polynomials of degree five and higher, like the quintic on the slide, it has been proved that no formula built from arithmetic and roots can exist. That is a theorem, not a gap in our knowledge.

Equations that mix different kinds of functions are worse still. Look at the figure: the line y equals x meets the curve y equals tan x at zero and again at about 4.4934. That crossing is a perfectly definite number, and this kind of equation turns up in physics problems about vibrating strings and heat flow, but no amount of algebra will produce it.

So in practice, most roots are approximated, not solved for. The question is how to approximate them efficiently and reliably, and that is the whole business of this section.

6. Swap the curve for its tangent line

Intuition

A curve's root is hard to find. A line's root is easy. Near a point, the tangent line is an excellent stand-in for the curve, so its root should be close to the curve's root.

Take the tangent's root as a new guess. It is typically closer than the old one, and from there you can draw a new tangent, whose root is closer still.

That repetition is the whole method: a sequence of guesses, each built from the last by the same one-line rule.

Here is the key idea in words, before any formula. Near a point on a smooth curve, the tangent line and the curve are almost indistinguishable. You found that out in Section 4.2 when you used linear approximation to estimate values.

Now ask a different question of the tangent line: not what its height is, but where it hits zero. Because the line hugs the curve, its zero should be near the curve's zero, as long as you started reasonably close.

The best part is that you can repeat it. The new guess gives a new point on the curve, which gives a new tangent line, which gives a newer guess. Each round costs one evaluation of the function and one of its derivative. When the method works, a handful of rounds gives more accuracy than you will ever need.

7. Follow the tangent down to the axis

Concept

Figure (svg): The curve y equals x cubed minus 3x plus 1 between x equals 1.3 and 2.15. From the point above x0 equals 2, the tangent line runs down to the x-axis at x1 equals 1.667; from the point above x1, a second tangent runs down to x2 equals 1.549, right beside the root at 1.532.

Each tangent line is followed down to where it meets the axis, and that landing point becomes the next guess. Two steps from 2 already land within 0.02 of the root.

Start at a guess, go up to the curve, slide down the tangent line to the axis. Where you land is the next guess.

\[ x_0 = 2 \;\longrightarrow\; x_1 \approx 1.667 \;\longrightarrow\; x_2 \approx 1.549 \;\longrightarrow\; \cdots \to x^* \approx 1.532 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, pp. 409-410 — Figure 4.77

Look at the picture from the top right. The first guess is 2, and above it the curve is at height 3. The orange tangent line at that point slides down to the axis at about 1.667, the second guess. From there, go straight up to the curve and follow the purple tangent down again, landing at about 1.549.

The green circle is the actual root, about 1.532. After just two tangents the guess is within two hundredths of it. Notice how the second tangent is much shorter than the first: as the guesses approach the root, the curve looks more and more like a straight line, so the tangent becomes a better and better stand-in.

The chain of arrows under the figure is the notation you will use all lesson: x zero is the start, x one the first improvement, and so on, with the sequence heading for the root, called x-star.

8. Where does the first tangent land?

Prediction

\[ f(x) = x^2 - 2, \quad x_0 = 2 \]

Predict first

The tangent to this parabola at x = 2 meets the x-axis at which point?

  • x = 1
  • x = 1.5
  • x = 1.414
  • x = 1.75

Correct: x = 1.5

Why: At 2 the height is 2 and the slope is 4, so the tangent falls 2 units over a run of one half: it lands at 1.5. It does not land on the root itself, about 1.414, because the tangent is only an approximation to the curve, but it is already much closer than 2.

Commit to an option before revealing. You can do this one in your head from the picture: at 2 the parabola is at height 2 and climbing with slope 4.

A slope of 4 means that going back half a unit drops the tangent by 2, exactly its height, so the tangent reaches the axis at 1.5. If you chose 1.414, you picked the root itself; the tangent is not the curve, so it cannot land exactly on the curve's root, but it gets surprisingly close.

If you chose something else, try drawing the triangle: the height is the rise, the slope is rise over run, so the run is the height divided by the slope. That one relationship is the formula the next part of the lesson derives.

9. Deriving the iteration

Section

Part 2

10. Setting the tangent line equal to zero

Concept

Write the tangent line at the current guess, set its height to zero, and solve for x. Call the solution the next guess.

\[ y = f(x_0) + f'(x_0)(x - x_0) \]

\[ 0 = f(x_0) + f'(x_0)(x_1 - x_0) \]

\[ x_1 - x_0 = -\frac{f(x_0)}{f'(x_0)} \qquad (f'(x_0) \ne 0) \]

\[ x_1 = x_0 - \frac{f(x_0)}{f'(x_0)} \]

The same algebra at the next guess gives the next one, and so on forever.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 410 — derivation of Equation 4.8

Read the lines one at a time. The first is the point-slope equation of the tangent line at x zero. The second says that the new guess x one is where that line has height zero. The third subtracts and divides by the slope, and the fourth adds x zero to both sides.

Notice the condition in brackets: you can only divide by the derivative if it is not zero. That small condition is where one of the method's failures will come from later. Geometrically, a zero derivative means a horizontal tangent, and a horizontal line other than the axis itself never meets the axis.

Nothing about this derivation used the particular guess x zero. Put x one in its place and you get x two, and so on. That is why the result is written once, for a general step, as Equation 4.8 on the next slides.

11. Rise over run in one triangle

Picture it

Figure (svg): The parabola y equals x squared minus 2 with its tangent line at x0 equals 2. The tangent, the vertical segment of height f(x0) equals 2 above x0, and the horizontal segment from x1 equals 1.5 to x0 form a right triangle whose slope is f prime of x0 equals 4.

The tangent's slope is rise over run: the rise is f(x₀) and the run is how far you step back, x₀ − x₁. Solving slope = rise/run for the run gives the whole method.

The tangent, the vertical drop from the curve and the stretch of axis form a right triangle. Slope is rise over run, so the run is the rise divided by the slope: that run is exactly the correction.

\[ f'(x_0) = \frac{f(x_0)}{x_0 - x_1} \;\Longrightarrow\; x_0 - x_1 = \frac{f(x_0)}{f'(x_0)} \]

This is the same derivation as a picture, and it is worth remembering in this form because it explains every behaviour of the method. The shaded right triangle has the tangent line as its slanted side. Its vertical side is the height of the curve at x zero, which is 2 here. Its horizontal side runs from x one to x zero.

The slope of the tangent is rise over run, so the run, the distance you step back, equals the rise divided by the slope. That is exactly the correction in Newton's formula: the value of f divided by the value of f prime.

Now you can predict the method's personality. A steep tangent turns even a large height into a short, careful step. A shallow tangent turns a small height into a long leap. Keep this triangle in mind when you reach the failures: most of them are the triangle becoming very flat.

12. Reading Equation 4.8 piece by piece

Notation

Annotate

On: \( x_n = x_{n-1} - \frac{f(x_{n-1})}{f'(x_{n-1})} \)

  • The new guess, the nth approximation. The subscript counts steps, not powers.
  • The previous guess: everything on the right is evaluated there, never at the new point.
  • How far the curve is from zero at the old guess. When this is tiny, the correction is tiny.
  • The tangent's slope. It is in the denominator, so it must not be 0, and when it is nearly 0 the step is enormous.
  • Subtract the correction. If f is positive and rising you step left; if f is positive and falling you step right.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 410 — Equation 4.8

Step through the annotations. The single most common slip is evaluating something at the wrong point: every piece on the right-hand side is evaluated at the old guess, and the result on the left is the new guess.

The fraction is the correction, and each part of it tells you something. A small function value means you are already near a root, so the correction is small and the guesses barely move. A derivative in the denominator means a flat curve produces a large step, and a perfectly flat one produces no step at all.

The minus sign handles direction automatically. You never have to decide whether to move left or right: if the function is positive and increasing, the fraction is positive and you move left; if it is positive and decreasing, the fraction is negative and you move right. In both cases you move downhill toward the axis along the tangent.

13. Newton's method is an iterative process

Concept

Every step uses the same rule, so package the rule as one function F and apply it again and again.

\[ F(x) = x - \frac{f(x)}{f'(x)} \]

\[ x_n = F(x_{n-1}): \quad x_1 = F(x_0), \; x_2 = F(x_1), \; x_3 = F(x_2), \; \ldots \]

iterative process — A list of numbers in which each is produced from the one before by repeating the same function.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 412 — iterative process

It helps to see Newton's method as one fixed machine, the function F, that you feed its own output. F takes a guess and returns the tangent line's root. Nothing inside F changes from step to step, which is what makes the method so easy to program.

The general name for this is an iterative process: a list of numbers where each one comes from the previous one by the same rule. You will meet other examples in Part 6, where F is not built from a tangent line at all.

Thinking in terms of F also sets up the key question of Part 6. If the list settles down to some number, what must be true of that number? Hold that question; the answer turns out to explain why Newton's method lands on roots.

14. Exercise 408: Newton's function F for sin x

Fill the middle

Section 4.9's exercises ask for Newton's formula written as the function F. Fill in the simplified form for the equation sin x = 0.

Fill in the blanks

F(x) = x - \frac\tan x___ = x - ___

Why: The derivative of sine is cosine, and sine over cosine is tangent, so F(x) is x minus tan x. Starting at 3, one step gives about 3.1425, already close to the root pi: tangent is tiny near pi, so the correction is tiny.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 417 — Exercise 408

The section's exercises begin by asking you to write Newton's formula as a single function F. For sine, the derivative is cosine, so the correction is sine over cosine, which simplifies to tangent.

Type your answer in the blank. Writing tan x, tan(x), or tanx will all be accepted.

It is worth trying a step to see the method in action. Start at 3, near pi. The tangent of 3 is about minus 0.1425, so F of 3 is about 3.1425, and pi is about 3.1416: one step has taken you from one correct digit to three. You have just built a way of computing pi from nothing but the sine and cosine keys.

15. Trap: adding the correction instead of subtracting

Trap

The trap

One step from 2 on the function below:

\[ f(x) = x^2 - 2 \]

\[ x_1 = 2 + \frac{f(2)}{f'(2)} \]

\[ x_1 = 2 + \frac24 = 2.5 \]

Wrong. The guess moved away from the root.

The fix

The tangent runs down to the axis, so from a point where f is positive and rising you must step left. The formula subtracts.

\[ x_1 = 2 - \frac24 = 1.5 \]

Quick test: the new f should be closer to zero. Here f(1.5) = 0.25 but f(2.5) = 4.25.

This slip is easy to make when you write the formula from memory, and it is easy to catch if you look at what happened. The guess moved from 2 to 2.5, away from the root near 1.414, and the function value went up from 2 to 4.25 instead of down toward zero.

The picture of the triangle tells you the sign. At 2 the parabola is above the axis and climbing. Its tangent line reaches the axis to the left, so the new guess must be smaller than the old one. The formula subtracts, and subtracting a positive correction moves left.

Make the check a habit for the first step of any Newton computation: the new function value should be much closer to zero than the old one. If it is not, look for a sign error or a derivative evaluated at the wrong point.

16. Order one Newton step

Ranking

Put in order

Put the moves of a single Newton step in order.

  1. Evaluate f at the current guess
  2. Evaluate f′ at the current guess
  3. Check that f′ is not zero (or nearly zero)
  4. Subtract f/f′ from the current guess
  5. Compare the new guess with the old one to decide whether to stop

Why: Both evaluations happen at the old guess, the derivative must be safe to divide by before you divide, and only after computing the new guess can you compare it with the old one to decide whether to stop.

Drag the moves into order before checking. The ordering is not arbitrary; each move depends on the one before.

Both evaluations, the function and its derivative, happen at the current guess, because the tangent line is drawn there. Then you look at the derivative before you divide by it: a zero or very small derivative is the early warning of a wild step. Only then do you form the new guess.

The last move is the one people skip. After each step, compare the new guess with the old one. If they agree to the accuracy you want, stop. If they are wildly different, or if the values seem to be repeating, something has gone wrong and it is time to think rather than keep pressing the button.

17. Newton's method at work

Section

Part 3

18. Sketch first: which root, and where to start

Concept

Figure (svg): The cubic y equals x cubed minus 3x plus 1 from x equals minus 2.5 to 2.5. It crosses the x-axis three times, near minus 1.879, 0.347 and 1.532; dashed lines mark the interval from 1 to 2, which contains exactly one of them.

Sketch first. The graph shows one crossing between 1 and 2, so x₀ = 2 is a sensible start for that root. The other two roots need other starts.

Newton's method needs a starting value, and the cheapest way to get one is a sketch. This cubic has three real roots; the interval from 1 to 2 contains exactly one of them.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 411 — Figure 4.78

Before you run Newton's method, you need a first guess, and the book's advice is simple: sketch the function. This cubic crosses the axis three times, near minus 1.879, 0.347 and 1.532. Newton's method finds one root at a time, and which one it finds depends on where you start.

The dashed lines mark the interval from 1 to 2 that Example 4.46 asks about. You can confirm there is a root inside without a picture: the cubic is minus 1 at x equal to 1 and plus 3 at x equal to 2, and a continuous function that changes sign must cross zero in between. That is the Intermediate Value Theorem from Section 2.5.

Starting at 2, the right end of the interval, is a reasonable choice because the curve is steep and smooth there. Starting at 1 would be a disaster, as you will see in Part 5: the curve is flat at 1.

19. Example 4.46: a root of x³ − 3x + 1 in [1, 2]

Worked example

Use Newton's method with x zero equal to 2 to approximate the root in the interval from 1 to 2.

\[ f(x) = x^3 - 3x + 1, \quad x_0 = 2 \]

Differentiate

Why: The iteration needs f prime.

\[ f'(x) = 3x^2 - 3 \]

Evaluate at x zero

Why: At 2 the cubic is 8 minus 6 plus 1.

\[ f(2) = 3, \quad f'(2) = 9 \]

First step

Why: Apply Equation 4.8 with n equal to 1.

\[ x_1 = 2 - \frac39 = \frac53 \approx 1.666666667 \]

Evaluate at x one

Why: Use exact fractions: five thirds cubed is 125 over 27.

\[ f\left(\tfrac53\right) = \frac{125}{27} - 5 + 1 = \frac{17}{27}, \quad f'\left(\tfrac53\right) = \frac{25}{3} - 3 = \frac{16}{3} \]

Second step

Why: Divide the two fractions and subtract.

\[ x_2 = \frac53 - \frac{17/27}{16/3} = \frac53 - \frac{17}{144} = \frac{223}{144} \approx 1.548611111 \]

Continue with a calculator

Why: Store each value and feed it back in.

\[ x_3 \approx 1.532390162, \quad x_4 \approx 1.532088989, \quad x_5 \approx 1.532088886 \]

Stop when two iterates agree

Why: The sixth iterate repeats the fifth to all ten digits shown.

\[ x_6 \approx 1.532088886 = x_5 \]

Check by substituting back

Why: The cubic at the answer is essentially zero. This root even has an exact form, twice the cosine of 40 degrees, which matches to all digits.

\[ f(1.532088886) \approx -1\times 10^{-9}, \qquad 2\cos 40^\circ = 1.532088886\ldots \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, pp. 410-411 — Example 4.46

The first two steps are worth doing with exact fractions once, so that you see exactly what the formula does. At 2, the value is 3 and the slope is 9, so the correction is one third and the new guess is five thirds. At five thirds, the value is seventeen over twenty-seven and the slope is sixteen thirds. Dividing gives seventeen over one hundred forty-four, and subtracting leaves two hundred twenty-three over one hundred forty-four.

After that, fractions become unwieldy and you should let a calculator do the work, storing every value rather than retyping a rounded one. The book's table shows the digits settling from the left: first 1.5, then 1.53, then 1.5320, until two successive values agree in all ten digits displayed.

The check has two parts. Substituting back gives a value of about one billionth, which is zero to the accuracy shown. And for this special cubic there is an exact answer to compare with, twice the cosine of 40 degrees, which matches every digit. That exact form comes from a trigonometric identity and is rare; usually the substitution is all the check you get.

20. Checkpoint 4.45: the root in [0, 1]

Worked example

Same cubic, now starting at x zero equal to 0 to find the root between 0 and 1.

\[ f(x) = x^3 - 3x + 1, \quad f'(x) = 3x^2 - 3, \quad x_0 = 0 \]

Evaluate at 0

Why: Only the constant term survives.

\[ f(0) = 1, \quad f'(0) = -3 \]

First step

Why: Subtracting a negative correction moves right.

\[ x_1 = 0 - \frac{1}{-3} = \frac13 \approx 0.333333 \]

Evaluate at one third

Why: One third cubed is one twenty-seventh.

\[ f\left(\tfrac13\right) = \frac1{27} - 1 + 1 = \frac1{27}, \quad f'\left(\tfrac13\right) = \frac13 - 3 = -\frac83 \]

Second step

Why: Again a negative derivative, so the step is to the right.

\[ x_2 = \frac13 - \frac{1/27}{-8/3} = \frac13 + \frac1{72} = \frac{25}{72} \approx 0.347222 \]

Check against more iterations

Why: Two more steps settle at 0.347296355, so x two is already right to three decimal places; and the cubic there is small.

\[ x_4 \approx 0.347296355, \qquad f(0.347222) \approx 0.00019 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 411 — Checkpoint 4.45

Same function, different starting point, different root. At zero, the cubic is 1 and its slope is minus 3. A negative slope with a positive value means the tangent runs down to the right, and the formula agrees: subtracting a negative correction moves you right, to one third.

At one third, the value is small, one twenty-seventh, so you are already close. The slope is still negative, so the correction again moves you right, by one seventy-second. Adding thirds and seventy-seconds needs a common denominator of seventy-two, giving twenty-five seventy-seconds.

The check shows how good two steps can be. The fully converged root is 0.347296, and twenty-five seventy-seconds is 0.347222, already correct to three decimal places. The function there is about two ten-thousandths, small but not yet zero, which tells you one or two more steps would be worth taking if you needed more digits.

21. Example 4.47: the square root of 2

Worked example

Approximate the square root of 2 as the positive root of a quadratic, starting from x zero equal to 2.

\[ f(x) = x^2 - 2, \quad f'(x) = 2x, \quad x_0 = 2 \]

Substitute into Equation 4.8

Why: Replace f and f prime.

\[ x_n = x_{n-1} - \frac{x_{n-1}^2 - 2}{2x_{n-1}} \]

Split the fraction

Why: Divide each term of the numerator by the denominator.

\[ x_n = x_{n-1} - \frac12 x_{n-1} + \frac{1}{x_{n-1}} \]

Collect

Why: Half of x plus one over x, written as an average.

\[ x_n = \frac12\left(x_{n-1} + \frac{2}{x_{n-1}}\right) \]

First step

Why: Average 2 with 2 over 2.

\[ x_1 = \frac12\left(2 + 1\right) = 1.5 \]

Second step

Why: Average 1.5 with 2 over 1.5.

\[ x_2 = \frac12\left(1.5 + \frac{2}{1.5}\right) = \frac{17}{12} \approx 1.416666667 \]

Continue

Why: The third iterate is 577 over 408.

\[ x_3 \approx 1.414215686, \quad x_4 \approx 1.414213562, \quad x_5 \approx 1.414213562 \]

Check by squaring

Why: Square the answer and compare with 2.

\[ (1.414213562)^2 = 1.9999999989\ldots \approx 2 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, pp. 411-412 — Example 4.47 and Figure 4.79

To compute a square root with Newton's method, find a function whose root is that square root. The square root of 2 is a root of x squared minus 2, so that is the function to use.

The simplification is worth doing carefully, because the result is famous. Splitting the fraction and collecting terms turns Newton's formula into an average: the new guess is the average of the old guess and 2 divided by the old guess. If your guess is too big, 2 divided by it is too small, and averaging the two lands in between. This rule is often called the Babylonian method, and Heron of Alexandria described it nearly two thousand years ago, long before calculus; Newton's method rediscovers it.

The iterates 1.5, then seventeen twelfths, then five hundred seventy-seven over four hundred eight, agree with the true value to more digits each time. Squaring the final answer gives 2 to within about one billionth, which is the limit of the ten digits shown.

22. Checkpoint 4.46: the square root of 3

Worked example

\[ f(x) = x^2 - 3, \quad x_0 = 3 \]

Build the iteration

Why: The same simplification as for 2, with 3 in its place.

\[ x_n = x_{n-1} - \frac{x_{n-1}^2 - 3}{2x_{n-1}} = \frac12\left(x_{n-1} + \frac{3}{x_{n-1}}\right) \]

First step

Why: Average 3 with 3 over 3.

\[ x_1 = \frac12(3 + 1) = 2 \]

Second step

Why: Average 2 with 3 over 2.

\[ x_2 = \frac12\left(2 + 1.5\right) = 1.75 \]

Check by squaring

Why: 1.75 squared is 3.0625, just above 3; one more step gives 1.732143, against the true 1.7320508.

\[ 1.75^2 = 3.0625, \qquad x_3 \approx 1.732143 \approx \sqrt3 = 1.7320508\ldots \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 412 — Checkpoint 4.46

This is the same computation with 3 in place of 2, so the averaging rule becomes: average the guess with 3 divided by the guess. The starting value 3 is a deliberately poor guess, since the root is about 1.73.

The first step averages 3 with 1 to give 2. The second averages 2 with one and a half to give 1.75. Notice that every guess after the first is too big. That is always true for this rule, because the average of two different positive numbers is at least the square root of their product, and the product here is always 3.

The check squares 1.75 to get 3.0625, just a little above 3, and one more step gives 1.732143 against the true 1.7320508. That is four correct digits after three steps from a bad start.

23. Division without dividing

Real world

Early computers and some chips today compute a reciprocal using only multiplication and subtraction. Newton's method applied to one over x minus a gives the rule.

\[ f(x) = \frac1x - a, \quad f'(x) = -\frac{1}{x^2} \;\Longrightarrow\; x_n = x_{n-1}\left(2 - a\,x_{n-1}\right) \]

Discussion prompt

Use the rule to approximate one seventh, starting from 0.1. How many steps until you have eight correct decimal places?

This is how some computer chips have actually computed reciprocals. Addition and multiplication are fast in hardware; division is slow. So choose a function whose root is one over a, and whose Newton formula needs no division. The function one over x minus a works, and after simplifying, the division in Newton's formula cancels completely.

Try the computation yourself before revealing. Starting at one tenth for one seventh, each step multiplies the guess by 2 minus 7 times the guess. The values 0.13, then 0.1417, then 0.14284777 close in on 0.142857.

Count the correct digits: one, then two, then four, then eight. That doubling pattern is not a coincidence; it is the signature of Newton's method whenever it works, and the next part of the lesson explains it.

24. How fast it converges

Section

Part 4

25. The correct digits roughly double

Concept

Line the square-root iterates up against the true value and count the digits that match.

nxₙerror xₙ − √2correct digits
020.5860
11.50.08581
21.4166666670.002453
31.4142156860.000002126
41.414213562…0.000000000001612

Each error is roughly a constant times the square of the one before, which is why the number of correct digits roughly doubles each step.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 414 — when it works, the sequence approaches the root very quickly

The table lines up the square root of 2 computation against the true value. Read the last column: 0, 1, 3, 6, 12 correct digits. Each step roughly doubles what you had.

The error column shows why. Going from 0.0858 to 0.00245 to 0.00000212, each error is about one third of the square of the previous one. Squaring a number less than one makes it much smaller, and the smaller it already is, the more dramatic the effect. Squaring one thousandth gives one millionth.

The book says only that when the method works it approaches the root very quickly, and leaves the exact rate to numerical analysis courses. You can see the rate for yourself in the table, and the next few slides put a number on it. The practical upshot: once Newton's method has two or three correct digits, one or two more steps usually finish the job.

26. The error falls off a cliff

Concept

Figure (svg): Errors plotted on a logarithmic vertical axis against the step number. Newton's iterates for the square root of 2 plunge from about 0.6 to about 10 to the minus 12 in four steps; the bisection midpoints drift down slowly and irregularly, reaching only about 10 to the minus 6 after 20 steps.

On a log scale, bisection's errors follow a straight line (halving every step), while Newton's fall off a cliff: each error is roughly the square of the one before.

On a logarithmic scale, halving the error each step is a straight line. Newton's errors bend sharply downward, because squaring a small number makes it very much smaller.

The vertical axis here is logarithmic: each gridline down is a thousand times smaller than the one above. On such an axis, an error that halves every step drops by the same amount each step, so it traces a straight line. That is the orange dashed line for bisection, with its dots scattered around it.

Newton's errors, the blue dots, do not follow a straight line. They start slowly, then fall faster and faster: the drop from step 3 to step 4 is twice as big as the drop from step 2 to step 3. On a log scale, squaring the error doubles its distance below 1.

By step 4 Newton is at about ten to the minus twelve. Bisection, after twenty steps, has not reached ten to the minus six. That is why nearly every root-finding routine in real software uses Newton's method, or something like it, once it gets close.

27. Watching the error square

Worked example

Test the claim that each error is about a constant times the square of the last, for the square root of 2 from x zero equal to 2.

List the errors

Why: Subtract the true value from each iterate.

\[ e_0 = 0.585786, \; e_1 = 0.085786, \; e_2 = 0.002453, \; e_3 = 2.1239\times 10^{-6} \]

Divide the first error by the square of the one before

Why: If the claim holds, these ratios settle down.

\[ \frac{e_1}{e_0^2} = \frac{0.085786}{0.343145} = 0.2500 \]

The next ratio

Why: Same computation, one step later.

\[ \frac{e_2}{e_1^2} = \frac{0.002453}{0.0073593} = 0.3333 \]

And the next

Why: The ratio is settling.

\[ \frac{e_3}{e_2^2} = \frac{2.1239\times 10^{-6}}{6.0177\times 10^{-6}} = 0.3529 \]

Compare with the theoretical constant

Why: Numerical analysis texts show the ratio tends to f double prime over twice f prime, at the root.

\[ \frac{f''(\sqrt2)}{2f'(\sqrt2)} = \frac{2}{4\sqrt2} = \frac{1}{2\sqrt2} \approx 0.3536 \]

Check by predicting the next error

Why: Use the constant to predict the fourth error, then compare with the real one.

\[ 0.3536\,(2.1239\times 10^{-6})^2 \approx 1.595\times 10^{-12} \quad\text{vs actual}\quad e_4 \approx 1.595\times 10^{-12} \]

This slide tests the claim of quadratic convergence with actual numbers. If each error is roughly a constant times the square of the last, then dividing each error by the square of the previous one should give nearly the same number every time.

The ratios come out 0.25, then 0.333, then 0.353. They are settling, and the value they settle on is predicted by a formula from numerical analysis: the second derivative over twice the first derivative, both at the root. For this function that is one over twice root 2, about 0.3536.

The check is a genuine prediction. Using 0.3536 and the third error, the formula predicts a fourth error of about 1.595 times ten to the minus twelve, and the actual error is the same to four digits. When you can predict an error in the twelfth decimal place, you understand a method.

28. Reading the error relation

Notation

Annotate

On: \( e_{n+1} \approx C\,e_n^{2}, \qquad C = \frac{f''(x^*)}{2f'(x^*)} \)

  • The error at step n: how far xₙ is from the true root x*.
  • Quadratic convergence. An error of 0.001 becomes roughly 0.000001: three correct digits become six.
  • A constant fixed by the function near the root. It matters little when the error is small, because the square wins.
  • In the denominator again. If the curve is flat at the root (a repeated root), this relation breaks and convergence slows to a crawl.

Step through the annotations. The relation says the new error is roughly a constant times the old error squared. It holds once you are close to the root, not necessarily from a poor start.

The constant matters less than you might think. If the constant were 10 instead of about one third, an error of one millionth would become ten to the minus eleven instead of about three times ten to the minus thirteen. Either way the digits roughly double. The square does the work.

The last annotation is the important caveat. The constant has the derivative at the root in its denominator. When the curve is flat at the root, as with a double root like x squared at zero, the relation breaks down and Newton's method converges much more slowly, gaining only about one binary digit per step, no better than bisection.

29. Predict the next error

Estimation

\[ e_n = 0.001, \quad C \approx 0.35 \]

Predict first

Newton's method on the square root of 2 has an error of 0.001 at some step. Roughly what will the error be one step later?

  • about 0.0005
  • about 0.0001
  • about 0.0000004
  • exactly 0

Correct: about 0.0000004

Why: Square the error to get one millionth, then multiply by about 0.35: roughly four ten-millionths. Halving, the first option, is what bisection would give. The error never becomes exactly zero in exact arithmetic, because the square root of 2 is irrational.

\[ e_{n+1} \approx 0.35\,(0.001)^2 = 3.5\times 10^{-7} \]

Commit to an estimate before revealing. The question gives you everything you need: square the error and multiply by about 0.35.

One thousandth squared is one millionth, and a third of that is a few ten-millionths. So an error of one thousandth becomes an error of roughly four ten-millionths: three correct digits become six. If you picked about five ten-thousandths, you applied bisection's rule, which only halves the error.

Nobody expects exactly zero, and it is worth seeing why. The square root of 2 is irrational, and every Newton iterate from a rational start is a fraction, so no iterate can ever equal it exactly. The error just becomes too small for any calculator to show.

30. Bisection: slow but certain

Concept

The Intermediate Value Theorem gives a completely different method. If f changes sign on an interval, a root lies inside; test the midpoint and keep the half where the sign still changes.

Figure (svg): Six horizontal bars stacked downward, each the bisection interval for the square root of 2: from 1 to 2, then 1 to 1.5, 1.25 to 1.5, 1.375 to 1.5, 1.375 to 1.4375, and 1.40625 to 1.4375. A dashed vertical line at 1.41421 stays inside every bar.

Each step keeps the half where the function changes sign, so the root can never escape. The price is speed: every step only halves the bracket.

\[ \text{after } k \text{ steps: } \quad |\text{error}| \le \frac{b - a}{2^{k}} \]

Bisection never fails once it has a sign change, but it gains only one binary digit per step, about one decimal digit every three or four steps.

Bisection is the method the Intermediate Value Theorem suggests. If the function is negative at one end of an interval and positive at the other, a root lies between. Evaluate at the midpoint, keep whichever half still has the sign change, and repeat.

The figure shows six rounds for the square root of 2, starting from the interval from 1 to 2. The dashed green line, the true root, stays inside every bar, and it always will. That is bisection's great strength: once you have a sign change it cannot fail, cannot jump to another root, and cannot cycle.

Its weakness is in the formula: after k steps the error is at most the starting width divided by 2 to the power k. Each step gains exactly one binary digit, and it takes about three and a third binary digits to make one decimal digit. That is steady, but slow.

31. Newton against bisection on the square root of 2

Worked example

How many steps does each method need to pin down the square root of 2, starting from the interval from 1 to 2, or from x zero equal to 2?

Bisection's error bound

Why: The bracket starts with width 1 and halves each step.

\[ |\text{error}| \le 2^{-k} \]

Ask for six decimal places

Why: Solve for k.

\[ 2^{-k} \le 10^{-6} \iff k \ge \log_2 10^6 \approx 19.93 \;\Longrightarrow\; k = 20 \]

Ask for twelve decimal places

Why: Twice the digits, twice the steps.

\[ 2^{-k} \le 10^{-12} \iff k \ge 39.86 \;\Longrightarrow\; k = 40 \]

Newton's steps for the same accuracy

Why: Read off the error table.

\[ e_3 \approx 2.1\times 10^{-6}, \quad e_4 \approx 1.6\times 10^{-12} \;\Longrightarrow\; 4 \text{ steps} \]

Check the bisection count directly

Why: Twenty halvings of 1 leave about one millionth; forty leave about one trillionth.

\[ 2^{-20} \approx 9.54\times 10^{-7}, \qquad 2^{-40} \approx 9.09\times 10^{-13} \]

This puts numbers on the comparison. For bisection the error bound is one over 2 to the k, so six decimal places need 2 to the k to exceed a million, which first happens at k equal to 20. Twelve decimal places need twice as many steps, 40.

Newton's method needs 3 steps for about six digits and 4 steps for about twelve, from the start at 2. Doubling the digits costs Newton one extra step and costs bisection twenty.

The check computes the bisection bounds directly: twenty halvings of 1 leave about one millionth, forty leave about one trillionth. In practice, good software combines the two: it uses bisection to trap the root safely in a small interval, then switches to Newton's method to finish quickly, falling back to bisection whenever a Newton step would leave the interval.

32. Newton or bisection?

Comparison

Comparison matrix

Newton's methodbisection
needsf′ and one starting valuean interval where f changes sign
error per steproughly squareshalves
can it fail?yesno, once bracketed
steps for 10⁻¹² on √2440

Fill in each blank before checking. The table summarises the trade-off between the two methods, and it is the comparison an exam question on this section will most likely ask about.

Bisection needs a sign change, meaning an interval where the function is negative at one end and positive at the other. Newton's method needs a derivative and a single starting guess. Newton's error roughly squares each step; bisection's halves. Newton can fail, in all the ways Part 5 shows; bisection cannot, once the root is bracketed.

The last row is the price of safety: 40 bisection steps against 4 Newton steps for twelve correct digits. Neither method is simply better. You choose based on whether you care more about speed or about guarantees, and often you use both.

33. When to stop iterating

Concept

The book's rule is to stop when two successive iterates agree to the digits you need. That works because, near the root, the step size is a very good estimate of the error.

\[ |x_n - x_{n-1}| < \text{tolerance} \]

Then confirm by substituting back: the function's value at your answer should be close to zero. Agreement without a small function value means something went wrong.

\[ f(x_n) \approx 0 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 411 — stopping when x5 and x6 agree

In Example 4.46 the book stopped when the fifth and sixth iterates agreed to all ten digits displayed, reasoning that further steps would give the same value. That is the usual stopping rule: stop when successive guesses agree to the accuracy you need.

Why does that work? Because of quadratic convergence. When the method is converging, the next error is much smaller than the current one, so the step you just took is almost exactly the error you had. A tiny step means you were already very close.

But agreement alone is not proof. A method stuck in a cycle has successive values that disagree forever, which you would notice, but a method crawling along a very flat curve can take tiny steps while still far from a root. So always finish by substituting back: the function's value at your answer should be close to zero.

34. How it fails

Section

Part 5

35. Three ways the book says it can fail

Concept

failurewhat happenscause
flat tangentf′(xₙ) = 0, so the formula divides by zerothe tangent is horizontal and never meets the axis
the wrong rootthe iterates settle on a root you did not wantx₀ too far from the root sought
no root at allthe iterates bounce in a cycle, or run awaytangents that keep overshooting

None of these announces itself with an error message. The iteration keeps producing numbers; only checking tells you whether they mean anything.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 413 — Failures of Newton's Method

The book lists three ways Newton's method can go wrong, and this table gives each a name, a symptom and a cause. The first is a flat tangent: if the derivative at some guess is zero, the formula divides by zero and the method stops.

The second is the wrong root: the method converges perfectly well, but to a root you did not want, because the start was too far from the one you were after. The third is no convergence at all, and the book's example is a cycle that bounces between two values forever. You will also see the method run away to infinity.

The sentence under the table is the one to remember. A calculator running Newton's method does not print a warning. It prints numbers. Every one of these failures looks like ordinary output until you check it against a sketch or substitute back.

36. A horizontal tangent: starting at a critical point

Worked example

Return to the cubic of Example 4.46 and suppose you start exactly at, or just beside, its valley.

\[ f(x) = x^3 - 3x + 1, \quad f'(x) = 3x^2 - 3 \]

Start at 1

Why: The derivative vanishes there, but f does not.

\[ f(1) = -1, \quad f'(1) = 0 \]

Try the formula

Why: Division by zero: there is no next guess.

\[ x_1 = 1 - \frac{-1}{0} \quad \text{undefined} \]

Start at 1.01 instead

Why: The derivative is not zero, only tiny.

\[ f(1.01) \approx -0.99970, \quad f'(1.01) = 0.0603 \]

Take the step

Why: A small value divided by a tiny slope is huge.

\[ x_1 = 1.01 - \frac{-0.99970}{0.0603} \approx 1.01 + 16.58 = 17.59 \]

Figure (svg): Two panels on the curve y equals x cubed minus 3x plus 1. Left: at x0 equals 1 the tangent is the horizontal line y equals minus 1, which never meets the x-axis. Right: at x0 equals 1.01 the tangent is almost horizontal and leaves the window heading for x equals 17.6.

A horizontal tangent has no x-intercept, and a nearly horizontal one has an x-intercept miles away. Both come from dividing by a derivative that is zero or almost zero.

Check that the method recovers

Why: From 17.59 the steps shrink again and eventually reach 1.532, but only after many wasted steps: a start near a critical point is a bad start.

\[ x_1 \approx 17.59 \to 11.76 \to 7.90 \to \cdots \to 1.532 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 413 — failure 1

Start the cubic from Example 4.46 at 1, the bottom of its valley. The derivative there is exactly zero, so the formula divides by zero. In the left panel you can see why: the tangent is the horizontal line at height minus 1, and it never meets the axis. There is no next guess.

Now start at 1.01, just beside the valley. The derivative is not zero, only 0.0603, tiny. The value is about minus 1, not tiny. Dividing gives a correction of about 16.6, and the next guess is 17.59, far off the right edge of the right panel. Recall the triangle: a nearly flat tangent turns a modest height into an enormous run.

The check shows the method does eventually recover from 17.59, because far out the cubic is steep and well behaved, but it wastes about eight steps crawling back. The lesson is not that Newton's method is broken; it is that a start near a critical point is a bad start.

37. Figure 4.80: a start near a hump finds the wrong root

Concept

Figure (svg): The cubic y equals x cubed minus 3x plus 1. From x0 equals 0.95, just left of the valley bottom, the nearly flat tangent runs far to the left and meets the axis at minus 2.44; the iterates then settle on the root minus 1.879 instead of the nearby roots 0.347 or 1.532.

The start sits closest to 1.532 and 0.347, yet the method lands on the root farthest away. Near a flat stretch, the tangent throws the next guess wherever its tiny slope points.

Start at 0.95, where the curve is nearly flat. The tangent is almost horizontal, so it meets the axis far to the left, and the iterates then home in on a root you were not aiming for.

\[ x_0 = 0.95 \to -2.4436 \to -2.0238 \to -1.8927 \to \cdots \to -1.8794 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 413 — Figure 4.80

This is the book's second failure, drawn on the same cubic. Suppose you want one of the roots near the start, 0.347 or 1.532, and you start at 0.95. That point is close to the valley bottom at 1, so the tangent is nearly flat, sloping gently down to the left.

Follow the red tangent: it reaches the axis at about minus 2.44, beyond all three roots. From there the curve is steep, and the method converges quickly, but to minus 1.879, the root farthest from where you started.

Nothing in the output warns you. The numbers settle beautifully. Only a sketch, or a comparison with what you expected, tells you that you found the wrong root. When a problem asks for a particular root, always check which root your answer is.

38. Trap: expecting the nearest root

Trap

The trap

A common belief about the method:

\[ x_0 = 0.95 \]

\[ \Longrightarrow\; \text{finds } 1.532 \]

Wrong. It finds the farthest root.

The fix

Newton's method follows tangents, not distances. Near a critical point, a tiny slope throws the next guess a long way, and from there it may settle anywhere.

\[ 0.95 \to -2.44 \to \cdots \to -1.879 \]

If you want a particular root, start close to it and away from any flat stretch, and check which root you actually got.

It is natural to think of Newton's method as sliding down to the nearest root, like a ball rolling into the nearest dip. It does not work that way. It follows tangent lines, and a tangent line near a flat stretch can point almost anywhere.

The start 0.95 is 0.58 from the root at 1.532 and 0.60 from the root at 0.347, and 2.83 from the root at minus 1.879. The method chooses minus 1.879. The next slides colour every starting value by the root it finds, and you will see that near the humps, tiny changes in the start switch between all three roots.

The practical rule: start close to the root you want, and keep away from places where the derivative is small.

39. Example 4.48: bouncing between 0 and 1

Worked example

Show that starting at 0 fails to approach a root of this cubic.

\[ f(x) = x^3 - 2x + 2, \quad f'(x) = 3x^2 - 2, \quad x_0 = 0 \]

Evaluate at 0

Why: Only the constant term and the linear coefficient survive.

\[ f(0) = 2, \quad f'(0) = -2 \]

First step

Why: Subtract the correction.

\[ x_1 = 0 - \frac{2}{-2} = 1 \]

Evaluate at 1

Why: One minus two plus two is one.

\[ f(1) = 1, \quad f'(1) = 1 \]

Second step

Why: Back where you started.

\[ x_2 = 1 - \frac11 = 0 \]

Figure (svg): The curve y equals x cubed minus 2x plus 2 from minus 2.3 to 2. The tangent at x equals 0 runs down to the axis at 1; the tangent at x equals 1 runs back down to the axis at 0. The real root, near minus 1.769, is marked far to the left.

Each tangent sends you exactly to the other point: 0, 1, 0, 1, forever. The method never gets near the real root on the far left.

Check that the cycle never ends

Why: The third step is the first step again, so the list is 0, 1, 0, 1 forever, and neither value is a root: f is 2 at 0 and 1 at 1. The real root lies between minus 2 and minus 1.

\[ x_0, x_1, x_2, x_3, \ldots = 0, 1, 0, 1, \ldots, \qquad f(0) = 2 \ne 0, \; f(1) = 1 \ne 0 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, pp. 413-414 — Example 4.48 and Figure 4.81

This is the book's example of the third failure. At zero, the cubic has value 2 and slope minus 2, so the tangent runs down to the right and meets the axis at exactly 1. At 1, the value is 1 and the slope is 1, so the tangent is the line y equals x, which meets the axis at exactly 0.

Look at the figure: the orange tangent goes from the point above 0 down to 1, and the purple tangent goes from the point above 1 back down to 0. You are back where you started, so the whole computation repeats: 0, 1, 0, 1, forever.

The check confirms that neither value is a root, since the cubic is 2 at 0 and 1 at 1, and that the real root is somewhere else entirely, between minus 2 and minus 1. The book calls this behaviour a 2-cycle. It is exact here because the numbers are so simple, but nearby starting values get trapped near the cycle for a long time too.

40. Checkpoint 4.47: a better start for the same cubic

Worked example

Start at minus 1.5 instead, near the root the sketch shows.

\[ f(x) = x^3 - 2x + 2, \quad f'(x) = 3x^2 - 2, \quad x_0 = -1.5 \]

Evaluate at minus 1.5

Why: Minus 3.375 plus 3 plus 2.

\[ f(-1.5) = 1.625, \quad f'(-1.5) = 4.75 \]

First step

Why: The correction is positive, so step left.

\[ x_1 = -1.5 - \frac{1.625}{4.75} \approx -1.842105 \]

Evaluate at x one

Why: Keep the stored value.

\[ f(x_1) \approx -0.566701, \quad f'(x_1) \approx 8.180055 \]

Second step

Why: A negative value over a positive slope: step right.

\[ x_2 \approx -1.842105 + 0.069278 = -1.772827 \]

Check by continuing and substituting

Why: Two more steps settle at minus 1.769292354, where the cubic is zero to nine decimal places. A good start makes all the difference.

\[ x_4 \approx -1.769292, \qquad f(-1.769292354) \approx 2\times 10^{-9} \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 414 — Checkpoint 4.47

The cure for Example 4.48 is the one the book suggests: start closer to the actual root. A sketch shows the cubic crossing the axis between minus 2 and minus 1, so minus 1.5 is a natural start.

At minus 1.5 the value is 1.625 and the slope is 4.75, so the correction is about 0.342 and the first step goes left to minus 1.842105. There the value is negative, about minus 0.567, and the slope is about 8.18, so the correction is negative and the second step moves right, to minus 1.772827. The guesses are straddling the root and closing in.

Two more steps settle at minus 1.769292354, where the cubic is zero to nine decimal places. Compare with Example 4.48: the same function and the same formula, and the only difference was where you started.

41. Running away: the cube root

Worked example

A root of a function that is steep, not flat, at the root. Apply Newton's method to the cube root of x from x zero equal to 1.

\[ f(x) = x^{1/3}, \quad f'(x) = \frac13 x^{-2/3} \]

Form the correction

Why: Divide f by f prime; the powers combine.

\[ \frac{f(x)}{f'(x)} = \frac{x^{1/3}}{\frac13 x^{-2/3}} = 3x \]

Write the iteration

Why: Subtract the correction.

\[ x_n = x_{n-1} - 3x_{n-1} = -2x_{n-1} \]

Iterate from 1

Why: Each step doubles the distance and flips the side.

\[ 1, \; -2, \; 4, \; -8, \; 16, \; \ldots \]

Figure (svg): The curve y equals the cube root of x from minus 3 to 5. The tangent at x equals 1 meets the axis at minus 2; the tangent at minus 2 meets the axis at 4. Each landing point is twice as far from the root at 0 as the last, on the opposite side.

The curve is vertical at its root, so every tangent is too shallow and overshoots to the other side, twice as far each time.

Check with the formula for the nth guess

Why: The guesses are minus 2 to the n, so their size doubles every step: no start other than the root itself converges.

\[ x_n = (-2)^n, \qquad |x_n| = 2^n \to \infty \]

Stewart, Calculus: Early Transcendentals 8e, §4.8 Newton's Method §4.8, pp. 345-349 — the cube-root failure is a standard Stewart exercise

This failure is the opposite of the flat tangent. The cube root is vertical at its root, so every tangent line is too shallow compared with the curve near zero, and it overshoots.

The algebra is clean. Divide the cube root by its derivative and the powers combine to give exactly 3x. So the Newton step is x minus 3x, which is minus 2x. From 1 the guesses go to minus 2, then 4, then minus 8, doubling in size and flipping sides every step. The figure shows the first two tangents doing exactly that.

The check writes the nth guess as minus 2 to the power n, whose size doubles without bound. Here no starting value helps, except starting at the root itself. This is why Newton's method can only be trusted when the derivative at the root is a sensible, nonzero, finite number.

42. Exercises 447 and 448: nothing to find

Counterexample

Discussion prompt

Apply Newton's method to e to the x equal to 0 from x₀ = 0, and to 1 + x² = 0 from x₀ = 0. What happens in each, and what do the two failures teach about the method?

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 418 — Exercises 447 and 448

Write your answer before revealing. Both equations have no real solution: e to the x is always positive, and 1 plus x squared is at least 1. Newton's method does not know that.

For e to the x, the function and its derivative are the same, so the correction is always exactly 1 and every step just subtracts 1. The guesses march off to minus infinity, and the function values really do shrink toward zero along the way, which could fool a careless check into thinking a root is being approached. For 1 plus x squared starting at zero, the tangent is horizontal and the method cannot even take a first step.

The lesson is that Newton's method assumes there is a root to find. It can approximate a root once you know one exists, but it cannot tell you whether one exists. A sketch, or a sign change and the Intermediate Value Theorem, has to do that job first.

43. Converges, stops, cycles or runs away?

Sorting

Sort into buckets

Sort each function and starting value by what Newton's method does.

Converges
x² − 2 from x₀ = 2; x³ − 2x + 2 from x₀ = −1.5
Stops: f′(x₀) = 0
x³ − 3x + 1 from x₀ = 1; 1 + x² from x₀ = 0
Cycles
x³ − 2x + 2 from x₀ = 0; x³ − x from x₀ = 1/√5
Runs away
∛x from x₀ = 1; eˣ from x₀ = 0
conv
The square root of 2 from 2 (Example 4.47), and the cubic from minus 1.5 (Checkpoint 4.47), both settle quickly.
stop
At 1 the cubic x³ − 3x + 1 has a horizontal tangent, and 1 + x² has one at 0: the formula divides by zero at once.
cyc
Example 4.48 bounces 0, 1, 0, 1. For x³ − x, the start 1/√5 is sent to −1/√5 and back again forever.
run
The cube root doubles and flips each step; eˣ has no root, and each step just subtracts 1.

Sort each case before checking. Most of them appeared earlier in the lesson, so this is a chance to see the four behaviours side by side.

The two cases that stop at once both have a horizontal tangent at the start: the cubic at 1, and 1 plus x squared at 0. The two that cycle are Example 4.48 and a new one: for x cubed minus x, the Newton function sends one over root 5 exactly to minus one over root 5 and back.

The two that run away do so for different reasons. The cube root has a vertical tangent at its root, so each step overshoots by a factor of two; e to the x has no root at all. Only the two with sensible starts near a genuine root, where the derivative is comfortably nonzero, converge.

44. Find the error: a cycle reported as convergence

Error analysis

Annotate

On: \( x_0 = 0,\; x_1 = 1,\; x_2 = 0 \;\Longrightarrow\; \text{the values repeat, so the root is } 0 \)

  • Correct: for x³ − 2x + 2 from 0, the iterates really are 0, 1, 0, 1, as in Example 4.48.
  • Converging means successive values get closer to each other and to one number. These jump by 1 every step, forever.
  • Substituting back exposes it at once: f(0) = 2, not 0. Zero is not a root.
  • Restart near the real root. A sketch shows it between −2 and −1; from −1.5 the method gives −1.769292.

Read the argument and spot the problem before revealing the annotations. The numbers themselves are right; this is Example 4.48.

The error is in the word repeat. Convergence means the values get closer and closer to one number, so that successive values eventually agree to more and more digits. Here successive values always differ by exactly 1. Repeating a pattern is not settling on a value.

The quickest way to catch this is the check you should always do: substitute the claimed root into the function. The cubic at 0 is 2, nowhere near zero. Then fix it the way the book suggests, by restarting near the true root, which a sketch places between minus 2 and minus 1.

45. Exercise 425: before you start on x³ − x

Step zero

\[ f(x) = x^3 - x = 0 \qquad \text{roots } -1, \; 0, \; 1 \]

Discussion prompt

Before choosing x₀ for Newton's method on x³ − x, which starting values should you steer clear of, and why?

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 418 — Exercise 425

Answer before revealing. The roots are easy here, minus 1, 0 and 1, which is exactly why this is a good place to study starting values: you can see what the method finds.

The derivative is zero at plus and minus one over root 3, about 0.577, the tops and bottoms of the two humps. Starting there is impossible, and starting near there sends the next guess a long way off.

The subtler trap is the 2-cycle. Simplify Newton's function to 2x cubed over 3x squared minus 1 and put in one over root 5: the output is exactly the negative of the input. So the start 0.447 goes to minus 0.447 and back forever. Even a start that is not near a flat tangent can go wrong, which is why checking the output is never optional.

46. Every start, coloured by the root it finds

Picture it

Figure (svg): The cubic y equals x cubed minus 3x plus 1 above a coloured strip. Each point of the strip is a starting value, coloured by the root Newton's method reaches from it: purple for minus 1.879, green for 0.347, orange for 1.532. Starts left of minus 1 mostly reach minus 1.879, starts between about minus 0.6 and 0.8 reach 0.347, starts right of 1 reach 1.532, and near minus 1 and 1 the colours alternate in thin bands.

Far from the humps, a start finds its nearest root. Close to the critical points at −1 and 1, where the tangent is nearly flat, the colours shatter into thin stripes: a tiny change in the start changes the answer.

Run the method from five hundred starting values between minus 2.5 and 2.5 and colour each start by the root it reaches. Away from the humps, each start finds its nearest root; near the critical points the colours break into thin, alternating stripes.

This figure ran Newton's method five hundred times on the cubic from Example 4.46, once from each starting value along the strip, and coloured the start by where it ended up. Purple starts reach minus 1.879, green starts reach 0.347, and orange starts reach 1.532.

Far from the humps the picture is simple. Everything left of about minus 1 goes to the left root, a band around zero goes to the middle root, and everything right of 1 goes to the right root. In those regions, a start finds its nearest root.

Near the dashed lines at minus 1 and 1, where the tangent is flat, the colours break into thin stripes of all three. There, a change in the start in the second decimal place changes which root you find. This is a small taste of the chaos the chaos project at the end of the section describes: sensitive dependence on the starting value.

47. Slide the starting value

Tweak it

Parameter explorer

The staircase shows the Newton iterates x₀, x₁, x₂, … for x³ − 3x + 1, plotted against the step number. Slide x₀. Where does the staircase settle quickly, and where does it leap before settling?

\[ x_0 = {x0}, \quad x_n = x_{n-1} - \frac{x_{n-1}^3 - 3x_{n-1} + 1}{3x_{n-1}^2 - 3} \]

  • x0 — from -2.5 to 2.5: starting value x₀

The graph plots the Newton iterates against the step number, as a staircase: the height on step n is the guess x n. Start at the default, 0.95, and watch the staircase leap down to about minus 2.44 before settling at minus 1.879.

Now slide the start slowly to the right, toward 1.5 and beyond. The staircase settles on 1.532 within two or three steps, almost flat. Slide it into the middle, between about minus 0.5 and 0.7, and it settles on 0.347.

Then make small moves near 1 and near minus 1. The staircase leaps off the screen and lands on different roots for starts that differ by only a twentieth. Compare with the coloured strip on the previous slide: you are walking along it by hand.

48. Other iterative processes

Section

Part 6

49. Iterating any function: fixed points

Concept

Newton's method is one example of a general idea: choose any function F and a start, then apply F over and over.

\[ x_n = F(x_{n-1}), \qquad n \ge 1 \]

If the list settles on a number, that number is unchanged by F, because applying F to the limit gives the limit again.

fixed point — A number x-star with F of x-star equal to x-star. On a graph it is where y equals F(x) crosses the diagonal y equals x.

\[ x_n \to x^* \;\text{ and } F \text{ continuous} \;\Longrightarrow\; x^* = F(x^*) \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 414 — Other Iterative Processes

Newton's method feeds a guess into the function F again and again. You can do that with any function, and the book's next examples do exactly that with simple linear functions.

Suppose the list does settle down to some number. Then applying F to a value very close to that number gives a value very close to that number, and in the limit, F leaves the number unchanged. A number that F leaves unchanged is called a fixed point. On a graph, fixed points are where the graph of F crosses the diagonal line y equals x.

This explains Newton's method too. Its F subtracts the value of f divided by the slope, so a fixed point of Newton's F is a place where the value of f is zero: a root. Newton's method is a way of turning the root problem into a fixed-point problem with a very good F.

50. Example 4.49: iterating F(x) = x/2 + 4

Worked example

Starting at 0, find the first few values and conjecture where the list is heading.

\[ F(x) = \tfrac12 x + 4, \quad x_0 = 0 \]

First three values

Why: Halve and add 4 each time.

\[ x_1 = \tfrac12(0) + 4 = 4, \quad x_2 = \tfrac12(4) + 4 = 6, \quad x_3 = \tfrac12(6) + 4 = 7 \]

Keep going

Why: The gaps to 8 are 4, 2, 1, one half, one quarter.

\[ x_4 = 7.5, \; x_5 = 7.75, \; x_6 = 7.875, \; x_7 = 7.9375, \; x_8 = 7.96875 \]

Conjecture

Why: The distance to 8 halves every step.

\[ x_n = 8 - \frac{8}{2^{n}} \to 8 \]

Solve for the fixed point

Why: Set x equal to F(x).

\[ x = \tfrac12 x + 4 \iff \tfrac12 x = 4 \iff x = 8 \]

Check

Why: F leaves 8 unchanged, and the formula reproduces the ninth value in the book.

\[ F(8) = 4 + 4 = 8, \qquad x_9 = 8 - \tfrac{8}{512} = 7.984375 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, pp. 414-415 — Example 4.49

Compute the first few values yourself: halve and add 4. From 0 you get 4, 6, 7, 7.5, 7.75, and the book continues to 7.984375. The list is plainly heading for 8.

The distances to 8 tell the story: 8, 4, 2, 1, one half, one quarter. Each distance is half the last, because halving the input halves its distance from the fixed point. That gives the exact formula 8 minus 8 over 2 to the power n.

To find the limit without computing, solve x equals F of x. Half of x plus 4 equals x exactly when x is 8. The check confirms that F sends 8 to itself, and that the formula reproduces the book's ninth value.

51. A cobweb diagram

Concept

Figure (svg): A cobweb diagram on square axes from 0 to 9. The line y equals x over 2 plus 4 and the diagonal y equals x cross at (8, 8). Starting at 0, a vertical segment rises to the line at 4, a horizontal one moves to the diagonal, then up to 6, across, up to 7, and so on, the staircase closing in on the crossing.

Up to the graph of F, across to the diagonal, up again. The steps shrink by half each time, trapped between the two lines, and close in on the point where they cross.

Go up from the start to the graph of F, then across to the diagonal: that turns an output into the next input. Repeat, and the path is trapped between the two lines, closing in on where they cross.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 415 — Figure 4.82

This is the book's graphical argument for Example 4.49, and it works for any iteration. Start on the x-axis at 0 and go straight up to the graph of F, the blue line: the height there is x one, 4. Then go horizontally to the orange diagonal. On the diagonal, height and x-coordinate are equal, so you are now standing above x one, ready for the next step.

Repeat: up or down to the graph of F, across to the diagonal. The yellow staircase is trapped between the two lines, and because the lines cross, the steps get shorter and close in on the crossing at 8.

This picture shows why the iteration converges. The blue line is flatter than the diagonal, with slope one half. When the graph of F is less steep than the diagonal where they cross, the staircase shrinks inward; when it is steeper, it would spiral outward. Keep that in mind for the logistic map at the end of this part.

52. Checkpoint 4.48: iterating F(x) = x/3 + 6

Worked example

\[ F(x) = \tfrac13 x + 6, \quad x_0 = 0, \quad x_n = F(x_{n-1}) \text{ for } n \ge 1 \]

First two values

Why: A third of the input, plus 6.

\[ x_1 = 6, \quad x_2 = 2 + 6 = 8 \]

Next three

Why: The gaps to 9 are now shrinking by a factor of three.

\[ x_3 = \tfrac{26}{3} \approx 8.6667, \quad x_4 = \tfrac{80}{9} \approx 8.8889, \quad x_5 = \tfrac{242}{27} \approx 8.9630 \]

Conjecture and solve

Why: Set x equal to F(x).

\[ x = \tfrac13 x + 6 \iff \tfrac23 x = 6 \iff x = 9 \]

Check with the pattern

Why: Each value is 9 minus 9 over a power of 3. The book prints the rule as starting at n equal to 2, a misprint: x one must be F of x zero, so the rule starts at n equal to 1.

\[ x_n = 9 - \frac{9}{3^{n}}: \quad x_5 = 9 - \tfrac{9}{243} = 8.9630 \to 9 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 415 — Checkpoint 4.48

This is the same computation with a third instead of a half. From 0 you get 6, then 8, then twenty-six thirds, eighty ninths and two hundred forty-two twenty-sevenths.

The distances to 9 are now 9, 3, 1, one third, one ninth: each is a third of the last, so this list converges faster than the one in Example 4.49. Solving x equals a third of x plus 6 gives 9, the conjectured limit.

One correction to the book. The checkpoint states the rule as applying for n from 2 on, but then x one would never be defined. Since the values are meant to start with x one equal to F of x zero, the rule must apply from n equal to 1, exactly as in Example 4.49. The computation on the slide uses that corrected reading.

53. Exercise 411: a c that works and a c that fails

Worked example

Solve x squared minus 4 equal to zero with the simpler iteration below, starting at 0, which uses a fixed constant c instead of one over the derivative.

\[ x_{n+1} = x_n - c\,(x_n^2 - 4), \quad x_0 = 0 \]

Try c equal to one quarter

Why: Iterate.

\[ c = \tfrac14: \quad 0, \; 1, \; 1.75, \; 1.984375, \; 1.999939, \; \ldots \to 2 \]

Try c equal to 1

Why: Iterate.

\[ c = 1: \quad 0, \; 4, \; -8, \; -68, \; -4688, \; \ldots \]

Explain with the slope of G at the root

Why: Write the iteration as G; the fixed point 2 attracts when the slope of G there is less than 1 in size.

\[ G(x) = x - c(x^2 - 4), \quad G'(2) = 1 - 4c \]

Check both values of c against Newton's choice

Why: One quarter makes the slope 0, the fastest possible; 1 makes it minus 3, which pushes the iterates away. Exercise 413: Newton's method uses c equal to one over f prime, here one over 2x, which is exactly one quarter at the root.

\[ c = \tfrac14: G'(2) = 0, \qquad c = 1: G'(2) = -3, \qquad c_{\text{Newton}} = \frac{1}{2x} \to \frac14 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 417 — Exercises 411 and 413

This iteration is a cousin of Newton's method. Instead of dividing by the derivative, which changes at every step, you multiply by a fixed constant c. The roots of x squared minus 4 are still the fixed points, since the correction is zero exactly at a root.

With c equal to one quarter the list 0, 1, 1.75, 1.984 closes in on 2 quickly. With c equal to 1 it explodes: 4, minus 8, minus 68. The explanation is the slope idea from the cobweb picture. The iteration function G has slope 1 minus 4c at the root 2. If that slope is smaller than 1 in size, the fixed point attracts the iterates; if it is larger, it repels them.

The check also answers Exercise 413. Newton's method is this iteration with c equal to one over the derivative, which for this function is one over 2x, and at the root that is exactly one quarter, the value that makes the slope of G zero. That zero slope is the deep reason Newton's method converges so fast.

54. Exercise 434: the fixed points of eˣ − 2

Worked example

Use Newton's method to find the fixed points of F(x) equal to e to the x minus 2, to three decimals.

Turn the fixed-point equation into a root problem

Why: Move x across.

\[ e^x - 2 = x \iff g(x) = e^x - 2 - x = 0, \quad g'(x) = e^x - 1 \]

Figure (svg): The curve y equals e to the x minus 2 and the line y equals x from minus 2.5 to 2. They cross twice, near minus 1.841 and near 1.146.

A fixed point is a crossing with the diagonal. Two crossings means two fixed points, and a different starting value is needed for each.

Newton from x zero equal to 2

Why: The sketch shows a crossing near 1.

\[ 2 \to 1.469553 \to 1.207329 \to 1.148806 \to 1.146198 \to 1.146193 \]

Newton from x zero equal to minus 2

Why: The other crossing is on the left.

\[ -2 \to -1.843482 \to -1.841406 \to -1.841406 \]

Round

Why: Three decimals.

\[ x^* \approx 1.146 \quad\text{and}\quad x^* \approx -1.841 \]

Check both are fixed

Why: Apply F and compare.

\[ e^{1.146193} - 2 \approx 1.146193, \qquad e^{-1.841406} - 2 \approx -1.841406 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 418 — Exercise 434

To find fixed points with Newton's method, rewrite the equation F of x equals x as a root problem by moving everything to one side. Here that gives e to the x minus 2 minus x, whose derivative is e to the x minus 1.

The figure shows the curve crossing the diagonal twice, so there are two fixed points and you need two starting values. From 2 the iterates settle on 1.146193, and from minus 2 they settle on minus 1.841406, within three steps. Rounding to three decimals gives 1.146 and minus 1.841.

The check applies F to each answer and gets the answer back, which is exactly what a fixed point means. Notice you could not simply iterate F itself to find the right-hand fixed point: the curve is steeper than the diagonal there, so plain iteration would move away from it. Newton's method has no such problem.

55. Exercise 442: Newton's method finds a maximum

Worked example

Exercise 436 shows that to find a critical point you apply Newton's method to f prime, which puts f double prime in the denominator. Use it for the local maximum of this cubic.

\[ f(x) = x^3 + 10x^2 + 15x - 2 \qquad x_{n+1} = x_n - \frac{f'(x_n)}{f''(x_n)} \]

Differentiate twice

Why: Power rule.

\[ f'(x) = 3x^2 + 20x + 15, \quad f''(x) = 6x + 20 \]

Figure (svg): The cubic y equals x cubed plus 10 x squared plus 15x minus 2 from minus 8 to 1, with a local maximum near x equals minus 5.805, height about 52.3. Its derivative is drawn dashed; the derivative crosses zero at exactly the x of the maximum.

The top of the hump and the zero of the dashed derivative line up. Newton's method on f′ finds that zero, and so finds the maximum.

First step from minus 6

Why: The sketch puts the hump near minus 6.

\[ x_1 = -6 - \frac{f'(-6)}{f''(-6)} = -6 - \frac{3}{-16} = -5.8125 \]

Second step

Why: Evaluate at minus 5.8125.

\[ x_2 = -5.8125 - \frac{0.105469}{-14.875} \approx -5.805410 \]

Third step

Why: The digits have settled.

\[ x_3 \approx -5.805399 \;\Longrightarrow\; x \approx -5.805 \]

Check it is a maximum and matches the exact root

Why: The second derivative is negative there, and the quadratic formula gives the same number.

\[ f''(-5.805) \approx -14.83 < 0, \qquad \frac{-10 - \sqrt{55}}{3} \approx -5.805399 \]

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 418 — Exercises 436 and 442

Local maxima and minima happen at critical points, where the derivative is zero. So to find one, apply Newton's method not to f but to f prime. Every derivative in the formula moves up a level, and the second derivative lands in the denominator.

The figure shows the cubic, with its hump near minus 6, and its derivative dashed. The derivative crosses zero exactly beneath the top of the hump. Starting at minus 6, the first step has f prime equal to 3 and f double prime equal to minus 16, which moves the guess to minus 5.8125. Two more steps settle at minus 5.805399.

The check does two things. The second derivative is negative there, so the critical point is a maximum rather than a minimum, which is the second derivative test from Section 4.5. And because f prime is a quadratic, the quadratic formula gives the exact answer, minus 10 minus root 55, all over 3, which matches every digit.

56. Trap: applying Newton to f to find a maximum

Trap

The trap

Looking for the maximum of f:

\[ x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)} \]

Wrong. This hunts for a zero of f, a place where the graph crosses the axis, not its peak.

The fix

A maximum is a zero of f′. Apply the method to f′, and every derivative moves up one level.

\[ x_{n+1} = x_n - \frac{f'(x_n)}{f''(x_n)} \]

Then check the sign of f″ to see whether you found a maximum or a minimum.

When a problem says use Newton's method, it is tempting to reach for the formula you have used all lesson. But that formula hunts for places where f is zero, and the height of a maximum has nothing to do with zero.

The fix is to ask what equation the maximum satisfies. A maximum of a differentiable function occurs where the derivative is zero, so the equation is f prime equals zero. Newton's method on that equation uses f prime over f double prime.

The same method finds minima too, so after you find a critical point, check which it is. A negative second derivative means a maximum and a positive one means a minimum.

57. The logistic map: match r to its long-run behaviour

Matching

The project Iterative Processes and Chaos iterates the logistic map below, starting from 0.2. Match each r to what the list of values does after many steps.

\[ f(x) = rx(1 - x), \quad x_0 = 0.2 \]

Match the pairs

  • a. r = 0.5
  • b. r = 2
  • c. r = 3.2
  • d. r = 3.5
  • e. r = 4
  • v. settles on 0
  • w. settles on 0.5
  • x. a 2-cycle: 0.513, 0.800
  • y. a 4-cycle: 0.501, 0.875, 0.383, 0.827
  • z. chaos: no settling, no cycle

Why: Small r shrinks every value toward zero. At r equal to 2 the fixed point one minus one over r, which is one half, attracts. As r grows the fixed point becomes unstable and the values split into a 2-cycle, then a 4-cycle, and at r equal to 4 into chaos, where changing the start from 0.2 to 0.201 gives a completely different list within about ten steps.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, pp. 415-417 — Student Project: Iterative Processes and Chaos

The project at the end of the section, Iterative Processes and Chaos, iterates the logistic map, a parabola that population biologists use for a species whose growth is limited by crowding. Its single number r controls everything. Match each r to its behaviour before checking.

For small r, everything shrinks to zero: the population dies out. At r equal to 2, the list settles on one half, the fixed point one minus one over r. At 3.2 the fixed point can no longer hold the iterates, and they settle into a 2-cycle alternating between about 0.513 and 0.800, just like the Newton cycle in Example 4.48 but stable. At 3.5 the cycle doubles again to four values.

At r equal to 4 there is no settling and no cycle at all: this is chaos. The project's last question compares starts of 0.2 and 0.201, and after about ten steps the two lists have nothing in common. A thousandth of difference in the start is enough to change everything, which is the same sensitivity you saw in the coloured strip of Newton starting values.

58. Cobwebs for the logistic map

Picture it

Figure (svg): Two cobweb diagrams for the logistic map f(x) equals r x times 1 minus x, starting at 0.2. Left, r equals 2.8: the cobweb spirals in to the crossing with the diagonal near 0.643. Right, r equals 3.2: the cobweb settles into a rectangle that bounces between about 0.513 and 0.800.

The same parabola-and-diagonal picture. At r = 2.8 the web spirals into a fixed point; raise r to 3.2 and it settles into a square loop instead, a 2-cycle.

The same up-and-across construction, now with a parabola. A fixed point attracts when the parabola is not too steep where it meets the diagonal; when it is too steep, the web cannot close in and settles into a loop instead.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, pp. 416-417 — Figure 4.84

The same cobweb construction as before, now with a parabola instead of a straight line. In the left panel, with r equal to 2.8, the web winds round and round, spiralling in to where the parabola meets the diagonal, near 0.643.

In the right panel, with r equal to 3.2, the web starts the same way but never closes in. It settles into a rectangle that visits two heights, about 0.513 and 0.800, over and over: a 2-cycle.

What changed is the steepness of the parabola where it crosses the diagonal. At r equal to 2.8 the slope there is a little less than 1 in size, so the web shrinks inward, very slowly. At 3.2 it is more than 1, so the fixed point repels, and the iterates settle on the cycle instead. It is the same slope test you used in Exercise 411.

59. Putting it together

Section

Part 7

60. Pattern: solving f(x) = 0 with Newton's method

Pattern

Figure (svg): A flow diagram: sketch f and choose a starting value; apply x next equals x minus f over f prime; if the steps are shrinking and the derivative is not near zero, keep going until successive values agree, then substitute back; if a derivative is near zero or the values jump or repeat, choose a new start or bracket the root first.

One loop, two exits: success, confirmed by substituting back, or a restart from a better guess.
  1. Sketch f (or find a sign change) to see roughly where the root is, and pick x₀ close to it and away from flat stretches.
  2. Write the iteration x − f(x)/f′(x), simplified if it helps.
  3. Iterate, keeping every digit your calculator stores.
  4. Stop when two successive values agree to the accuracy you need.
  5. Substitute back: f should be nearly zero. If the values jump, repeat or run away, choose a new start or bracket the root with bisection first.

This is the routine to follow whenever you use Newton's method, and the diagram shows its loop and its two exits. The first move happens before any formula: sketch the function, or find a sign change, so that you know a root exists and roughly where it is.

Then iterate, keeping every digit, and watch the steps. Shrinking steps mean it is working; when two successive values agree to the accuracy you need, stop and substitute back. If the values jump wildly, repeat, or run away, or if a derivative is close to zero, do not keep pressing the button. Choose a better start, or use bisection to trap the root in a small interval first.

The substitution at the end is not optional. It is the only step that distinguishes a root from a cycle, a runaway, or an arithmetic mistake.

61. Check: one Newton step

Check

Check your understanding

Apply one step of Newton's method to f(x) = x² − 5 from x₀ = 2. What is x₁?

  • A. 2.25 (correct)
  • B. 1.75
  • C. 2.236
  • D. 2.5

Answer: A

Why: At 2 the function is 4 minus 5, which is minus 1, and the slope is 4. Subtracting minus one quarter gives 2.25, which is already close to the square root of 5, about 2.236.

Why B tempts people
This adds the correction instead of subtracting it; the step went the wrong way.
Why C tempts people
That is the true root, the square root of 5. One Newton step gets close but does not land exactly on it.
Why D tempts people
This divides the function value by the guess rather than by the derivative; the derivative at 2 is 4, not 2.

At 2 the function is minus 1 and the derivative is 4, so the correction is minus one quarter and subtracting it gives 2.25. The square root of 5 is about 2.236, so one step has already brought you from an error of about 0.24 to about 0.014.

If you chose 1.75, you added the correction instead of subtracting it, the exact trap from Part 2. If you chose 2.236, you gave the root rather than the first step.

62. Check: why the method stops

Check

Check your understanding

For which of these does Newton's method fail at the very first step?

  • A. f(x) = x² − 2 with x₀ = 2
  • B. f(x) = x³ − 3x + 1 with x₀ = −1 (correct)
  • C. f(x) = x³ − 2x + 2 with x₀ = −1.5
  • D. f(x) = eˣ − 2 − x with x₀ = 2

Answer: B

Why: At minus 1 the derivative of the cubic is 3 minus 3, which is zero, while the function value is 3. The formula divides by zero, and the horizontal tangent never meets the axis.

Why A tempts people
The derivative at 2 is 4, so the first step is fine and leads to 1.5; this is Example 4.47.
Why C tempts people
The derivative at minus 1.5 is 4.75, and the iteration converges to minus 1.769; this is Checkpoint 4.47.
Why D tempts people
The derivative at 2 is e squared minus 1, about 6.39, and the iteration converges to 1.146.

Look for a start where the derivative is zero. The cubic x cubed minus 3x plus 1 has derivative 3x squared minus 3, which is zero at 1 and at minus 1. At minus 1 the function is 3, not zero, so the tangent is horizontal and misses the axis entirely.

The other three all begin with a healthy derivative, and each appeared in this lesson converging nicely: the square root of 2, the good start for Example 4.48's cubic, and the fixed-point equation from Exercise 434.

63. Check: the fixed point

Check

Check your understanding

The iteration xₙ = F(xₙ₋₁) with F(x) = x/4 + 3 starts at x₀ = 0. What number does the list approach?

  • A. 4 (correct)
  • B. 3
  • C. 12
  • D. It does not approach a number

Answer: A

Why: A limit must be a fixed point: setting x equal to a quarter of x plus 3 gives three quarters of x equal to 3, so x is 4. The list 0, 3, 3.75, 3.9375 closes in on it, the gap shrinking by a factor of four each step.

Why B tempts people
Three is the first value, F of 0, not the limit; the next values are 3.75 and 3.9375.
Why C tempts people
Twelve solves x over 4 equal to 3, which drops the x on the left-hand side of the fixed-point equation.
Why D tempts people
The multiplier one quarter is smaller than 1 in size, so the gaps shrink and the list does converge.

If the list converges, its limit must be a fixed point, so solve x equals a quarter of x plus 3. Three quarters of x equals 3, so x is 4.

You can confirm by iterating: 0, 3, 3.75, 3.9375, with the distance to 4 shrinking by a factor of four each time. That factor is the slope of F, one quarter, which is smaller than 1, so the iteration is guaranteed to close in.

64. Explain the derivative in the denominator

Explain it to yourself

Discussion prompt

In two or three sentences, explain why f′(xₙ) sits in the denominator of Newton's formula, and what that predicts about starting near a horizontal tangent.

Write your explanation before revealing. The best answers use the right triangle from Part 2: height of the curve over slope of the tangent equals the horizontal distance to the axis.

From that picture everything else follows. A steep tangent means a small step, which is safe. A shallow tangent means a long step, which can overshoot everything. A horizontal tangent means no step at all. If your explanation connects the size of the derivative to the size of the jump, you have understood why starting near a critical point is dangerous, and why a flat curve at the root slows convergence.

65. Exit ticket

Exit ticket

\[ x + 5\cos x = 0 \qquad \text{(Exercise 426)} \]

Discussion prompt

Write the Newton iteration and compute from x₀ = −1 until the value is fixed to four decimal places. Then check your answer.

OpenStax Calculus Volume 1, §4.9 Newton's Method §4.9, p. 418 — Exercise 426

This exercise from the section brings everything together. The derivative of x plus 5 cosine x is 1 minus 5 sine x, so the iteration is the one shown. Work it out before revealing.

Where should you start? A quick sketch, or just plugging in a couple of values, shows the function is positive at minus 1 and negative at minus 1.5, so a root lies between, and minus 1 is a sensible start. The iterates go to minus 1.326752, then minus 1.306484, then minus 1.306440, and the next value agrees to six decimals.

Rounded to four decimal places the root is minus 1.3064, and substituting back gives zero to the accuracy shown. Notice the error shrinking the Newton way: about two hundredths, then about four hundred-thousandths, then about two ten-billionths.

66. Recap

Recap

ideawhat to remember
the iterationxₙ = xₙ₋₁ − f(xₙ₋₁)/f′(xₙ₋₁): the root of the tangent line
speedwhen it works, the error roughly squares, so correct digits double
failuresf′ = 0, the wrong root, a cycle, a runaway: always substitute back
starting valuesketch first; start near the root, away from flat stretches
bisectionslow (error halves) but certain once a sign change brackets the root
iterative processesxₙ = F(xₙ₋₁); a limit must be a fixed point, F(x) = x

\[ x_n = x_{n-1} - \frac{f(x_{n-1})}{f'(x_{n-1})} \]

Next, Section 4.10 turns derivatives around: finding a function from its derivative, the antiderivative.

Stewart, Calculus: Early Transcendentals 8e, §4.8 Newton's Method §4.8, pp. 345-349 — the same material in Stewart

Newton's method replaces a curve by its tangent line and solves the line instead. Repeated, that gives the iteration at the bottom of the slide, which is the one formula from this section to know by heart and to be able to derive from the tangent line.

When it works, it works spectacularly: the error roughly squares at each step, so the correct digits double. When it fails, it fails silently, through a flat tangent, a jump to the wrong root, a cycle, or a runaway. So sketch before you start, choose a start near the root and away from flat stretches, and substitute back at the end. Bisection is the slow but certain alternative when you need a guarantee.

More generally, any rule applied repeatedly is an iterative process, and if its values settle, they settle on a fixed point. The next section reverses differentiation, asking which functions have a given derivative.

Sources

  1. OpenStax Calculus Volume 1, §4.9 Newton's Method — Strang & Herman, OpenStax / Rice University, CC BY-NC-SA 4.0, pp. 409-418
  2. Stewart, Calculus: Early Transcendentals 8e, §4.8 Newton's Method — James Stewart, Cengage Learning, 2016, pp. 345-349

Want this taught 1-on-1? Alexander tutors Calculus I — $55/session, free consultation.

Book on Wyzant · Text (657) 465-8108