Introduction to Numerical Analysis

In mathematics we usually aim for an exact answer using tools like Algebra, Geometry and Calculus.

But sometimes we can't find an answer that way, or the answer takes too long to find.

Numerical Analysis to the rescue!

Numerical Analysis gives us an approximate answer using simple calculations, and we can often use it again and again getting better answers until the error is small.

Error does not mean a mistake or anything, it is how close we think we are to the true answer. Our goal is to have the lowest error with limited calculation time. Understanding how errors grow or shrink is important!

Numerical Methods

The methods are also called "algorithms" and are like recipes.

One of the famous algorithms is Newton's method.

It is very useful for finding roots (where a function crosses the axis).

We start with a guess x₀, then find where that puts us on the curve.

Using that point (x₀, f(x₀)) and it's slope we find a new point on the axis x₁ and that becomes our new guess. Repeat using x₁ etc.

It does not always work (sometimes the slope sends us way away) but when it does work it is great!

Example: Newton's Method for √10

Newton's Method finds where a function equals zero (a "root").

So to find x = √10 we can use x² − 10 = 0

The slope of x² − 10 is 2x (using deriviatives)

Start with an initial guess

x₀ = 3

The point on the curve is:

(3, 3² − 10) = (3, −1)

At x=3 the slope is

2x = 6

Update the guess using

x₁ = x₀ − y₀/slope

Calculate:

x₁ = 3 − (−1)/6 = 3.166...

After one round we have √10 ≈ 3.1666...

Our calculator says √10 = 3.16227766..., good so far.

Let's try another round:

Start with

x₁ = 3.166...

The point on the curve is:

(3.166..., 3.166...² − 10) = (3.166..., 0.027...)

At x=3.166... the slope is

2x = 6.33...

Update the guess using

x₂ = x₁ − y₁/slope

Calculate:

x₂ = 3.166... − 0.027.../6.3... = 3.16228...

That is really good!

Try one more round yourself and see what you get.

Other Methods

There are many clever and interesting methods. Each has strengths and weaknesses.

Another way to find roots is the Bisection Method, or "binary chop". We start either side of the root, one side with positive y, the other with negative y. We choose a value in between, calculate its value and then reduce the interval, making sure to keep one negative and one positive y value.

There are also many methods for doing integrals such as Simpson's Rule (see Integral Approximations and Integral Approximation Calculator).

Also have a play with a simple computer model of reflection inside an ellipse. The errors accumulate but it is still fun to watch, or the single pendulum or double pendulum animation (which also suffers from accumulating errors).

It is actually heaps of fun seeing answers getting better and better (or worse and worse) as the computer works through them.

More complex examples include:

Weather prediction
Economic Models (predicting interest rates, unemployment, etc)
Public health vs infectious diseases
Models of how large structures behave under stress (bridges, skyscrapers, etc)
Many more ...

If you become an expert in any of those you will have a good and satisfying career.