Sequences and Series: Patterns, Sums and Limits

1. Introduction

Patterns are everywhere in numbers. The days of the week repeat, a savings balance climbs month by month, a bouncing ball rises a little less each time, and a model's error shrinks a little with every training step. Mathematics captures these patterns with two companion ideas: a sequence, an ordered list of numbers, and a series, the sum of such a list.

These ideas connect the other topics in this series. An arithmetic sequence is a linear function whose input is limited to whole numbers. A geometric sequence is an exponential function sampled at whole numbers. Adding up the terms leads to formulas that save enormous effort, and adding up infinitely many terms leads to one of the most surprising facts in mathematics: an endless sum can have a finite answer.

This article covers the two main types of sequence, the matching series formulas, infinite series and when they converge, recursive sequences such as Fibonacci, and practical applications. We finish with where these ideas appear in machine learning, from gradient descent to reinforcement learning.

2. What is a sequence?

A sequence is an ordered list of numbers called terms. We name the first term a₁, the second a₂, and the general term aₙ, where n is the position. Unlike a set, order matters and repeats are allowed. A sequence is really a function whose input is the positive whole numbers: n goes in and aₙ comes out.

A sequence can be described in two ways. An explicit rule gives aₙ directly from n, for example aₙ = 3n + 2, which produces 5, 8, 11, 14, and so on. A recursive rule says how each term follows from the previous one, for example a₁ = 5 and aₙ = aₙ₋₁ + 3. Both describe the same list.

3. Arithmetic sequences

An arithmetic sequence changes by the same amount each time, the common difference d. Starting at a₁, the terms are a₁, a₁ + d, a₁ + 2d, …, so the nth term is aₙ = a₁ + (n − 1)d. The (n − 1) appears because the first term has had no steps added yet.

For 5, 8, 11, 14, …, we have a₁ = 5 and d = 3, so aₙ = 5 + 3(n − 1). The tenth term is 5 + 3(9) = 32. You can test whether a list is arithmetic by checking that all neighbouring differences are equal.

An arithmetic sequence is a linear function with the input restricted to whole numbers: aₙ = dn + (a₁ − d) has slope d. That is why its dots lie on a straight line.

4. Geometric sequences

A geometric sequence multiplies by the same factor each time, the common ratio r. The terms are a₁, a₁r, a₁r², …, so aₙ = a₁ · rⁿ⁻¹. This is an exponential function sampled at whole numbers.

For 3, 6, 12, 24, …, a₁ = 3 and r = 2, so aₙ = 3 · 2ⁿ⁻¹ and the eighth term is 3 · 2⁷ = 384. Check by dividing neighbours: 6/3 = 12/6 = 2. If |r| > 1 the terms grow in size, if |r| < 1 they shrink toward zero, and a negative r makes the signs alternate.

5. Series and sigma notation

A series is what you get by adding the terms of a sequence. The sum of the first n terms is the partial sum Sₙ = a₁ + a₂ + … + aₙ. Mathematicians abbreviate sums with the Greek letter sigma: Σ. The notation Σ (k = 1 to 5) k² means "add the value of k² as k runs from 1 to 5," which is 1 + 4 + 9 + 16 + 25 = 55.

Three sums come up so often that their formulas are worth knowing: the sum of the first n whole numbers, of their squares, and of their cubes.

6. Arithmetic series

The story goes that a young schoolboy, Carl Friedrich Gauss, was asked to add the numbers 1 to 100 and found the answer in seconds. His idea was to write the sum twice, once forwards and once backwards. Every column then adds to the same total, 101, and there are 100 columns. So twice the sum is 100 × 101, and the sum is 5050.

The same trick works for any arithmetic series. Pair the first term with the last, the second with the second last, and so on: each pair has the same sum, a₁ + aₙ. This gives Sₙ = n/2 · (a₁ + aₙ), or equivalently Sₙ = n/2 · (2a₁ + (n − 1)d). In words: the number of terms times the average of the first and last.

For 5 + 8 + … + 32, there are 10 terms, so S₁₀ = 10/2 · (5 + 32) = 185. A slightly harder case: how many terms are in 7, 10, 13, …, 100? Solve 7 + 3(n − 1) = 100 to get n = 32, so the sum is 32/2 · (7 + 100) = 1712.

7. Geometric series

For a geometric series, a different trick works. Write Sₙ = a₁ + a₁r + … + a₁rⁿ⁻¹. Multiply by r to get rSₙ = a₁r + … + a₁rⁿ. Subtracting, almost everything cancels, leaving Sₙ − rSₙ = a₁ − a₁rⁿ, so Sₙ = a₁(1 − rⁿ) / (1 − r), valid when r ≠ 1.

For 3 + 6 + 12 + … with 8 terms, S₈ = 3(2⁸ − 1)/(2 − 1) = 3 × 255 = 765. The same formula explains an old legend of grains of rice on a chessboard, doubling from 1 grain on the first square. Over all 64 squares the total is 2⁶⁴ − 1 = 18,446,744,073,709,551,615 grains, a number beyond any real harvest.

8. Infinite series and convergence

What if a series never stops? An infinite series adds endless terms. We look at the partial sums S₁, S₂, S₃, …. If they settle toward a single number, the series converges to that number. If they grow without limit or never settle, it diverges.

For a geometric series with |r| < 1, rⁿ fades to zero, so the finite formula becomes S = a₁ / (1 − r). Take 1 + 1/2 + 1/4 + 1/8 + …. Here a₁ = 1 and r = 1/2, so S = 1/(1 − 1/2) = 2. Each term fills half of what remains between the running total and 2, so the total gets ever closer but never passes it.

If |r| ≥ 1, the geometric series diverges. The series 1 + 2 + 4 + 8 + … does not equal −1, which the formula would wrongly suggest. The sum simply grows without limit. The formula is valid only when |r| < 1.

A bouncing ball makes this concrete. Dropped from 10 metres, it rebounds to 60 percent of its previous height each time. The rebound heights are 6, 3.6, 2.16, and so on, a geometric series with sum 6/(1 − 0.6) = 15. The ball travels up and down each time, so the total distance is 10 + 2 × 15 = 40 metres, even though it bounces infinitely often.

Be careful: terms shrinking toward zero is necessary for convergence but not sufficient. The harmonic series 1 + 1/2 + 1/3 + 1/4 + … has terms that shrink to zero, yet it diverges, because its partial sums keep climbing, very slowly. By contrast, 1 + 1/4 + 1/9 + 1/16 + … converges to about 1.645.

9. Recursive sequences and Fibonacci

Sometimes a sequence is naturally defined by a recursive rule. The most famous example is the Fibonacci sequence: F₁ = 1, F₂ = 1, and Fₙ = Fₙ₋₁ + Fₙ₋₂. Each term is the sum of the two before it, giving 1, 1, 2, 3, 5, 8, 13, 21, and so on. It appears in the branching of plants and the spiral arrangement of seeds.

It is neither arithmetic nor geometric, yet it hides a geometric pattern. The ratio of neighbouring terms, Fₙ₊₁/Fₙ, settles toward the golden ratio, φ = (1 + √5)/2 ≈ 1.618. For large n, the Fibonacci numbers grow almost like a geometric sequence with ratio 1.618.

10. Real-world patterns

Series formulas solve practical problems. Suppose a worker starts at 40,000 dollars and receives a 2,000 dollar raise every year. Her pay forms an arithmetic sequence, and over 10 years she earns a total of 10/2 · (2 · 40000 + 9 · 2000) = 490,000 dollars.

A savings plan is a geometric series. Deposit 100 dollars at the end of each month into an account earning 0.5 percent monthly. The deposit made k months before the end has grown by 1.005ᵏ, so the balance after 12 months is 100(1 + 1.005 + … + 1.005¹¹) = 100 · (1.005¹² − 1)/0.005 ≈ 1,233.56 dollars, slightly more than the 1,200 deposited.

11. AI and machine learning

Machine learning is full of sequences and series. Training itself is a sequence: the weights w₀, w₁, w₂, … are produced by repeated updates. For the loss (w − 3)² + 2 from the quadratics article, with learning rate 0.25, each step halves the distance to the minimum at 3. The weights run 0, 1.5, 2.25, 2.625, …, and the error 3 · (1/2)ᵗ is a geometric sequence with ratio ½.

The step sizes 1.5, 0.75, 0.375, … form a geometric series whose sum is 1.5/(1 − 0.5) = 3, exactly the total distance from the starting weight 0 to the minimum at 3. The infinite series explains why the process ends at a finite place.

Reinforcement learning uses discounted sums. An agent that receives reward r at each future step values them with a discount factor γ between 0 and 1: G = r + γr + γ²r + …. This is geometric with ratio γ, so a steady reward of 1 is worth 1/(1 − γ). With γ = 0.9 that is 10, and with γ = 0.99 it is 100. A discount close to 1 makes the agent look much further ahead.

The exponential moving average used by many optimisers weights the past by (1 − β)βᵏ. Those weights are geometric and sum to exactly 1, so the average stays on the right scale. Series also compute functions: a calculator or library can approximate eˣ by the series 1 + x + x²/2! + x³/3! + …. For x = 1 the partial sums close in on e ≈ 2.71828 quickly.

Finally, nearly every loss is a finite series. The total squared error over a dataset is a sum of N terms, and its average divides by N. Text, audio and sensor streams are sequences, and models for them, from recurrent networks to transformers, are built to process one term in relation to the others.

12. Common mistakes

13. Practice problems

Try these before looking at the answers.

1. Find the 20th term of 4, 7, 10, ….
2. Find the 6th term of 5, 10, 20, ….
3. Find the sum of the first 50 positive integers.
4. Find the sum of the first 6 terms of 2 + 6 + 18 + ….
5. Find the sum of 8 + 4 + 2 + 1 + … (infinitely many terms).
6. Evaluate Σ (k=1 to 4) (3k − 1).
7. Does 5 + 7.5 + 11.25 + … converge? Explain.
8. An agent earns a reward of 1 every step with discount γ = 0.8. What is the total discounted return?

Answers.

1. 4 + 3(19) = 61.
2. 5 · 2⁵ = 160.
3. 50 · 51 / 2 = 1275.
4. 2(3⁶ − 1)/(3 − 1) = 728.
5. 8/(1 − 1/2) = 16.
6. 2 + 5 + 8 + 11 = 26.
7. No. The ratio is r = 1.5, and |r| ≥ 1, so the series diverges.
8. 1/(1 − 0.8) = 5.

14. Conclusion

A sequence is an ordered list, and a series is its sum. Arithmetic sequences add a constant difference and have the nth term a₁ + (n − 1)d; geometric sequences multiply by a constant ratio and have the nth term a₁rⁿ⁻¹. Their sums have clean formulas, and an infinite geometric series converges to a₁/(1 − r) exactly when the ratio is smaller than 1 in size.

In machine learning these ideas describe training sequences that converge, discounted rewards, moving averages and total losses. The next article moves from patterns of numbers to comparisons between them: inequalities.