software-design/unit-tests-why-four-passing-tests-don-t-prove-a-function-works.md

Unit tests: why four passing tests don't prove a function works

Building software · 4 min read ·

Your function gave the right answer four times. Is it correct? Not necessarily: passing tests only show the bug isn't where you looked.

Using a leap-year function that looks right, this article explains what a unit test is, why edge cases catch what typical cases miss, and how to turn every bug into a test that guards it forever. Edit the code and run the tests live in the page.

Dark card titled Unit tests with three green PASS boxes for 2024, 2023, 2028 and one orange FAIL box for 1900

You wrote a function and tried it three times. It gave the right answer every time. Is it correct? Not necessarily, and that gap between "worked when I tried it" and "is correct" is exactly why programmers write tests.

What a test is

A test is a tiny program that calls your code with a chosen input and checks that the result matches the answer you expected. A test that checks one small piece (usually one function) is called a unit test. If the result matches, the test passes. If not, it fails, and you have found a bug before a user did.

Real projects use a test framework (Jest or Vitest for JavaScript, pytest for Python), but the idea is no more than this: input in, compare output with expected.

A function that looks right

Let's write isLeapYear(year). Most people remember "a leap year is divisible by 4", so:

function isLeapYear(year) {
  return year % 4 === 0;
}

(% is the remainder operator: 2024 % 4 is 0 because 4 divides 2024 with nothing left over.) Now try it on four years:

console.log(isLeapYear(2024), isLeapYear(2023), isLeapYear(2028), isLeapYear(2025));
// true false true false

All four are right. If you stopped here you would ship it. But the real rule of the Gregorian calendar (the calendar most of the world uses) has two more clauses: a year divisible by 100 is not a leap year, unless it is also divisible by 400. So 1900 was not a leap year, but 2000 was.

console.log(isLeapYear(1900), isLeapYear(2000));
// true true     <- 1900 should be false

Four passing tests did not prove the function correct; they only failed to find the bug, because every one of those years happens to be an ordinary case. That is the most important lesson about testing: passing tests show the bug is not in the places you looked.

Try it: break it, then fix it

The box below holds the function and a list of tests. Press Run tests: all pass. Then use Add a test to try 1900 and see the failure. Edit the code to fix it (or press Load the fix) and run again. Your own code runs right here in the page; nothing is sent anywhere.

The fix, and checking it

The corrected function handles the exceptions first, from the most specific to the least specific:

function isLeapYear(year) {
  if (year % 400 === 0) return true;
  if (year % 100 === 0) return false;
  return year % 4 === 0;
}
console.log(isLeapYear(1900), isLeapYear(2000), isLeapYear(2100), isLeapYear(1600));
// false true false true

As an extra check, I compared both versions against JavaScript's own date handling (asking whether February 29 exists) for every year from 1583 to 3000. The fixed function agreed every time. The naive version was wrong for 11 years in that range: 1700, 1800, 1900, 2100, 2200, 2300, 2500, 2600, 2700, 2900 and 3000.

How to choose good tests

You cannot test every possible input, so you pick inputs on purpose. Three habits cover most of the value:

KindQuestion it asksExample here
Typical caseDoes the normal path work?2024, 2023
Edge caseWhat happens at the boundary of a rule?1900, 2000, 2100
Past bugWill this mistake come back?keep 1900 forever

The edge cases are where bugs live, because rules change there. Whenever you find a bug, first write a test that fails because of it, then fix the code until it passes. The test stays in your project and guards that bug for good.

What a test cannot do

A test only checks what you thought to ask, and it trusts the expected answer you typed. If you believed 1900 was a leap year, you would have written a test that agrees with the bug. So look up the rule from a trustworthy source first, then write the test. Tests make you far more confident, but they are evidence, not proof.

The next time you write a function, try the two-minute version of this habit: call it with one normal input, then with the weirdest input you can think of. The second call is the one that teaches you something.

$ ls software-design/

see all →
software-design/the-observer-pattern-how-a-shop-tells-many-functions-about-one-order-w.md The observer pattern: how a shop tells many functions about one order without knowing them A shop that must email a receipt, update stock and add loyalty points ends up with one function that knows about everything. Every new… 5 min read · 27 Oct software-design/refactoring-how-to-tidy-working-code-without-breaking-it.md Refactoring: how to tidy working code without breaking it Refactoring means changing how code is built on the inside without changing what it does. It sounds risky, but there is a simple routine… 6 min read · 24 Oct software-design/idempotency-why-clicking-pay-twice-should-charge-you-once.md Idempotency: why clicking Pay twice should charge you once You tap Pay, the screen shows an error, you tap again, and your card is charged twice. Networks drop messages, so a phone often cannot tell… 6 min read · 18 Oct software-design/state-machines-why-an-order-can-t-be-delivered-before-it-s-paid.md State machines: why an order can't be delivered before it's paid An online order can be in the cart, paid, shipped, delivered or cancelled, but what stops the code from delivering an order nobody paid… 4 min read · 15 Oct software-design/dependency-injection-why-a-test-passes-at-9am-and-fails-at-3pm.md Dependency injection: why a test passes at 9am and fails at 3pm A test that passes in the morning and fails in the afternoon, with no change to the code, is a puzzle many beginners meet. The cause is… 5 min read · 12 Oct software-design/pure-functions-why-the-same-call-can-give-two-different-answers.md Pure functions: why the same call can give two different answers Call a function twice with the same input and get two different answers? Python isn't broken. The function is doing something extra, called… 5 min read · 9 Oct