100% Code Coverage Does Not Mean Your Code Is Tested

The Coverage Theater Problem

Your CI pipeline reports 100% code coverage. Every line, every branch. The coverage badge on the README is green. And yet, a bug that has been in production for two weeks — a discount calculation that rounds the wrong way on amounts over $10,000 — is not caught by any of those tests.

How? Because the test that executes the discount function passes in 100.0 and asserts the output is 90.0. It covers the line. It does not test the behavior at scale.

Code coverage tools — Istanbul, JaCoCo, Coverage.py, simplecov — measure execution, not verification. A line is "covered" if it ran during a test. The test does not have to assert anything meaningful about what that line did.

How 100% Coverage Can Mean Nothing

Here is a concrete demonstration:

def apply_discount(price: float, rate: float) -> float:
    if rate < 0 or rate > 1:
        raise ValueError("Rate must be between 0 and 1")
    discounted = price * (1 - rate)
    return round(discounted, 2)

# This test achieves 100% line coverage
def test_apply_discount():
    result = apply_discount(100.0, 0.1)
    assert result is not None  # ← This assertion is useless

Every line in apply_discount executes. The branch for invalid rates does not execute, but let's say you add another test to cover that. Still 100%. But the actual behavior — what the function returns for various inputs — is never meaningfully verified. The assertion is not None passes for any non-crashing output.

This is not a contrived example. Codebases accumulate tests like this through coverage mandates: teams are required to hit a coverage threshold, so developers write tests designed to satisfy the tool, not to validate behavior.

What Coverage Metrics Are Actually Useful For

Coverage is useful as a floor, not a ceiling. If your coverage is 30%, there are almost certainly whole classes and modules that have never been exercised under any test condition. That is a problem worth knowing about. The coverage report tells you where the obvious gaps are.

What coverage cannot tell you:

• Whether the assertions in your tests are meaningful
• Whether your tests cover realistic input ranges
• Whether edge cases and failure modes are exercised
• Whether the behavior under test is the behavior users actually depend on

Branch coverage (also called condition coverage) is strictly more useful than line coverage. It requires that both the true and false paths of every conditional are executed. Istanbul and JaCoCo both support it. But branch coverage still does not tell you if your assertions are meaningful.

// Branch coverage requires both paths — but look at the assertions
@Test
void testGetUserRole() {
    User admin = new User("alice", true);
    User regular = new User("bob", false);

    // Both branches covered, but assertions tell us nothing useful
    assertNotNull(getRole(admin));
    assertNotNull(getRole(regular));
}

// This is what testing the branches actually looks like
@Test
void testGetUserRoleBehavior() {
    User admin = new User("alice", true);
    User regular = new User("bob", false);

    assertEquals("ADMIN", getRole(admin));
    assertEquals("USER", getRole(regular));
}

The first version hits 100% branch coverage. The second version actually tests the behavior.

Mutation Testing: The Coverage Check for Your Coverage

If you want to know whether your tests are actually verifying behavior, mutation testing is the right tool. Tools like PIT (Java), mutmut (Python), or Stryker (JavaScript/TypeScript) systematically introduce small changes — mutations — into your code: flipping a > to >=, changing a + to -, deleting a return value. Then they run your test suite.

If a test suite with 100% coverage fails to catch these mutations, it means your assertions are not actually validating the logic. A high mutation score means your tests would catch real bugs. A low mutation score means your coverage is decorative.

PIT for a Java project:

<plugin>
    <groupId>org.pitest</groupId>
    <artifactId>pitest-maven</artifactId>
    <configuration>
        <targetClasses>
            <param>com.example.billing.*</param>
        </targetClasses>
        <mutators>
            <mutator>DEFAULTS</mutator>
        </mutators>
        <mutationThreshold>80</mutationThreshold>
    </configuration>
</plugin>

Setting a mutation threshold of 80% means the build fails unless 80% of introduced mutations are caught by the test suite. That is a much more meaningful gate than a coverage percentage.

If you are going to enforce a number in CI, enforce mutation score, not line coverage. Or enforce nothing and just use the coverage report to identify completely untested code — the 0% modules — and address those. Beyond that, the number is noise.

The question that matters is not "did this line run?" It is "would a test fail if this line did something different?"