The Unit Test That Passes Locally and Fails in CI Is a Design Problem

The Most Demoralizing Test Failure

The test passes on your machine. It passes on your colleague's machine. It fails in CI every third run. You spend 45 minutes adding debug logging and re-running it locally. It passes every time. You merge with a "should be fine" comment and watch it fail in CI again.

This experience is so common it has a name — "works on my machine" — and it is usually dismissed as an environment problem. Install the right version of the runtime. Fix the CI container configuration. Set the right environment variables.

Sometimes that is the fix. But the underlying cause is almost always a design problem in the test or the code under test: a hidden dependency on something in the environment that is different between local and CI.

The Five Hidden Dependencies That Cause This

1. System time. Code that calls new Date(), DateTime.now(), or time.time() inside a function under test will behave differently depending on when the test runs. Tests that assert on current time, check whether something is expired, or calculate durations relative to "now" will pass at certain times of day or day of month and fail at others.

The fix is always the same: inject time as a parameter or through a clock interface. Never call a system time function inside logic you want to test.

// Depends on system time — passes Monday, fails Sunday
func IsWeekend() bool {
    return time.Now().Weekday() == time.Saturday ||
           time.Now().Weekday() == time.Sunday
}

// Clock interface — fully testable
type Clock interface {
    Now() time.Time
}

func IsWeekend(clock Clock) bool {
    day := clock.Now().Weekday()
    return day == time.Saturday || day == time.Sunday
}

// In tests, inject a fixed time
type FixedClock struct{ t time.Time }
func (c FixedClock) Now() time.Time { return c.t }

func TestIsWeekend(t *testing.T) {
    saturday := time.Date(2024, 1, 6, 12, 0, 0, 0, time.UTC) // Known Saturday
    assert.True(t, IsWeekend(FixedClock{t: saturday}))
}

2. Environment variables. Code that reads os.getenv(), System.getenv(), or process.env directly inside functions under test will behave differently in CI if those variables are not set the same way. The fix is to read environment variables at startup and inject the values through configuration objects, not to call getenv deep inside business logic.

3. File system state. Tests that depend on files existing in specific paths, temporary directories with predictable names, or working directories relative to the project root will fail in CI if the container mounts the workspace differently. Use absolute paths derived from the test file's location, or use in-memory filesystems for tests (e.g., afero in Go, pyfakefs in Python).

4. Ordering and parallelism. Tests that depend on execution order — setting state in one test that another test reads — will pass when run sequentially in the order they were written and fail when CI runs them in parallel or alphabetical order. Every test must set up its own state and tear it down. Shared mutable state between tests is the cause, not the runner configuration.

5. Randomness. Code that uses random numbers, random shuffles, or non-deterministic iteration (iterating over a HashMap in Java, iterating over a dict in Python before 3.7) will produce non-deterministic test results. Seed your random number generator in tests, or inject a random source.

The Design Implication

Each of these hidden dependencies points to the same design issue: the code under test has an implicit dependency on the environment that was not made explicit in the function signature. If the function needs to know the current time, it should accept a clock. If it needs an API key, it should accept a configuration struct. If it needs the filesystem, it should accept a filesystem abstraction.

When dependencies are explicit — passed in, not fetched from global state — tests can control them precisely. The test is deterministic because the test controls all the inputs. The test that passes locally but fails in CI is a test whose inputs are not fully under control.

The fix is not to add environment variables to the CI configuration. The fix is to remove the implicit dependency from the code and make it explicit. That change makes the code more testable everywhere — and also makes it more understandable, because the function's dependencies are visible in its signature rather than buried in its implementation.

When CI fails and local passes, treat it as a code review note, not a CI configuration ticket.