You Don't Need a Complex Pipeline to Start. You Need a Working One.

The Pipeline That Does Too Much and Catches Too Little

You've seen this pipeline. It has 14 stages. There's a separate job for linting, one for unit tests, one for integration tests, one for security scanning, one for container building, one for pushing to ECR, two for staging deployments (blue and green), a manual approval gate, and a production deployment that runs a shell script someone wrote in 2021 that nobody fully understands anymore. It takes 55 minutes. It flakes about twice a week. When it fails, diagnosing which stage actually matters takes longer than fixing the underlying problem.

This pipeline was built by ambitious engineers who read all the right articles. It is genuinely useless as a safety net because nobody trusts it, nobody understands it end to end, and the cognitive cost of maintaining it exceeds the cognitive cost of just being careful at deploy time.

What a Working Pipeline Actually Requires

A working pipeline has one property that overrides everything else: the team trusts it. Trust means that when it's green, developers believe the code is safe to ship. When it's red, they believe the failure is real, not a flake or an environment problem.

Building to that trust level requires starting much smaller than feels comfortable.

For a backend service, a pipeline that earns trust might look like this:

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up JDK 21
        uses: actions/setup-java@v4
        with:
          java-version: '21'
          distribution: 'temurin'
          cache: 'gradle'

      - name: Build and run tests
        run: ./gradlew build

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: test-results
          path: build/reports/tests/

That's it to start. One job. Runs on every push and PR. Compiles, runs all tests, uploads results. Under 5 minutes on typical hardware for most Spring Boot services. The Gradle build cache (cache: 'gradle') gives you free incremental builds.

Is this pipeline missing things? Yes. Security scanning. Container builds. Deployment. A lot. But it runs reliably, developers see the results, and they can trust the green checkmark to mean something. That is the foundation everything else gets added on top of.

The Cost of Complexity You Haven't Earned

Complexity has a carrying cost that compounds. Every stage added to a pipeline needs to be maintained when underlying tools change versions, when the environment image gets updated, when a new developer joins and has to understand what all of it does, and when it breaks at 11pm on a Tuesday.

Teams that add stages before they've validated that each stage provides real signal end up with pipelines full of checks that never fail — which means they either never catch anything (the check was unnecessary) or they've trained the team to ignore failures (the check is always flaky). Neither outcome is useful.

The test of whether a stage belongs in your pipeline: has it caught a real problem in production that would have shipped without it? If you can't point to a specific incident, the stage is hypothetically useful, not actually useful. Hypothetically useful stages are the primary ingredient in 55-minute pipelines.

How to Grow the Pipeline Correctly

Start with build and unit tests. Get that reliable and fast. Then add stages in response to actual pain:

• Had a production incident caused by a dependency vulnerability? Add Trivy or Grype for container scanning.
• Had a production regression that integration tests would have caught? Add the integration tests.
• Had a bad deploy because the container image wasn't built correctly? Add the container build step.

Each addition is justified by a real failure mode, not a best practice checklist. This sounds slower than building the "ideal" pipeline upfront, but it produces a pipeline where every stage has a known, named purpose — and where the team knows exactly why each check exists.

The Upgrade Path

Once your simple pipeline is trusted and stable, add complexity in isolated, reviewable increments. One new stage per sprint, not a pipeline rewrite. Review the build times before and after. Track the flake rate. If a new stage consistently flakes more than once per 20 runs, it's not ready — fix the test environment stability before relying on that check as a gate.

The pipeline that serves your team in 18 months looks nothing like your pipeline today. But it has to be built through iterations of a working pipeline, not by designing the final state upfront and hoping it holds together.

Start simple. Ship it. Trust it. Grow it from there.