Writing Code That Works Is the Easy Part

by Eric Hanson, Backend Developer at Clean Systems Consulting

The Part They Don't Warn You About

You spent months learning your language. You practiced algorithms, studied design patterns, worked through tutorials until you could build something from scratch. Then you got a job and discovered that writing code that works — actually works, passes CI, ships to production — is the straightforward part. What nobody prepared you for is everything else.

This isn't a complaint about the industry being hard. It's a reframe that changes what you spend your learning budget on. If you're still optimizing primarily for "how do I write better code," you're working on a relatively solved problem. The leverage is elsewhere.

What "Working" Actually Means at Scale

A function that returns the right value in unit tests is working. A service that handles 50 requests per second during business hours, degrades gracefully when a downstream dependency is slow, recovers cleanly after a deploy, and doesn't silently corrupt data when it receives a malformed payload — that is a different kind of working.

The gap between those two things is filled with concerns that have nothing to do with whether your algorithm is correct:

Observability: When something behaves unexpectedly at 3am, can you tell what happened? Structured logs with correlation IDs, meaningful metric names, and distributed traces are not features you add later. They are part of the work.

Operability: Can someone who didn't write this code deploy it, roll it back, and investigate an incident without reading your Slack messages? This is a design constraint, not a documentation task.

Graceful degradation: What does your service do when the database connection pool is exhausted? When a third-party API returns 503 for thirty seconds? "Throw an exception and hope" is a choice. A circuit breaker with a fallback is also a choice. These are not equivalent.

The Invisible Work That Holds Systems Together

Here is a partial list of things that matter more than code quality once a system reaches production:

  • The data model: Wrong abstractions in your schema will haunt every feature for the lifetime of the product. Bad column names, missing indexes, denormalization that made sense then but doesn't now — these cost more than bad code, because they're harder to refactor.
  • The deployment pipeline: Code that cannot be deployed safely and quickly is code that creates risk. Feature flags, blue-green deployments, automated rollback triggers — these are the difference between "we can ship ten times a day" and "we have a two-hour deploy window on Friday nights."
  • The contracts between services: An API is a promise. A message schema is a promise. Breaking these without coordination creates cascading incidents. Versioning, deprecation policies, and consumer-driven contract tests (Pact is the standard tool here) are how you avoid this.
  • The on-call experience: If your service pages someone every other night, it doesn't matter how elegant the code is. Noisy alerts, missing runbooks, and opaque error states are engineering failures as much as any bug.

The Skill Distribution Nobody Talks About

Most engineers invest heavily in the skills that got them hired — language proficiency, algorithmic thinking, framework knowledge. These plateau. After a certain point, writing better Kotlin or knowing more Spring Boot APIs doesn't dramatically change your impact.

The skills with compounding returns — the ones that separate engineers who run systems from engineers who just write code — are:

  • Incident analysis and post-mortem thinking
  • Schema and data model design
  • API design and versioning strategy
  • System decomposition: knowing where to draw service boundaries
  • Production debugging without a local reproducer

None of these appear prominently in most interview processes. All of them matter enormously in the day-to-day of a senior engineer.

A Concrete Example

Consider a service that processes webhook events from a payment provider. Writing the handler function is trivial. The hard problems are:

1. Idempotency — the provider may send the same event twice. Do you
   deduplicate? How? Using a database unique constraint on event_id is
   reliable. Using an in-memory set is not.

2. Ordering — events may arrive out of order. Does your processing
   logic assume sequence? What happens if a refund event arrives before
   the charge event it refunds?

3. Poison pills — a malformed payload causes your handler to throw on
   every retry, blocking the queue. Do you have a dead-letter queue?
   Does it alert? Does anyone know how to reprocess it?

4. Backpressure — the provider sends a burst of 10,000 events during
   a batch job. Does your service fall over, or does it slow down
   gracefully and catch up?

None of these problems are about whether your code is clean. They're about whether you asked the right questions before you wrote it.

The Practical Takeaway

Pick the next service or feature you're building and spend thirty minutes before writing any code answering: how will I know when this is broken? How will someone debug it without me? What happens to data in flight if this process crashes mid-operation?

Write those answers down. Then build to satisfy them. That discipline — not cleaner code — is what separates the work that just works from the work that lasts.

Scale Your Backend - Need an Experienced Backend Developer?

We provide backend engineers who join your team as contractors to help build, improve, and scale your backend systems.

We focus on clean backend design, clear documentation, and systems that remain reliable as products grow. Our goal is to strengthen your team and deliver backend systems that are easy to operate and maintain.

We work from our own development environments and support teams across US, EU, and APAC timezones. Our workflow emphasizes documentation and asynchronous collaboration to keep development efficient and focused.

  • Production Backend Experience. Experience building and maintaining backend systems, APIs, and databases used in production.
  • Scalable Architecture. Design backend systems that stay reliable as your product and traffic grow.
  • Contractor Friendly. Flexible engagement for short projects, long-term support, or extra help during releases.
  • Focus on Backend Reliability. Improve API performance, database stability, and overall backend reliability.
  • Documentation-Driven Development. Development guided by clear documentation so teams stay aligned and work efficiently.
  • Domain-Driven Design. Design backend systems around real business processes and product needs.

Tell us about your project

Our offices

  • Copenhagen
    1 Carlsberg Gate
    1260, København, Denmark
  • Magelang
    12 Jalan Bligo
    56485, Magelang, Indonesia

More articles

The Cost of Interruptions in Remote Software Development

Every ping, message, or unexpected call might feel small—but in reality, interruptions quietly destroy focus and slow progress. Remote developers face more than just deadlines; they battle a constant stream of distractions.

Read more

How to Recover a Branch You Accidentally Deleted

Deleting a branch does not delete its commits. Git keeps everything in the object store until garbage collection runs — and reflog tells you exactly where to find it.

Read more

SSRF, Path Traversal, and Other Spring Boot Vulnerabilities That Don't Get Enough Attention

SQL injection and XSS get attention. SSRF, path traversal, ReDoS, XXE, and deserialization vulnerabilities are less discussed but appear regularly in penetration tests and bug bounty reports. Here is how each manifests in Spring Boot and how to prevent it.

Read more

When Architecture Decisions Get Messy Because Nobody Oversees Them

Without someone guiding architectural choices, small decisions pile up and create chaos. Messy systems grow quietly until they become a nightmare to maintain.

Read more