You're Fetching More Data Than You Need and It's Slowing You Down

The invisible tax on every request

Your API endpoint responds in 400ms. You profile it and find the database query takes 80ms. The rest of the time is spent serializing a 2MB payload that the frontend uses to display a 12-row table with three columns. The query itself is fast. The problem is that you're fetching the entire products table — all 40 columns including description, raw_import_data JSONB, and legacy_notes TEXT — and discarding 37 of those columns before the response is sent.

Over-fetching is the dominant form of database waste in CRUD-heavy applications. It's not a query plan problem. It's a "what did you ask for" problem. And it compounds: more data means more memory pressure, more serialization time, more network bytes, and larger caches that evict faster.

The SELECT * habit

SELECT * is fine for exploratory queries in a database console. It is not fine in application code.

-- What the ORM generates by default
SELECT * FROM products WHERE category_id = 5;

-- What you almost certainly need
SELECT id, name, price, stock_quantity FROM products WHERE category_id = 5;

The cost difference depends on your column set. If products has a description TEXT column averaging 2KB and you're fetching 500 rows per page, that's 1MB of text you're discarding. If there's a metadata JSONB column with nested attributes for analytics, the bloat is worse. And the database has to read all of that from disk or buffer cache regardless.

Most ORMs offer projection support. Use it:

// JPA / Spring Data — explicit projection interface
public interface ProductSummary {
    Long getId();
    String getName();
    BigDecimal getPrice();
}

List<ProductSummary> findByCategoryId(Long categoryId);

# Django — values() for column selection
Product.objects.filter(category_id=5).values('id', 'name', 'price')

The friction of writing explicit projections is real. The cost of not doing it is also real, just invisible until traffic grows.

Unbounded result sets

-- In a background job processing "all pending items"
SELECT * FROM events WHERE processed = false;

If events has 2 million unprocessed rows, this query returns all 2 million to the application layer at once. Memory usage spikes. The database holds a cursor open for the duration. Network transfer takes seconds. If the application process crashes mid-processing, you've made no progress and have to restart.

Batch it:

-- Process in chunks of 1000
SELECT id, payload FROM events
WHERE processed = false
ORDER BY id
LIMIT 1000;

-- After processing, advance the cursor
SELECT id, payload FROM events
WHERE processed = false AND id > :last_processed_id
ORDER BY id
LIMIT 1000;

This is keyset pagination applied to batch jobs. It keeps memory bounded, allows restartability, and lets you control throughput.

N+1 queries: fetching related data one at a time

This is the most widely documented over-fetching pattern, and it's still prevalent:

# Fetch 100 orders, then fetch the user for each one
orders = Order.objects.filter(status='pending')[:100]
for order in orders:
    print(order.user.name)  # N additional queries

This executes 101 queries: one for the orders, and one per order for the user. With 100 orders, the database is hit 101 times. With 1,000 orders, 1,001 times.

The fix is eager loading:

# Django: select_related fetches users in a single JOIN query
orders = Order.objects.filter(status='pending').select_related('user')[:100]

// JPA: JOIN FETCH
@Query("SELECT o FROM Order o JOIN FETCH o.user WHERE o.status = 'PENDING'")
List<Order> findPendingOrdersWithUsers(Pageable pageable);

The distinction between select_related (JOIN, good for single related objects) and prefetch_related (separate query + in-memory join, good for many-to-many) in Django maps to a real execution difference — choose based on cardinality.

Fetching for validation when existence is enough

-- You just want to know if the record exists
SELECT * FROM subscriptions WHERE user_id = 42 AND status = 'active';

If you're using the result for an existence check, fetch a constant:

SELECT 1 FROM subscriptions WHERE user_id = 42 AND status = 'active' LIMIT 1;

With a covering index on (user_id, status), this is an index-only scan that never touches the heap. The difference matters in hot code paths called thousands of times per second.

Pagination defaults that are too large

Default page sizes of 100 or 500 rows were set by someone who didn't think about them. For a dashboard showing a table, 20–50 rows is the practical maximum. For an API consumed by mobile clients, even 20 is often too many.

Set conservative defaults and let clients request more explicitly with a hard cap:

-- Server enforces maximum regardless of what client requests
SELECT id, name, price
FROM products
WHERE category_id = :category_id
ORDER BY name
LIMIT LEAST(:requested_limit, 50)
OFFSET :offset;

The immediate action

Run your three highest-traffic queries through EXPLAIN ANALYZE and check the width field in the output (PostgreSQL). This tells you the average row size in bytes being processed at each stage. Then check what your application actually uses from those rows. The gap between those two numbers is your over-fetching tax. Eliminate it by projecting only the columns you need, and if the width is still large, investigate whether your column types are appropriately sized for the data they store.