Allocator Internals: Size Class, Fragmentation, Scavenger

Size classes balance speed against waste. How the Go allocator slices pages into slots, where fragmentation comes from, and why RSS lags behind pprof.

Size classes balance speed against waste. Allocation is fast, but the slot isn’t always a perfect fit.

Four-Level Allocation Ladder

mcache (per-P): each P holds a local cache of spans per size class. Most allocations hit this path: no lock, no syscall (GMP scheduler).
Size classes (8B, 16B, 24B, 32B…): a 10-byte object goes into the 16B class. Same-sized objects in the same span → vacated slots are immediately reusable by same-class allocations.
Bitmap free-slot search: each span tracks slot availability as a bitmap. Finding a free slot = finding the lowest set bit via bitwise ops, O(1) per word.

Fragmentation: Internal and External

Internal: size-class rounding (17B → 24B). Go’s 67 size classes keep waste under ~12%. Struct field order matters too: {int8, int64, int8} = 24B, {int64, int8, int8} = 16B.
External: scattered live objects pin entire spans. Free space exists but not as reclaimable contiguous pages. Go uses non-move GC: lower CPU cost, but no compaction.

Scavenger and madvise

A background scavenger periodically returns idle heap pages to the kernel via madvise. MADV_DONTNEED (Go ≥1.16 default) = immediate return, RSS drops, but reuse triggers a page fault. MADV_FREE (Go <1.16) = lazy reclaim, RSS looks stuck.
Go 1.16 switched back to MADV_DONTNEED because too many “why doesn’t RSS drop” false alarms.

Role in This OOM

This OOM wasn’t caused by allocator fragmentation; it was browser memory. But understanding the allocator explains why “pprof inuse dropped but RSS didn’t”. The allocator holds memory it hasn’t returned to the kernel yet (memory three-layer model).

The allocator is the middleman between kernel and heap. Without understanding it, the gap between RSS and pprof never adds up.

References:

Related: see memory three-layer model and OOM diagnostic decision tree, or go back to the series overview.

Tags #go #concurrency #reliability