OOM Diagnostic Decision Tree: Drill Down from pprof

Start with the cheapest tool. Only drill to runtime → cgroup when pprof (Go’s built-in heap profiler, go tool pprof) can’t explain it.

The Decision Tree

Three Layers

1. App layer (pprof): inuse_space identifies live heap hogs by function. If small, alloc_space catches high-churn functions (rapid alloc → GC → alloc that inuse misses at sample time).
2. Runtime layer (Go metrics): heap_inuse − heap_alloc = fragmentation. heap_idle − heap_released = pages the allocator holds but hasn’t returned to the kernel (memory three-layer model, allocator internals).
3. Kernel layer (cgroup): container_memory_usage_bytes − container_memory_rss = kernel-charged memory. A large gap means unclosed sockets, heavy file I/O, or kernel buffer bloat.

Role in This OOM

The culprit here wasn’t the Go heap. It was the browser sidecar’s memory. This decision tree helps rule out “Go’s own problems” and confirm the issue is outside the process (shared resource blast radius).

The order of diagnostic tools is the order of their cost: pprof (free) → runtime (low) → cgroup (needs Prometheus).

References:

• https://go.dev/doc/gc-guide
• https://go.dev/blog/pprof

Related: see memory three-layer model, allocator internals, and shared resource blast radius, or go back to the series overview.