Skip to content
All writing ‹ Part 18 of 18 · A Cache Is a Chain, Not a Box
Engineering · 4 min read

What I Learned: A Cache Is a Chain, Not a Box

A cache is a chain, not a box. Three lessons from chasing a CDN cache loop, and three questions to ask at every layer a request passes through.

In interviews I only said “add a CDN layer and it gets faster.” After this chase, I can read every cache layer that a request passes.

Before:clientCDNserver Now:clientedgetiered cacheload balancerLB cachestorage /compute inside the CDN our own origin Each layer: its own TTL, which headers it reads, whether a purge reaches it
The one CDN box I used to draw is a chain of layers, and each layer has its own TTL, headers, and purge reach.

Three Things I Understand Now

  1. Every component has its own cache setting: the TTL passes from storage to the client and can only go down on the way. If IaC does not say it, it does not mean there is no setting. The defaults are always there (TTL Is Set in Three Places).
  2. A tiered cache and an LB cache are two different things: a tiered cache is the second layer inside the CDN. The LB cache sits on our own origin. The CDN cannot skip it, and a purge cannot reach it (CDN Terms on One Page, Two Cache Layers: What I Believed).
  3. A custom header works only where a rule reads it: X-Cache-Bypass and the proxy header are not HTTP standards. A component with no rule to read them simply ignores them (Cache Headers: Who Sets, Who Reads, A Bypass Header Works Only Where a Rule Reads It).

What This Chase Brought

  • I turned the findings into one direction: lower the cost and raise the serving performance at the same time.
  • I proposed this direction to the team. With fewer duplicate caches and failed refreshes, the origin gets fewer requests, and users load pages faster.
  • Lower cost and faster loads help the team and the revenue of the company.

The biggest gain was not finding one bug. It was learning to follow one request along the whole path (Five Parts Make One Loop). At each layer, I ask three questions:

  1. Is there a cache here?
  2. Which headers does it read?
  3. Can a purge reach it?

A cache is not a box. It is a chain, and each link has its own rules.

Related: CDN Terms on One Page, Cache Headers: Who Sets, Who Reads, Two Cache Layers: What I Believed, TTL Is Set in Three Places, A Bypass Header Works Only Where a Rule Reads It, Five Parts Make One Loop. Or go back to the series overview.

Tags #cdn-cache-chain #cdn #http-caching
↑↓ navigate ↵ open esc close