Skip to content
All writing ‹ Part 03 of 18 · A Cache Is a Chain, Not a Box
Engineering · 5 min read

Load Balancer: One Entrance, Many Backends by Rule

A load balancer is the single entrance to the origin. Longest prefix match versus priority order, and why a cache can hide inside the load balancer itself.

A load balancer is the single entrance to the origin. It reads the host, path, and headers, then picks a backend.

I used to think the origin was “one server.” Later I learned the origin is often a load balancer with several kinds of backends behind it.

shop.example.com(HTTP load balancer)reads host / path / header /images/*storage(returns image files only) /api/*application(can rewrite content) otherdefault backend
One load balancer reads each request and sends it to a different backend by rule.
  • HTTP load balancer (layer 7): it understands URLs and headers, so it can route by path. A layer 4 load balancer sees only IP and port.
  • A backend can be storage or a group of programs. Every cloud platform has both kinds of backend, but the names differ.

Two Ways to Match

Longest prefix match: the longer prefix wins, order does not matter /images/*image storage /images/icons/*icon storage GET /images/icons/cart.pngicon storage(same result in either order) Priority order (first match): compare from rule 1, stop at the first match 1header beta: 1beta service 2/images/*image storage GET /images/a.png + beta: 1beta service(rule 1 matches first) swap 1 and 2 → the same request goes to image storage
With longest prefix the order does not matter, but with priority order a swap sends the same request elsewhere.
  • The location prefix in nginx is the first kind. When a rule must read a header, you can usually use only the second kind, and then order matters.
  • Our platform allows only one way to match in one set. One of our rules reads a header, so all rules must be ordered. If the order is wrong, some paths cannot reach the right backend (How the Load Balancer Picks a Backend).

The Cache May Live Inside the Load Balancer

  • Some platforms make the cache a switch on each backend of the load balancer. Each backend has its own cache rules.
  • Then, under the same load balancer, some backends have a cache and some do not. The rules also differ (A Bypass Header Works Only Where a Rule Reads It).

A load balancer does more than route. It can also be another cache layer that you did not notice.

Sources: nginx: location

Related: The Path of One Request, How the Load Balancer Picks a Backend, A Bypass Header Works Only Where a Rule Reads It. Or go back to the series overview.

Tags #cdn-cache-chain #cdn #http-caching
↑↓ navigate ↵ open esc close