Engineering Sep 13, 2026 · 9 min read

Image Format Benchmark: 5,625 Runs Later, WebP Wins

We benchmarked 75 production images across WebP and AVIF with 6,075 automated runs. WebP won on decode speed, coverage, and time-to-paint. This is the overview and the verdict.

The goal: faster paint, not smaller files

Every millisecond between request and paint is a millisecond the user stares at nothing. On a mid-tier phone over slow 3G, that gap decides whether the image shows at all.

We wanted to answer one question. Can a format change make the serving experience measurably better? Not “can we save bytes.” Can the user see the image sooner, on more devices.

When the image reaches the screen faster, three things improve:

The image paints earlier. The user sees it sooner. In ad serving, the ad renders before the user scrolls past it.
Slow networks stop timing out. A 900 ms download on slow 3G becomes 530 ms. The image arrives in time.
Weak CPUs keep up. A phone in low-battery mode throttles its CPU. A lighter decode means the image still finishes before the next frame.

CDN cost also drops, but that is a side effect, not the goal.

The formats: what we serve today vs what exists

Today we serve three formats: JPEG (photos), GIF (animation), PNG (transparency). Two modern alternatives exist:

WebP (Google, 2010). Lossy and lossless modes. Animated WebP replaces GIF. Browser support: 97% globally.
AVIF (AOM, 2019). Based on the AV1 video codec. Higher compression ceiling than WebP. Browser support: 95%.

The tradeoff is not just “which compresses more.” Three axes matter for serving:

WebP wins on three of three axes. AVIF wins only on compression.

Terms you will see in this series

This series uses metrics from web performance and image encoding. Each term appears again with an inline definition at first use in every part. This table is a reference card.

Term	Plain words
LCP	Largest Contentful Paint. The moment the user sees the image. The main speed metric of this series.
Decode time	CPU work to turn downloaded bytes into pixels. Heavier compression costs more decode work.
SSIM	Structural Similarity, 0 to 1. How close the converted image looks to the original. Above 0.95, most people see no difference.
TTFB	Time To First Byte. How fast the server starts sending. The image format has no effect on this metric.
CPU throttle	Chrome DevTools can slow the CPU by N times to simulate a weaker phone. 1x = desktop. 4x = mid-tier. 6x = low-end.
Cold / Warm	Does the CDN already hold the file? Warm = fast. Cold = first load, slower.
q (quality)	The encoder quality knob, 0 to 100. Higher keeps more detail and makes a larger file.
CDP	Chrome DevTools Protocol. The programmatic interface we used to control Chrome.

The experiment machine: how CDP automated 6,075 runs

Running 75 images times 3 formats times 3 CPU tiers times 5 repeats by hand is 3,375 clicks. We built a pipeline instead.

The core idea: every run follows the same steps, and CDP makes those steps programmable. One test page loads one image, collects the metrics, and exits. Playwright (headless Chrome) drives that page thousands of times with CDP controlling four variables:

CPU throttle via Emulation.setCPUThrottlingRate. Three tiers: 1x (desktop), 4x (mid-tier phone), 6x (low-end phone).
Network emulation via Network.emulateNetworkConditions. Two profiles: fast 4G (9 Mbps, 60 ms) and slow 3G (1.6 Mbps, 300 ms).
Fresh incognito context per run. This gives cache isolation without restarting the browser.
Metric collection from four browser APIs. Element Timing gives load and render timestamps. Resource Timing gives download duration. The LCP observer gives time-to-paint. img.decode() gives the isolated CPU decode cost.

The same test page served all three experiments. Only the controlled variable changed:

CDP turned a manual spot-check into 6,075 measured runs from one machine.

Experiment A answers: “What is the pure CPU cost of each format?” It runs on localhost, so download time is near zero.

Experiment A2 answers: “On a constrained network, does the byte savings outweigh decode cost?” It adds network throttle on localhost.

Experiment B answers: “Through the real CDN, which format paints fastest?” It runs on the production path with cold/warm cache classification per run.

Statistical validation uses the paired Wilcoxon signed-rank test across all 75 images per condition. This is a non-parametric test. It does not assume the data follows a normal distribution.

Key results (quality q80, median values)

75 production images: 25 JPEG, 25 GIF, 25 PNG. All files come from the production environment. Pixel sizes range from 200x200 to 1200x627.

Overall compression

Source	WebP savings (p50)	WebP SSIM	AVIF savings (p50)	AVIF SSIM
JPEG	47.1%	0.979	38.2%	0.986
GIF	51.0%	0.980	54.0%	0.994
PNG	82.9%	0.982	81.0%	0.992

WebP compresses JPEG and PNG more than AVIF at the same quality setting. AVIF compresses GIF slightly more. The difference is 3 percentage points.

Decode time (CPU cost to turn bytes into pixels)

CPU tier	Original (p50)	WebP (p50)	AVIF (p50)
Desktop (1x)	1.2 ms	2.2 ms	7.6 ms
Mid-tier phone (4x)	1.6 ms	2.5 ms	3.9 ms
Low-end phone (6x)	2.3 ms	3.4 ms	31.9 ms

WebP adds 1 ms of decode cost on a low-end phone. AVIF adds 29.6 ms. That gap matters on real devices. A phone in low-battery mode throttles its CPU the same way. Every extra millisecond of decode eats into the time the user waits.

Combined: download + decode on slow 3G, mid-tier phone

Format	Min (ms)	Median (ms)	Max (ms)
Original	438	939	6,777
WebP	410	534	1,466
AVIF	413	547	1,406

The median image paints 405 ms earlier with WebP. That is 43% faster. AVIF is close to WebP here because the smaller file offsets the heavier decode. The max column shows the worst case per format. WebP and AVIF are similar, but the median favors WebP.

Deeper breakdowns (per source format, per CPU tier, per network profile) are in Part 02: WebP vs AVIF.

The verdict

Replace all GIF with animated WebP. Convert JPEG and PNG to WebP as a second batch. Ship AVIF later, behind a <picture> fallback.

Conversion	Recommendation	Basis
GIF to animated WebP	Replace all	51% smaller, animation preserved, 25/25 integrity pass
JPEG to WebP	Second batch	47% smaller, 0.979 SSIM, encode 24x faster than AVIF
PNG to WebP	Second batch	83% smaller, 0.982 SSIM
AVIF	Next milestone (M2)	Higher compression but 32 ms decode on low-end, 95% coverage
Quality setting	q80 for JPEG/GIF, q95 for PNG	Lossy sources hold limited detail. See Part 03.

What to take away

Benchmark your own traffic. Published comparisons use synthetic test images. Your production images have a different size distribution and device profile. Test with your own data.
Decode cost matters as much as file size. AVIF compresses more. Its decode time on low-end devices (31.9 ms at 6x throttle) can eat the download savings. The format with the best compression is not always the format with the fastest paint.
CDP turns spot-checks into thousands of runs. One Playwright script and one test page produced 6,075 data points. Without CDP automation, this benchmark takes weeks of manual work.
If the format lacks full browser support, direct replacement breaks some users. WebP support is 97%, so 3% of browsers show a broken image. Read your traffic data first. The <picture> element closes this gap. That is why AVIF (95%) sits in M2.