Cost per million tokens and throughput for Llama 3.3 70B on 8x H100 as the batch fills. A single-tenant node (B=1) runs ~25 tok/s at ~$258/M; a filled batch (B=256) runs ~2,800 tok/s at ~$2.30/M — same silicon, 112x cheaper. Pick a batch; read the tax you pay for the GPU you can't fill.
Verification overhead gets the blame, but the real tax on decentralized LLM inference is the batch a single-tenant node can't fill. On identical 8x H100s, Llama-3.3-70B costs about $258 per million tokens at batch 1 and $2.30 at batch 256 — a 112x spread. Why, and how networks fight it.