vLLM’s Disaggregated Serving Cuts GPU Interference, Delivering 2.5x Higher Goodput on the Same Hardware
The Problem: Prefill and Decode Fighting Over the Same GPUs Standard LLM inference collocates two fundamentally different workloads on the same GPU resources. Prefill is compute-bound—it processes the entire input prompt in parallel using large matrix multiplications, with cost scaling directly by input length. Decode is memory-bandwidth-bound—it generates tokens one at a time, repeatedly loading…