A model can finish crossing the network before its new replica can serve a single request.
vLLM merged daemon-to-daemon weight seeding on November 16. A new replica can copy model weights from an already loaded replica instead of reading the checkpoint from storage again. Across hosts, it uses Mooncake and remote direct memory access (RDMA).
The contributor reports moving 502 GiB of DeepSeek-V4.1-Flash weights between GB200 nodes in about 9.3 seconds, roughly 54 GiB/s across four RoCE network interfaces. RoCE carries RDMA over Ethernet.
Read the footnote. That DeepSeek mirror couldn’t reach READY because a kernel lacked support for the preprocessed weights. The measurement validates the transfer path, not a nine-second inference startup.
A separate Qwen3-30B-A3B test did complete startup: 100 seconds with seeding versus 230 seconds from disk.
For network operators, this adds another burst to account for when replicas scale out. The PR lists no bandwidth throttling. I’d measure weight-transfer time, time to READY, and interference with existing inference traffic separately. A fast copy is useful. It doesn’t tell us whether the next request can run.


