llm-d 1.0 arrives: Google's open inference stack cuts median TTFT 85%
Google's llm-d 1.0 cuts median TTFT over 85% with 2.4x throughput on Ironwood TPUs, and scales 2.9x across a 17,000-node GKE benchmark — all vendor-reported.
Topic
Llm D
Topic
Topic
Topic
Google's llm-d 1.0 cuts median TTFT over 85% with 2.4x throughput on Ironwood TPUs, and scales 2.9x across a 17,000-node GKE benchmark — all vendor-reported.