MiTAC Splits AI Inference Across Two Servers With 2 TB Decode Memory
MiTAC and Skymizer propose splitting prefill and decode inference across two servers, with 2 TB of decode-side accelerator memory to run Kimi K3 on premises.
Topic
Topic
MiTAC and Skymizer propose splitting prefill and decode inference across two servers, with 2 TB of decode-side accelerator memory to run Kimi K3 on premises.