最高第 8 名 已下榜
SGLang 适配英伟达 Rubin
SGLang与英伟达合作,在早期Rubin硬件上优化注意力、MoE与投机验证内核,Kimi K3推理最高加速20%,端到端提速5.9%。
原推
We brought SGLang to NVIDIA Vera Rubin and accelerated Kimi K3 inference.
Working closely with @NVIDIA, we optimized attention, MoE, and speculative verification kernels on early-access Rubin hardware.
Highlights:
• Up to 20% faster FP8 MLA at batch 1 / 128K context
• 20% faster KDA verification, with bitwise-identical output
• 5.9% end-to-end inference speedup from MoE tail fusion, removing 276 kernel launches per decode step
SGLang also powers rollouts for Miles' end-to-end RL training on Rubin, including agentic RL with 64 concurrent sandboxes on the Vera CPU.
Full results and engineering details 👉 https://t.co/xGwlR0ax4L
讨论 0
注册 登录 后参与讨论
还没有人讨论,来说第一句。