Boosting vLLM Inference on Huawei NPU with Ray Compiled Graphs

Ray Summit 2025November 5, 2025San Francisco, CASpeaker, with Zhilong Chen and FengChun Hua

Play video

How we extended Ray Compiled Graphs with an NPU backend to speed up vLLM inference on Huawei Ascend NPUs.

• Session page