DynamiaDynamia
Product insights

Dynamia and MetaX Complete llm-d Integration: The First Chinese GPU in CNCF's Distributed Inference Ecosystem

DynamiaProduct insights
Dynamia x MetaX llm-d fitProduct capabilities and release notes
Dynamia · Open-source AI infrastructure

Dynamia has partnered with MetaX to complete full adaptation of MetaX C-series GPUs for the llm-d community, validating single-GPU, multi-GPU, and Prefill/Decode disaggregated inference deployments — the first Chinese GPU ever added to llm-d's officially supported accelerator list.

On this page

Dynamia recently partnered with MetaX to complete the full adaptation of MetaX C-series GPUs for the llm-d community, validating single-GPU, multi-GPU, and Prefill/Decode disaggregated inference deployments on C-series hardware. With this work, the cloud-native distributed inference project llm-d now officially supports MetaX C-series GPUs — the first Chinese GPU ever added to llm-d's officially supported accelerator list.

In this effort, Dynamia owned the llm-d-side feature implementation, end-to-end validation, and ongoing engagement with the upstream community, while MetaX provided the complete test environment, the vLLM-MetaX inference image, and technical support at the MXMACA software-stack level. Together, the two teams carried the proposal through the international community's technical review. The llm-d accelerator maintainer list now registers two maintainers — one from each company — who will jointly own the long-term maintenance and evolution of this integration path.

From Runnable to Production-Grade

Getting a large model to run is only the starting point; delivering it as a scalable service is what determines real-world efficiency, and that hinges on whether the chip integrates smoothly with a cloud-native inference scheduling stack. Historically, default configurations in international open-source projects of this kind assumed foreign silicon. Even when a Chinese GPU could already run mainstream inference engines, users who wanted to reuse the community's mature deployment recipes still had to work through countless hidden adaptation details on their own.

llm-d is a cloud-native distributed inference framework initiated by Red Hat, built on top of inference engines such as vLLM and SGLang and on Kubernetes, and now hosted by the CNCF (Cloud Native Computing Foundation). The project focuses on the capabilities that matter for production LLM serving — prefix-cache-aware routing, load-aware scheduling, and Prefill/Decode disaggregation — turning single-node inference engines into multi-node, coordinated, production-grade distributed inference systems. Since its inception, llm-d has earned more than 4,400 GitHub stars, and major compute platforms including NVIDIA GPUs, Google TPUs, AMD GPUs, and Intel XPUs have each established deployment guides in the community.

This adaptation closes exactly that gap for the MetaX C-series, providing three out-of-the-box deployment paths.

Three Out-of-the-Box Deployment Paths

  • Optimized baseline, single GPU: Qwen3-14B with tensor parallelism TP=1, validating the C-series' basic inference capability on the standard llm-d model-serving stack.
  • Optimized baseline, eight GPUs: DeepSeek-R1-Distill-Llama-70B with tensor parallelism TP=8, validating large-model serving on a single eight-GPU node.
  • Prefill/Decode disaggregation: Qwen3-14B on a 1-Prefill + 1-Decode topology, using the vLLM NixlConnector together with the llm-d routing sidecar to enable cross-instance KV-cache transfer. P/D disaggregation splits the two computationally distinct phases of inference onto separate instances optimized independently — one of the key techniques for raising LLM inference resource utilization today. This path is built on MIXL, a transport library adapted from and optimized for NIXL, which substantially improves transfer efficiency and lowers the adoption barrier for users.

Establishing a Performance Baseline with Real Measurements

The team also completed measured performance calibration of the llm-d inference router on C-series GPUs: peak prefill throughput of 5,773 tokens/s for the single-GPU Qwen3-14B configuration, and 5,468 tokens/s for the eight-GPU DeepSeek-R1-Distill-Llama-70B configuration. Both results have been contributed to the llm-d community's public benchmark matrix, so users deploying on the MetaX platform can cite a performance baseline drawn from real hardware.

The vLLM-MetaX inference image required for the adaptation has been published to a public image registry and is available for direct use.

From Joint Adaptation to Long-Term Stewardship

This collaboration marks a starting point. Dynamia will continue to maintain and iterate the llm-d integration path for MetaX C-series GPUs together with MetaX, and — through sustained upstream collaboration — help Chinese compute silicon participate more deeply in the world's mainstream open-source infrastructure. Dynamia will also partner with more chip and ecosystem players to keep strengthening the full chain from domestic silicon to cloud-native foundation to industry deployment, providing solid support for scaling AI across industries.

KEEP READING

Keep reading

All articles
DynamiaCommunity & updates
HAMi positioning in CNCF x PyTorch baseBuilding together · discussions · event notes
Dynamia · Open-source AI infrastructure

HAMi Meetup Shanghai Deep Dive (1): Where Does HAMi Fit in the CNCF × PyTorch Cloud Native AI Foundation?

The model has been able to run and the operation is slow to go online. Keith Chan, Director General of the China District, gave an opening speech at the HAMMI Meetup Shanghai Station, giving a structured collaborative idea of how to create a base for the development of claud-native AI: Hami is located in Scheduling with resource management, assuming the role of heterogenouscompute resources sharing and managing.

Read article
DynamiaCommunity & updates
HAMi Community Round Table: From Users to Co-ConstructorsBuilding together · discussions · event notes
Dynamia · Open-source AI infrastructure

HAMi Meetup Shanghai Deep Dive (7): From User to Co-Builder — Why Enterprises Need a Sustainable HAMi Community

Once an enterprise uses open source software in its production environment, the concerns will gradually extend beyond code: can it be responded to when problems are encountered? How do new needs enter public routes? The Hami Meetup Shanghai Station round-table discussion revolves around real use, contribution thresholds, AI ' s liability boundary for assistive development and community knowledge deposition, providing a different set of judgement than technology selection.

Read article
DynamiaCommunity & updates
HAMi 2.10: Soolation, Scheduling and ObservationBuilding together · discussions · event notes
Dynamia · Open-source AI infrastructure

HAMi Meetup Shanghai Deep Dive (2): After GPU Sharing — Isolation, Scheduling, and Observability in HAMi 2.10

The distribution of a GPU to multiple tasks is only the beginning of resource sharing. Dynamia co-founder and CTO, HAMi Mainlainer Mengxuan Li, at the HAMi Meetup Shanghai Station, presented the progress of V2.10 capabilities, DRA ecological appliance, and v2.11 planning (Remote GPU, CPU NUMA alignment). The main thread for these changes is to enable shared compute resources to be distributed on demand and to be subject to monitoring, observation and diagnosis during their operation.

Read article
Image viewer