DynamiaDynamia
Integration & Ecosystem

Dynamia Expands Its GPU Ecosystem with Iluvatar CoreX llm-d Integration

DynamiaIntegration & Ecosystem
Dynamia Expands Its GPU Ecosystem with Iluvatar CoreX llm-d Integration
Dynamia · Open-source AI infrastructure

Dynamia and Iluvatar CoreX have completed integration and end-to-end validation of disaggregated Prefill/Decode deployment for llm-d on products including Tiangai 300, covering model loading, distributed scheduling, and inference output, with the changes submitted upstream.

On this page

Deploying large language models on Chinese GPU platforms requires coordination between hardware, inference engines, and cluster management frameworks. Successful model loading, request distribution across inference instances, and data exchange between instances are concrete challenges in building distributed inference services.

To address these deployment requirements, Dynamia and Iluvatar CoreX have completed integration and end-to-end validation of disaggregated Prefill/Decode deployment for llm-d on products including Tiangai 300. The work covers the full process from model loading and distributed scheduling to inference output, with the integration changes submitted to the upstream llm-d community.

Connecting the Deployment Workflow for Disaggregated Prefill/Decode Inference

llm-d is an open-source distributed inference framework built on Kubernetes and is a CNCF Sandbox project. It provides request routing, inference scheduling, and Prefill/Decode disaggregation around inference engines, helping platform teams organize multiple inference instances into a coordinated model-serving system.

Prefill/Decode disaggregation separates LLM inference into two stages: Prefill processes the input, while Decode generates the output incrementally. Deploying the stages separately allows their resources to be configured independently, but also requires request coordination and KV-cache transfer between instances. Supporting a new hardware platform means connecting these components with the inference engine and its underlying runtime.

The collaboration covered joint debugging and validation across this workflow, using Qwen3.8-27B and DeepSeek V4 Flash as examples to test model loading, distributed scheduling, and inference output. It brings Iluvatar CoreX hardware and software into the llm-d deployment workflow and establishes an integration foundation for distributed model services on Chinese GPUs.

Joint Framework Implementation, Validation, and Upstream Integration

Dynamia handled llm-d-side implementation, end-to-end validation, and ongoing communication with the upstream community. The team worked on inference-instance deployment, routing configuration, and Prefill/Decode coordination, collaborating with upstream contributors to review and merge the changes into the main branch.

Iluvatar CoreX provided the hardware runtime environment, inference images, and full-stack software support, helping connect hardware drivers, communication libraries, and the inference framework. The two teams worked respectively on the framework and the hardware/software platform to validate the complete deployment workflow.

The code and configuration were merged into the main branch through llm-d PR #2381. They include accelerator documentation, disaggregated Prefill/Decode deployment configurations, and maintainer information. Developers can review the implementation, refer to the deployment configurations, and follow ongoing maintenance.

An Open, Neutral Approach to Ecosystem Collaboration

Inference frameworks, models, and hardware/software stacks continue to evolve, and integrations require maintenance as versions change. Both companies have assigned technical specialists to maintain this integration, follow community feedback, and support feature development and future iterations.

From the earlier collaboration with MetaX to this joint validation with Iluvatar CoreX, Dynamia continues connecting Chinese compute platforms with cloud-native inference frameworks. In line with the open and neutral positioning of Dynamia and HAMi, we will keep working with chip vendors, inference frameworks, and platform partners on integration, deployment validation, and long-term maintenance. This cooperation supports interoperability across hardware and software, giving enterprises more flexibility to choose and combine compute resources around their business needs.

For the integration changes and subsequent progress, see the integration pull request. Framework information and deployment documentation are available in the llm-d project repository.

KEEP READING

Keep reading

All articles
DynamiaCommunity & updates
HAMi positioning in CNCF x PyTorch baseBuilding together · discussions · event notes
Dynamia · Open-source AI infrastructure

HAMi Meetup Shanghai Deep Dive (1): Where Does HAMi Fit in the CNCF × PyTorch Cloud Native AI Foundation?

The model has been able to run and the operation is slow to go online. Keith Chan, Director General of the China District, gave an opening speech at the HAMMI Meetup Shanghai Station, giving a structured collaborative idea of how to create a base for the development of claud-native AI: Hami is located in Scheduling with resource management, assuming the role of heterogenouscompute resources sharing and managing.

Read article
DynamiaCommunity & updates
HAMi Community Round Table: From Users to Co-ConstructorsBuilding together · discussions · event notes
Dynamia · Open-source AI infrastructure

HAMi Meetup Shanghai Deep Dive (7): From User to Co-Builder — Why Enterprises Need a Sustainable HAMi Community

Once an enterprise uses open source software in its production environment, the concerns will gradually extend beyond code: can it be responded to when problems are encountered? How do new needs enter public routes? The Hami Meetup Shanghai Station round-table discussion revolves around real use, contribution thresholds, AI ' s liability boundary for assistive development and community knowledge deposition, providing a different set of judgement than technology selection.

Read article
DynamiaCommunity & updates
HAMi 2.10: Soolation, Scheduling and ObservationBuilding together · discussions · event notes
Dynamia · Open-source AI infrastructure

HAMi Meetup Shanghai Deep Dive (2): After GPU Sharing — Isolation, Scheduling, and Observability in HAMi 2.10

The distribution of a GPU to multiple tasks is only the beginning of resource sharing. Dynamia co-founder and CTO, HAMi Mainlainer Mengxuan Li, at the HAMi Meetup Shanghai Station, presented the progress of V2.10 capabilities, DRA ecological appliance, and v2.11 planning (Remote GPU, CPU NUMA alignment). The main thread for these changes is to enable shared compute resources to be distributed on demand and to be subject to monitoring, observation and diagnosis during their operation.

Read article
Image viewer