DynamiaDynamia
Product

Dynamia Partners with Biren Technology to Advance BR166M GPU Adaptation for llm-d

DynamiaProduct
Dynamia Partners with Biren Technology to Advance BR166M GPU Adaptation for llm-dProduct capabilities and release notes
Dynamia · Open-source AI infrastructure

Completing a distributed inference deployment path for Chinese compute

On this page

Dynamia recently partnered with Biren Technology to contribute a Biren™ 166M (BR166M) adaptation to the open-source distributed inference project llm-d, covering both standard inference and Prefill/Decode (PD) disaggregated deployment configurations. The collaboration focuses on inference instance deployment, request coordination, and cross-node cache transfer—connecting Biren general-purpose GPUs with cloud-native inference frameworks and providing a deployment reference for enterprises building distributed model serving on Chinese compute.

This work is one practice in Dynamia’s ongoing multi-hardware adaptation for llm-d. By jointly adapting with chip vendors and contributing deployment configs and documentation upstream, Dynamia continues to help more hardware platforms join open inference infrastructure.

Adapting Standard Inference and PD Disaggregation

When enterprises deploy large models as continuously running services, the inference engine, request routing, and cluster resources need to work together. Facing different chips’ software stacks and communication conditions, platform teams also need to define how inference instances start, how requests are handed off, and how computation-produced cache moves between nodes.

llm-d is a Kubernetes-based open-source distributed inference project that provides request routing and distributed service orchestration on top of inference engines such as vLLM. PD disaggregation deploys the Prefill phase (processing input) and the Decode phase (generating output step by step) separately, so platforms can configure resources independently for each load type.

This contribution provides both standard inference and PD disaggregation configs for the BR166M GPU. The PD path targets the INT8 quantized version of Qwen2.5-72B in a “1 Prefill + 1 Decode” layout: the two instances run on different nodes, each using four BR166M GPUs with tensor parallelism (TP=4), for eight GPUs in total.

Along this path, the Prefill instance handles input computation and produces the KV cache used for subsequent output; the Decode instance receives the cache and continues generation. The setup reuses llm-d’s routing sidecar for request coordination and transfers KV cache via vLLM NixlConnector, so the two nodes handle input processing and output generation respectively.

Cross-node communication uses InfiniBand RDMA through UCX, with a CPU buffer for KV data transfer. These configs document the communication path used in the current adaptation and provide a basis for later performance validation and optimization around target models, concurrency, and network environments.

Bringing Adaptation Work into Community Collaboration

This collaboration connects Biren BR166M GPUs, the BIRENSUPA™ software stack, and the adapted vLLM inference environment with llm-d’s deployment flow. Dynamia contributed on the llm-d side—team members submitted changes covering inference image configuration, deployment parameters, accelerator documentation, and a PD disaggregation deployment guide.

For platform teams planning to adopt Biren general-purpose GPUs, this material offers a concrete starting point: what runtime is required, how to organize inference instances, and how to transfer cache are all documented in configs and guides. Moving the adaptation into upstream collaboration also lets hardware vendors, framework maintainers, and users iterate on the same implementation with shared feedback.

The changes were submitted as llm-d PR #2505, with maintainer information for the adaptation path added in the PR. The PR has since been merged by the community.

Expanding Compute Choices Through Open Neutrality

Around llm-d, Dynamia will continue expanding adaptation partnerships with more chip vendors and helping different hardware platforms join the open-source distributed inference ecosystem. Connecting each platform’s runtime, deployment configs, and communication paths to existing community workflows gives enterprises more concrete implementation references when evaluating compute options, and a foundation for software and hardware partners to maintain and optimize together.

Dynamia remains technically neutral and works openly with chip vendors, open-source communities, and platform partners around real enterprise deployment needs. Going forward, Dynamia will keep advancing joint adaptation, deployment validation, and upstream contribution with upstream and downstream partners, follow compatibility and maintenance needs as versions evolve, and help more compute platforms work with mainstream open-source infrastructure—so enterprises can choose compute based on business needs and actual adaptation conditions.

Full adaptation configs and further progress are available in this adaptation PR.

KEEP READING

Keep reading

All articles
DynamiaEngineering
As Models Grow Larger, Why Do Enterprises Still Need Fine-Grained Compute Scheduling?Deep dives and engineering detail
Dynamia · Open-source AI infrastructure

As Models Grow Larger, Why Do Enterprises Still Need Fine-Grained Compute Scheduling?

In a shared resource pool, enterprises must support full-GPU and multi-GPU deployments for large models while also enabling on-demand sharing for lightweight inference, R&D, and batch jobs. Drawing on model usage data and production cluster research, this article discusses how to allocate GPUs by workload needs and manage heterogeneous compute shared across teams.

Read article
DynamiaCommunity
Technical Progress and Hands-on Practice: HAMi at KubeCon NA 2026Building together · discussions · event notes
Dynamia · Open-source AI infrastructure

Technical Progress and Hands-on Practice: HAMi at KubeCon NA 2026

At KubeCon + CloudNativeCon North America 2026, the HAMi community joined Red Hat to discuss GPU sharing and inference architecture, covered scheduling policies and Kubernetes DRA progress, and led developers through GPU virtualization in ContribFest.

Read article
DynamiaCommunity
Making the Most of Every GPU: Dynamia Shares Practices at ContainerDays SingaporeBuilding together · discussions · event notes
Dynamia · Open-source AI infrastructure

Making the Most of Every GPU: Dynamia Shares Practices at ContainerDays Singapore

At ContainerDays & AI Context Singapore 2026, Dynamia shared HAMi GPU sharing and scheduling practices in Kubernetes through hands-on labs and production cases, plus GPU utilization optimization for Neocloud scenarios.

Read article
Image viewer