HAMi Meetup Shanghai Deep Dive (6): How UCloud Turns HAMi Sharing into Ready-to-Use Development Environments

The GPUs are in place; the algorithm team is still waiting for environments. Drivers, images, networking, storage — configured one by one, and re-adapted for every new cluster. Even with GPU sharing in place at the bottom, the platform still has to connect resource specs, scheduling configuration, and development tools so users get an environment they can start working in immediately.

This is the sixth deep-dive in our HAMi Meetup Shanghai series. Peng Peng, Senior R&D Engineer at UCloud, placed HAMi inside this environment delivery chain in "Engineering Challenges and Practices of AI Computing Platforms." The team adopted HAMi vGPU for private AI computing scenarios while lowering the usage barrier through resource abstraction, multi-cluster access, tool injection, and image acceleration. For platform teams, the reference value of this practice is how to organize low-level resource capability into a service users can use and ops can manage.

Key Takeaways

  • UCloud adopted HAMi vGPU for private AI computing scenarios, covering sharing needs like mixed large/small model deployment.
  • Unified instance specs, multi-cluster access, tool injection, and image acceleration together shorten the dev-environment delivery chain.
  • HAMi user-space sharing, public-cloud kernel-level vGPU, and QEMU passthrough correspond to different deployment conditions — not one interchangeable solution.

Speaker: Peng Peng | Senior R&D Engineer, UCloud

Whole Cards and vGPUs Under One Instance Spec

UCloud's platform-building needs came from three directions: internal algorithm teams needing fast environment provisioning, self-built inference clusters needing unified management, and some GPU-owning customers wanting lightweight management.

What users submit is usually clear: image, CPU and memory, GPU model and count, and whole-card or vGPU allocation. But those fields can't directly become executable Kubernetes configuration — different hardware brings resource keys, allocation annotations, node constraints, and runtime parameters.

The team uses Instance Specs to define stable instance specifications, expressing whole-card or vGPU allocation uniformly, with an adaptation layer translating to the underlying configuration. Resource claims, vendor annotations, scheduling constraints, and runtime parameters are handled centrally, so each workload doesn't rewrite hardware adaptation logic. HAMi-provided shared resources enter the platform's unified environment creation flow the same way.

Stable instance specs and an adaptation layer for underlying differences (slide 9)
Figure 1: Stable instance specs and an adaptation layer for underlying differences (slide 9)

Users still choose the right model, count, and image. What the platform unifies is spec expression and translation — not the assumption that all GPUs are interchangeable. When new devices arrive, adaptation work can also start and validate around this layer early.

Multi-Cluster Management: Trade-Offs Along Real Business Boundaries

Clusters sit in different regions, with different storage, network, and registry conditions. Managing multiple clusters means continuously handling access credentials, capacity and health, job lifecycle, and log/metric aggregation.

The team evaluated full multi-cluster orchestrators like Karmada, and ultimately chose to access each cluster's Kubernetes API via kubeconfig. The reason: their scenario is bounded — a task picks one target cluster at creation and needs no cross-cluster distribution, migration, or disaster recovery afterward.

Under that premise, lightweight access avoids the maintenance burden of an extra control plane and resource model. Logs and metrics still need separate adaptation; credentials must be managed as sensitive information with access control enforced.

The lightweight-access trade-off for single-task, single-cluster scenarios (slide 11)
Figure 2: The lightweight-access trade-off for single-task, single-cluster scenarios (slide 11)

The choice has a clear applicability range. When business truly needs cross-cluster migration and disaster recovery, the orchestration solution and data conditions must be re-evaluated — lightweight access is not a substitute for full multi-cluster capability.

Tool Decoupling and On-Demand Loading Cut Environment Wait

Algorithm frameworks and project dependencies want stability; dev tools like SSH and VSCode update constantly. Baking everything into one image means every tool upgrade drags along image rebuilds and environment revalidation.

The team moved to assembling tools at environment creation — injecting development capability via sidecars and supplementing configuration through rules. The algorithm container and platform tools can be maintained separately, while shared files, network interfaces, and configuration boundaries still need defining.

Assembling dev tools at runtime, decoupling algorithm images from tool updates (slide 13)
Figure 3: Assembling dev tools at runtime, decoupling algorithm images from tool updates (slide 13)

Image size itself slows startup. The traditional flow downloads and decompresses every layer before assembling the filesystem. The acceleration approach in the talk scans the original image's compressed layers and generates independent file indexes. At runtime, the system fetches metadata and indexes first, mounts remote layers, and reads needed chunks — writing them to local cache — only when files are actually accessed.

Downloading hasn't disappeared; some reads have just moved after container start. If the application immediately reads large files, the wait is still there; layers without usable indexes or unsuited to lazy loading fall back to regular pulling.

Image indexing, on-demand reads, and the fallback path to regular pulling (slide 14)
Figure 4: Image indexing, on-demand reads, and the fallback path to regular pulling (slide 14)

So judging whether environments got faster should follow through to the application actually being usable — not stop at "container started."

HAMi for Private Deployments; Separate Isolation Paths for Public Cloud

Peng recounted that the team tried MIG early on, but under their device and business conditions ran into inflexible slice profiles and hardware that was hard to adapt; they then chose HAMi for private AI computing scenarios. He also shared practice improving GPU utilization with HAMi in mixed large/small model deployment. Concretely: private AI computing uses HAMi vGPU — GPU sharing plus memory and compute quotas in Kubernetes; public-cloud consumer-GPU scenarios use kernel-level vGPU, mapping logical devices to QEMU via VFIO; public-cloud enterprise-GPU scenarios involve passing different GPUs of a whole machine through to different VMs.

The selection logic is clear: HAMi carries user-space resource sharing and quota control in private scenarios, fitting Kubernetes resource usage; kernel-level vGPU and QEMU per-card passthrough are UCloud's engineering solutions for the other two deployment environments. The three paths face different hardware and tenant-isolation conditions, and each needs its own design and validation.

Different deployment scenarios map to different GPU management approaches (slide 16)
Figure 5: Different deployment scenarios map to different GPU management approaches (slide 16)

Enterprise-card passthrough must also consider multi-card communication. Keeping NVLink requires a trusted Fabric service managing the shared NVSwitch, coordinated with VM lifecycle; the alternative disables NVLink and passes GPUs through as independent PCIe devices — simpler engineering, but losing that high-speed interconnect.

Which path to take depends on hardware conditions, deployment form, isolation requirements, and communication needs. Getting a GPU allocated is only one link in the delivery chain.

Peng also listed model weight loading, inference resource coordination, idle CPU reuse, and heterogeneous compute selection as future exploration directions, extending the platform's focus toward model runtime and resource operations.

For enterprises evaluating HAMi, UCloud's practice offers a clear order of checks: can sharing specs enter the environment-creation flow; can images and tools be delivered smoothly; can quotas and lifecycle be managed continuously at runtime. The enterprise value Dynamia emphasizes lives in this complete chain: making HAMi's resource sharing work together with platform integration, compatibility validation, and operational management — cutting the wait from "got a GPU" to "can start working."

Video and Slides

Series

Share this article

Back to Blog