Case Study | CETC Cloud Builds a Heterogeneous GPU Shared Resource Foundation for a Portable Knowledge Base with HAMi
CETC Cloud (Beijing) Technology runs text generation, embedding, and rerank production models alongside Kubernetes dev environments on a single 8-GPU portable device, raising concurrent dev Pod capacity from 2 to 30.
Customer Overview
CETC Cloud (Beijing) Technology needed to bring its intelligent knowledge base to project sites, running text generation, embedding, rerank, knowledge processing, and R&D debugging workloads together on a single set of portable devices equipped with heterogeneous GPUs. The team adopted Kubernetes and HAMi to turn these limited GPUs, previously bound long-term to individual services, into resources that production models and Kubernetes dev environments can request, share, and schedule on demand.
Portable intelligent knowledge base appliance that runs independently on site
Knowledge base production pipeline and Kubernetes dev environments coexist on one 8-GPU heterogeneous device
Built on Kubernetes-native resource declarations plus HAMi device sharing and unified scheduling
CETC Cloud (Beijing) Technology Co., Ltd.
The cloud and intelligent computing brand of CETC, responsible for product development and engineering delivery of the portable intelligent knowledge base — on-site local operation, knowledge processing, model serving, and dev environments.
Case Information
This case is adapted from the official CNCF Case Study: CNCF Case Study
Challenge: One Device, Two GPU Usage Paths
Online Q&A depends on text generation, vector retrieval, and rerank; incoming documents must go through parsing, chunking, vectorization, knowledge extraction, and index building; and R&D engineers debug models, validate dependencies, and run experiments inside Kubernetes dev Pods/containers. These workloads use GPUs in very different ways — yet they must coexist on the same portable device.
Very different workload profiles
Text generation must stay online with a high VRAM baseline and latency sensitivity; embedding creates periodic peaks during bulk import; rerank does short but frequent compute; knowledge processing and index rebuilds are batch jobs that must be isolated from online services; dev Pods are periodic and interactive.
Whole-card binding exhausts devices fast
If each service or dev environment keeps a whole card bound long-term, the limited devices are quickly exhausted logically, and the remaining capacity cannot be reused by other workloads.
Over-partitioning introduces new risks
Pursuing only the finest possible partitioning amplifies VRAM shortages, scheduling jitter, and fault propagation.
A need for differentiated resource policies
The team needed each workload type to describe its own needs, then let the platform complete resource placement based on device state and service tier.
Solution: A Common Resource Contract on HAMi
The value of HAMi is not merely representing one physical card as multiple resource shares. More importantly, it establishes a common resource contract: workloads declare the device resources they need, scheduling selects based on available device state, multiple Pods share one heterogeneous GPU within their own resource boundaries, and Pods, containers, devices, and scheduling results can be correlated back together.
Kubernetes-Native Resource Contract
Keep the Kubernetes application delivery model while letting the platform understand whether heterogeneous GPU devices can satisfy workload requests, and letting dev environments suited to sharing reuse the same physical GPU.
Model services and dev Pods declare device resources themselves — no more long-term manual card binding
Scheduling selects based on available device state, not just GPU presence on a node
Pods, containers, devices, and scheduling results are traceably linked for capacity observation and config correction
Differentiated Provisioning by Workload Profile
The team built resource profiles around model loading, peak usage, concurrency, latency, and long-running behavior, then translated validated requirements into Kubernetes resource requests with different policies per workload type.
Stable boundaries for the online path: text generation, online retrieval, and rerank get stable resources first
Periodic tasks queue off-peak: vectorization, knowledge extraction, and index rebuilding run when online load is low
Configuration corrected by real load: granularity comes from baseline validation and runtime observation, not a preset ratio
Dev Environments Share a Single Heterogeneous GPU
Multiple Kubernetes dev Pods/containers reuse one heterogeneous GPU through HAMi virtualization instead of each holding a device exclusively; VRAM quotas are requested per task by users rather than set to one fixed platform value.
Multiple dev Pods share one heterogeneous GPU, each with its own resource boundary
Reuses the same resource declarations and scheduling as production services
Validated with 30 dev Pods running concurrently on the same device
Results
On the same portable device equipped with 8 heterogeneous GPUs, the team compared resource capacity before and after introducing HAMi on a consistent basis:
Dev Pod Capacity
2 → 30
Concurrent dev Pods on the same device rose from 2 to 30 — 15× the original
VRAM Reclaimed
87.5%
Embedding went from a whole 64 GB GPU to an 8 GB allocation, freeing 56 GB for reuse
Compute Reclaimed
80%
Embedding was allocated 20% of GPU compute; the remaining 80% stays schedulable
Performance Impact
< 5%
Token throughput stayed within 5% of whole-card use; load time and P95 latency essentially unchanged
Embedding Model: Single-Card Resource Comparison
Same model image, request set, and concurrency — whole-card exclusive use vs. HAMi partitioning on the same card:
| Metric | Without HAMi | With HAMi |
|---|---|---|
| VRAM allocation | Exclusively occupies the whole 64 GB GPU | 8 GB allocated on demand |
| VRAM reusable by other tasks | 0 GB | 56 GB — 87.5% of the whole card |
| Compute allocation | Whole card exclusively | 20% allocated, remaining 80% schedulable |
| Token throughput (tokens/s) | Baseline | Within 5% of whole-card use |
| Model load time / P95 latency | Baseline | Essentially identical to whole-card use |
Lessons & Next Steps
Heterogeneous GPU sharing should start from workload profiles, not a preset uniform partitioning ratio — production models, batch tasks, and dev environments differ in latency, VRAM, throughput, and isolation requirements, so their resource policies should differ too.
R&D environments are a major consumer of shared GPUs. Running dev workloads as Kubernetes Pods/containers reuses the same resource declarations and scheduling as production services, while avoiding each dev environment holding a device exclusively long-term.
Next, CETC Cloud will keep refining capacity and health views for production and R&D workloads, and add reproducible experimental data across different models, networks, and calling patterns.
Get More from Limited Heterogeneous Compute
From whole-card exclusivity to on-demand sharing, CETC Cloud used Kubernetes and HAMi to serve a knowledge base production pipeline and 30 concurrent dev environments on a single portable device — with essentially no loss of production performance. The same resource contract applies to any team that needs production and R&D to coexist on limited heterogeneous compute.