An enterprise AI company develops generative AI systems for document intelligence, enterprise knowledge management, and domain-specific language models. Its ML engineering team performs model training, fine-tuning, evaluation, synthetic-data generation, embedding generation, and large-scale inference. Initially, most GPU workloads ran in public cloud environments.<\/span><\/p>\n Cloud GPUs worked well while GPU usage was intermittent. As the company moved from experimentation to continuous production and R&D workloads, its infrastructure requirements changed. The engineering team increasingly needed eight H100-class GPUs simultaneously for distributed workloads.<\/span><\/p>\n Key requirements included:<\/span><\/p>\n Cloud capacity remained useful for temporary bursts, but the company’s baseline GPU consumption had become almost continuous.<\/span><\/p>\n The company moved its baseline AI compute capacity to a dedicated multi-GPU platform.<\/span><\/p>\n AZM-12<\/b> Multi-GPU Dedicated Server – Netherlands: 2x Intel Xeon Gold 6448Y, 8x NVIDIA H100 NVLink, 640 GB total HBM2e, 2 TB RAM, 2 TB NVMe, 10 Gbps network.<\/span><\/p>\n Used for: distributed LLM training, fine-tuning, preference optimization, model evaluation, synthetic-data generation, embeddings, and large-scale inference.<\/span><\/p>\n The eight H100 GPUs can be assigned to one distributed workload or divided between multiple smaller jobs.<\/span><\/p>\n The company maintains its ML software environment permanently on the server, including:<\/span><\/p>\n Public cloud GPUs remain part of the architecture, but primarily for temporary capacity beyond the dedicated baseline.<\/span><\/p>\n Hybrid GPU architecture<\/span><\/p>\n Average utilization of paid GPU capacity increased from approximately 35 – 45% to more than 70%. A persistent GPU environment also reduced the operational overhead associated with repeatedly creating cloud instances, synchronizing datasets, restoring checkpoints, downloading containers, and rebuilding model caches. Large jobs can use all eight H100 GPUs continuously, while unused GPUs can be reassigned to inference, evaluation, or development workloads. Most importantly, baseline AI compute spending became predictable. For workloads requiring continuous H100 usage, dedicated GPU infrastructure provides a fundamentally different cost model from purchasing GPU capacity by the hour.<\/span><\/p>\n The company plans to separate training and production inference as workloads increase. A second 8-GPU system could also form the basis of a multi-node GPU cluster. <\/span>Results at a glance: 8x NVIDIA H100 NVLink, 640 GB GPU memory, 70%+ GPU utilization, persistent training environment, predictable GPU capacity.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":" About the project An enterprise AI company develops generative AI systems for document intelligence, enterprise knowledge management, and domain-specific language models. Its ML engineering team performs model training, fine-tuning, evaluation, synthetic-data generation, embedding generation, and large-scale inference. Initially, most GPU workloads ran in public cloud environments. Challenge Cloud GPUs worked well while GPU usage was […]<\/p>\n","protected":false},"author":7,"featured_media":8986,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[46,49],"tags":[],"class_list":["post-8985","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-case-study","has-post-title","has-post-date","has-post-category","has-post-tag","has-post-comment","has-post-author",""],"yoast_head":"\nChallenge<\/b><\/h3>\n
\n
Solution<\/b><\/h3>\n
\n
\n
Results<\/b><\/h3>\n
What’s next<\/b><\/h3>\n