How Asteria AI Built an 8x H100 Platform for Large-Scale Model Training

About the client

Asteria AI develops enterprise generative AI solutions for document intelligence, knowledge management, and domain-specific language models.

The company works with large proprietary datasets and builds customized models for customers in financial services, legal technology, and enterprise software. Its engineering team performs model fine-tuning, continuous evaluation, large-scale inference, and experimentation with open-weight foundation models. As model sizes and datasets increased, GPU infrastructure became one of the company’s most important technical and financial resources.

Challenge

During its early development stage, Asteria AI relied primarily on public cloud GPU instances. This approach was flexible for experiments. Engineers could allocate GPUs for several hours or days, complete a training job, and terminate the environment. But the workload changed as the company grew, GPU utilization became almost continuous.

The team was now running:

• distributed model training;
• fine-tuning of large language models;
• synthetic-data generation;
• embedding pipelines;
• evaluation workloads;
• production batch inference;
• customer-specific model experiments.

Large training jobs increasingly required 8 H100-class GPUs simultaneously, and obtaining identical GPU capacity at the required time was not always predictable. The company also found that running large GPU instances continuously changed the economics of the public cloud. Its requirements were therefore very different from those of an early-stage experimental AI project:

• 8 high-end GPUs in a single server;
• high-speed inter-GPU communication;
• hundreds of gigabytes of GPU memory;
• large system RAM capacity;
• fast local storage;
• high-speed networking;
• guaranteed GPU availability;
• predictable monthly infrastructure cost.

Solution

Asteria AI moved its baseline training and inference capacity to a dedicated 8x NVIDIA H100 server with Unihost in the Netherlands. The company selected the AZM-12, a large multi-GPU configuration designed for demanding AI workloads. Unlike the temporary cloud instances previously used by the team, the entire GPU platform is dedicated to Asteria AI and remains permanently available for its ML engineers and production workloads.

Unihost solution used

AZM-12 Multi-GPU Dedicated Server in Netherlands

• 2x Intel Xeon Gold 6448Y
• 8x NVIDIA H100 NVLink
• 2 TB RAM, 2 TB NVMe
• 10 Gbps network

Used for: distributed LLM training, fine-tuning, reinforcement and preference optimization workloads, large-scale inference, synthetic-data generation, embeddings, and model evaluation. The 8x NVIDIA H100 NVLink configuration allows Asteria AI to distribute large training workloads across multiple GPUs within a single physical machine while maintaining fast communication between accelerators. The server’s 2 TB of system memory provides substantial capacity for preprocessing and large datasets, while the 10 Gbps connection is used for transferring datasets, model checkpoints, and results to the company’s storage environment.

Results

Moving to dedicated H100 infrastructure changed how the engineering team used GPU resources. Previously, engineers planned experiments around cloud GPU availability and frequently terminated environments immediately after jobs were completed.

With the dedicated 8-GPU node, the team can maintain a permanent training environment and keep datasets, Docker images, model caches, and development tooling ready for the next workload. Average GPU utilization increased from approximately 35 – 45% to more than 70% because unused capacity could immediately be reassigned between training, evaluation, inference, and research jobs. Asteria AI also reduced the time required to start large training workloads. Provisioning an 8-GPU environment, synchronizing datasets, installing dependencies, and restoring checkpoints previously took anywhere from tens of minutes to several hours depending on cloud capacity and dataset size.

With a persistent dedicated environment, new workloads can begin almost immediately when GPU resources become available.

Training larger models

The biggest architectural change was the ability to treat eight H100 GPUs as a permanent compute pool.

Asteria AI uses the server for distributed training and fine-tuning of models that would be impractical to run on one or two GPUs.

The environment supports several workload types:

Large training jobs

→ all 8x H100 GPUs assigned to one distributed workload

Model fine-tuning

→ 4-8 GPUs depending on model size

Evaluation and inference

→ GPUs divided between multiple jobs

Development and experimentation

→ available capacity allocated dynamically to engineering teams

This allows the company to keep expensive GPU hardware productive even when a large training run is not active.

Predictable AI infrastructure costs

Cost predictability was another major benefit. Cloud GPU pricing worked well when the team used accelerators intermittently. It became less attractive once H100 workloads were running continuously. With dedicated hardware, Asteria AI knows the cost of its baseline GPU capacity in advance and can optimize utilization around a fixed infrastructure budget. The company still uses public cloud GPU instances when it needs to temporarily exceed the capacity of its dedicated environment. The resulting architecture is therefore hybrid:

Unihost 8× H100 server

→ permanent training capacity

→ continuous inference workloads

→ fine-tuning

→ evaluation

→ development

Public cloud GPU capacity

→ temporary large-scale experiments

→ exceptional demand

→ additional GPUs during major training cycles

“Once eight GPUs are busy most of the week, GPU infrastructure stops being something you provision for individual experiments. It becomes core infrastructure. Having a dedicated H100 node means our researchers can schedule workloads around available GPUs instead of scheduling their work around cloud capacity.”

Operational impact

The dedicated environment also simplified MLOps.

Instead of recreating temporary environments for each major workload, the team maintains one standardized GPU platform with:

• CUDA and NVIDIA drivers;
• PyTorch;
• distributed training frameworks;
• containerized ML environments;
• local model and dataset caches;
• monitoring;
• centralized job scheduling.

This reduced setup overhead and made experiments easier to reproduce across the engineering team.

What’s next

Asteria AI expects its GPU requirements to continue growing as it works with larger models and larger customer datasets. The company is evaluating a second multi-GPU server to separate production inference from research and training. Longer term, the architecture can evolve from a single 8x H100 node into a multi-node GPU cluster, allowing the company to increase compute capacity without changing its existing ML workflow.

Results at a glance: 8x NVIDIA H100 NVLink, 70%+ GPU utilization, permanent large-model training environment, faster job startup, predictable baseline GPU capacity.