HPC Cooling Technologies: Air Cooling, Direct Liquid Cooling, and Immersion Cooling

Cooling is not a peripheral concern in HPC infrastructure — it is a primary constraint. Modern GPU nodes (DGX H100, HGX H100) dissipate 10–14 kW per 4U chassis. A 42U rack of GPU servers can generate 80–120 kW of heat. Air cooling at this density is physically impractical. Even for CPU-only clusters, thermal management directly affects performance: processors that exceed their temperature limits automatically reduce clock speed (thermal throttling), degrading simulation throughput with no warning.

Why Thermal Management Is Critical for HPC

Thermal throttling: Modern processors (Intel Xeon, AMD EPYC) and GPUs implement hardware thermal protection by reducing operating frequency when die temperature approaches the maximum specified value (TjMax for CPUs, typically 95°C; NVML thermal shutdown for GPUs at 90°C). A throttling processor delivers less compute per watt than a properly cooled one. An HPC node that throttles for 10% of its operating time delivers only 90% of its purchased compute capacity.

Component temperature limits:

Reliability impact: The Arrhenius equation shows that every 10°C reduction in operating temperature approximately doubles component lifetime (the rule of thumb for semiconductor failure rates). Operating GPU clusters at 70°C inlet temperature (allowable in A2 ASHRAE class) vs. 25°C shortens DRAM and capacitor lifetime significantly.

Air Cooling

Air cooling moves heat from servers to room air, which is then cooled by Computer Room Air Conditioning (CRAC/CRAH) units.

How it works: Servers draw cool air from the front (cold aisle) and exhaust hot air from the rear (hot aisle). CRAC units supply conditioned air to the cold aisle and return hot exhaust. Hot/cold aisle containment (physical barriers separating the two air streams) is essential — without containment, hot and cold air mix, forcing CRAC units to over-cool to compensate.

Power density limit: Air cooling becomes impractical above approximately 15–20 kW per rack for standard raised-floor deployments, or 25–30 kW with high-velocity perforated tiles and precision containment. A 4U DGX H100 at 14 kW is already challenging for air cooling in a shared row.

Advantages:

• Lowest infrastructure cost for moderate density
• Standard IT operations — no coolant plumbing
• Immediate scalability (just add more CRAC capacity)

Limitations:

• PUE rarely below 1.4 (40% overhead for cooling infrastructure)
• Cannot handle > 20–30 kW/rack cost-effectively
• High airflow creates significant noise (75+ dBA) and vibration

Direct Liquid Cooling (DLC)

Direct liquid cooling delivers chilled water directly to the server chassis, where cold plates in contact with CPUs and GPUs transfer heat to the coolant. The heated coolant is returned to a building chiller or cooling tower.

DLC cooling circuit:

Building chilled water supply (10–18°C)
         ↓
Manifold / CDU (Cooling Distribution Unit)
         ↓
Flexible hoses to rack rear
         ↓
In-server manifold
         ↓
Cold plates on CPU / GPU / memory
         ↑
Return manifold → CDU → building return loop

The CDU is an intermediate heat exchanger that separates the building water from the server water, handling pressure differential and preventing building water contamination from entering servers.

Thermal performance: A properly designed DLC system maintains CPU and GPU junction temperatures 15–25°C lower than air cooling under the same load, completely eliminating thermal throttling on high-density HPC workloads.

Rear-door heat exchanger (RDHx): A simpler variant of DLC where a passive or active heat exchanger in the rack rear door absorbs server exhaust heat. No cold plates on CPUs/GPUs are required; existing servers work without modification. Effective to approximately 30–50 kW per rack.

Advantages of DLC:

• Handles 50–150 kW per rack
• PUE approaching 1.1–1.2 (versus 1.4–1.6 for air)
• Eliminates server fan noise (some DLC servers run fanless)
• Higher server inlet temperature is allowed (reduces cooling energy)

Installation requirements:

• Chilled water distribution infrastructure in the data center
• CDU units (one per rack row, or one per rack)
• Qualified plumbing for installation and maintenance
• Leak detection system around server manifolds

Immersion Cooling

Immersion cooling submerges servers directly in electrically non-conductive dielectric fluid. The fluid absorbs heat from all components simultaneously — not just CPUs and GPUs but also VRMs, capacitors, and PCIe cards that are unreachable by cold plates.

Single-phase immersion: Servers submerge in a fluid (engineered fluids from 3M Novec or similar) that remains liquid throughout the process. A pump circulates fluid from the bath through an external heat exchanger.

Two-phase immersion: Servers submerge in a fluid with a lower boiling point (~49°C for 3M Novec 7100). The fluid boils at the hot components, vapor rises, condenses on a water-cooled condenser at the top of the tank, and falls back as liquid. No pump needed for fluid circulation.

Immersion cooling advantages:

• Lowest PUE of any cooling approach (1.02–1.05)
• Servers can run overclocked beyond TDP specifications
• No air circulation required — significant noise reduction
• Components can last 2–3× longer at lower temperatures
• Highest possible rack density

Immersion cooling challenges:

• Significant infrastructure investment (tanks, fluid, secondary cooling)
• Fluid is expensive and must be managed carefully
• Not all hardware is certified for immersion
• Fluid contamination from component failures requires costly fluid changes
• Not suitable for environments where frequent hardware changes are expected

Decision Criteria

Monitoring Thermal Metrics

# CPU temperatures via lm-sensors
sensors | grep -E "Core|Package|temp"

# GPU temperatures via nvidia-smi (all GPUs)
nvidia-smi --query-gpu=index,name,temperature.gpu --format=csv

# Continuous GPU monitoring with DCGM
dcgmi dmon -e 203,252,225 -d 5
# 203 = GPU temperature, 252 = power draw, 225 = SM utilization

# IPMI outlet and ambient temperature
ipmitool sdr type Temperature

# Alert rules in Prometheus (GPU temperature)
# gpu_alert.yml
groups:
  - name: gpu_thermal
    rules:
      - alert: GPUTemperatureHigh
        expr: DCGM_FI_DEV_GPU_TEMP > 85
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "GPU {{ $labels.gpu }} on {{ $labels.instance }} at {{ $value }}°C"
      - alert: GPUTemperatureCritical
        expr: DCGM_FI_DEV_GPU_TEMP > 88
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "GPU {{ $labels.gpu }} approaching thermal shutdown"

Cooling infrastructure is a long-term commitment that constrains which compute hardware can be deployed. The transition from air to liquid cooling for high-density GPU clusters is not a future concern — it is a present requirement for any deployment above 20–30 kW per rack. Plan cooling before finalizing server specifications. Contact Mevasis for HPC data center cooling design and thermal management consulting.