Inside the Modern Data Center: Power, Cooling, and the Density Wall

The thermal design assumption that held for twenty-five years — that a rack dissipates somewhere between 5 and 15 kW and air can carry that away — no longer holds. Dense accelerator racks now draw more power than an entire row did a decade ago, and the consequence is not that cooling becomes harder. It is that air stops working entirely, and the facility architecture changes with it.

Why air runs out, in one calculation

Heat removal by a flowing fluid is governed by a single relationship, and everything about the air-to-liquid transition follows from the properties in it.

Q = ṁ · c_p · ΔT = ρ · V̇ · c_p · ΔT

Q — heat removed (W) · ṁ — mass flow (kg/s) · V̇ — volumetric flow (m³/s) · c_p — specific heat · ΔT — temperature rise across the load

The determining quantity is volumetric heat capacity, ρ·c<sub>p</sub>. For air at rack inlet conditions that is roughly 1.2 kJ/m³·K. For water it is about 4,170 kJ/m³·K. Water therefore carries approximately 3,400 times more heat per unit volume for the same temperature rise.

( ρ c_p )_water / ( ρ c_p )_air ≈ ( 997 × 4.18 ) / ( 1.2 × 1.005 ) ≈ 3.4 × 10³

This ratio is why the transition is a cliff rather than a slope: removing 100 kW with a 12 K rise needs roughly 7 m³/s of air, or about 2 litres per second of water.

Seven cubic metres per second through a single rack is not an engineering challenge, it is an impossibility — the face velocity required exceeds what the equipment can accept and the fan power to move it starts consuming a meaningful fraction of the IT load itself. This is the actual limit. Air cooling remains practical to roughly 20–30 kW per rack with rigorous containment, becomes exotic to about 50, and stops.

Rack density	Approach	Notes
< 10 kW	Room air, hot-aisle containment	Conventional CRAH, raised floor optional
10–25 kW	Contained aisle, in-row cooling	Containment becomes mandatory, not optional
25–50 kW	Rear-door heat exchanger	Water to the rack, air inside the chassis
50–130 kW	Direct-to-chip cold plate + CDU	Liquid to the die; residual air load remains
> 130 kW	Immersion or full single-phase DLC	Facility designed around liquid from day one

Cooling approach by rack density. The rear-door heat exchanger is the pragmatic middle step: it introduces facility water to the row without requiring servers designed for cold plates.

Warm water is the counter-intuitive part

The instinct is that colder coolant is better. For total facility energy it is usually the opposite. Silicon junction temperature limits permit facility water supply well above ambient — ASHRAE's liquid cooling classes go up to 45 °C supply — and once the required supply temperature exceeds the local wet-bulb for most of the year, mechanical chilling can be replaced by dry coolers or a cooling tower running in economiser mode.

The chiller is typically the single largest non-IT consumer in a facility. Eliminating its run hours does more for PUE than any amount of airflow optimisation, and the cost is accepting a warmer coolant that the hardware was designed to tolerate anyway.

PUE = E_facility / E_IT = 1 + ( E_cooling + E_power loss + E_other ) / E_IT

Written this way it is clear that PUE improvement is entirely about shrinking the numerator terms. Well-executed warm-water DLC reaches 1.05–1.10 annualised; a legacy air facility with chillers running year-round sits at 1.5–1.8.

Warm return water has a second property worth designing for: at 45–60 °C it is genuinely useful. District heating, industrial process pre-heat and absorption chilling can all take it. Low-grade heat at 25 °C is waste; at 55 °C it has a buyer, and in several European jurisdictions the heat offtake agreement now materially affects project economics.

The electrical side, where the constraint has moved

Power distribution efficiency compounds, because every conversion loss must itself be cooled. The dominant design responses are raising distribution voltage to reduce I²R losses, minimising conversion stages, and choosing a UPS topology honestly.

415 V three-phase to the rack, 240 V line-to-neutral to the PSU — removes a transformer stage and its losses relative to legacy 208 V distribution.
Line-interactive or eco-mode UPS operation reaches 98–99% efficiency against roughly 94–96% for always-on double conversion; the trade is transfer time, which must be verified against the PSU hold-up time.
Lithium-ion UPS batteries over VRLA: higher energy density, far longer service life, and materially less floor space — at the cost of fire-code requirements that must be engaged early.
48 V DC rack distribution for the highest-density deployments, cutting conversion stages between busway and point-of-load.

But the constraint that most often decides a project is none of these. It is grid interconnection. Utility connection queues for large loads now run to multiple years in most major markets, which means capacity, not capital, is the scarce resource — and it changes the calculus toward on-site generation, staged energisation and, increasingly, siting decisions made on the basis of available headroom rather than land price or latency.

Designing for a load you cannot yet specify

The hardware roadmap outruns the building. A facility commissioned today will host at least three accelerator generations, each denser than the last, and no one can specify the third. The defensible response is to build the distribution to a density the current IT load does not need — oversized busway, liquid distribution to every row whether or not it is used, floor loading and structural provision for immersion tanks — because the marginal cost during construction is a fraction of the retrofit cost, and the retrofit requires taking revenue-generating space offline.

The mistake is not building too much headroom. It is building exactly enough, and discovering in year four that the constraint is a busway rating that cannot be changed without a shutdown.