Rack-scale AI and liquid cooling: what the GB300 and Vera Rubin specs mean for facilities

For most of the last decade, buying AI compute meant buying servers. The unit of purchase is now a rack: 72 GPUs wired into a single high-bandwidth fabric, liquid-cooled, delivered and serviced as one system. That changes what a facility has to provide, and it is why cooling has moved from an operations footnote to a procurement constraint.

What a rack-scale system is

NVIDIA lists the GB300 NVL72 as 72 Blackwell Ultra GPUs and 36 Grace CPUs, with 130 TB/s of NVLink bandwidth, about 20 TB of HBM3E GPU memory and 37 TB of "fast memory" in total ^[1]. The point of the 72-GPU NVLink domain is that a very large model can be spread across the whole rack as if it were one accelerator, rather than stitched together over slower network links.

Lenovo's version of the same platform shows the physical build: 18 1U compute trays and nine 1U NVLink switch trays in a 48U rack, 800 Gb/s networking, and up to eight 33 kW power shelves with a 48 V DC busbar ^[3]. Eight shelves at 33 kW is 264 kW of shelf capacity, which is a design ceiling for power conversion, not a statement of what the rack draws in operation; NVIDIA's page publishes no rack power figure ^[1].

The next generation on paper

NVIDIA's Vera Rubin NVL72 page lists the same 72-GPU layout with Rubin GPUs and Vera CPUs ^[2]. The headline specifications, as published by NVIDIA, compare as follows:

Capacity barely moves: about 20 TB of GPU memory in both. Bandwidth is what changes, with memory bandwidth listed at more than double and NVLink at roughly 1.7 times. That matters because serving large models is often limited by how fast weights and context can be moved, not by raw compute. NVIDIA's performance and cost claims for Vera Rubin, including one-tenth the cost per million tokens versus GB200 NVL72 (measured on a specific model test, Kimi-K2-Thinking at 32K/8K input/output lengths), carry "subject to change" disclaimers on its page ^[2], and we treat them as vendor claims.

What cooling actually has to do

Concentrating 72 accelerators in one rack concentrates heat. NVIDIA describes the GB300 NVL72 as a "fully liquid-cooled, rack-scale architecture" ^[1]. Lenovo's brief is more precise about its implementation: a hybrid design that relies primarily on warm-water liquid cooling across CPUs, GPUs, NVLink switches and power subsystems, with supplemental airflow handling the remaining heat. It lists warm-water operation at 18 to 45 degC, blind-mate manifolds so trays can be serviced without draining coolant, and internal leak detection ^[3].

Two planning consequences follow. First, a facility needs a liquid loop to the rack, with the heat exchange equipment that implies, even if it does not need chilled water ^[3]. Second, some air handling remains. A hall planned as "fully liquid" may still have residual air-side heat to manage, a detail worth confirming in the vendor's integration guide rather than the marketing page.

The chiller debate

At CES in January, NVIDIA CEO Jensen Huang said a rack running its newest chips "doesn't need a chiller," citing 45 degC coolant, the same as Grace Blackwell, and describing the Vera Rubin rack as 100% liquid-cooled. He said Vera Rubin uses about twice the overall power of Grace Blackwell while delivering five times the peak inference performance ^[4]. Shares of HVAC companies including Johnson Controls fell on the remark, and most except Johnson Controls later recovered ^[4].

LiquidStack CEO Joe Capes pushed back: chillers could still be needed in halls running only Vera Rubin because of high secondary-loop temperatures, although those temperatures may allow more free cooling with dry coolers. He also expects Vera Rubin to increase demand for liquid cooling, since denser chips concentrate heat ^[4]. Both statements can be true: the rack itself tolerates warm water, while the site design and climate decide whether mechanical cooling is still needed.

What buyers should check

• Power: ask for the rack's rated and typical draw, and the busway and breaker provisioning the vendor requires. Shelf capacity is not a load number.

• Coolant: confirm supply temperature range, coolant chemistry (Lenovo lists PG25 or deionized water ^[3]) and heat-exchanger sizing.

• Air residual: ask how much heat the rack still rejects to air.

• Serviceability: blind-mate manifolds and leak detection reduce risk but make site plumbing standards part of your warranty conversation ^[3].

• Rental alternative: most organizations do not need to own a rack to use one.

Practical implication

Unless you operate your own data halls, the useful consequence of rack-scale systems is on the price sheet and the cloud region list: capacity is gated by power and cooling, not just chip supply. If you are evaluating on-premises AI, get the facility requirements in writing before you price the hardware.