GPU server cooling is no longer a technical detail – it is a precondition for deployment. A single NVIDIA B300 GPU draws up to 1,100 W, so a system with eight of them needs 8.8 kW for the GPUs alone. Here is when air is still enough, when you need liquid, and why this class of hardware does not belong in an office server room.
We write this as a company that supplies GPU servers (we are a Supermicro and NVIDIA partner) and operates them in our Tier III data centre in Poland, including in liquid-cooled racks. You will not find benchmark results or savings percentages here. We stick to vendor data and physics: every watt a server draws has to be removed as heat.
Why GPU server cooling became a problem
Practically all the electrical energy a server consumes turns into heat. A few datasheet numbers show the scale:
- NVIDIA HGX B300 (Blackwell Ultra): up to 1,100 W per GPU, 8 GPUs per system. That is 8.8 kW for the GPUs alone – before CPUs, memory, network cards, drives and fans.
- NVIDIA RTX PRO 6000 Blackwell Server Edition: up to 600 W per card (configurable). An eight-card server means up to 4.8 kW for the GPUs alone.
- NVIDIA GB300 NVL72: 72 GPUs and 36 Grace CPUs in a single rack, liquid-cooled only.
- By contrast, DGX Spark: a GB10 chip with a 140 W TDP and a 240 W power supply – a book-sized unit that needs no special cooling.
So the problem is not “AI servers” in general, but a specific class of hardware: dense HGX and rack-scale systems. That is where the line between air and liquid cooling runs.
Air cooling: when it is still enough
The classic model works like this: precision air conditioning supplies cold air to a cold aisle, server fans pull it through the heatsinks, and hot air returns through the hot aisle to the cooling units. It is proven, easy to service, and every sysadmin knows it.
Air has physical limits, though. To remove more heat you have to move more air, which means faster fans and taller chassis. That is why air-cooled HGX systems are tall: Supermicro offers HGX B200 in 8U and 10U chassis and HGX B300 in 8U. Fans running flat out consume a noticeable share of the server’s power and are very loud. And when cooling cannot keep up, GPUs throttle their clocks, so you pay for performance you do not get.
Air remains the right choice for PCIe servers with RTX PRO 6000 (these are passive cards cooled by chassis airflow), for smaller configurations with a few GPUs, and for HGX systems when there are only a few of them in a rack.
Direct liquid cooling (DLC): how it works
With direct liquid cooling, cold plates are mounted on the hottest components – the GPUs and CPUs – and coolant flows through them. It picks up heat straight from the chips and carries it to a coolant distribution unit (CDU), and from there through a heat exchanger to the building’s water loop. Liquid moves heat far more effectively than air, so large heatsinks and fans are no longer needed.
You can see the effect in the hardware dimensions. Supermicro offers HGX B200 and B300 in 4U liquid-cooled versions, and B300 also in a 2-OU OCP format. Intel Gaudi 3 comes on two baseboard variants: the air-cooled HL-325L and the liquid-cooled HL-335. The rack-scale GB300 NVL72 has no air-cooled version at all.
Bear in mind that DLC usually does not capture all the heat. Power supplies, memory, drives and network cards still release heat into the air, so the data hall still needs air conditioning – just less of it relative to rack power.
Air vs liquid cooling compared
| Criterion | Air cooling | Direct liquid cooling (DLC) |
|---|---|---|
| B200 GPUs per rack (per Supermicro) | up to 32 | up to 96 |
| HGX system height | B200: 8U/10U; B300: 8U | B200 and B300: 4U; B300 also 2-OU OCP |
| Facility requirements | Precision air conditioning, hot/cold aisles | Plus CDUs, water loop, leak detection, service procedures |
| Fan noise and fan power | High under full GPU load | Clearly lower |
| Installation complexity and cost | Lower, widely available | Higher, requires prepared infrastructure |
| Servicing | Standard | Needs trained staff and spare parts for the loop |
| When it makes sense | PCIe servers, RTX PRO 6000, individual HGX systems | Dense Blackwell deployments, B300, rack-scale, training clusters |
Rack density: 32 or 96 B200 GPUs
According to Supermicro, an air-cooled rack fits up to 32 B200 GPUs (four 8-GPU systems), while a liquid-cooled rack fits up to 96 (twelve systems). A threefold difference in density has several consequences.
- Fewer racks, shorter links. The same cluster takes roughly a third of the racks, and the 800 Gb/s network cables between nodes are shorter.
- Far more power per rack. Three times as many GPUs in a rack means roughly three times the power you have to deliver to it and remove as heat. Power distribution (feeds, PDUs, UPS, generators) becomes as important as cooling.
- Different floor-space economics. With DLC you pay more for cooling infrastructure and less for floor space and rack count.
For comparison: if you are considering a few RTX PRO 6000 cards or DGX Spark, rack density barely affects you. For help choosing the right class of hardware, see the best GPU for LLMs in 2026.
Why an office server room cannot handle it
Office server rooms were designed for servers drawing hundreds of watts, not many kilowatts per box. With an HGX system, several things are usually missing at once:
- Power. An 8-GPU system needs dedicated, typically three-phase circuits and redundant power supplies connected to independent feeds.
- Heat removal. Comfort air conditioning is not designed for continuous operation at this load. If it fails, temperatures rise quickly and there is nothing to cool the hardware.
- A water loop. Liquid cooling needs CDUs, plumbing and leak detection – in an office building that is a construction project.
- UPS and generator capacity sized for GPUs, not for a few switches and a file server.
- Physical conditions. 8U–10U systems are heavy, and fan noise at full load rules out working next to them.
- 24/7 oversight – monitoring of temperature, power and hardware, with someone responding at night and at weekends.
Add physical security, fire suppression and connectivity on top. We look at the cost of running infrastructure in-house versus in colocation in our TCO comparison of cloud and GPU colocation.
What to check before ordering a GPU server
The cooling decision is made when you order, not after delivery. Air-cooled and liquid-cooled versions are usually different chassis and baseboard models, so changing your mind later effectively means replacing hardware. Before you order, establish:
- the maximum power draw of the whole system in the target configuration – from the vendor’s documentation, not just the GPU TDP,
- the cooling variant and whether the data centre that will host the hardware supports it,
- the power budget per rack and power redundancy (number of power supplies, independent feeds),
- expansion plans – if a multi-node cluster is likely within a year or two, consider liquid cooling from the start,
- networking – HGX B300 has integrated ConnectX-8 800 Gb/s adapters and Gaudi 3 has six 800 GbE ports; switches and cabling also produce heat and take up rack space,
- servicing – who replaces failed components and how quickly they respond.
GPU server cooling is in practice a joint project between the hardware supplier and the data centre operator. The earlier both sides agree on the parameters, the fewer surprises at installation.
Colocation in a Tier III data centre with liquid cooling
The more sensible route is to put the hardware in a data centre built for these loads. Our data centre in Poland is Tier III – redundant power and cooling that can be maintained without shutting equipment down – and ISO 27001 certified. Data stays in Poland and the EU.
What you get:
- GPU server colocation – we house your hardware in a rack matched to its power draw, including liquid-cooled racks for dense Blackwell systems.
- Hardware supply – as a Supermicro and NVIDIA partner, we specify, supply and install HGX B200/B300 systems, RTX PRO 6000 servers and Intel Gaudi 3 platforms in the right cooling variant.
- Operations – 24/7 monitoring, full administration, or root access if you prefer to manage things yourself.
If you would rather not buy hardware, you can rent a GPU server as a service – power, cooling and networking are then on us. We explain how that works in GPU as a Service.
FAQ: frequently asked questions
Can GPU servers be air-cooled?
Yes. PCIe servers with RTX PRO 6000 cards and individual HGX B200 or B300 systems are available in air-cooled versions. Liquid becomes necessary at high density – for example beyond 32 B200 GPUs per rack – and for rack-scale systems such as GB300 NVL72.
How much power does an 8-GPU B300 server draw?
The GPUs alone draw up to 1,100 W each, i.e. up to 8.8 kW. CPUs, memory, networking, drives and fans come on top, so the whole system draws more. The vendor specifies the exact figure for each configuration.
What is direct liquid cooling (DLC)?
Cold plates are mounted on the GPUs and CPUs, and coolant flows through them. The heat goes to a CDU and on to the building’s water loop. The remaining server components are still air-cooled.
Is liquid cooling safe for servers?
With a properly designed installation, yes. It does require leak detection, coolant quality control and trained staff – one of the reasons these systems are better placed in a professional data centre than in an office.
Does DGX Spark need special cooling?
No. DGX Spark has a 240 W power supply and the GB10 chip has a 140 W TDP. It is a desktop unit that only needs an ordinary ventilated spot or a place in a server rack.
Planning Blackwell servers or a GPU cluster and not sure how to power and cool them? Have a look at our server colocation with liquid-cooled racks and get in touch for a custom quote.