Climate change is throwing a wrench into an unexpected place: the workings of data centres. As the frequency of heat waves grows worldwide, these critical pieces of technology infrastructure are melting down more and more often, threatening a foundational element of the internet.
In July, Google’s and Oracle’s London-based data centres were forced offline when Britain experienced record-high temperatures of over 40 degrees celsius. The heat wave rendered their data centre cooling systems useless and caused website outages for many customers. Namely, the Google outage impacted WordPress-hosted websites across Europe.
These cooling systems are designed to regulate heat emitted by servers and other data centre equipment. But when the internal heat meets the high external temperature of heat waves, cooling systems become overwhelmed and cannot do their job. As a result, vital data centre equipment overheats and goes out.
With heat waves now creeping into the autumn months, technology companies face prolonged disruption to their IT operations. In September, Twitter found itself in a “non-redundant state” when intense heat caused an outage at its Sacramento data centre, according to a company memo by former Twitter vice president of engineering Carrie Fernandez. She called the incident “unprecedented” and said the heat wave caused “the total shutdown of physical equipment”.
Despite heat waves becoming a common occurrence globally, companies such as Twitter are grossly underprepared for the havoc that intense heat can wreak on the technology industry. Peiter “Mudge” Zatko, ex-head of security at Twitter, revealed in a whistleblowing disclosure in August that Twitter is put at risk by “insufficient data centre redundancy”. He warned that “a temporary but overlapping outage of a small number of data centres” could cause Twitter to be knocked “offline for weeks, months, or permanently”. Now that Elon Musk has acquired Twitter and laid off large swathes of staff, it’s likely that the firm is even less prepared for heat waves than before.
Heat waves: A growing IT headache
Heat waves can cause major harm to businesses that rely heavily on IT services or that offer digital products. Steve Wright, chief operating officer at 4D Data Centres , warns that environmental conditions like intense heat have the ability to “damage IT equipment and cause power outages due to overloaded power grids”.
Companies that fail to properly maintain their data centres in the face of soaring heat can experience “server failure, hard drive crashes, and data loss”, according to Wright. “Any lapse in power can be devastating for a customer, with critical data files getting corrupted or lost, mainframes malfunctioning, and money being lost when systems overheat,” he says.
There are, however, solutions to this growing IT headache. Wright explains that data centre operators can begin by installing backup generators, which ensure power supply continues during an outage. He adds that they can also extend the lifespan of data centre servers and hard drives by tracking temperature and humidity.
Wright points to Microsoft’s success testing underwater data centres as a means to counter overheating. The tech giant called the concept “reliable and practical”, slashing its failure rate considerably compared to on-land data centres.
“For data centres, it is necessary to use high-energy cooling systems to combat rising temperatures, especially as a 2021 survey on US-based data centres revealed that 45% of data centre owners and operators responsible for managing infrastructure at the world’s largest IT organisations said extreme weather had threatened their continuous operations,” he explains.
Data centres’ cooling systems are weak points for the whole system
Cooling systems are designed to prevent data centre equipment from overheating, but the reality is that they’re simply not equipped to deal with record-breaking heat waves. Intense heat places strain on compressors, pumps, fans, and other cooling equipment, according to Daniel Bizo, research director at data centre think tank Uptime Institute Intelligence.
“Without going into technical depth, compressors, of which there are many types, are at the heart of mechanical refrigeration systems, such as air conditioners and water chiller systems. They use electrical power to compress a gaseous coolant which then later in the cycle expands (cooling coils exposed to ambient air or water) and cools down dramatically to create a cooling effect,” he explains.
“A pump in this example is a water pump that circulates facility water (in a chilled water system) around the data centre as a coolant (cooled by the compressors in the water chillers) to remove heat from computer room air handlers, in-row cooling units, and other heat exchange units. The harder they work, the greater the likelihood of failure.”
Unfortunately, cooling equipment isn’t the only vital component of data centres vulnerable to extreme heat. Bizo says backup generators and external power equipment can also be affected by heat waves. This “can reduce their ability to support the full capacity of the data centre, if called upon, should the grid experience heat-induced issues”.
Although prominent tech companies like Google, Oracle, and Twitter saw significant disruption to their services due to intense heat in 2022, there is some hope for the industry. Bizo explains that, minus a few notable exceptions, the vast majority of data centres survived “extreme temperatures without significant problems” during the summer. He attributes this to “appropriate power and cooling redundancy and good equipment maintenance hygiene.”
“Additionally, most data centres typically run at only moderate utilization levels. Operators can take advantage of spare cooling capacity to combat extreme heat,” he says. “ In contrast, cloud providers are more inclined to push their infrastructure closer to the limits and have less margin for error during extreme weather events.”
How to protect data centres from heat waves
As heat waves become more common and catastrophic for global technology infrastructure, data centre operators must shore up their defences against this very real threat. Luckily, Bizo is confident that there are lots of solutions for mitigating the fallout of extreme heat.
For starters, data centre operators can invest in evaporative and adiabatic cooling systems. Or they can supplement existing air conditioning and chiller units with sprinkler systems.
“Tolerating a few degrees higher temperature in the data hall helps lessen the stress on cooling systems; if your system uses ambient air cooling only (just cooled by outside air), an operator may want to consider upgrading it with an evaporation effect,” he says.
“An example could be to mist air around the cooling coils of the air-conditioner / chiller. New builds/major refurbishments can opt for cooling systems that by design use evaporation (or adiabatic effect, another physical phenomenon that relies on water absorbed into air), to cool ambient air down, as long as it is not too humid, for cooling effect.”
But he says “a more strategic, long-term response” to the climate crisis would be to adopt liquid-cooled IT systems. “Liquid cooled IT helps too, because it allows higher temperatures across the cooling ‘chain’. This is because unlike air, which needs to be supplied in the rage of 18-27C per industry recommendation, liquid (water, engineered fluid) directly to the servers can be 30+ C, even 40+ C in some cases, depending on implementation,” he says. “That means that, say, a chilled water system can be designed to deliver data centre cooling water at 32C. This requires massively less energy than cooling to under 15C (as an example), which is typical in many implementations.
With data centre operators already experiencing the harmful effects of heatwaves, Uptime’s stance is that they should conduct regular assessments to identify climate-related vulnerabilities and enforce solutions before time runs out.
Bizo adds: “As extreme weather events and other consequences of climate change become more severe and widespread, addressing climatic resilience is a modern business imperative.”
How data centres can beat the heat
Even though the technology industry is extremely vulnerable to heat waves, there’s no shortage of technological solutions to this problem.
CyrusOne, a provider of global data centres, has responded to extreme heat with closed-loop chilled water systems and air-cooled chillers. Kyle Myers, vice president of environmental health, safety, and sustainability at CyrusOne, describes them as an “energy-efficient means of providing cool water to our equipment”.
This system comprises a loop containing under 30,283 l of water and only needs to be filled once, whereas other operators typically consume tens of millions of gallons of water annually to cool each of their data centres. This one-time source of water is then cooled by an integrated compressor and condenser. Once the water is cold, it can lower temperatures inside the data centre.
He tells Gizmodo: “This process cools the IT gear in different temperature regions around the nation. Our air-cooled chillers come with economizers that allow us to leverage colder temperatures to reject heat from our chilled water more efficiently – while eliminating the need for makeup water sources to maintain operation.”
While limiting the effects of extreme heat on vital data centre equipment, this technology is also good for the environment. It doesn’t require a constant water source, and because there’s no need for a sewage pipe, pollutants aren’t released from the data centre.
Myers says: “For facilities that are dependent on water for cooling, they can burn through a tremendous amount of water during these periods to keep data centres cool. Fortunately, our modern build standard uses water-free cooling, so while our total electrical load can increase during this time, we’re not depleting water resources in the drought-stricken Phoenix region.”
But without investing in fancy systems like chilled-water systems, organisations can mitigate increased heat by making smarter decisions. For instance, Cirrus Nexus CIO Kelly Fleming recommends that organisations looking to move workloads to the cloud choose data centre regions that use renewable energy.
She also recommends: “Servers that don’t need to run 24/7 can be spun up and down when the energy consumed in their data centre region is at its cleanest, which can vary significantly depending on the energy sources powering it.”
The last few months have proved just how devastating heat waves can be for the global technology industry. And as increased heat shows no signs of slowing down, technology companies are clearly at a crossroads. If tech firms fail to monitor and mitigate extreme heat, worse outages seem almost certain.