Share This:

It is a really good idea to run your data center warmer these days. Except when it isn't.


Data centers need capacity planning, just like the computers that run in them. One size does not fit all. There are standards like ASHRAE that give great ideas and guidance, but at the end of it all, building a green data center requires more than just blind adherence to some standards. At the same time, physics underlies it all. There are some things you can do, and others that are ... Suboptimal.


Example: the most recent set ASHREA standards indicate that it is good to run your data center warmer than in the past. The range of allowable humidity is higher too.


If I were to blindly follow that, I would find computers failing all over the place. The reason is simple: no one told the older computers about it being ok to run warmer. A natural consequence of BMC's heterogenous and deep platform support is that we have quite a number of computers designed and built before the new standards were set. They like it cold.


It is even more complicated than that though. You knew it had to be.


Fans and Power


Fans use power as the cube of their rotational speed. Not the square. Not linear. Airflow is linear.


The best example of that I have seen was when I was building a new data center in San Jose. The DC was going into an existing structure, but we had more or less gutted the building and custom built out the space to meet our needs. I had loaded up a fair number of servers into the shiny new data center, as we had certified and commissioned the new DC before we had started moving the people into the adjacent office space.


Building management had scheduled the fire marshal to come over one evening to look at the fire panel for the occupied space. Someone tried the alarm on the panel, which was connected incorrectly to the UPS, and the UPS dropped the line power to the building. I guess this was a good thing, as we found out about the incorrect configuration of the firepanel / UPS.


There was no backup generator, so the data center HVAC went off line, but because the UPS was working, the servers stayed up and running. I happened to have been in the UPS room looking at the UPS control screens because it was a new model that I was not familiar with, and I was learning how to drive the various displays. Set up SMTP and SNMP, etc.


From the UPS room, I could hear the sounds of the data center, and therein began a howl. Slowly at first, but building to a nearly deafening roar, every cooling fan in every server came online, and sped up in increments to its maximum speed. I had no idea how quiet the room had been till it wasn't.


I watched the power drain on the UPS with interest. It increased from 160 KVA to 270 KVA. To drive the fans at their maximum speed was chewing power at an incredible rate. Even for all that, and for the outage not being all that long... about 20 minutes... I had two older computers lose hard drives from the heat.


The point here is that ASHRAE says that it is OK to warm your cold aisles, and it is to some degree, but what degree that is will depend utterly on what type of computers you have, and at what temperature they are going to start cranking up their fans to stay cool.


It is not more power efficient spend less on A/C power if you are spending more on making fans spin faster. What that inflection point is I can not say. Data required. I know for us it is not very far away from the old ASHRAE standards.


Too Much of a Good Thing


Your data center A/C, be it CRAC's or CRAH's like temperature differential. Give or take and elephant, 30 degrees F is good. In Hot/Cold aisles, with no air mixing, that means that if your cold aisle is 68 degrees, your hot aisle is 98 degrees. No one is going to want to spend much time in the hot aisle, and when they do they will be wearing Hawaiian shirts and Bermuda Shorts. By keeping the air from mixing, your A/C can be as much as 50% more efficient. Since DC HVAC is 40% of the power bill (and therefore contributes hugely to your CO2 emissions) running the HVAC at maximum efficiency it paramount to a Green DC.


Some people like to go to warmer climate's, especially during the winter, but there are limits. Increase the temperature in the cold aisle to the ASHAE maximum (assuming all your gear is new enough to be able to run at that temperature) and your cold aisle is at 80 degrees. People are not utterly uncomfortable at that temperature, though they will probably be thinking about putting on the shorts and tennis shoes. The hot aisle is another story. At 110 degrees, there will have to be hazard pay to go in there. Next to the fire extinguisher will be hydration stations.


To drive that kind of temperature differential also requires the servers be racked at a density that can generate that level of heat. Lots of ifs and caveats here, and so once again you have to know exactly what kinds of servers and densities are even possible with your specific mix of servers. In some cases, our gear is old enough that even though the power supplies are not very efficient, the server is so large that I can not pack them together that close. In the case of the old Tandems, I can't easily rack them at all...



Too much HVAC is inefficient and a poor use of power and therefore carbon-footprint intensive. Not enough and your servers run too hot and fail. It is fairly easy to figure out how much A/C is the right amount, as noted in my two previous posts here ("By The Numbers" and "Whats in a Name(plate)"), but not addressed there is the idea of failure, in the sense of what to do when you lose an HVAC unit of some sort. Things fail. Entropy is law.


A recent case for us was where a 10 ton CRAC failed in one of the R&D DC's. There were three other units in the DC: two 20 ton units, and another 10 ton. The problem was that there was not enough cooling left in the surviving units to deal with the heat load until the 10 ton could be repaired. 


In theory there should have been one more 20 ton unit available: powered down, but piped into the common plenum so that it could assume the workload of the largest possible single failure. Alternatively (and what we did) about 10 tons of heat load had to be powered off till the HVAC technical crew had a chance to get the unit repaired. In this case, parts had to be ordered, and there was a multi-day wait. Supplemental air was brought in. Not pretty.




We had some latitude in how this was dealt with because the room is normally cooled to about 70 F in the cold aisle. That extra 10 degrees bought us time, and meant that we did not have to power down quite as many systems. We could, for a short time, run warmer.


For this lab, an idle 20 ton unit would be a hugely expensive investment relative to the rooms workload. A spare HVAC unit may make sense when it is 5% of the total room or something, but not when it is 30%. Then it is expensive insurance.


The fans ran faster while the 10 ton was being repaired, and we used more power because of that for the duration, but it was a short duration.



I am not saying that one should pay no heed to ASHRAE: Far from it. I am saying that in the effort to both design and run a green data center, understand that ASHRAE issues guidelines, not rules of nature like second law of thermodynamics. Apply them knowledgably to your particular set of servers, and also to the future plans for the data center.


The lessons of BSM are clear: You can not manage what you can not measure (in this case, the potential heat load), and to manage effectively (I.E., to be efficient) not only saves you money, it makes your DC greener.