Abstract

Through the application of cryogenic cooling via liquid nitrogen (LN2), the power consumption of a CPU was substantially reduced. Using a digitally controlled solenoid valve and an additively manufactured cold plate, the manual process of LN2 cooling was automated for precise control of cold plate temperature. The power consumption and frequency relationship of the processor were established across three different thermal solutions to demonstrate the effect of temperature on this relationship. It was found that power consumption of the processor decreased at lower temperatures due to a reduction in current leakage and the core voltage necessary for stable operation. This culminated in a reduction of up to 10.7% in processor power consumption for the automated solution and 21.5% for the manual LN2 solution when compared to the air-cooled baseline. Due to the binary nature of the solenoid valve used, flow rate was tuned via an in-line needle valve to increase thermal stability. It was found that for lower flow rates, approximately 5.0 g/s, temperatures oscillated within a range of ±11.5 °C while for higher flow rates of 10–12 g/s, generated amplitudes are as small as ±3.5 °C. Additionally, several tests measured the rate of LN2 consumption and found that the automated solution used 230%–280% more coolant than the manual thermal solution, implying there is room for improvement in the cold plate geometry, LN2 vapor exhaust design, and coolant delivery optimization.

References

1.
Ellsworth
,
M. J.
, and
Iyengar
,
M. K.
,
2009
, “
Energy Efficiency Analyses and Comparison of Air and Water Cooled High Performance Servers
,”
ASME
Paper No. InterPACK2009-89248.10.1115/InterPACK2009-89248
2.
Tuma
,
P. E.
,
2010
, “
The Merits of Open Bath Immersion Cooling of Datacom Equipment
,” 26th Annual IEEE Semiconductor Thermal Measurement and Management Symposium (
SEMI-THERM
), Santa Clara, CA, Feb. 21–25, pp.
123
131
.10.1109/STHERM.2010.5444305
3.
Avelar, V., Azevedo, D., and French, A., The Green Grid
,
2007
, “
The Green Grid Power Efficiency Metrics; PUE and DCiE
,” Beaverton, OR, accessed July 24, 2020, www.thegreengrid.org
4.
Eiland
,
R.
,
Fernandes
,
J.
,
Vallejo
,
M.
,
Agonafer
,
D.
, and
Mulay
,
V.
,
2014
, “
Flow Rate and Inlet Temperature Considerations for Direct Immersion of a Single Server in Mineral Oil
,”
Proceedings of the IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems
, Orlando, FL, May 27–30, pp.
706
714
.10.1109/ITHERM.2014.6892350
5.
Xu
,
H.
,
Feng
,
C.
, and
Li
,
B.
,
2013
, “
Temperature Aware Workload Management in Geo-Distributed Datacenters
,”
Proceedings of the 10th International Conference on Autonomic Computing
, San Jose, CA, June 26–28, pp.
303
314
.https://www.usenix.org/system/files/conference/icac13/icac13_xu_1.pdf
6.
Dou
,
H.
, and
Qi
,
Y.
,
2017
, “
An Online Electricity Cost Budgeting Algorithm for Maximizing Green Energy Usage Across Data Centers
,”
Front. Comput. Sci.
,
11
(
4
), pp.
661
674
.10.1007/s11704-016-5420-y
7.
Lei
,
N.
, and
Masanet
,
E.
,
2020
, “
Statistical Analysis for Predicting Location-Specific Data Center PUE and Its Improvement Potential
,”
Energy
,
201
, p.
117556
.10.1016/j.energy.2020.117556
8.
Alkharabsheh
,
S.
,
Fernandes
,
J.
,
Gebrehiwot
,
B.
,
Agonafer
,
D.
,
Ghose
,
K.
,
Ortega
,
A.
,
Joshi
,
Y.
, and
Sammakia
,
B.
,
2015
, “
A Brief Overview of Recent Developments in Thermal Management in Data Centers
,”
ASME J. Electron. Packag.
,
137
(
4
), p.
040801
.10.1115/1.4031326
9.
Turkmen
,
I.
,
Mercan
,
C. A.
, and
Erden
,
H. S.
,
2020
, “
Experimental and Computational Investigations of the Thermal Environment in a Small Operational Data Center for Potential Energy Efficiency Improvements
,”
ASME J. Electron. Packag.
,
142
(
3
), p.
031116
.10.1115/1.4047845
10.
Kasukurthy
,
R.
,
Rachakonda
,
A.
, and
Agonafer
,
D.
,
2021
, “
Design and Optimization of Control Strategy to Reduce Pumping Power in Dynamic Liquid Cooling
,”
ASME J. Electron. Packag.
,
143
(
3
), p.
031001
.10.1115/1.4049018
11.
Demetriou
,
D. W.
,
Kamath
,
V.
, and
Mahaney
,
H.
,
2016
, “
A Holistic Evaluation of Data Center Water Cooling Total Cost of Ownership
,”
ASME J. Electron. Packag.
,
138
(
1
), p.
010912
.10.1115/1.4032494
12.
Choo
,
K.
,
Galante
,
R. M.
, and
Ohadi
,
M. M.
,
2014
, “
Energy Consumption Analysis of a Medium-Size Primary Data Center in an Academic Campus
,”
Energy Build.
,
76
, pp.
414
421
.10.1016/j.enbuild.2014.02.042
13.
Petrongolo
,
J.
,
Nemati
,
K.
, and
Fouladi
,
K.
,
2020
, “
Simulation-Based Assessment of Performance Indicator for Data Center Cooling Optimization
,”
ASME J. Therm. Sci. Eng. Appl.
,
12
(
5
), p.
051009
.10.1115/1.4045962
14.
Jin
,
C.
,
Bai
,
X.
, and
Yang
,
C.
,
2019
, “
Effects of Airflow on the Thermal Environments and Energy Efficiency in Raised-Floor Data Centers: A Review
,”
Sci. Total Environ.
,
695
, p.
133801
.10.1016/j.scitotenv.2019.133801
15.
Demetriou
,
D. W.
, and
Khalifa
,
H. E.
,
2012
, “
Optimization of Enclosed Aisle Data Centers Using Bypass Recirculation
,”
ASME J. Electron. Packag.
,
134
(
2
), p.
020904
.10.1115/1.4005907
16.
Song
,
Z.
,
2016
, “
Numerical Cooling Performance Evaluation of Fan-Assisted Perforations in a Raised-Floor Data Center
,”
Int. J. Heat Mass Transfer
,
95
, pp.
833
842
.10.1016/j.ijheatmasstransfer.2015.12.060
17.
Cataldo
,
F.
,
Amalfi
,
R. L.
,
Marcinichen
,
J. B.
, and
Thome
,
J. R.
,
2020
, “
Implementation of Passive Two-Phase Cooling to an Entire Server Rack
,”
Proceedings of the 19th Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm)
, Orlando, FL, July 21–23, pp.
396
401
.10.1109/ITherm45881.2020.9190327
18.
Huang
,
H.
,
Quan
,
G.
, and
Fan
,
J.
,
2010
, “
Leakage Temperature Dependency Modelling in System Level Analysis
,”
Proceedings of the 11th International Symposium on Quality Electronic Design
(
ISQED
), San Jose, CA, Mar. 22–24, pp.
447
452
.10.1109/ISQED.2010.5450539
19.
Gough
,
C.
,
Steiner
,
I.
, and
Saunders
,
W.
,
2015
, “
CPU Power Management
,”
Energy Efficient Servers
, Apress, Berkeley, CA, pp.
21
70
. 10.1007/978-1-4302-6638-9
20.
Sadiqbatcha
,
S.
,
Zhao
,
H.
,
Amrouch
,
H.
,
Henkel
,
J.
,
Sheldon
,
X.-D.
, and
Tan
,
2019
, “
Hot Spot Identification and System Parameterized Thermal Modelling for Multi-Core Processors Through Infrared Thermal Imaging
,” Proceedings of Design, Automation, and Test in Europe Conference and Exhibition, Florence, Italy, Mar. 25–29, pp.
48
53
.10.23919/DATE.2019.8714918
21.
Jones
,
J.
,
2000
,
Contact Mechanics
,
Cambridge University Press
,
Cambridge, UK
, Chap.
6
.
22.
Aung
,
W. W.
,
2014
, “
Speedup Thin Clients With Overclocking Method for Resource Intensive Tasks
,”
IEEE International Conference on MOOC
,
Innovation and Technology in Education (MITE)
, Patiala, India, Dec. 19–20, pp.
42
46
.10.1109/MITE.2014.7020238
23.
Gizopoulos
,
D.
,
Papadimitriou
,
G.
,
Chatzidimitriou
,
A.
,
Reddi
,
V. J.
,
Salami
,
B.
,
Unsal
,
O. S.
,
Kestelman
,
A. C.
, and
Leng
,
J.
,
2019
, “
Modern Hardware Margins: CPUs, GPUs, FPGAs Recent System-Level Studies
,” IEEE 25th International Symposium on On-Line Testing and Robust System Design (
IOLTS
), Rhodes, Greece, July 1–3, pp.
129
134
.10.1109/IOLTS.2019.8854386
24.
Koutsovasilis
,
P.
,
Parasyris
,
K.
,
Antonopoulos
,
C. D.
,
Bellas
,
N.
, and
Lalis
,
S.
,
2020
, “
Dynamic Undervolting to Improve Energy Efficiency on Multicore X86 CPUs
,”
IEEE Trans. Parallel Distributed Syst.
,
31
(
12
), pp.
2851
2864
.10.1109/TPDS.2020.3004383
25.
Kim
,
H. S.
,
Vijaykrishnan
,
N.
,
Kandemir
,
M.
, and
Irwin
,
M. J.
,
2003
, “
Adapting Instruction Level Parallelism for Optimizing Leakage in VLIW Architectures
,”
SIGPLAN Not.
38
(
7
), pp.
275
283
.10.1145/780731.780770
26.
Fallah
,
F.
, and
Pedram
,
M.
,
2005
, “
Standby and Active Leakage Current Control and Minimization in CMOS VLSI Circuits
,”
IEICE Trans. Electron.
,
E88–C
(
4
), pp.
509
519
.10.1093/ietele/e88-c.4.509
27.
WikiChip
,
2019
, “
Thermal Velocity Boost (TVB) – Intel
,” accessed July 26, 2021, https://en.wikichip.org/wiki/intel/thermal_velocity_boost
28.
Patterson
,
M. K.
,
2008
, “
The Effect of Data Center Temperature on Energy Efficiency
,”
2008 11th Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems
, Orlando, FL, May 28–31, pp.
1167
1174
.10.1109/ITHERM.2008.4544393
29.
Singh
,
N.
, and
Dhir
,
V.
,
2019
, “
Competitive Analysis of Energy Aware VW Migration Algorithms in Cloud Computing
,”
Int. J. Develop. Admin. Res.
,
2
(
2
), pp.
36
49
.https://so02.tcithaijo.org/index.php/ijdar/article/view/247223
30.
Chun
,
B.-G.
,
Iannaccone
,
G.
,
Iannaccone
,
G.
,
Katz
,
R.
,
Lee
,
G.
, and
Niccolini
,
L.
,
2010
, “
An Energy Case for Hybrid Datacenters
,”
ACM SIGOPS Operat. Syst. Rev.
,
44
(
1
), pp.
76
80
.10.1145/1740390.1740408
31.
Energy Efficiency Baselines for Data Centers,
2013
, Statewide Customized New Construction and Customized Retrofit Incentive Programs Integral Group, Oakland, CA.
32.
Campbell
,
L.
,
Chu
,
R.
,
David
,
M.
,
Ellsworth
,
M.
,
Iyengar
,
M.
, and
Simons
,
R.
,
2011
, “
Multi-Fluid, Two-Phase Immersion-Cooling of Electronic Component(s)
,” U.S. Patent No. US8619425B2.
33.
Woltman
,
G.
, and
Kurowski
,
S.
,
2008
, “
GIMPS the Great Internet Mersenne Prime Search
,” accessed Aug. 24, 2020, https://www.mersenne.org/
34.
Byun
,
I.
,
Min
,
D.
,
Lee
,
G.-h.
,
Na
,
S.
, and
Kim
,
J.
,
2020
, “
Cryocore: A Fast and Dense Processor Architecture for Cryogenic Computing
,”
Proceedings of ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA)
, Valencia, Spain, May 30–June 3, pp.
335
348
.10.1109/ISCA45697.2020.00037
35.
Travers
,
M.
,
CPU Power Consumption Experiments and Results Analysis of Intel i7-4820K
,
Newcastle University
, Newcastle upon Tyne, UK.
36.
Wang
,
S. L.
,
Chen
,
C. A.
,
Lin
,
Y. L.
, and
Lin
,
T. F.
,
2012
, “
Transient Oscillatory Saturated Flow Boiling Heat Transfer and Associated Bubble Characteristics of FC-72 Over a Small Heated Plate Due to Heat Flux Oscillation
,”
Int. J. Heat Mass Transfer
,
55
(
4
), pp.
864
873
.10.1016/j.ijheatmasstransfer.2011.10.022
37.
Khalili
,
S.
,
Rangarajan
,
S.
,
Sammakia
,
B.
, and
Gektin
,
V.
,
2020
, “
An Experimental Investigation on the Fluid Distribution in a Two-Phase Cooled Rack Under Steady and Transient Information Technology Loads
,”
ASME J. Electron. Packag.
,
142
(
4
), p.
041002
.10.1115/1.4048180
38.
Mehta
,
B.
, and
Khandekar
,
S.
,
2015
, “
Local Experimental Heat Transfer of Single-Phase Pulsating Laminar Flow in a Square Mini-Channel
,”
Int. J. Therm. Sci.
,
91
, pp.
157
166
.10.1016/j.ijthermalsci.2015.01.008
39.
Chandratilleke
,
T. T.
,
Narayanaswamy
,
R.
, and
Jagannatha
,
D.
,
2011
, “
Thermal Performance Evaluation of a Synthetic Jet Heat Sink for Electronic Cooling
,”
2011 IEEE 13th Electronics Packaging Technology Conference
, Singapore, Dec. 7–9, pp.
79
83
.10.1109/EPTC.2011.6184390
40.
Ohadi
,
M. M.
,
Dessiatoun
,
S.
,
Choo
,
V.
,
Pecht
,
K. M.
, and
Lawler
,
J. V.
,
2012
, “
A Comparison Analysis of Air, Liquid, and Two-Phase Cooling of Data Centers
,” 2012 28th Annual IEEE Semiconductor Thermal Measurement and Management Symposium (
SEMI-THERM
), San Jose, CA, Mar. 18–22, pp.
58
63
.10.1109/STHERM.2012.6188826
You do not currently have access to this content.