OpenStack Neat: a framework for dynamic and energy-efficient consolidation of virtual machines in OpenStack clouds
Corresponding Author
Anton Beloglazov
Cloud Computing and Distributed Systems (CLOUDS) Laboratory, Department of Computing and Information Systems, The University of Melbourne, Melbourne, Australia
Correspondence to: Anton Beloglazov, CLOUDS Lab, Department of Computing and Information Systems, The University of Melbourne, Melbourne, Australia.
E-mail: [email protected]
Search for more papers by this authorRajkumar Buyya
Cloud Computing and Distributed Systems (CLOUDS) Laboratory, Department of Computing and Information Systems, The University of Melbourne, Melbourne, Australia
Search for more papers by this authorCorresponding Author
Anton Beloglazov
Cloud Computing and Distributed Systems (CLOUDS) Laboratory, Department of Computing and Information Systems, The University of Melbourne, Melbourne, Australia
Correspondence to: Anton Beloglazov, CLOUDS Lab, Department of Computing and Information Systems, The University of Melbourne, Melbourne, Australia.
E-mail: [email protected]
Search for more papers by this authorRajkumar Buyya
Cloud Computing and Distributed Systems (CLOUDS) Laboratory, Department of Computing and Information Systems, The University of Melbourne, Melbourne, Australia
Search for more papers by this authorSummary
Dynamic consolidation of virtual machines (VMs) is an efficient approach for improving the utilization of physical resources and reducing energy consumption in cloud data centers. Despite the large volume of research published on this topic, there are very few open-source software systems implementing dynamic VM consolidation. In this paper, we propose an architecture and open-source implementation of OpenStack Neat, a framework for dynamic VM consolidation in OpenStack clouds. OpenStack Neat can be configured to use custom VM consolidation algorithms and transparently integrates with existing OpenStack deployments without the necessity of modifying their configuration. In addition, to foster and encourage further research efforts in the area of dynamic VM consolidation, we propose a benchmark suite for evaluating and comparing dynamic VM consolidation algorithms. The proposed benchmark suite comprises OpenStack Neat as the base software framework, a set of real-world workload traces, performance metrics and evaluation methodology. As an application of the proposed benchmark suite, we conduct an experimental evaluation of OpenStack Neat and several dynamic VM consolidation algorithms on a five-node testbed, which shows significant benefits of dynamic VM consolidation resulting in up to 33% energy savings. Copyright © 2014 John Wiley & Sons, Ltd.
References
- 1 Koomey J. Growth in Data Center Electricity Use 2005 to 2010. Analytics Press: Oakland, CA, 2011.
- 2 Gartner, Inc. Gartner Estimates ICT Industry Accounts for 2 Percent of Global CO2 Emissions. Gartner Press: Stamford, CT, USA, Release (April 2007).
- 3 The Open Compute project—energy efficiency. (Available from: http://opencompute.org/about/energy-efficiency/) [Accessed on 21 November 2012].
- 4 Hermenier F, Lorca X, Menaud J, Muller G, Lawall J. Entropy: a consolidation manager for clusters. Proceedings of the 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Washington, DC, USA, 2009; 41–50.
- 5 Feller E, Rilling L, Morin C. Snooze: a scalable and autonomic virtual machine management framework for private clouds. Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), Ottawa, Canada, 2012; 482–489.
- 6 Verma A, Dasgupta G, Nayak TK, De P, Kothari R. Server workload analysis for power minimization using consolidation. Proceedings of the 2009 USENIX Annual Technical Conference, San Diego, CA, USA, 2009; 28–41.
- 7 Speitkamp B, Bichler M. A mathematical programming approach for server consolidation problems in virtualized data centers. IEEE Transactions on Services Computing (TSC) 2010; 3(4): 266–278.
- 8 Cardosa M, Korupolu M, Singh A. Shares and utilities based power consolidation in virtualized server environments. Proceedings of the 11th IFIP/IEEE Integrated Network Management (IM), Long Island, NY, USA, 2009; 327–334.
- 9 Kumar S, Talwar V, Kumar V, Ranganathan P, Schwan K. vManage: loosely coupled platform and virtualization management in data centers. Proceedings of the 6th International Conference on Autonomic Computing (ICAC), Barcelona, Spain, 2009; 127–136.
- 10 Kusic D, Kephart JO, Hanson JE, Kandasamy N, Jiang G. Power and performance management of virtualized computing environments via lookahead control. Cluster Computing 2009; 12(1): 1–15.
- 11
Kusic D,
Kandasamy N,
Jiang G. Combined power and performance management of virtualized computing environments serving session-based workloads. IEEE Transactions on Network and Service Management (TNSM) 2011; 8(3): 245–258.
10.1109/TNSM.2011.0726.100045 Google Scholar
- 12 Jung G, Hiltunen MA, Joshi KR, Schlichting RD, Pu C. Mistral: dynamically managing power, performance, and adaptation cost in cloud infrastructures. Proceedings of the 30th International Conference on Distributed Computing Systems (ICDCS), Genoa, Italy, 2010; 62–73.
- 13 Wang X, Wang Y. Coordinating power control and performance management for virtualized server clusters. IEEE Transactions on Parallel and Distributed Systems (TPDS) 2011; 22(2): 245–259.
- 14 Guenter B, Jain N, Williams C. Managing cost, performance, and reliability tradeoffs for energy-aware server provisioning. Proceedings of the 30th Annual IEEE International Conference on Computer Communications (INFOCOM), Shanghai, China, 2011; 1332–1340.
- 15 Ardagna D, Panicucci B, Trubian M, Zhang L. Energy-aware autonomic resource allocation in multitier virtualized environments. IEEE Transactions on Services Computing (TSC) 2012; 5(1): 2–19.
- 16 Berral JL, Goiri Í, Nou R, Julià F, Guitart J, Gavaldà R, Torres J. Towards energy-aware scheduling in data centers using machine learning. Proceedings of the 1st International Conference on Energy-Efficient Computing and Networking, Passau, Germany, 2010; 215–224.
- 17 Yang J, Zeng K, Hu H, Xi H. Dynamic cluster reconfiguration for energy conservation in computation intensive service. IEEE Transactions on Computers 2012; 61(10): 1401–1416.
- 18 Carrera D, Steinder M, Whalley I, Torres J, Ayguadé E. Autonomic placement of mixed batch and transactional workloads. IEEE Transactions on Parallel and Distributed Systems (TPDS) 2012; 23(2): 219–231.
- 19 Goiri Í, Berral J, Oriol Fitó J, Julià F, Nou R, Guitart J, Gavaldà R, Torres J. Energy-efficient and multifaceted resource management for profit-driven virtualized data centers. Future Generation Computer Systems (FGCS) 2012; 28(5): 718–731.
- 20
Nathuji R,
Schwan K. VirtualPower: coordinated power management in virtualized enterprise systems. ACM SIGOPS Operating Systems Review 2007; 41(6): 265–278.
10.1145/1323293.1294287 Google Scholar
- 21 Zhu X, Young D, Watson BJ, Wang Z, Rolia J, Singhal S, McKee B, Hyser C, Gmach D., Gardner R., Christian T., Cherkasova L. 1000 Islands: integrated capacity and workload management for the next generation data center. Proceedings of the 5th International Conference on Autonomic Computing (ICAC), Chicago, IL, USA, 2008; 172–181.
- 22 Bobroff N, Kochut A, Beaty K. Dynamic placement of virtual machines for managing SLA violations. Proceedings of the 10th IFIP/IEEE International Symposium on Integrated Network Management (IM), Munich, Germany, 2007; 119–128.
- 23 Nandi B, Banerjee A, Ghosh S, Banerjee N. Stochastic VM multiplexing for datacenter consolidation. Proceedings of the 9th IEEE International Conference on Services Computing (SCC), Honolulu, HI, USA, 2012; 114–121.
- 24 Wood T, Shenoy P, Venkataramani A, Yousif M. Black-box and gray-box strategies for virtual machine migration. Proceedings of the 4th USENIX Symposium on Networked Systems Design & Implementation, Cambridge, MA, USA, 2007; 229–242.
- 25 Feller E, Rohr C, Margery D, Morin C. Energy management in IaaS clouds: a holistic approach. Proceedings of the 5th IEEE International Conference on Cloud Computing (IEEE CLOUD), Honolulu, HI, USA, 2012; 204–212.
- 26 Beloglazov A, Buyya R, Lee YC, Zomaya A. A taxonomy and survey of energy-efficient data centers and cloud computing systems. In Advances in Computers, Vol. 82, M Zelkowitz (ed.). Academic Press: Waltham, MA, USA, 2011; 47–111.
- 27 Beloglazov A, Abawajy J, Buyya R. Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Future Generation Computer Systems (FGCS) 2011; 28(5): 755–768.
- 28 Beloglazov A, Buyya R. Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers. Concurrency and Computation: Practice and Experience (CCPE) 2012; 24(13): 1397–1420.
- 29 Beloglazov A, Buyya R. Managing overloaded hosts for dynamic consolidation of virtual machines in cloud data centers under quality of service constraints. IEEE Transactions on Parallel and Distributed Systems (TPDS) 2013; 24(7): 1366–1379. DOI: 10.1109/TPDS.2012.240.
- 30 Rackspace hosting reports second quarter 2012 results. (Available from: http://ir.rackspace.com/phoenix.zhtml?c=221673&p=irol-newsArticle&ID=1723357) [Accessed on 6 November 2012].
- 31
Park KS,
Pai VS. CoMon: a mostly-scalable monitoring system for PlanetLab. ACM SIGOPS Operating Systems Review 2006; 40(1): 65–74.
10.1145/1113361.1113374 Google Scholar
- 32 Beloglazov A, Piraghaj SF, Alrokayan M, Buyya R. Deploying OpenStack on CentOS using the KVM hypervisor and GlusterFS distributed file system. Technical Report, CLOUDS-TR-2012-3, CLOUDS Laboratory, The University of Melbourne: Australia, 2012.
- 33 Meisner D, Gold B, Wenisch T. PowerNap: eliminating server idle power. ACM SIGPLAN Notices 2009; 44(3): 205–216.
- 34 MySQL Cluster. (Available from: http://www.mysql.com/products/cluster/) [Accessed on 23 November 2012].
- 35
Yue M. A simple proof of the inequality FFD (L) < 11/9 OPT (L)+ 1, for all l for the FFD bin-packing algorithm. Acta Mathematicae Applicatae Sinica (English Series) 1991; 7(4): 321–331.
10.1007/BF02009683 Google Scholar
- 36 Beloglazov A. Scripts for setting up and analyzing results of experiments using OpenStack Neat. (Available from: http://github.com/beloglazov/ccpe-2014-experiments) [Accessed on 10 November 2013].
- 37 Ubuntu 12.04 (Precise Pangolin) Cloud images. (Available from: http://uec-images.ubuntu.com/precise/current/) [Accessed on 22 November 2012].
- 38 Fan X, Weber WD, Barroso LA. Power provisioning for a warehouse-sized computer. Proceedings of the 34th Annual International Symposium on Computer Architecture (ISCA), San Diego, CA, USA, 2007; 13–23.
