Transcript
FAES TELUS GARDEN THERMAL ENERGY SYSTEM
EXHIBIT C2-2
REQUESTOR NAME: BC Sustainable Energy Association and Sierra Club of B.C. INFORMATION REQUEST ROUND NO: 1 TO: FortisBC Alternative Energy Services (FAES) DATE: November 13, 2012 PROJECT NO: N/A APPLICATION NAME: Certificate of Public Convenience and Necessity Application for Telus Garden Thermal Energy System (TGTES) 1.0
Topic: Longevity of Data Centre Waste Heat supply Reference: Exhibit B-1, CPCN Application “The opportunity to make use of energy that would otherwise be wasted is attractive and has been shown to be superior to other heat source alternatives. Further, although all parties anticipate that the data centre will operate and produce waste heat for many years; prudent system design has included contingency planning in the event that use of other heat sources becomes necessary to continue to meet the needs of the Development.” [p.12, underline added]
2.0
1.1
Please provide evidence supporting the assertion that the Telus Data Centre will operate and produce waste heat (at the present location) for many years.
1.2
In the quotation above, FAES states: “prudent system design has included contingency planning in the event that use of other heat sources becomes necessary to continue to meet the needs of the Development.” [underline added] Please describe these contingency plans. Are the contingency plans limited to obtaining steam from Central Heat Distribution Ltd.?
Topic: Contract terms re Data Centre Waste Heat supply Reference: Exhibit B-1, CPCN Application 2.1
What commitment does FAES, or the Partnership, have from Telus that the Data Centre Waste Heat will be available and provided to the TGTES over the lifetime of the project?
2.2
What contracts or agreements set out the terms and conditions governing the supply of Data Centre Waste Heat to the Partnership? Please specify the parties to these contracts or agreements.
2.3
Please describe all the material terms and conditions governing the supply of Data Centre Waste Heat to the Partnership (or to FAES). 2.3.1
Please confirm that the price of the Data Centre Waste Heat is zero (“free” – p.24).
2.3.2
Is the (zero) price of the Data Centre Waste Heat guaranteed for the lifetime of the TGTES? For some other period of time? Is Telus (or its successors) precluded from reopening the price of the Data Centre Waste Heat at some time in the future?
BCSEA-SCBC IR1 FAES Telus Garden TES
3.0
November 13, 2012 Page 2 of 5
2.3.3
Are the Partnership’s (or FAES’s) contractual rights regarding the Data Centre Waste Heat protected in the event of a change of ownership of Telus?
2.3.4
What terms and conditions apply in the event that the Data Centre Waste Heat becomes permanently unavailable, for example due to the Telus Data Centre being moved to a different location?
2.3.5
Does the Partnership (or FAES) have a right, or obligation, to receive all of the Data Centre Waste Heat? Does Telus have a right to sell or otherwise transfer some of the Data Centre Waste Heat to a third party?
Topic: Longevity of Data Centre Waste Heat Reference: Exhibit B-1, CPCN Application, Appendix D, Cobalt Screening Study, 7.4 Option #4: Data Centre Waste Heat “The Data Center rejects a base heat load of 1,400kW at a reasonably constant rate 24/7, year round.” [p.2 of 43] 3.1
4.0
Did the Cobalt study address the availability of Data Centre Waste Heat over the lifetime of the project? 3.1.1
If so, what were the results? What specific assumptions and/or sources of data were utilized in this estimate?
3.1.2
If not, why not?
3.2
Please provide a graph and table showing the base heat load in kW produced by the Telus Data Centre over the past, say, ten years, and forecasted for the lifetime of the project.
3.3
How will the amount of Data Centre Waste Heat in future years be affected by: 3.3.1
Telus Data Centre energy efficiency improvements due to changes in server technology or changes in energy management practices, and
3.3.2
increases or decreases in the number of servers and other heatproducing equipment within the Telus Data Centre?
Topic: Forecast of Data Centre Waste Heat Reference: “ECOS Final Report: Canadian market Analysis for Servers and Data Centres, Nov 5, 2012” (Ecos Report), Attachment BCSEA IR 4.0 4.1
Is FAES familiar with the Ecos Report? Does FAES agree that the Ecos Report provides useful information regarding energy conservation and efficiency opportunities for data centres in Canada?
4.2
Please discuss the extent to which the energy conservation and efficiency measures identified in the Ecos Report are applicable to the Telus Data Centre? Have these measures already been implemented in the TDC?
BCSEA-SCBC IR1 FAES Telus Garden TES
November 13, 2012 Page 3 of 5
What plans does Telus have to implement energy efficiency and conservation measures at the TDC? Would implementing energy conservation and efficiency measures at the TDC reduce the Data Centre Waste Heat significantly? 4.3
Please discuss whether the use of Data Centre Waste Heat for the TGTES reduces the motivation to implement energy conservation and efficiency measures at the TDC.
“One fundamental area where energy can be saved in data centres is in server power supplies [SPS]… In actual operation, [SPSs] can waste 20 percent to 30 percent of the electricity that flows through them, turning it into heat during normal operation. … [SPSs] exist today that can achieve greater than 90 percent operational efficiency.”” [Ecos Report, p 26] 4.4
What percentage of the Telus Data Centre SPSs have been brought up to 2012 standards?
4.5
What impact on available energy (waste heat) will result if the percentage of SPSs at the TDC is substantially increased?
“Comatose servers are those that run applications no longer needed or run no applications at all, yet remain installed and operating continuously. …[S]uch servers account for up to 30 percent of total servers installed in some data centres.” [Ecos Report, p 26] 4.6
What percentage of servers, if any, are “comatose” at the TDC?
4.7
What impact on available energy will result if the percentage of comatose servers at the TDC is substantially reduced?
“According to a recent study… the average server is no more than 6 percent utilized… Maximizing the utilization of existing servers [through virtualization] therefore represents one of the most significant opportunities for energy savings….Based on these estimates, virtualization can save 57 percent of energy use.” [Ecos Report, pp. 26-27] 4.8
What percentage of servers have been virtualized at the TDC?
4.9
What impact on available energy will result if the percentage of servers at the TDC is substantially increased?
“A significant improvement in efficiency is also possible when replacing a linear [regulator] with a switcher [power regulator]. Most linear regulators have efficiencies in the 60 to 70 percent range. … Efficiency for switches… typically exceeds 95 percent and can result in significant power savings.” [Ecos Report, p 28] 4.10
Has power regulation been addressed at the TDC? What percentage of regulators have been updated to switchers?
4.11
What impact on available energy will result if the percentage of updated regulators at the TDC is substantially increased?
BCSEA-SCBC IR1 FAES Telus Garden TES 5.0
November 13, 2012 Page 4 of 5
Topic: Data Centre Waste Heat efficiencies References: 42U, Hot Aisle Containment Cooling Solutions, http://www.42u.com/cooling/hot-aisle-containment.htm Fontecchio, M. (2009, January 21). Data Center Air Conditioning Fans Blow Savings Your Way. Search Data Center: http://searchdatacenter.techtarget.com/news/article/0,289142,sid80_gci1345 584,00.html Miller, R. (2007, September 24). Data Center Cooling Set Points Debated. Data Center Knowledge: http://www.datacenterknowledge.com/archives/2007/09/24/data-centercooling-set-points-debated “Up to 37% of a data center's energy costs can be attributed to cooling infrastructure.” [Fontecchio] “Data center managers can save 4 percent in energy costs for every degree of upward change in the set point….If you’re running at 68 °F [20°C], you’re running at the bottom level of most of those ranges. ... There’s no reason why you can’t move to 78°C [26°C].” Mark Monroe, Director of Sustainable Computing at Sun Microsystems (JAVA). [quoted in Miller] 5.1
At what temperature is the TDC running its cooling setpoints?
5.2
What impact on available energy will result if the TDC was to implement substantially higher cooling setpoints?
The norm for data centre heat shedding is to provide hot and cold isles where cold air is supplied in a “cold” isle and removed in another “hot” isle. However, without containment of the isles the tendency is for the hot and cold air to mix through short circuiting, resulting in an average cold room temperature. One increasingly popular concept is to physically isolate hot isles from cold isles in order to prevent mixing of hot air with the cold supply. The aim is the reduction of supplied cold air volume and removed hot air volume. [Source: 42U, Hot Aisle Containment Cooling Solutions]
6.0
5.3
To what extent have the TDC supply and return fan volumes been optimized with variable frequency drives and isle containment?
5.4
Has Telus considered this strategy?
5.5
If Telus implements this strategy, to what extent would the resulting cooling system energy savings impact the available energy supply to the TGTES?
Topic: TES Load forecast Reference: Exhibit B-1, CPCN Application, Table 3-2: Summary of TGES Load Requirements 6.1
Please confirm that Domestic Hot Water (DHW) load is expected to be zero for Retail customers of the TES.
BCSEA-SCBC IR1 FAES Telus Garden TES 6.2
7.0
8.0
November 13, 2012 Page 5 of 5
If so, does that imply that the TES is designed not to provide DHW to Retail customers? Why is the system designed not to provide DHW to Retail? How will Retail DHW be provided?
Topic: Residential cooling load Reference: Exhibit B-1, CPCN Application, Table 3-2: Summary of TGES Load Requirements 7.1
Please confirm that the TGTES is designed to provide space cooling to residential customers.
7.2
Please confirm that the PCI Marine Gateway TES is designed not to provide space cooling to residential customers.
7.3
Please explain why the Telus Garden TES provides residential space cooling. Was consideration given to the energy conservation benefits of not providing residential space cooling?
7.4
Has the availability of free Data Centre Waste Heat influenced the building design toward less than optimal energy efficiency?
Topic: Cost of Data Centre Waste Heat Reference: Exhibit B-1, CPCN Application 8.1
Will Telus save money by providing Data Centre Waste Heat to the TGTES rather than disposing of the waste in some other way?
8.2
Has FAES considered whether there is a rationale for Telus to pay the Partnership for the service of receiving Data Centre Waste Heat? If so, what was the conclusion? If not, why not?
Attachment BCSEA IR 4.0
TM
Making a World of Difference
Portland, OR | San Francisco, CA | Seattle, WA | Durango, CO
Final report presented to Natural Resources Canada Canadian Market Analysis for Servers and Data Centres Submitted by: Catherine Mercier, Jonathan Hebert, Peter MayOstendorp, Joel Watkins, and Mike Bailey Ecos 1199 Main Avenue Suite 242 Durango, CO 81301 970.259.6801 November 5, 2012
Table of Contents 1. Executive Summary ................................................................................................................ 1 2. Introduction ............................................................................................................................. 2 3. Market Analysis ....................................................................................................................... 4 3.1. Literature Review .............................................................................................................. 4 Estimating Energy Used by Servers – World and U.S. ..................................................... 5 Definitions and Taxonomies ............................................................................................... 5 3.2. Characterizing the Canadian Server Market ................................................................ 10 Data and Methodology ....................................................................................................... 10 Calculating the Installed Base of Servers ........................................................................ 11 2. Canadian Server Market by CPU Type: x86 vs. Non-x86 Servers .............................. 13 3. Canadian Server Market by Form Factor ..................................................................... 14 4. Canadian Server Market by Application ....................................................................... 14 3.3. Annual Sales, Electricity Consumption and GHG Emissions .................................... 15 Annual Shipment Data by Unit .......................................................................................... 15 Annual Electricity Consumption Estimates ..................................................................... 18 Annual GHG Emissions ..................................................................................................... 21 3.4. Trends of Server Growth and Associated Electricity Use in Canada ........................ 22 3.5. Canadian Server Market Contact List ........................................................................... 26 3.6. Conclusion ...................................................................................................................... 26 Implications and Recommendations ................................................................................ 26 4. References ............................................................................................................................. 29 5. Appendices ............................................................................................................................ 32 Appendix 1: Glossary ............................................................................................................ 32 Appendix 2: Estimating Energy Use by Servers in Canada – Previous Works ............... 33 Appendix 3: IDC Canada Data Source ................................................................................. 36 Appendix 4: Workload Taxonomy (IDC Canada, 2008a) .................................................... 37 Appendix 5: CPU Type Taxonomy (IDC Canada, 2008a).................................................... 38 Appendix 6: Calculation for % of Replacement and Average Age of Installed Base ..... 39
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
i
Index of Figures Figure 1: Scope of this Study .................................................................................................................... 2 Figure 2: Ecos estimate of Canadian Installed Base of Server by Server Class, 2004-2007 ............ 12 Figure 3: Ecos Estimate of Installed Base of Servers by Application in 2007.................................... 15 Figure 4: Schematic of Server Energy Use Modeling Approach (Koomey, 2007a, U.S. EPA, 2007) 18 Figure 5: Ecos Estimate of Electricity Use by Canadian Servers by Server Class (MWh) ................ 20 Figure 6: Annual Electricity Consumption by Major Application/Workload ....................................... 21 Figure 7: Ecos Forecasted Installed Base of Servers by Server Class, 2008-2012 ........................... 23 Figure 8: Latitude for Energy Efficiency Improvements ....................................................................... 29
Index of Tables Table 1: Data Centre Definitions ................................................................................................................ 6 Table 2: Server Categorization by Size ..................................................................................................... 8 Table 3: Ecos Estimate of Canadian Server Installed Base by Server Class, 2004-2007 .................. 12 Table 4: Ecos estimate of Installed Base of Servers in Canada by Server CPU Type ....................... 13 Table 5: Ecos Estimate of Canadian Shipment of Servers by Province ............................................. 16 Table 6: Ecos Estimate of Canadian Server Shipments by Application ............................................. 17 Table 7: Ecos estimate of Replacement vs. new sales ......................................................................... 17 Table 8: Ecos Estimate of Average Power and Energy Consumption by Server Class in Canada .. 18 Table 9: Ecos Estimate of Annual Electricity Used by Servers in MWh .............................................. 19 Table 10: GHG Emissions in Thousands of kg of CO2 eq. .................................................................... 22 Table 11: Ecos Forecasted Canadian Server Installed Base, by Server Class .................................. 22 Table 12: Ecos Estimated Canadian Server Installed Base and Growth Rates in 2007 and 2012 .... 23 Table 13: Forecasted Annual Electricity Used by Servers in MWh, 2007-2012 .................................. 25
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
ii
1. Executive Summary This report was commissioned by Natural Resources Canada (NRCan). The objective of this study is to assess the current trends in energy use and related operational costs for servers and data centres in the Canadian market, and to outline existing and emerging opportunities for improved energy efficiency. This information will enable the Office of Energy Efficiency (OEE) and NRCan to conduct economic, energy savings and environmental impact analysis and to aid the design of market transformation programs for servers in Canada. Ecos relied on IDC Canada data provided by NRCan to perform this analysis. The following table is required by the statement of work, and contains the key statistics from the report. Mandatory Analysis Information
Comments
Energy source used
Electricity
Sector
Industrial
Annual shipments (units)
2002: 116 054 2003: 130 017 2004: 158 578 2005: 172 934 2006: 178 719 2007: 174 706
Installed base (units)
2004: 567 839 2005: 628 600 2006: 693 523 2007: 745 409 2008: 777 415 2009: 793 425 2010: 803 848 2011: 818 395 2012: 839 296
Forecast data (server shipments) (units)
2008: 178 389 2009: 183 259 2010: 187 902 2011: 191 581 2012: 198 483
Source: IDC Canada, Quarterly Server Tracker, Forecast View, 2008a
Average life expectancy (years)
x86 servers: 4.42 years Non-x86 servers: 4.75 years
Source: IDC Canada, Infrastructure Hardware Research, 2008b
Incremental annual energy savings (MWh) for volume servers
2009: 249 140 2010: 504 100 2011: 764 188 2012: 1 033 835
Estimated annual energy consumption of popular, low efficiency unit (volume server)
1 963 kWh per volume server
Estimated annual energy consumption of popular, high efficiency unit (volume server)
1 463 kWh per volume server
Source: IDC Canada, Quarterly Server Tracker ,2008a
Source: Ecos’ analysis
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
1
2. Introduction The amount of electricity used by data centres in Canada has risen significantly in recent years. This increase in data centre energy use is part of a world-wide trend; other studies around the world now show data centres as one of the fastest growing electrical end uses. Natural Resources Canada (NRCan) has estimated that energy use by servers and data centres has doubled in the past five years and is expected to double again to close to 10 billion kWh by 2011. 1 A number of large data centre operators are evaluating Canada as a possible location for major projects, ensuring sector growth for years to come. Many factors, including the low electricity costs and the cool temperatures in Canada, could explain their interest. 2 Figure 1 depicts the main power-consuming components of a data centre. The largest single power consuming component is the server. Therefore, the scope of this report is limited to the server, omitting the associated cooling and auxiliary equipment, data storage, and network equipment. In order to quantify the entire data centre power consumption, similar research on the cooling and auxiliary equipment, data storage and network equipment should be conducted. Figure 1: Scope of this Study
Focus of this study
Data Server power
storage power
Network equipment power
Cooling and auxiliaries (C&A) associated with server power
C&A for data storage
C&A for networking
Total data center power Source: Modified from Koomey, 2007a
Ecos conducted a preliminary analysis of the Canadian server market on behalf of NRCan in January 2008 (see Appendix 2). The first analysis, or Phase I, included rough estimates of the Canadian server installed base and energy use based on a top-down approach, scaling U.S. data to the Canadian market. Phase I took the ratio of the Canadian and U.S. economies in terms of gross domestic products, which is eight percent, and estimated that the Canadian server market is therefore eight percent of the U.S. server 1 2
NRCan, 2008 (internal estimates) http://www.datacenterknowledge.com/archives/2008/Mar/26/manitoba_new_frontier_for_huge_data_centers.html Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
2
market in terms of shipments. Ecos concluded Phase I by recommending that NRCan purchase Canadian server market research data from IDC Canada to support this market analysis. This report constitutes Phase II, which improves on Ecos’ previous analysis by estimating Canadian server installed base and energy use using Canadian historical and forecasted market data from IDC Canada. The resulting Phase II Market Analysis is divided into six main sections:
Literature review
Characterizing the Canadian server market
Annual sales of servers, electricity consumed annually and annual GHG emissions
Trends of server growth and associated electricity use in Canada
List of contact information for Canadian server market information sources
Conclusion - Implications and Recommendations
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
3
3. Market Analysis
3.1. Literature Review The literature review uncovered no reports that focus on Canadian server or data centre usage. However, the following reports have been completed for the United States and the global market. These reports are extremely relevant in assessing the validity of energy use models for the Canadian server and data centre market:
EPA Report to Congress and Data Center Energy Efficiency (U.S. EPA, 2007): The U.S. Environmental Protection Agency (EPA) developed this report in response to the request from Congress stated in Public Law 109-431. This report assesses opportunities for energy efficiency improvements for government and commercial computer servers and data centres in the U.S.
Estimating Total Power Consumption by Servers in the U.S. and in the World (Koomey, 2007a): This study estimates total electricity used by servers in the U.S. and the world by combining detailed data from IDC on the server installed base and shipments with measured data and estimates of the power used per unit for the most common server models in each server class in the U.S. and the world. Koomey shows that total world electricity used by servers and the associated cooling and infrastructure equipment doubled from 2000 to 2005, to approximately 123 billion kWh. The U.S. accounted for about 40 percent of that total.
Estimating Regional Power Consumption by Servers: A Technical Note (Koomey, 2007b): This technical note builds on previous analysis of total electricity used by servers in the U.S. and the world (Koomey 2007a) to estimate the regional distribution of electricity used by servers. The results reveal the predominance of the U.S. and Europe in total server electricity use as well as much greater than average annual growth rates in the Asia Pacific region (excluding Japan) over the 2000 to 2005 period. If current trends continue, the U.S. share of total world server electricity use will likely decline from 40 percent in 2000 to about one-third by 2010, while the Asia Pacific region (excluding Japan) will increase from 10 percent to about 16 percent. Other regions will maintain roughly constant shares of the world total.
Energy Consumption by Office and Telecommunications Equipment in Commercial Buildings-Volume I: Energy Consumption Baseline (Roth, et al. 2002): This study was prepared by Arthur D. Little for the U.S. Department of Energy in 2002. Arthur D. Little carried out a “bottom-up” study to quantify the annual electricity consumption of more than 30 types of non-residential office and telecommunications equipment, including computer servers.
Server Power Supplies (Ton and Fortenbery, 2005): Ton and Fortenbery gathered market data on servers to determine which types of configurations and manufacturers had the largest market penetration. Starting from the basic methodology established in the Roth et al. (2002) analysis, they constructed a revised estimate of the annual energy consumption of servers in the U.S.
Report on Energy Efficient Servers in Europe: Part 1, Energy consumption, saving potentials, market barriers and measures (Schäppi et al., 2007a) – This report presents the interim results of the international project Efficient Servers, which is conducted within the EU programme Intelligent Energy Europe. The project aims at demonstrating the high potential for energy savings and cost reductions for servers in practice and at supporting the market development for energy-efficient servers. In the first project phase, the European market for servers, the server
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
4
energy consumption and energy saving potentials were analysed. Furthermore the total electric power consumption and the saving potentials in data centres were estimated.
Report on Energy Efficient Servers in Europe: Part 2, Energy consumption, saving potentials, market barriers and measures (Schäppi et al., 2007b) – The report covers the project results on market barriers and on technical measures and options to support energy efficient solutions in practice.
Report on Energy Efficient Servers in Europe: Part 3, Energy efficiency criteria and benchmarks (Schäppi et al., 2007c) – This report classifies benchmark environments to assess the energy efficiency, and describe a measurement protocol to assess the energy efficiency in general.
Energy Efficiency in Data Centers: A New Policy Frontier (Loper and Parr, 2007) – This report discusses various energy saving opportunities and recommends a range of government policies and programs to encourage improvements to the energy efficiency of technologies and practices in data centres.
Estimating Energy Used by Servers – World and U.S. There are several estimates of the amount of electricity used by servers and data centres (Kawamoto et al., 2001, Mitchell-Jackson et al., 2001, Roth et al., 2002, Ton and Fortenbery, 2005, Koomey, 2007a). In 2002, Roth et al. used aggregate data from IDC by server class and measured power data from a representative server to estimate electricity used by each server class. This study also assessed the electricity used by data storage systems and network equipment. Ton and Fortenbery (2005) used a similar approach with updated data to estimate electricity used by server class. In a recent study, Koomey (2007a) estimated total electricity used by servers in the U.S. and the world. This study focused on the server loads and the infrastructure energy use associated with those servers. This analysis relied on detailed data from IDC installed base and shipments of servers in combination with measured data and estimates of the power used per unit of the most common server models in each server class in the U.S. and the world. Koomey reported a U.S. server installed base of more than 10.3 million, almost twice the 2000 installed base of 5.6 million. Total electricity used by servers in the U.S. in 2005 is estimated to be about 23 billion kWh. If electricity used by cooling and auxiliary equipment is included, that total rises to 45 billion kWh. Definitions and Taxonomies Data Centre Definitions There are no government or private sector umbrella organization covering all data centres, however, definitions are available from a number of sources: ACEEE and CECS 2001; Aebischer et al. 2002; Brown et al. 2001; Gruener 2000; Intel 2002; Mitchell-Jackson 2001. Table 1 summarizes some of the relevant definitions of data centres.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
5
Table 1: Data Centre Definitions
Organization
Definition “A data center contains primarily electronic equipment used for data processing (servers), data storage (storage equipment), and communications (network equipment)*. Collectively, this equipment processes, stores, and transmits digital information and is known as ’information technology’ (IT) equipment. Data centers also usually contain specialized power conversion and backup equipment to maintain reliable, high-quality power, as well as environmental control equipment to maintain the proper temperature and humidity for the IT equipment.”
U.S. Environmental Protection Agency “This study excludes from the definition of “data center” any facilities that are primarily devoted to communications (e.g., telephone exchanges), including network equipment located in telecom data centers. The definition does, however, follow market research firm IDC’s convention of including in the definition any room that is devoted to data processing servers, i.e., server closets and rooms (Bailey et al. 2006).”
1
Source
U.S. EPA 2007. Report to Congress on Server and Data Center Energy Efficiency Public Law 109-431.
Telecommunications Industry Association
“Data center: a building or portion of a building whose primary function is to house a computer room and its support areas.”
Telecommunications Infrastructure Standard for Data Centers. April 12, 2005.1
Lawrence Berkeley National Laboratory
In “High-performance data centers- A research roadmap,” the Lawrence Berkeley National Laboratory adopts a broad definition of the term data centre. As mentioned in the report, “we generally use the term data centre to be a facility that contains concentrated equipment to perform one or more of the following functions: store, manage, process and exchange digital data and information. We do not consider spaces that primarily house office computers, including individual workstations, servers associated with workstations, or small server rooms, to be data centers.”
Tschudi, Xu, Sartor and Stein, 2003
Code of Conduct on Data Centres Version 0.7
The term “data centres” includes all buildings, facilities, offices and rooms which contain enterprise servers, server communication equipment, cooling equipment and power equipment, and provide some form of data service (e.g., largescale mission-critical facilities all the way down to small server rooms located in office buildings).
European Commission, 2007
This Standard specifies the minimum requirements for telecommunications infrastructure of data centres and computer rooms including single-tenant enterprise data centres and multitenant Internet hosting data centres. The topology proposed in this document is intended to be applicable to any size data centre. For further information see: http://ftp.tiaonline.org/TR-42/Tr425/Public/TR425-05-10010a_working_dictionary.pdf
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
6
Data Centre Categories/Taxonomy There are four common ways to categorize data centres. 1. Categorization by End-Application (Koomey et al., 2005) •
Multi-tenant hosting facilities, which include data centres owned by third parties that house servers owned by companies (collocation), as well as data centres that sell server services to companies that do not want to manage their own servers (managed hosting).
•
Corporate or enterprise data centres, owned by corporations and managed in-house. These data centres are often housed within existing facilities and may make up only a small portion of the total floor area associated with those facilities.
•
Institutional and government data centres, owned and operated by federal, state and local government or by non-profit institutions.
•
Educational data centres, serving students and faculty in post-secondary institutions.
2. Categorization by Size or Square Footage Data centres range in size from small rooms (server closets of less than 20 square meters) within a conventional building to large buildings (enterprise class data centres in the hundreds of square meters) dedicated to housing servers, storage devices and network equipment, with the largest being called enterprise-class. Large data centres are becoming increasingly common as smaller data centres consolidate (Carr, 2005). As summarized in Table 2, server closets and server rooms can have significantly different IT equipment and infrastructure characteristics than larger data centres.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
7
Table 2: Server Categorization by Size Space type
Server closet
Server room
Localized data centre
Mid-tier data centre
Enterpriseclass data centre
Typical size
1-2 servers
<200 ft2
No external storage
A few to dozens of servers
<500 ft2
<1 000 ft
Typical IT equipment characteristics
No external storage
2
Dozens to hundreds of servers Moderate external storage
<5 000 ft2
5 000 + ft2
Hundreds of servers Extensive external storage
Hundreds to thousands of servers Extensive external storage
Typical site infrastructure system characteristics Typically conditioned through an office HVAC system. To support VoIP and wireless applications, UPS and DC power systems are sometimes included in servers maintained as for other data centre types. HVAC energy efficiency associated with server closet is probably similar to the efficiency of office HVAC systems. Typically conditioned through an office HVAC system, with additional cooling capacity, probably in the form of a split system specifically designed to condition the room. The cooling system and UPS equipment are typically of average or low efficiency because there is insignificant economy of scale to make efficient systems more first-cost competitive. Typically used for underfloor or overhead air distribution systems and a few in-room CRAC units. CRAC units in localized data centres are more likely to be air-cooled with constant-speed fans, and are thus relatively low efficiency. Operational staff is likely to be minimal, which makes it likely that equipment orientation and airflow management are not optimized. Air temperature and humidity are tightly monitored. However, power and cooling redundancy reduce overall system efficiency. Typically use under-floor air distribution and in-room CRAC unit. The larger size of the centre relative to those listed above increases the probability that efficient cooling (e.g., a central chilled water plant and central air handling units with variable speed fans) is used. Staff at this size data centre may be aware of equipment orientation and airflow management best practices. However, power and cooling redundancy may reduce overall system efficiency. The most efficient equipment is expected to be found in these large data centres. Along with efficient cooling, these data centres may have energy management systems. Equipment orientation and airflow management best practices are most likely implemented. However, enterpriseclass data centres are designed with maximum redundancy, which can reduce the benefits gained from the operational and technological efficiency measures.
Source: U.S. EPA, 2007
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
8
3. Categorization per Uptime Institute One of the main obstacles to the adoption of energy efficiency solutions in data centres has been a perception that some energy-efficiency strategies may have a negative effect on data centre performance, including reliability, which is a key attribute of high-quality data centre equipment. Manufacturers will not compromise performance or reliability to achieve energy-efficiency improvements (U.S. EPA, 2007). Further research is therefore needed to identify areas of uncertainty about performance and reliability and provide information to manufacturers, purchasers, and operators. The Uptime Institute Tier Classification system is based on performance standards and site infrastructure. Uptime’s classification has been in use since 1995 and has been updated in 2008.
Tier 1: composed of a single path for power and cooling distribution, without redundant components, providing 99.671 percent availability.
Tier II: composed of a single path for power and cooling distribution, with redundant components, providing 99.741 percent availability
Tier III: composed of multiple active power and cooling distribution paths, but only one path active, has redundant components, and is concurrently maintainable, providing 99.982 percent availability
Tier IV: composed of multiple active power and cooling distribution paths, has redundant components, and is fault-tolerant, providing 99.995 percent availability.
For more details of the Uptime Classification System classification, see Turner IV and al., 2008. Server Definitions As mentioned in the introduction, servers are the largest energy-using components of data centres. A server generally provides common functions to a group of users or performs back-end processing invoked on a scheduled basis or by other computers (Roth et al., 2002). ENERGY STAR ® is currently developing a new product specification for enterprise servers. Since releasing its Draft 1 server specification in February 2008, the U.S. EPA has engaged in several industry discussions including a stakeholder online meeting on April 1, 2008. During the online meeting, the U.S. EPA discussed several key stakeholder comments and next steps toward releasing a subsequent draft. One of the next steps included revising and releasing a draft definition and scope document, which was distributed to stakeholders for review and comment on April 25. The goal in distributing a revised definitions document prior to Draft 2 document was to develop a consensus regarding the types of products to be covered by the specification. On July 9, 2008, in Redmond WA, the U.S. EPA held a stakeholder meeting to discuss with manufacturers and other interested stakeholders the latest developments for the ENERGY STAR ® server specification under development. Following discussions held during the Redmond meeting and comments and data received over the last few weeks, the U.S. EPA released its Draft 2 server specification on August 15, 2008. The computer server definition has been revised to address the full range of servers in the marketplace that are used in data centres and business environments. Below is the latest computer server definition, as proposed in the ENERGY STAR ® Draft 2 server specification: 3
3
Computer Server: A computer that provides services and manages networked resources for client devices, e.g., desktop computers, notebook computers, thin clients, wireless devices, PDAs, IP telephones, other computer servers and other networked devices. Computer servers are sold through enterprise channels for use in data centers and office/corporate environments.
Further information can be found at http://www.energystar.gov/index.cfm?c=new_specs.enterprise_servers. Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
9
Computer servers are designed to respond to requests and are primarily accessed via network connections rather than through direct user input devices such as a keyboard, mouse, etc. In addition, computer servers must include all of the following characteristics:
Marketed and sold as a server;
Designed for and listed as supporting Server Operating Systems and/or Hypervisors, and targeted to run user-installed enterprise applications;
Designed and capable of supporting one or more processor sockets and/or one or more processor boards in the device; and
Support for error-correcting code (ECC) and/or buffered memory (including both buffered DIMMs and buffered on board (BOB) configurations).
Server Subcategories/Taxonomy Servers themselves can be categorized in many ways (e.g., by shipments, revenue, geographic region, manufacturer, price range, operating system, processor, U-rating, 4 architecture, etc.). IDC’s Worldwide and Regional Quarterly Server Tracker and Forecast research programs are probably the most comprehensive data source for statistics on server shipments. IDC’s server tracker research programs provide actual quarterly data for more than 15 data categories (including those mentioned above). In his recent work, Jonathan Koomey (2007a, 2007b) used IDC’s data on aggregate installed base split into three server classes (volume, mid-range and high-end). IDC’s server class taxonomy segments the server market into three server classes (based on the cost of the system): volume server market with an average selling value (ASV) of below $25 000 U.S. per unit, mid-range enterprise server market with an ASV from $25 000 to $500 000 U.S. per unit and high-end enterprise server market with an ASV of $500 000 U.S. and above per unit.
3.2. Characterizing the Canadian Server Market One of the main tasks of this market analysis was to present the most common types, sizes and applications of servers in Canada, which required information about the installed base of servers. The Canadian server market can be characterized by cost, CPU type, form factor or application. Data and Methodology We estimated the installed base of servers in Canada using data on shipments provided by IDC Canada (2008a). Established in 1984, IDC Canada has more than 30 analysts exclusively focused on the Canadian market and is the leading provider of research, consulting and marketing services to the Canadian information and communication technologies industry. IDC Canada’s data included historical shipments of servers segmented by server class, vendor, and architecture into the Canadian market over the period of 2002 to 2007, as well as forecasts for server shipments through 2012. For more information on data provided by IDC Canada, see Appendix 3. Ecos estimated the installed base of servers by adding the IDC Canada provided shipment data over the assumed useful life of servers. Guidance on the average life expectancy of shipped product was provided by IDC Canada through its Canadian Infrastructure Hardware research program (2008b).
4
The “U” rating is based on the height of the server: A 1-U server is 1.75 in. tall. Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
10
Server shipment data was not available for 2000 and 2001. To calculate the Canadian installed base of servers in 2005 and 2004, we used worldwide shipment data from J. Koomey’s research (2007a), and assumed that Canada represents about the same as the ratio of the 2002 Canadian server market and the worldwide server market in terms of shipments (see Appendix 7). Calculating the Installed Base of Servers 1. Canadian Server Market by Server Class IDC’s server class taxonomy segments the server market into three server classes (based on the cost of the system), where volume class servers have an average selling value (ASV) below $25 000 U.S., midrange servers have an ASV between $25 000 and $500 000 U.S., and high-end servers have an ASV above $500 000 U.S. Table 3 shows Ecos’ estimates of the Canadian installed base of servers broken out by server class from 2004 to 2007. As shown in Figure 2, Ecos estimates that volume servers represent 96 percent of the current total installed base of servers in Canada. According to Koomey (2007a), volume servers represented 96 percent of the U.S. installed servers on a unit basis in 2005. In the world, volume servers represented 95 percent of the installed servers on a unit basis in 2005 (Koomey, 2007a).
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
11
Table 3: Ecos Estimates of Canadian Server Installed Base by Server Class, 2004-2007
2000
2001
2002
2003
2004
2005
2006
2007
CAGR
Volume
101 960
103 388
108 660
122 722
152 443
166 766
173 690
170 361
9%
Mid-range
9 725
7 079
7 010
7 022
5 870
5 938
4 816
4 139
-10%
High-end
531
425
384
273
265
230
213
206
-12%
Total
112 215
110 892
116 054
130 017
158 578
172 934
178 719
174 706
9%
Volume
532 374
596 385
663 750
718 692
11%
Mid-range
33 724
30 748
28 507
25 963
-8%
High-end
1 740
1 468
1 266
1 115
-13%
Total
567 839
628 600
693 523
745 409
10%
Shipments
Installed base
To calculate server installed base for 2004 and 2005, we used worldwide shipment data from Koomey (2007a), and assumed that Canada represents about the same as the ratio of the 2002 Canadian server market and the worldwide server market in terms of shipments.
Figure 2: Ecos Estimates of Canadian Installed Base of Server by Server Class, 2004-2007
700
High-end server
Installed base (thousands of units)
Mid-range server
600
Volume server
500 400 300 200 100 0 2004
2005
2006
2007
Sources: To calculate server installed base for 2004 and 2005, we used worldwide shipment data from Koomey (2007a), and assumed that Canada represents about the same as the ratio of the 2002 Canadian server market and the worldwide server market in terms of shipments.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
12
2. Canadian Server Market by CPU Type: x86 vs. Non-x86 Servers Server shipment data provided by IDC Canada included information on the architecture of the servers being shipped into Canada. Understanding the architecture of the servers being shipped provides a useful insight into the types of applications for which the servers are being used and customer buying habits. Server architecture can be split into two simple groups: x86 servers – whose processors operate using instruction sets widely compatible with most Windows-compatible personal computers – and nonx86 servers – whose processors operate using other types of instruction sets, such as CISC, RISC, and EPIC and are used for more proprietary and task-specific functions (see Appendix 5 for more information on IDC’s CPU type taxonomy). The generic x86 server architecture is similar to typical PC architecture and is usually compatible with PC software. Non-x86 servers include a wide range of server architectures and software platforms designed to perform a particular task-specific function, usually handling many instructions at the same time (e.g., database servers, IT infrastructure servers, network routers). The instruction sets used by non-x86 servers have specialized computer instruction sets that drive different operating systems usually aimed at optimizing to process large amounts of data. Servers were traditionally designed around the non-x86 architecture but have been steadily shifting toward x86, utilizing typical processor instruction sets common to Intel and AMD processors. As shown in Table 4, the x86 servers represent 90 percent of the installed base of servers in Canada today. Forecasts show that this trend is likely to continue. The x86 servers are expected to account for over 94 percent of the Canadian installed base of servers by 2012. Table 4: Ecos Estimates of Installed Base of Servers in Canada by Server CPU Type
Server Type
2007
2008
2009
2010
2011
2012
CAGR
Non-x86 Server x86
74 550 670 859
66 165 711 250
57 753 735 672
51 081 752 766
47 692 770 703
46 428 792 868
-9% 3%
Total
745 409
777 415
793 425
803 848
818 395
839 296
2%
Dell’s chief technology officer (CTO), Kevin Kettler, has said that x86 server technologies will dominate the market in the years to come. He believes that x86 type scale-out server system design– incrementally building server system capacity over time – is preferable to overbuilding server systems, as is common, based on the projected system’s needs years out. Incrementally building a system reduces the amount of sunk cost needed for a system upgrade and can also save on operational expenses through energy savings. Dell’s CTO also predicts that rapidly improving processor technology and virtualization will contribute to the dominance of x86 server technologies. 5 There are some dissenting opinions concerning the prevalence of x86 type scale-out system design architecture in the future of the data centre market. 6 The argument is based on two premises: First, that enterprise type scale-up system designs are better for high input/output rates and large workloads; and second, that scale-out system designs are often obsolete and no longer fulfil the system’s needs by the time the infrastructure is updated.
5
http://searchservervirtualization.techtarget.com/news/article/0,289142,sid94_gci1226067,00.html# Kevin Kettler advocates Dell’s focus on “scale-out” architecture, contrasting with the other three major vendors – IBM, Hewlett-Packard and Sun Microsystems – which focus on scale-out and scale up architectures. http://searchservervirtualization.techtarget.com/news/article/0,289142,sid94_gci1226067,00.html# 6
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
13
3. Canadian Server Market by Form Factor Servers come in a number of physical form factors, including rack-optimized and blade servers. A rackoptimized server is a server designed to be installed in a framework called a rack. A blade server is a server chassis housing multiple, thin, modular electronic circuit boards, known as server blades. Blade servers integrate servers, storage, networking and applications into one system. In 2007, rack-optimized servers accounted for 62 percent of the Canadian server market in terms of shipments. In the future, the number of rack-optimized servers is expected to remain stable, while the number of blade servers grows. IDC Canada predicts that blade servers will account for 22.4 percent of the market in 2012 (2008b) (for more information on the Canadian server market trend, see section 3.4). The move towards more blade servers will tend to increase power density in data centers. With multiple, highly compact blade servers, a single blade server chassis can deliver more processing capacity than rack-optimized servers. This increased density can translate into higher power drawn per square foot. In general, for the same processing capacity, blade servers tend to be more energy efficient than individually rack mounted servers due to better utilization of shard components (blades tend to have shared power supplies, data storage, and I/O support). The associated total energy consumption will depend on how the overall processing capacity changes. On one hand, if the overall processing capacity stays the same, we can expect the total energy consumption to decrease as more blade servers are used. One the other hand, if the overall processing capacity increases, the total energy consumption could stay the same or even increase if the processing capacity increase is greater than the efficiency gains. 4. Canadian Server Market by Application Servers are extremely versatile and can be used for applications ranging from the processing of financial transactions to handling digitized phone calls. IDC Canada provided historical annual shipment data split by the following applications for 2004 (see Appendix 4 for IDC’s Workload Taxonomy):
Business processing: Enterprise resource planning (ERP) (including enterprise wide line-ofbusiness applications, other business commerce applications that facilitate business transactions or other task automation over networks, departmental transactional applications that run on servers but don't tie directly to other applications), customer relationship management (CRM), online transaction processing (OLTP), traditional legacy mainframe-type processes that execute business process transitions in a batch process
Decision support: data warehousing/data mart, data analysis/data mining
Collaborative: email and applications that let users collaborate and share information
Application/software: traditional application development work
IT infrastructure: file/print sharing, networking, proxy catching, security, system management
Web infrastructure: Web serving and streaming media
Technical: scientific/engineering workloads
Other: all other applications not covered by the above definitions
Assuming that the market share of each application remained constant over the period of 2002-2007, we calculated the Canadian estimated installed base of servers split by application; see Figure 3. Over 40 percent of servers in Canada are being used for IT infrastructure. The other categories pertinent to this research include business processing, decision support and Web infrastructure. The combined market share of these server categories makes up about 30 percent the Canadian server market; these categories are likely to consist of the data centre type IT systems infrastructure covered by this report.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
14
Figure 3: Ecos Estimates of Installed Base of Servers by Application in 2007
IT Infrastructure
Application/Workload
Collaborative Business Processing Web infrastructure Decision Support Application Development Technical Other 0
50000
100000
150000
200000
250000
300000
Installed base of servers (units)
3.3. Annual Sales, Electricity Consumption and GHG Emissions Ecos has outlined the quantitative effects of the Canadian server market by annual electricity usage and the respective GHG emissions by province. It is important to remember that the scope of this analysis only examines energy use as a function of electricity used by servers, excluding the associated cooling and auxiliary equipment needed to support IT equipment. The inclusion of infrastructure equipment would roughly double the overall energy use figures. Further energy analysis of data centres could include IT infrastructure energy usage. Canada’s unique northern climate might, for example, result in lower infrastructure-related energy use for cooling, causing servers to dominate the energy use equation. Annual Shipment Data by Unit Ecos used information on population size and distribution by province to split Canadian shipments by province/region. We then validated this assumption by comparing the results to a similar spread using gross domestic product by province, active population by province, computer system design and related services by province, as well as employment by major industry groups by province. Ecos determined that population density scaling provides an adequate view of server installed base across region. Our team also researched a number of other scaling methodologies based on installed footage of raised floor. We determined that the lack of information available for these alternate scaling methodologies outweighed any additional accuracy that those methods might provide. Table 5 summarizes annual sales by province and by the market as a whole.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
15
Table 5: Ecos Estimates of Canadian Shipment of Servers by Province 2002 Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Yukon Territory Northwest Territories Nunavut Canada
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
1 843
2 065
2 519
2 747
2 839
2 775
2 833
2 911
2 984
3 043
3 153
496
555
677
738
763
746
762
783
802
818
848
3 361
3 766
4 593
5 009
5 176
5 060
5 167
5 308
5 442
5 549
5 749
2 696 27 286 45 066 4 217 3 565 11 842 15 309 111 153 108 116 054
3 020 30 568 50 489 4 725 3 994 13 267 17 151 125 171 121 130 017
3 684 37 283 61 579 5 762 4 872 16 181 20 919 152 209 147 158 578
4 017 40 659 67 154 6 284 5 313 17 646 22 813 166 228 161 172 934
4 152 42 019 69 401 6 494 5 491 18 236 23 576 171 235 166 178 719
4 059 41 075 67 842 6 348 5 367 17 827 23 046 167 230 162 174 706
4 144 41 941 69 272 6 482 5 481 18 202 23 532 171 235 166 178 389
4 257 43 086 71 164 6 659 5 630 18 699 24 175 176 241 170 183 259
4 365 44 178 72 966 6 828 5 773 19 173 24 787 180 247 175 187 901
4 451 45 043 74 395 6 962 5 886 19 549 25 272 184 252 178 191 582
4 611 46 666 77 076 7 212 6 098 20 253 26 183 190 261 185 198 483
Sources: Ecos used information on population size and distribution by province to split Canadian shipments by province/region.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
16
IDC Canada also provided shipment data by application/workload for the year 2004. Ecos assumed that the market share of each application remained constant over the period of 2002-2007. Table 6 tabulates Canadian shipments by application. Table 6: Ecos Estimates of Canadian Server Shipments by Application
Application Development Business Processing Collaborative Decision Support IT Infrastructure Other Technical Web Infrastructure Total
2002 8 056 12 660 20 286 10 650 48 766 292 4 458 10 886 116 054
2003 9 025 14 183 22 727 11 931 54 634 327 4 994 12 196 130 017
2004 11 008 17 299 27 719 14 552 66 635 399 6 091 14 875 158 578
2005 12 005 18 865 30 229 15 869 72 668 435 6 642 16 221 172 934
2006 12 406 19 496 31 240 16 400 75 099 450 6 864 16 764 178 719
2007 12 128 19 058 30 539 16 032 73 412 439 6 710 16 388 174 706
Replacement vs. New Sales No systematic data on server retirement and replacement vs. new sales was available for Canada at the time this report was created. However, using shipment data provided by IDC Canada (IDC, 2008a), Ecos estimated that replacements are about 70 percent of total server sales; see Table 7. For detail information on how we estimated the replacement vs. new sales see Appendix 6. Table 7: Ecos Estimates of Replacement vs. new sales
Years
Shipment
Replacement/ retirement/
Installed bases
Share of current shipments that are new sales
2007
174 706
123 874
745 409
About 30%
Source: 2007 shipment data comes from IDC Canada’s Quarterly Server Tracker research program(IDC, 2008a)
Penetration Rate by Efficiency Manufacturers of servers, processors and other components have increased the energy efficiency of their product over the last few years. According to the U.S. EPA (2007), in the U.S., energy efficient servers was estimated to represent 5 percent of volume server shipments in 2007 and expected to represent 15 percent of shipments in 2011. Estimated Age of Equipment Use The average age of the server installed base in 2007 is estimated to be 2.13 years (see Appendix 6). To calculate the average age of the server installed we used the server lifetime estimates provided by IDC Canada (IDC, 2008b). The server lifetime estimates are based on research conducted as part of IDC Canada’s Infrastructure Hardware program. It is important to note that many servers are never removed even when they are no longer needed, yet they continue to use electricity. The average age of the server installed base may therefore be underestimated.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
17
Annual Electricity Consumption Estimates Data and Methodology Ecos determined the annual electricity consumption of servers in Canada using the approach used by J. Koomey to estimate the total energy consumption for the U.S. server market (Koomey, 2007a). Figure 4 below show a schematic of Koomey’s approach. Figure 4: Schematic of Server Energy Use Modeling Approach (Koomey, 2007a, U.S. EPA, 2007) # of installed base volume servers in Canada
# of installed base mid-range servers in Canada
# of installed base high-end servers in Canada
X
X
X
Average volume server energy use
Average mid-range server energy use
Total energy use by servers in Canada
Average high-end server energy use
First, the Canadian installed base of servers is split by server class into provinces/regions by using information on Canadian population distribution and size. Next, average energy use per server is calculated by taking the average power used per server for the rest of the world in 2005 directly from Koomey (2007b), 7 and multiplying the power usage by the number of hours in a calendar year (assuming that servers operated 100 percent of the year); see Table 8. Table 8: Ecos Estimates of Average Power and Energy Consumption by Server Class in Canada
Average power use (watts) Annual energy consumption per server (kWh)
Volume
Mid-range
High-end
224
598
8 378
1 963
5 241
73 433
Sources: Average power used estimates by server class come from Koomey (2007b). Energy consumption per server was calculated assuming 8 765 hours/year and that servers operated 100 percent of the year
7
Publicly available Canadian data on power use per server in Canada was not available for this analysis. Koomey’s general approach to determine the average power used per server was to develop a representative sample of the servers across markets and then use a combination of the rated input power from spec sheets, typical operational power available from manufactures, and a set of power usage multipliers to determine power use per unit, for a given generation of server hardware. He developed a weighted average power use figure based on the market share of each server and characterized the average power consumption for each class of servers – volume, midrange and high-end – for the U.S. and the world markets. Finally, he used a standard duty cycle of 100 percent, assuming that servers operated all year, to calculate the energy use. Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
18
Finally, we multiplied the estimated average energy used per server by the installed base by server class and province/region for the period 2004 to 2007 to calculate total electricity use by server class and province/region; see Appendix 8. Estimated Annual Electricity Used by Servers in Canada – Results Table 9 shows the resulting electricity consumption by server class and province/region for the period 2004-2007. As shown Table 8, high-end and mid-range servers consume significantly more power than volume servers, but Table 9 and Figure 5 show that, given the share of volume servers that make up the Canadian market, their energy use ultimately dominates Table 9: Ecos Estimates of Annual Electricity Used by Servers in MWh Volume – total Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Yukon Territory Northwest Territories Nunavut
2004 1 045 243 16 602 4 463 30 274 24 282 245 748 405 892 37 982 32 113 106 655 137 883 1 001 1 376 972
2005 1 170 919 18 598 5 000 33 914 27 201 275 296 454 695 42 549 35 974 119 479 154 462 1 122 1 541 1 089
2006 1 303 181 20 698 5 565 37 744 30 274 306 392 506 055 47 355 40 037 132 975 171 909 1 248 1 715 1 212
2007 1 411 051 22 412 6 026 40 869 32 780 331 754 547 944 51 274 43 351 143 981 186 139 1 352 1 857 1 312
CAGR (2004-2007) 11%
Mid-range – total Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Yukon Territory Northwest Territories Nunavut
176 763 2 808 755 5 120 4 106 41 559 68 641 6 423 5 431 18 037 23 318 169 233 164
161 167 2 560 688 4 668 3 744 37 892 62 585 5 856 4 951 16 445 21 260 154 212 150
149 418 2 373 638 4 328 3 471 35 130 58 022 5 430 4 590 15 246 19 710 143 197 139
134 195 2 131 573 3 887 3 117 31 551 52 111 4 876 4 123 13 693 17 702 129 177 125
-8%
High-end – total Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Yukon Territory Northwest Territories Nunavut
127 797 2 030 546 3 701 2 969 30 046 49 626 4 644 3 926 13 040 16 858 122 168 119
107 710 1 711 460 3 120 2 502 25 324 41 826 3 914 3 309 10 991 14 208 103 142 100
92 930 1 476 397 2 692 2 159 21 849 36 087 3 377 2 855 9 482 12 259 89 122 86
81 863 1 300 350 2 371 1 902 19 247 31 789 2 975 2 515 8 353 10 799 78 108 76
-13%
1 349 802
1 439 795
1 545 528
1 627 108
7%
Total
Sources: Koomey’s data on power use for individual servers (Koomey, 2007b). Electricity use is calculated by multiplying the installed base by the average power used per server. Direct electricity consumption assumes 8 765hours/ year and 100 percent load.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
19
Figure 5: Ecos Estimates of Electricity Use by Canadian Servers by Server Class (MWh) 1800000 High-end servers
1600000
Mid-range servers
Electricity use of servers (MWh)
Volume servers
1400000 1200000 1000000 800000 600000 400000 200000 0 2004
2005
2006
2007
Sources: Koomey’s data on power use for individual servers (Koomey, 2007b). Electricity use is calculated by multiplying the installed base by the average power used per server. Direct electricity consumption assumes 8 765 hours/year and 100 percent load.
As of 2007, the electricity use attributable to servers in Canada is estimated at about 1 627 108 MWh, or 0.3 percent of the total Canadian electricity consumption. These findings concur with our phase one analysis (see Appendix 2). In our phase one analysis, we estimated that total electricity used by servers in Canada was about 1 800 000 MWh in 2005. Recall, this phase II study focused on the servers only, and the total electricity used with servers in data centres also includes the electricity used by cooling and auxiliary equipment. If we assume as Koomey did (2007a) that every kWh of electricity use for IT loads means another kWh of electricity use for infrastructure, including cooling and auxiliary equipment, that increases the total electricity consumption to about 3 254 216 MWh. 8 Koomey (2007a) estimated that electricity use of U.S. servers in 2005 was roughly 22 500 000 MWh. When electricity use for cooling and auxiliary equipment is included, that total rises to 45 000 000 MWh. Total electricity consumption for the world as a whole is about 61 400 000 MWh, or 122 900 000 MWh when electricity use for cooling and auxiliary equipment is included (Koomey, 2007b). The Canadian share of the world server electricity use would therefore be about 2.6 percent. In 2007, volume servers were responsible for the majority of the electricity used by Canadian servers. With a CAGR of 11 percent, volume servers also experienced the greatest growth in electricity use among all server classes. The electricity use by Canadian servers was disaggregated into eight workloads (see Appendix 4 for a definition of each workload). This allowed a better characterization of energy use by servers. Figure 6 shows the estimates of energy use by major application/workload.
8
Further research is needed for cooling and auxiliary equipment, so that the total electricity use of data centres can be estimated. Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
20
Figure 6: Annual Electricity Consumption by Major Application/Workload
Application Development
Applications/workloads
Business Processing Collaborative Decision Support IT Infrastructure Other Technical Web Infrastructure 0
100000
200000
300000
400000
500000
600000
700000
Electricity use (MWh)
Sources: Installed base of servers was calculated by Ecos. Ecos used Koomey’s data on power use for individual servers (Koomey, 2007b). Electricity use is calculated by multiplying the installed base by the average power used per server. Direct electricity consumption assumes 8 765 hours/year and 100 percent load.
Annual GHG Emissions Canada's greenhouse gas emissions vary from region to region. This is because the distribution of natural resources and heavily fossil fuel dependent industry within the country. Alberta, with an energy generation system that is predominantly coal-based, has the highest GHG intensity in Canada. The Atlantic region, which relies on coal and nuclear, has a GHG intensity that is somewhat lower than that of Alberta, and Quebec, Manitoba and British Columbia, where energy generation is dominated by hydro, have the lowest GHG intensities. Ontario lies somewhere between the two extremes, with its mix of hydro, nuclear and fossil fuels, and is very close to the Canadian average. To calculate GHG emissions by servers in Canada, Ecos therefore used different GHG intensities; see Table 10. GHG emissions were calculated by multiplying each province’s GHG intensity by the estimated electricity used by the server installed base in the province.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
21
Table 10: GHG Emissions in Thousands of kg of CO2 eq.
Canada Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Yukon Territory Northwest Territories Nunavut
GHG intensities g CO2 eq per kWh
2004
2005
2006
2007
314 094 63 1 419 29 436 12 065 2 820 112 614 671 33 289 118 634 2 956 38 52 37
343 621 69 1 552 32 203 13 200 3 085 123 201 734 36 419 129 787 3 234 41 57 40
368 264 74 1 663 34 513 14 146 3 307 132 036 786 39 031 139 094 3 466 44 61 43
387 703 78 1 751 36 334 14 893 3 481 139 006 828 41 091 146 436 3 649 47 64 45
3 252 771 394 9.1 220 14 822 882 17 30 30 30
Sources: The GHG intensities by province is from Environment Canada, National Inventory Report, 1990-2005: Greenhouse Gas Source and sinks in Canada, http://www.ec.gc.ca/pdb/ghg/inventory_report/2005_report/a9_eng.cfm#ta9_1, August 14 2008.
3.4. Trends of Server Growth and Associated Electricity Use in Canada The total number of servers in Canada is expected to grow with a CAGR of 2.4 percent from 745 409 units in 2007 to around 839 296 units by 2012, which is almost 1.5 times the number of the installed base in 2004; see Table 11. Current trends toward server virtualization, which allows IT demand to be met through fewer physical servers, may explain why the growth of the installed base is predicted to be slower in the next five years, compared to previous years. Virtualization technology can help improve server utilization by allowing multiple, software-based “virtual machines” to run on a single server, allowing that server to run two to 20 times the number of applications. Consequently, the base of “virtual servers” in Canada will continue to grow rapidly through virtualization software packages; however, the growth in the number of physical servers will slow because each new physical server might house multiple virtual servers. As shown in Table 11 and Figure 7, volume servers represent 96 percent of the current total installed base of servers in Canada. Forecasts show that this trend is likely to continue. In 2012, Ecos expects that the volume servers will represent 97.5 percent of the Canadian installed base of servers. Table 11: Ecos Forecasted Canadian Server Installed Base, by Server Class 2007
2008
2009
2010
2011
2012
CAGR
Volume Server
718 692
753 841
771 815
783 443
798 104
818 536
2.6%
Mid-range Server
25 602
22 541
20 640
19 440
19 329
19 816
-5.0%
High-end Server
1 115
1 033
970
964
962
944
-3.4%
Total
745 409
777 415
793 425
803 848
818 395
839 296
2.4%
Sources: The installed base of servers was estimated by adding the shipments over the assumed useful life of servers. The server life expectancy estimates by server class come from Ecos’ analysis of IDC’s server life expectancy data by CPU type.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
22
Figure 7: Ecos Forecasted Installed Base of Servers by Server Class, 2008-2012 1000 High-end server
Installed base (thousands of units)
900
Mid-range server Volume server
800 700 600 500 400 300 200 100 0
2008
2009
2010
2011
2012
If the current IDC Canada server shipment forecast holds true, Ecos estimates that the installed base for volume servers will grow by more than 14 percent from 2007 by 2012, while the mid-range and high-end installed bases will decline by 23 percent and 16 percent, respectively; see Table 12. Koomey (2007a) concluded that if the IDC worldwide 2006 forecast holds true, the installed base for volume server would grow by more than 50 percent from 2005 levels by 2010, while the mid-range and high-end installed bases will decline 20 percent to 30 percent. Table 12: Ecos Estimated Canadian Server Installed Base and Growth Rates in 2007 and 2012
718 692
2007 Installed base Composition 96.4%
Mid-range
25 602
High-end
1 115 745 409
Server Class Volume
Total
Installed base in 2007
818 536
2012 Installed base Composition 97.5%
3.4%
19 816
2.4%
-23%
0.2%
944
0.1%
-16%
100%
839 296
100%
13%
Installed base in 2012
Growth 2007-2012
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
14%
23
As discussed previously, the move towards more blade servers will tend to increase power density in data centers. According to IDC, blade server shipments are expected to increase at a CAGR of 16.4 percent (IDC Canada, 2008a).This is expected to result in higher power densities in many data centres. If the per unit energy consumption for blade servers follows the same trends as the energy use of rackoptimized servers, electricity use by servers in Canada will increase by 10 percent from 2007 to 2012, largely due to the increased sales of blade servers. The forecasted electricity used by servers from 2007 to 2012, and the percentage growth rates in server electricity use from 2007 to 2012 are presented in Table 13.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
24
Table 13: Forecasted Annual Electricity Used by Servers in MWh, 2007-2012 2007 1 411 051
2008 1 480 061
2009 1 515 351
2010 1 538 181
2011 1 566 965
2012 1 607 081
22 412
23 508
24 068
24 431
24 888
25 525
6 026 40 869 32 780 331 754 547 944 51 274 43 351 143 981 186 139 1 352 1 857 1 312
6 320 42 867 34 383 347 979 574 742 53 782 45 471 151 023 195 242 1 418 1 948 1 376
6 471 43 890 35 203 356 276 588 445 55 065 46 555 154 624 199 897 1 452 1 995 1 409
6 568 44 551 35 733 361 643 597 311 55 894 47 257 156 954 202 909 1 474 2 025 1 430
6 691 45 384 36 402 368 411 608 489 56 940 48 141 159 891 206 706 1 501 2 063 1 457
6 863 46 546 37 334 377 842 624 067 58 398 49 374 163 984 211 998 1 540 2 116 1 494
Mid-range –total Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Yukon Territory Northwest Territories Nunavut
134 195
118 150
108 184
101 895
101 314
103 865
2 131
1 877
1 718
1 618
1 609
1 650
573 3 887 3 117 31 551 52 111 4 876 4 123 13 693 17 702 129 177 125
505 3 422 2 745 27 778 45 880 4 293 3 630 12 056 15 586 113 156 110
462 3 133 2 513 25 435 42 011 3 931 3 324 11 039 14 271 104 142 101
435 2 951 2 367 23 957 39 568 3 703 3 130 10 397 13 442 98 134 95
433 2 934 2 354 23 820 39 343 3 682 3 113 10 338 13 365 97 133 94
444 3 008 2 413 24 420 40 333 3 774 3 191 10 598 13 701 100 137 97
High-end –total Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Yukon Territory Northwest Territories Nunavut
81 863
75 847
71 255
70 813
70 611
69 328
1 300
1 205
1 132
1 125
1122
1 101
350 2 371 1 902 19 247 31 789 2 975 2 515 8 353 10 799 78 108 76
324 2 197 1 762 17 833 29 453 2 756 2 330 7 739 10 005 73 100 71
304 2 064 1 655 16 753 27 670 2 589 2 189 7 271 9 400 68 94 66
302 2 051 1 645 16 649 27 498 2 573 2 176 7 226 9 341 68 93 66
302 2 045 1 640 16 602 27 420 2 566 2 169 7 205 9 315 68 93 66
296 2 008 1 611 16 300 26 922 2 519 2 130 7 074 9 145 66 91 64
1 627 108
1 674 058
1 694 790
1 710 889
1 738 891
1 780 275
Volume – total Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Yukon Territory Northwest Territories Nunavut
Total
CAGR 2.6%
-5.0%
-3.3%
1.8%
Sources: Koomey’s data on power use for individual servers (Koomey, 2007). Electricity use is calculated by multiplying the installed base by the average power use per server.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
25
3.5. Canadian Server Market Contact List Contact information on Canadian server market information sources has been provided as an electronic Excel spreadsheet attached to the electronic version of this report.
3.6. Conclusion Implications and Recommendations The amount of electricity used by data centres in Canada has risen significantly in recent years. In this analysis, we relied on IDC shipment data and server life assumptions to assess current trends in energy use for servers and data centres in the Canadian market. As discussed previously, many servers are never removed even when they are no longer needed, and continue to use electricity longer than their useful life. Therefore, the installed base and the power impact of servers may be underestimated in this study. Previous work indicates substantial latitude for improving the design and operation of servers in data centres. Improve Efficiency of Power Supplies One fundamental area where energy can be saved in data centres is in server power supplies. These devices convert alternating current (ac) power at the plug (often at “high line” voltages of 208 V to 230 V) into direct current (dc) power that server can use (often at 12 V or lower). In actual operation, server power supplies can waste 20 percent to 30 percent of the electricity that flows through them, turning it into heat during normal operation (Ecos and EPRI, 2008). Server power supplies exist today that can achieve greater than 90 percent operational efficiency (Ecos and EPRI, 2008). According to Ecos and EPRI (2008), annual savings from 73 to 195 kWh/ yr could be achieved per volume server depending on the level of efficiency of the power supply. The national electricity savings with 719 000 volume class servers in use is about 52-140 GWh. Decommissioning Comatose Servers Comatose servers are those that run applications no longer needed or run no applications at all, yet remain installed and operating continuously. According to a recent study jointly conducted by Uptime Institute and McKinsey (McKinsey and Company, 2008), such comatose servers account for up to 30 percent of total servers installed in some data centres. One way to improve data centre efficiency is simply to decommission comatose servers. Turning off comatose servers and removing them from use can save approximately 2 628 kWh per server removed. The national electricity savings with 719 000 volume class servers in use would be about 567 GWh. Virtualization of Existing Servers The widespread underutilization of servers is one of the most often-cited reasons for suboptimal energy efficiency in data centres. According to a recent study jointly conducted by the Uptime Institute and McKinsey (McKinsey and Company, 2008), the average server is no more than 6 percent utilized. The tendency to waste server Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
26
capacity by overbuilding systems and underutilizing them is widely spread in the general server market. Poor planning, overbuilding and lack of infrastructure management are the key contributors to energy waste in the data centre market. Facility utilization, meaning the cooling and floor layout and other infrastructure facilities, was found to be on average only 56 percent utilized. The report also concludes that with a growing need for data centres and IT infrastructure it will become an increasingly important issue for companies that seek to prevent IT costs from cutting into their profitability. Uptime expects that the use of virtualization technology will improve server utilization as well as reduce the number of physical servers needed (McKinsey and Company, 2008). Maximizing the utilization of existing servers therefore represents one of the most significant opportunities for energy savings. Virtualization technology can help improve utilization by allowing multiple, softwarebased “virtual machines” to run on a single server, allowing that server to run two to 20 times the number of applications (depending on application usage and demand on server computational capacity). The savings from virtualization come first from eliminating comatose servers and then from consolidating computation from servers that are underutilized. Based on these estimates, virtualization can save 57 percent of energy use. The national electricity savings with 719 000 volume class servers in use is about 1 581 GWh. Processor Efficiency Improvements Three key trends in server microprocessor technology can significantly reduce server energy use in the near future (Loper and Parr, 2007; U.S. EPA, 2007): the shift to multiple core processors; the development of dynamic frequency and voltage scaling capabilities; and the development of virtualization capabilities. Organizations like the Standard Performance Evaluation Corporation (SPEC) are working on methods to measure processor performance per watt as a metric for energy efficiency. 9 This should help IT managers to consider power characteristics along with other selection criteria to increase the efficiency of data centres. Memory Power Reductions Recent advancements in memory technology may also help to improve the energy efficiency of future servers. Ecos and EPRI found that in desktop computer differences in memory efficiency can save up to 1-2 watts on the motherboard (Beck et al, 2008). 10 Assuming that servers are operating 100 percent of the time, this would translate into 8 to 16 kWh of annual savings per server. The national electricity savings with 719 000 volume class servers in use is about 6 to 12 GWh. Improve Efficiency of Secondary Power Supplies, including Voltage Regulator Downs (VRDs), Point-of-Load Converters (POLs) and other Power Supplies Integrated into Motherboard Secondary power supplies are devices that convert dc electricity from the primary power supply into even lower dc voltages. Electricity entering servers is converted from ac to low-voltage dc power in the server power supply unit. The low-voltage dc power is used by the server’s internal components of, such as the central processing unit (CPU), memory, disk drives, chipset, and fans. The dc voltage serving the CPU is adjusted by the server’s secondary power supply before reaching the CPU.
9
http://www.spec.org/power_ssj2008/ http://efficientproducts.org/reports/computers/1337_EnergyEfficientComputerWhitePaper_FINAL_20Mar08.pdf
10
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
27
Although the server’s secondary power supply might only consume 50 to 80 watts of power itself, its power passes through a large number of other power conversion devices, including the primary power supply and the uninterruptible power supply, each with its own power conversion losses (EPRI and Ecos, 2008). Saving a watt at the secondary power supply saves an additional 0.3 to 0.5 watts in upstream devices because less power passes through them. Using best available measurement data and knowledge of secondary power supply design, EPRI and Ecos (2008) found that for a server that is effectively utilized 100 percent of the time, the annual energy savings opportunity for thicker copper traces is estimated at about 28 kWh per year. A significant improvement in efficiency is also possible when replacing a linear with a switcher. Most of the linear regulators have efficiencies in the 60 to 70 percent range (EPRI and Ecos, 2008). Efficiency figures for switchers often quoted by manufacturers (e.g. National semiconductor, Maxim IC, or Linear Technologies) in their dataset typically exceeds 95 percent and can result in significant power savings (EPRI and Ecos, 2008). Reduce Hard Drive Power Consumption A number of new and emerging technologies promise to decrease hard drive power consumption. Osterberg predicts that the average power use per drive will decrease by approximately 7 percent from 2007 to 2010 as a result of these trends (U.S. EPA, 2007). Technologies currently available can yield significant energy savings by allowing servers to “sleep” during times when data centre traffic is low (for example, in financial institutions where the servers only need to be on during active trading hours). Ecos and EPRI carried out tests on desktop computers to evaluate the performance and energy consumption differences between efficient drives and a more conventional 7,200 rpm, 3.5-inch desktop hard drive, and found that some technologies used 5W to 6W less power than the stock drive, while demonstrating comparable performance. The hybrid hard drive is a recent innovation that utilizes a flash memory buffer to cache a user’s most frequently used files, allowing magnetic portion of the hard drive to spin down. The national electricity savings with 719 000 volume class servers in use is about 31 GWh. Figure 8 below illustrates potential energy efficiency improvements.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
28
Figure 8: Latitude for Energy Efficiency Improvements
Estimated annual energy used / saved by servers (MWh)
1800000 1600000 1400000 Hard Drive
1200000 SPS
1000000
Secondary Pow er Supply
800000
Virtualization
600000
Pow er Supply
400000 200000 0 2004
2005
2006
2007
2008
2009
2010
2011
2012
4. References Aebischer, B., R. Frischknecht, C. Genoud, and F. Varone, Energy Efficiency Indicator for High ElectricLoad Buildings, The Case of Data Centres, Proceedings of the IEECB 2002, 2nd International Conference on Improving Electricity Efficiency in Commercial Buildings, Nice, France, 2002. ACEEE, and CECS. Funding prospectus for Analysis of Data Centers and their implications for energy demand, Washington, DC, American Council for an Energy Efficient Economy (ACEEE), Center for Energy and Climate Solutions (CECS), 2001. Bailey, M., M. Eastwood, T Grieser, L. Borovick, V. Turner, and R.C. Gray. 2007. Special Study: Data Center of the Future. New York, NY: IDC. IDC #06C4799. April. Brown, E., R. N. Elliott, and A. Shipley, Overview of Data Centers and Their Implications for Energy Demand. Washington, DC, American Council for an Energy Efficient Economy, Center for Energy & climate Solutions (CECS). 2001. Carr, N. G., The End of Corporate Computing, MIT Sloan Management Review, vol. 46, no. 3, pp. 67-73, Spring, 2005. Ecos and EPRI, Efficient Power Supplies for Data Center and Enterprise Servers, Final report, 2008, http://www.80plus.org/documents/ServerResearchReportFinal.pdf. EPRI and Ecos, Challenges and Energy Saving Opportunities in Measuring, Reporting, and Promoting High Efficiency Secondary Power Supplies, PIER final report prepared for the California Energy Commission, 2008. European Commission, Code of Conduct on Data Centres Version 0.7, October 2007, http://re.jrc.ec.europa.eu/energyefficiency/pdf/meeting%20Data%20Centers%20CoC%204%20December %202007/CoC%20Data%20Centres-v07-WORKING-DRAFT.pdf. Frost and Sullivan, World UPS Markets A569-27. 238 p., 2004
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
29
Gruener, J., Building High-Performance Data Centers. Dell Magazines - Dell Power Solutions (Issue 3 Building Your Internet Data Center, 2000. IDC Canada, Server Data Cut from Canadian Quaterly Server Tracker for Natural Resources Canada, April 17th 2008a. IDC Canada, Server Life Expectancy Guidance as per Canadian Infrastructure Hardware Research Program, May 2008b. Intel, Planning and Building a Data Center - Meeting the e-Business Challenge, Intel Corp, 2002. http://www.intel.com/network/idc/doc_library/white_papers/data_center /. Aug 01, 2002. Kawamoto, K., J. Koomey, M. Ting, B. Nordman, R.E. Brown, M. Piette, and A. Meier, Electricity used by Ofiice Equipment and Network Equipment in the U.S.: Detailed Report and Appendices. Berkeley, CA: Lawrence Berkeley National Laboratory. LBNL-45917. 2001. Koomey, G.K., O.Sezgen, and R. Steinmetz, Estimating Total Power Used by Data Centers in California, 2005, http://hightech.lbl.gov/library.html Koomey, J., Estimating Total Power Consumption by Servers in the U.S. and the World, 2007a Koomey, J., Estimating Regional Power Consumption by Servers: A Technical Note, 2007b. Requires actual IDC report # - can this be provided for this reference? Loper, J., and S. Parr, Energy Efficiency in Data Centers: A New Policy Frontier, Washington, DC: Alliance to Save Energy, 2007. McKinsey and Company, Revolutionizing Data Center Efficiency—Key Analyses, Presentation at the 2008 Uptime Symposium, 2008, http://uptimeinstitute.org/. Mitchell-Jackson, J., J. Koomey, M. Blazek, and B. Nordman, National and Regional Implications of Internet Data Center Growth, 2001, http://enduse.lbl.gov/Info/National_Implications.rev2.pdf Mitchell-Jackson, J., Energy Needs in an Internat Economy: A closer Look at Data Centers, Masters thesis, 2001. Roth, K., F. Goldstein, and J. Kleinman. Energy Consumption by Office and Telecommunications Equipment in Commercial Buildings--Volume I: Energy Consumption Baseline. Washington, D.C.: Prepared by Arthur D. Little for the U.S. Department of Energy, A.D. Little Reference no. 7289, 2002. Schäppi, B., F. Bellosa, B. Przywara, T. Bogner, S. Weeren, A. Anglade, Energy Efficient Servers in Europe - Energy Consumption, Saving Potentials, Market Barriers and Measures Part I: Energy Consumption and Saving Potentials, 2007a, http://www.efficient-server.eu/fileadmin/docs/reports/EServer_PartI_SavingPotentials_and_Scenarios_28112007.pdf. Schäppi, B., F. Bellosa, B. Przywara, T. Bogner, S. Weeren, A. Anglade, Energy Efficient Servers in Europe - Energy Consumption, Saving Potentials, Market Barriers and Measures - Part II, Draft Version, 2007b, http://www.efficient-server.eu/fileadmin/docs/reports/E-Server-Report_PartII.pdf. Schäppi, B., F. Bellosa., B. Przywara, T. Bogner, S. Weeren, A. Anglade, Energy Efficient Servers in Europe Report Part III Energy Efficiency Criteria and Benchmarks, 2007c, http://www.efficientserver.eu/fileadmin/docs/reports/E-Server-Report_Part3.pdf. The Economist, Pocket World in Figures 2006, 2006. TIA, Compilation of Definition of Terms, Acronyms and Abbreviations, Units of Measure, and Symbols from TR-42 and TR-41.7.2 Telecommunications Standards Modified and Accepted by the TR-42.5 Subcommittee, 2005, http://ftp.tiaonline.org/TR-42/Tr425/Public/TR425-05-10010a_working_dictionary.pdf Ton, M., and B. Fortenbery, Server Power Supplies, Berkeley CA: Report to Lawrence Berkeley National Laboratory by Ecos Consulting and EPRI Solutions, 2005. Tschudi, W.F., Xu, D. Sartor, and D. Stein, High Performance Data Centers: A Research Roadmap, LBNL-53483, 2003, http://hightech.lbl.gov/library.html Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
30
Turner IV, P., J.H. Seader, and K. G. Brill, Tier Classifications Define Site Infrastructure Performance, The Uptime Institute, 2008. U.S. Environmental Protection Agency, Report to Congress on Server and Data Center Energy Efficiency Public Law 109-431, 2007.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
31
5. Appendices
Appendix 1: Glossary Blade servers: Server chassis housing multiple, thin modular electronic circuit boards, known as servers blades. CAGR (Compound annual growth rate): CAGR is an average annual growth rate over a period of several years. Comatose servers: Comatose servers are those that run applications no longer needed or run no applications at all yet remain installed and operating continuously. High-end servers: Servers with an average selling value above $500 000. Mid-range servers: Server with an average selling value between $25 000 and $499 000. Non-x86 servers: Non-x86 servers include a wide range of server architectures and software platforms designed to perform a particular task-specific function, usually handling many instructions at the same time. Rack-optimized servers: Servers designed to be installed in a framework called a rack. Sunk cost: In economics and in business decision-making, sunk costs are costs that have been incurred and which cannot be recovered to any significant degree. Virtualization: Virtualization technology allows multiple, software based “virtual machines” to run on a single server, allowing that server to run 20 times the number of application. Volume servers: Server with an average selling value below $25 000. Workload: In computing, the workload is the amount of processing that the computer has been given to do at a given time. The workload consists of some amount of application programming running in the computer and usually some number of users connected to and interacting with the computer's applications. x86 servers: x86 refers to the most commercially successful instruction set architecture in the history of personal computing. .
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
32
Appendix 2: Estimating Energy Use by Servers in Canada – Previous Works There are no publicly available server energy use estimates for Canada. Ecos’ first analysis of the Canadian server market on behalf of NRCan included estimates of Canadian installed base of servers and energy use by servers. The results of this first analysis were discussed at the Energy Efficient Data Centres Executive Round Table in Ottawa, 24 January 2008. For this preliminary analysis, Canadian estimates of the number of servers installed in Canada were derived using a top-down approach from Koomey’s 2007 research, which provides worldwide and U.S. installed base estimates for servers (Koomey, 2007a), as shown in Figure 1. We assumed that Canada represents about 8 percent of the U.S. server market in terms of shipments, or about the same as the ratio of the Canadian and the U.S. economies in terms of gross domestic product. A2-Figure 1: Phase I – Ecos’ Energy Scaling Methodology
2005 Canadian Server Stock 826 thousand units
=
2005 United States Server Stock 10 306 thousand units
Canadian % of United States Servers Market about 8%
×
Source: 2005 U.S. server Installed base comes from Koomey (2007a). Canadian percent of U.S. server market is based on Ecos’ analysis of the Canadian server market (Ecos, 2007).
Other factors considered included the ratio of the Canadian and U.S. population and the ratio of the number of “secure servers” 11 in Canada and in the U.S. (see A2-Table 1, below). A2-Table 1: Scaling Factors Considered for Canadian Server Market Analysis
GDP (trillion U.S. $) Population Number of secure servers (Frost & Sullivan, 2004)
Canada
U.S.
Scaling Factor (U.S. to Canada)
0.856
10.9
8%
32 976 000
303 284 578
11%
6 841
187 673
4%
Sources: The Economist, 2006.
11
A secure server is a Web server that supports any of the major security protocols, like SSL, that encrypt and decrypt messages to protect them against third-party tampering (Frost & Sullivan, 2004). Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
33
On a population basis, Canada would account for about 11 percent of the U.S. shipment of products. On the basis of the number of secure servers, Canada would account for about 4 percent of the U.S. shipment of products (Frost & Sullivan, 2004). Another factor used to validate this 8 percent estimate was that it fell neatly between the 4 percent number of secure servers and the 11 percent population ratio. Following this methodology, we estimated the installed base of servers in Canada at 826 000 units. A2Table 1 compares Ecos’ estimate of the number of servers in Canada with Koomey’s estimates of the number of servers in the U.S. and worldwide in 2005. As indicated in A2-Table 1, volume servers represented over 95 percent of the market in terms of units. We used Koomey’s U.S. weighted average power use per unit for each server class as estimates of power use per unit for each server in Canada (Koomey, 2007a). The product of the estimated installed base by class and the average power use per server class yields the electricity used for that server summarizes the installed base, average power use and electricity consumption by server class for Canada, the U.S. and the world. In 2005, total electricity consumption for all servers in Canada was about 1.81 billion kWh. A2-Table 2: Estimated 2005 Server Installations, Power Use and Energy Use
Region
Server class
Canada
Volume Mid-range High-end Total
Installed units (thousands) 792 32 2 826
Average power use per server (watts) 219 625 7 651 -
Total electricity consumption (billion kWh/yr) 1.52 0.17 0.12 1.81
U.S.
Volume Mid-range High-end Total
9 897 387 22 10 306
219 625 7 651 -
19 2.1 1.5 22.6
World
Volume Mid-range High-end Total
25 959 1 264 59 27 282
222 607 8 106 -
50.5 6.7 4.2 61.45
Sources: Koomey 2007a for installed base of servers in the U.S. and in the world, average power use per server, and total electricity consumption of servers in the U.S and in the world. The Canadian server installed base numbers were derived form Koomey 2007a assuming that Canada represents about 8 percent of the U.S. server market.
Figure 2, below, shows the Canadian energy consumption by server class for 2000 and 2005, including cooling and auxiliary equipment. As Koomey (2007a), we multiplied the total electricity consumption by servers by a factor of two to get the total electricity consumption including electricity use for cooling and auxiliary equipment. As more detailed data become available, this hypothesis should be tested for Canada.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
34
A2-Figure 2: Estimated Allocation of Server Energy Consumption in Canada, 2000 and 2005 4 Cooling and auxillary equipment High-end servers Mid-range servers
Billion kWh/year
3
Volume servers
2
1
0
2000
2005
Sources: U.S. energy consumption estimates come from Koomey (2007a). Canadian energy consumption by server class comes from Ecos’ analysis of Koomey’s U.S. data assuming that Canada represents about 8 percent of the U.S. server market. As Koomey (2007a), we multiplied the total electricity consumption by servers by a factor of two to get the total electricity consumption including electricity use for cooling and auxiliary equipment.
Aggregate electricity use for servers doubled over the period 2000 to 2005. Most of this growth was the result of growth in the number of volume servers. The findings of our first analysis are based on best estimates from a variety of sources. Following this first analysis, Ecos recommended that NRCan purchase data from IDC Canada to support this market analysis.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
35
Appendix 3: IDC Canada Data Source The server data cut provided by IDC Canada came from three main sources:
IDC’s Canadian Quarterly Server Tracker
IDC Canada tracks Canadian server shipments into the market on a quarterly basis as part of its Canadian Quarterly Server Tracker service. Data inputs to this service include quarterly server manufacturer and their channels shipment data as well as demand side research conducted by IDC.
IDC’s Canadian Server Tracker, Forecast View
Based on ongoing observation of server shipments and related trends through the Canadian Quarterly Server Tracker , along with primary research conducted with IT managers in Canadian business as part of the Canadian Infrastructure Hardware research program, IDC generated the Server Tracker Forecast View worksheet that was provided. IDC also provides this forecast to its server manufacturer clients.
IDC’s Canadian Server Workloads 2004 Study
The Server Workloads 2004 data provided by IDC Canada is summary data collected as part of a syndicated study conducted by IDC in 2004. The study's goal was to help the sponsors of the study (server manufacturers) better understand the business needs that drive server deployment. Guidance on Canadian server life expectancy was provided by IDC Canada from research conducted within its Canadian Infrastructure Hardware research program
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
36
Appendix 4: Workload Taxonomy (IDC Canada, 2008a) Workload
BUSINESS PROCESSING
ERP. Includes the following transactional applications: Enterprise wide line-of-business applications (e.g., Oracle, PeopleSoft and SAP) Other business commerce applications that facilitate business transactions or other task automation over networks Departmental transactional applications that run on servers but don't tie directly to other applications CRM. Automates the customer-facing business processes within an organization: sales, marketing, and customer service (Collectively, these applications serve to manage the entire life cycle of a customer in helping an organization to build and maintain successful relationships.) OLTP. Uses a database; is not ERP Batch. Traditional legacy mainframe-type processes that execute business process transitions in a batch process
DECISION SUPPORT
Data warehousing/data mart. Tools that are used to create and run data warehouses and data marts Data analysis/data mining. Tools used to access data warehouses: online analytical processing (OLAP), data mining, data visualization, Web query tools, and so on
COLLABORATIVE
APPLICATION/SOFTWARE AADEVELOPMENT
IT INFRASTRUCTURE
WEB INFRASTRUCTURE
TECHNICAL
OTHER
Email. Traditional email applications Workgroup. Applications that let users collaborate and share information, such as groupware applications
Traditional application development work
File/print sharing. Includes infrastructure applications whose purpose is primarily as an endpoint (A transaction is sent to the network to be printed or stored on a disk, or a request is made get information from the network. This category also would include cache servers) Networking. Includes the following networking applications: directory, security/authentication, network data/file transfer, communication, and system data/file transfer Proxy caching. Includes applications that improve data centre performance by storing and serving content from the edge of the network Security. Includes applications specifically designed for authentication and identification and typically performs "firewall" services Systems management. Includes applications that monitor and account for systems performance, resource planning, and resource allocation Web serving. Utilizes HTTP protocols to accept requests from other servers and then search file systems according to the request Streaming media. Video and audio multimedia applications for streaming, including an Internet component This definition of technical workloads does not include infrastructure activities, which are included as part of a primary usage model of scientific computing. Note: This document uses the terms scientific/engineering and technical interchangeably. This category includes any other application types not covered by the above definitions.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
37
Appendix 5: CPU Type Taxonomy (IDC Canada, 2008a)
Complex instruction set computer (CISC). This design is the traditional type of computer processor. CISCs have large instruction sets with simple and complex instructions of variable lengths. Although x86 chip types (such as those produced by Intel and AMD) are CISC processor designs, they are separated into their own chip-type categories because of their high volume. In IDC's server taxonomy, CISCbased systems refer to proprietary systems, such as those produced by IBM for its zSeries servers.
Reduced instruction set computer (RISC). This processor design is produced by IBM, HP, Sun Microsystems, and others. RISCs have smaller instruction sets, and each instruction is usually of limited function with fixed-length formats. RISC servers typically support Unix and other (e.g., Tandem) platform software. Sun's SPARC, HP's PA-RISC, and IBM's POWER processors are all examples of RISC architecture.
Explicitly Parallel Instruction Computing (EPIC). This processor design is produced by Intel and represents its 64-bit Itanium processor family (codeveloped by Intel and HP). Several server OEMs have products that utilize the 64-bit Itanium processor.
x86. What was once referred to as the Standard Intel Architecture Server (SIAS) market will now encompass all x86(32)- and x86(64)-based systems, and will be referred to as the x86 server market in IDC publications and databases. The x86 server market includes all systems that fit IDC chip-type definitions, regardless of form factor (such as blades) and regardless of price band or CPU capacity (i.e., systems with price points above $25,000 and/or containing more than eight processors will still be in the x86 server category as long as they meet chip-type definitions):
x86(64). This processor design refers to x86 architecture systems that have 64-bit extensions. x86(64) processor designs enable 64-bit computing while remaining compatible with existing x86 software infrastructure. AMD's Opteron processor and Intel's EM64T are examples of x86(64) processors.
x86(32). x86(32) refers to volume 32-bit CISC processors developed and produced by companies such as Intel and AMD. Intel's Pentium and XEON processor families and AMD's Athlon processors are examples of this architecture. x86(32)-based systems run Microsoft Windows, Novell NetWare, Linux, Unix, and other operating system environments
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
38
Appendix 6: Calculation for % of Replacement and Average Age of Installed Base of Servers in 2007 Replacement vs. New Sales The difference between the installed base of servers in 2007 and the installed base of servers in 2006 equals the new server capacity:
installedbase06 = sales 06 + sales 05 + sales 04 + sales 03 + 0.49 * sales 02 installedbase07 = sales 07 + sales 06 + sales 05 + sales 04 + 0.49 * sales 03
installedbase07 − installedbase06 = sales 07 − 0.51 * sales 03 − 0.49 * sales 02
We estimated the installed base of servers using data on shipments provided by IDC Canada (2008a). We calculated that the average server life expectancy is 4.49 years. The average server life expectancy was calculated by using a weighted average of the x86 and non-x86 server life expectancies provided by IDC (IDC, 2008b).
The % of new sales in 2007 is equal to:
∆installedbase2007 − 2006 sales 2007
% of replacement = 1-% of new sales
Average age of the server installed base in 2007 The average age of servers was calculated based on a weighted average of the age of servers which compose the 2007 installed base of servers; see the following equation:
AverageAgeend 07 =
sales 07 (1 / 2) + sales 06 (1.5) + sales 05 (2.5) + sales 04 (3.5) + (1 / 2) sales 03 (4.25) installedbaseend 07
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
39
Appendix 7: Ecos Estimates of Canadian Shipment Data for 2000 and 2001 A7-Table 1: Koomey’s World Server Installed Base Information (Koomey, 2007a)
Volume Mid-range High-End Total
2000
2001
2002
3 926 000
3 981 000
4 184 000
283 000
206 000
204 000
13 000
10 400
9 400
4 222 000
4 197 400
4 397 400
A7-Table 2: Ecos Canadian Percentage of the World Shipment (2002) Canada Percentage (2002) Volume
2.6%
Mid-range
3.4%
High-End
4.1%
A7-Table 3: Ecos Canadian Shipments for 2000 and 2001
Volume
2000
2001
101 960
103 388
Mid-range
9 725
7 079
High-End
531
425
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
40
Appendix 8: Energy Analysis, 2008 Units
Volume
Mid-range
High-end
Installed base Canada Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Yukon Territory Northwest Territories Nunavut
753 841 11 973 3 219 21 834 17 512 177 236 292 734 27 393 23 160 76 921 99 443 722 992 701
22 541 358 96 653 524 5 300 8 753 819 693 2 300 2 974 22 30 21
1 033 16 4 30 24 243 401 38 32 105 136 1 1 1
Watts/server
224
598
8 378
kWh/server
1 963
5 241
73 433
GWh/year GWh/year GWh/year GWh/year GWh/year GWh/year GWh/year GWh/year GWh/year GWh/year GWh/year GWh/year GWh/year GWh/year
1 480 061 23 508 6 320 42 867 34 383 347 979 574 742 53 782 45 471 151 023 195 242 1 418 1 948 1 376
118 150 1 877 505 3 422 2 745 27 778 45 880 4 293 3 630 12 056 15 586 113 156 110
75 847 1 205 324 2 197 1 762 17 833 29 453 2 756 2 330 7 739 10 005 73 100 71
Average power used per server
Annual energy consumption per server
Direct electricity consumption Canada Newfoundland and Labrador Prince Edward Island Nova Scotia New Brunswick Quebec Ontario Manitoba Saskatchewan Alberta British Columbia Yukon Territory Northwest Territories Nunavut
(1) The installed base for Canada was calculated by adding the server shipments over the useful life of servers. We used information on population size and distribution by province to split the Canadian shipments by province/region. (2) The average power used by server class for Canada was taken from Koomey 2007b. We used the average power used for the rest of the world in 2005 directly from Koomey (2007b). (3) Annual energy consumption per server was calculated by multiplying the average power usage by the number of hours in a calendar year. We assumed 8765 hours/year and 100% load factor/ (4) Direct electricity consumption was calculated by multiplying the server installed base by province by the average energy consumption per server.
Final Report | Canadian Market Analysis for Servers and Data Centres | November 5, 2012
41
