Transcript
Reference Architecture
Cisco Solution for EMC® VSPEX™ Server Virtualization VMware® vSphere™ 5 for 250 Virtual Machines Enabled by Cisco Unified Computing System, Cisco Nexus Switches, VMware VMware vSphere 5, EMC VNX5500™, and EMC Next-Generation Backup
EMC VSPEX April 2012
Copyright © 2012 EMC Corporation. All rights reserved. Published in the USA. Published April 2012 EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. For the most up-to-date regulatory document for your product line, go to the technical documentation and advisories section on the EMC online support website. Part Number: h10713.1
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
2
Table of contents Reference architecture overview .................................................................................... 5 Document purpose ..................................................................................................................... 5 Solution purpose ........................................................................................................................... 5 The business challenge ............................................................................................................... 6 The technology solution .............................................................................................................. 6 Solution benefits ........................................................................................................................... 7
Key components ............................................................................................................... 8 Introduction................................................................................................................................... 8 Cisco UCS B-Series Servers .......................................................................................................... 8 Cisco Nexus 5000 Series Switches .............................................................................................. 9 VMware vSphere 5....................................................................................................................... 9 Cisco Nexus 1000v Series Switches ............................................................................................ 9 EMC Virtual Storage Integrator for VMware ............................................................................ 9 VNX VAAI support ........................................................................................................................ 9 EMC VNX Series .......................................................................................................................... 10 Software suites available...................................................................................................... 11 Software packs available .................................................................................................... 11 VNX FAST Cache ........................................................................................................................ 11 EMC Avamar .............................................................................................................................. 11
Solution architecture ....................................................................................................... 13 Logical architecture diagram .................................................................................................. 13 Reference architecture overview ........................................................................................... 13 Hardware resources................................................................................................................... 17 Software resources..................................................................................................................... 18
Server and Network architecture .................................................................................. 20 Topology...................................................................................................................................... 20 Server details............................................................................................................................... 20 Network Layout .......................................................................................................................... 20
Storage architecture ....................................................................................................... 21 Storage layout ............................................................................................................................ 21 Storage layout overview ........................................................................................................... 21
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
3
High availability and failover ......................................................................................... 23 Introduction................................................................................................................................. 23 Storage layer .............................................................................................................................. 23 Connectivity layer ...................................................................................................................... 23 Host layer ..................................................................................................................................... 23
Validated environment profile ....................................................................................... 24 Profile characteristics................................................................................................................. 24
Backup environment profile ........................................................................................... 25 Backup characteristics .............................................................................................................. 25 Backup layout ............................................................................................................................ 25
Conclusion ....................................................................................................................... 26 References ....................................................................................................................... 27 Cisco ............................................................................................................................................ 27 EMC documentation ................................................................................................................. 27 Other documentation ............................................................................................................... 27
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
4
Reference architecture overview Document purpose
This document describes the reference architecture of the Cisco Solution for EMC® VSPEX™ server virtualization for 250 virtual machines. This document provides guidelines for building similar solutions.
Solution purpose
VSPEX pre-validated and modular architectures are built with proven bestof-breed technologies to create complete virtualization solutions that enable you to make an informed decision in the hypervisor, compute and networking layers. VSPEX eliminates server virtualization planning and configuration burdens. VSPEX infrastructures accelerate your IT Transformation by enabling faster deployments, greater flexibility of choice, efficiency, and lower risk. This reference architecture is not intended to be a comprehensive guide to every aspect of this solution. Server capacity is provided in generic terms for required minimums of CPU, memory and network interfaces; the customer is free to select the server and networking hardware of their choice that meet or exceed the stated minimums. The Cisco solution for VSPEX features the enterprise-class Cisco Unified Computing System (UCS) Servers and Nexus switches delivering performance, versatility, and density without compromise. Cisco’s portfolio of products and technologies offers a holistic approach to the design, operation, and delivery of IT services. This approach makes it easier for IT to integrate existing technology silos and: •
Respond more quickly to business demands
•
Reduce capital expenditures and operating expenses
•
Simplify IT operations
These platforms optimized for Virtualization, manageability, performance and scalability are ideal for: •
Consolidation and virtualization
•
Private cloud computing
•
Application or infrastructure-as-a-service
For more detailed information on performance and scalability testing, refer to the EMC VSPEX Server Virtualization Solution VMware vSphere 5 for 250 Virtual Machines Enabled by VMware vSphere 5, EMC VNX5500— Deployment Guide of this solution and associated documentation for best practices and specific usage requirements.
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
5
The business challenge
Customers require a scalable, tiered, and highly available infrastructure on which to deploy their business and mission-critical applications. There are several new technologies available to assist them in consolidating and virtualizing their server infrastructure, but they need to know how to use these technologies to maximize their investment, support service-level agreements, and reduce their total cost of ownership (TCO). This solution addresses the following challenges:
The technology solution
•
Availability — Standalone servers incur downtime for maintenance or unexpected failures. Clusters of redundant standalone nodes are an inefficient in use of CPU, disk and memory resource.
•
Server management and maintenance — Individually maintained servers require significant repetitive activities for monitoring, problem resolution, patching, and other common activities. This is labor intensive, costly, error-prone, and inefficient. Security, downtime, and outage risks are elevated.
•
Ease of solution deployment — While small and medium businesses must address the same IT challenges as larger enterprises, staffing levels, experience, and training are generally more limited. IT generalists are often responsible for managing the entire IT infrastructure, or reliance is placed on third-party sources for maintenance or other tasks. Perceived complexity of the IT function raises fear of risk and may block adoption of new technology. Therefore, simplicity of deployment and management are highly valued.
•
Storage efficiency— Storage added locally to physical servers or provisioned directly from a shared resource or array leads to overprovisioning and waste.
•
Backup — Traditional backup approaches are slow and frequently unreliable. There tends to be inflection points (or plateaus) in the virtualization adoption curve when the number of virtual machines increases. With a few virtual machines, the situation seems manageable and most organizations can get by with existing tools and processes. However, when the virtual environment grows the backup and recovery become the limiting factor in deployment.
This solution uses the Cisco UCS B200 M3 Server, Cisco Nexus 5548 UP and 1000v switches, Cisco UCS Manager, Cisco VM-FEX technology, EMC VNX5500™ array and VMware vSphere 5 to provide storage and server hardware consolidation. The new virtualized infrastructure is centrally managed, allowing efficient deployment and management of a scalable number of virtual machines and associated shared storage.
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
6
Solution benefits
This VSPEX solution allows the customer that is new to server virtualization to realize the following benefits: •
Application Availability — Highly available, scalable infrastructure that makes most efficient use of CPU, memory, and storage resources and grows with the business
•
Simplified and centralized management of server and storage deployment, monitoring, patching, and other normal activities, resulting in lower management overhead, reduced risk, and reduced downtime
•
Simplicity of solution deployment and use, targeting the IT generalist in the small or medium business (SMB) and providing ease of use, integration points, and ability to mask complex tasks
•
Storage Efficiency — EMC VNX™ array features optimized storage efficiency, reduced over-provisioning and simplified storage management
•
Backup — Integration with EMC’s next-generation backup solutions offering virtual machine and application backup and recovery capabilities.
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
7
Key components Introduction
This section briefly describes the key components of this solution. •
Cisco UCS B-Series Servers
•
Cisco Nexus 5000 Series Switches
•
VMware vSphere 5
•
Cisco Nexus 1000v Series Switches
•
EMC Virtual Storage Integrator for VMware
•
EMC VNX Series
•
VNX
•
VNX FAST Cache
•
VNX FAST Cache, a part of the VNX FAST Suite, enables flash drives to be used as an expanded cache layer for the array. FAST Cache is an array-wide, non-disruptive cache, available for both file and block storage. Frequently accessed data is copied to the FAST Cache in 64kB increments and subsequent reads and/or writes to the data chunk are serviced by FAST Cache. This enables immediate promotion of very active data to flash drives. This dramatically improves the response times for the active data and reduces data hot spots that can occur within the LUN.
•
EMC Avamar
The Solution architecture on page 12 section provides details on all the components that make up the reference architecture. Cisco UCS B-Series Cisco Unified Computing System is the first truly unified data center Servers platform that combines industry-standard, x86-architecture blade and rack servers with networking and storage access into a single system. Key innovations in the platform include a standards-based unified network fabric, Cisco Virtualized Interface Card (VIC) support, and Cisco Extended Memory Technology. The system uses a wire- once architecture with a selfaware, self-integrating, intelligent infrastructure that eliminates the timeconsuming, manual, error-prone assembly of components into systems. Cisco UCS B-Series Blade Servers provide a comprehensive line of two
and four-socket servers to deliver world-record-setting performance to a
wide range of workloads. Based on Intel® Xeon® processor E7 and E5 product families, these servers are ideal for virtualized and non-virtualized applications. These servers: •
Reduce capital and operating expenses with converged network fabrics and integrated systems management
•
Deliver performance, versatility, and density without compromise, Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
8
Cisco Nexus 5000 Series Switches
•
Address a broad set of workloads, from IT and web infrastructure through distributed database for both virtualized and nonvirtualized environments
•
Increased IT staff productivity and business agility through just-intime provisioning and mobility support for both virtualized and nonvirtualized environments
Cisco Nexus® 5000 Series Switches deliver an innovative architecture to simplify data center transformation by enabling a high-performance, standards-based, multi-protocol, multi-purpose, Ethernet-based fabric. They help consolidate separate LAN, SAN, and server cluster network environments into a single 10 Gigabit Ethernet fabric. This unification enables network consolidation and greater utilization of previously separate infrastructure and cabling, reducing by up to 50 percent the number of adapters and cables required and eliminating redundant switches. This infrastructure displacement also lowers power and cooling costs significantly.
VMware vSphere 5 VMware vSphere 5 transforms a computer’s physical resources, by virtualizing the CPU, RAM, hard disk, and network controller. This transformation creates fully functional virtual machines that run isolated and encapsulated operating systems and applications just like physical computers The high-availability features of VMware vSphere 5 such as vMotion and Storage vMotion enable seamless migration of virtual machines and stored files from one vSphere server to another with minimal or no performance impact. Coupled with vSphere DRS and Storage DRS, virtual machines have access to the appropriate resources at any point in time through load balancing of compute and storage resources. Cisco Nexus 1000v Series Switches
Cisco Nexus 1000v Switches deliver highly secure, multitenant services by adding virtualization intelligence to the data center network. These softswitches are integrated with VMware vCloud Director. They are built to scale for cloud networks, with support for Virtual Extensible LAN (VXLAN). This addresses the requirements for scalable LAN segmentation and helps to enable broader virtual machine mobility.The VNXe™ series is powered by Intel Xeon processor, for intelligent storage that automatically and efficiently scales in performance, while ensuring data integrity and security.
EMC Virtual Storage Integrator for VMware
EMC Virtual Storage Integrator (VSI) for VMware vSphere is a plug-in to the vSphere client that provides a single management interface that is used for managing EMC storage within the vSphere environment. Features can be added and removed from VSI independently, which provides flexibility for customizing VSI user environments. Features are managed by using the
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
9
VSI Feature Manager. VSI provides a unified user experience, which allows new features to be introduced rapidly in response to changing customer requirements. The following features were used during the validation testing: •
Storage Viewer (SV) – Extends the vSphere client to facilitate the discovery and identification of EMC VNX storage devices that are allocated to VMware vSphere hosts and virtual machines. SV presents the underlying storage details to the virtual datacenter administrator, merging the data of several different storage mapping tools into a few seamless vSphere client views.
•
Unified Storage Management – Simplifies storage administration of the EMC VNX unified storage platform. It enables VMware administrators to provision new Network File System (NFS) and Virtual Machine File System (VMFS) datastores, and RDM volumes seamlessly within vSphere client.
Refer to the EMC VSI for VMware vSphere product guides on the EMC Online Support website for more information.
EMC VNX Series
The EMC VNX™ family is optimized for virtual applications delivering industry-leading innovation and enterprise capabilities for file, block, and object storage in a scalable, easy-to-use solution. This next-generation storage platform combines powerful and flexible hardware with advanced efficiency, management, and protection software to meet the demanding needs of today’s enterprises. The VNX™ series is powered by Intel Xeon processor, for intelligent storage that automatically and efficiently scales in performance, while ensuring data integrity and security. The VNXe™ series is purpose-built for the IT manager in smaller environments and the VNX series is designed to meet the highperformance, high-scalability requirements of midsize and large enterprises. Table 1.
VNX customer benefits
Feature Next-generation unified storage, optimized for virtualized applications
Capacity optimization features including compression, deduplication, thin provisioning, and application-centric copies
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
10
Feature High availability, designed to deliver five 9s availability
Automated tiering with FAST VP (Fully Automated Storage Tiering for Virtual Pools) and FAST Cache that can be optimized for the highest system performance and lowest storage cost simultaneously
Multiprotocol support for file, block, and object with object access through Atmos™ Virtual Edition (Atmos VE)
Simplified management with EMC Unisphere™ for a single management interface for all network-attached storage (NAS), storage area network (SAN), and replication needs
Up to three times improvement in performance with the latest Intel Xeon multicore processor technology, optimized for Flash
Software suites available • FAST Suite — Automatically optimizes for the highest system performance and the lowest storage cost simultaneously. •
Local Protection Suite — Practices safe data protection and repurposing.
•
Remote Protection Suite — Protects data against localized failures, outages, and disasters.
•
Application Protection Suite — Automates application copies and proves compliance.
•
Security and Compliance Suite — Keeps data safe from changes, deletions, and malicious activity.
Software packs available • Total Efficiency Pack — Includes all five software suites. • VNX VMware vStorage API for Array Integration Support
Total Protection Pack — Includes local, remote, and application protection suites.
Hardware acceleration with VMware vStorage API for Array Integration (VAAI) is a storage enhancement in vSphere 5 that enables vSphere to offload specific storage operations to compatible storage hardware such as the VNX series platforms. With storage hardware assistance, vSphere performs these operations faster and consumes less CPU, memory, and storage fabric bandwidth.
VNX FAST Cache VNX FAST Cache, a part of the VNX FAST Suite, enables flash drives to be used as an expanded cache layer for the array. FAST Cache is an arraywide, non-disruptive cache, available for both file and block storage. Frequently accessed data is copied to the FAST Cache in 64kB increments and subsequent reads and/or writes to the data chunk are serviced by Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
11
FAST Cache. This enables immediate promotion of very active data to flash drives. This dramatically improves the response times for the active data and reduces data hot spots that can occur within the LUN. EMC Avamar
EMC’s Avamar® data deduplication technology seamlessly integrates into virtual environments, providing rapid backup and restoration capabilities. Avamar’s deduplication results in vastly less data traversing the network, and greatly reduces the amount of data being backed up and stored – translating into realized capital savings and operational benefits. The following are two of the most common recovery requests made to backup administrators: •
File-level recovery — Object-level recoveries account for the vast majority of user support requests. Common actions requiring file-level recovery are–individual users deleting files, applications requiring recoveries, and batch process-related erasures.
•
System recovery — Although complete system recovery requests are less frequent in number than those for file-level recovery this bare metal restore capability is vital to the enterprise. Some common root causes for full system recovery requests are–viral infestation, registry corruption, or unidentifiable unrecoverable issues.
The Avamar’s functionality in conjunction with VMware implementations adds new capabilities for backup and recovery in both of these scenarios. Key capabilities added in VMware such as the vStorage API integration and change block tracking (CBT) enables the Avamar software to protect the virtual environment more efficiently. Leveraging CBT for backup and recoveries with virtual proxy server pools, this functionality minimizes management needs. Coupling that with Data Domain as the storage platform for image data, this backup solution enables the most efficient integration with two of the industry leading nextgeneration backup solutions.
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
12
Solution architecture Logical architecture diagram
The architecture diagram characterizes the infrastructure validated for support of up to 250 virtual machines. Figure 1 shows the overall logical architecture of the solution.
Figure 1. Reference architecture overview
Logical architecture
The architecture includes the following components: Cisco UCS B200 M3 Server - The Cisco UCS B200 M3 Series Blade Server delivers performance, versatility, and density without compromise. It addresses the broadest set of workloads, from IT and web infrastructure, through distributed database. Building on the success of the Cisco UCS B200 M2 Series Blade Server, the enterprise-class Cisco UCS B200 M3 Blade Series Server further extends the capabilities of the Cisco UCS portfolio in a half-width blade form factor. The Cisco UCS B200 M3 harnesses the power of the latest Intel® Xeon® processor E5-2600 product family, with up to 384 GB of RAM (using 16-GB DIMMs), two disk drives, and up to dual 4x 10 Gigabit Ethernet throughput. In addition, Cisco Unified Computing System has the architectural advantage of not having to power and cool excess switches in each blade chassis. With a larger power budget per blade server, Cisco can design uncompromised expandability and capabilities in its blade servers, as evidenced by the new Cisco UCS B200 M3, with its leading memory slot and drive capacity. Cisco Nexus 5548UP switches - The Cisco Nexus 5548UP is a 1RU 10 Gigabit Ethernet, Fibre Channel, and FCoE switch offering up to 960 Gbps of throughput and up to 48 ports. The switch has 32 unified ports and one Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
13
expansion slot. The Cisco Nexus 5500 platform is equipped with expansion modules that can be used to increase the number of 10 Gigabit Ethernet and FCoE ports or to connect to Fibre Channel SANs with 8/4/2/1-Gbps Fibre Channel switch ports, or both. The Cisco Nexus 5548UP supports one expansion module from the following offerings : • Ethernet module that provides sixteen 1/10 Gigabit Ethernet and FCoE ports using the SFP+ interface • Fibre Channel plus Ethernet module that provides eight 1/10 Gigabit Ethernet and FCoE ports using the SFP+ interface, and eight ports of 8/4/2/1-Gbps native Fibre Channel connectivity using the SFP+/SFP interface • A unified port module that provides up to sixteen 1/10 Gigabit Ethernet and FCoE ports using the SFP+ interface or up to sixteen ports of 8/4/2/1Gbps native Fibre Channel connectivity using the SFP+ and SFP interface. The use of 1/10 Gigabit Ethernet or 8/4/2/1-Gbps Fibre Channel on a port is mutually exclusive but selectable for any of the 16 physical ports per module. In addition to these expansion modules, the Cisco Nexus 5548UP supports a Layer 3 daughter card that can be ordered with the system or as a spare (field upgradable). This daughter card provides up to 160 Gbps of Layer 3 forwarding capability (240 million packets per second [mpps]) that can be shared by all 48 ports in the chassis. Layer 3 daughter card does not take up one of the expansion slots on the rear of the chassis, but instead is installed by replacing the Layer 2 I/O module (N55-DL2) that is located on the front of the chassis. • In addition to three expansion module, the Cisco Nexus 5596UP supports Layer 3 module, which provides up to 160 Gbps of Layer 3 forwarding capability (240 mpps) that can be shared by all the I/O ports in the chassis. • Version 2 of Layer 3 Daughter Cards and Expansion Modules have enhanced hardware capability that increases host table size from 8K to 16K or multicast routes from 4K to 8K (enabled in future software release). Cisco UCS™ Manager – The Cisco UCS™ Manager provides unified, embedded management of all software and hardware components of the Cisco Unified Computing System™ across multiple chassis, Cisco UCS BSeries Blade Servers, Cisco UCS C-Series Rack-Mount Servers, and thousands of virtual machines. By enabling better automation of processes, Cisco UCS Manager allows data center managers to achieve greater agility and scale in their server operations while reducing complexity and risk. Cisco UCS Manager provides flexible role- and policy-based management using service profiles
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
14
and templates and facilitates processes based on IT Infrastructure Library (ITIL) concepts. It also provides integration with industry-leading systems management solutions to ease adoption of Cisco Unified Computing System within IT departments. Cisco UCS Manager also enables custom development with an extensive XML API that exposes more than 9000 objects and provides increased system visibility and control. VMware vSphere 5 –Provides a common virtualization layer to host a server environment. The specifics of the validated environment are listed in Table 2. vSphere 5 provides highly available infrastructure through such features as: •
vMotion – Provides live migration of virtual machines within a virtual infrastructure cluster, with no virtual machine downtime or service disruption.
•
Storage vMotion – Provides live migration of virtual machine disk files within and across storage arrays with no virtual machine downtime or service disruption.
•
vSphere High Availability (HA) – Detects and provides rapid recovery for a failed virtual machine in a cluster.
•
Distributed Resource Scheduler (DRS) – Provides load balancing of computing capacity in a cluster.
•
Storage Distributed Resource Scheduler (SDRS) – Provides load balancing across multiple datastores, based on space use and I/O latency.
Cisco Nexus 1000v technology - Cisco Nexus® 1000V Series Switches provide a comprehensive and extensible architectural platform for virtual machine and cloud networking. The switches are designed to accelerate server virtualization and multitenant cloud deployments in a secure and operationally transparent manner. Integrated into the VMware vSphere hypervisor and fully compatible with VMware vCloud Director, the Cisco® Nexus 1000V Series provides: •
Advanced virtual machine networking based on Cisco NX-OS operating system and IEEE 802.1Q switching technology
•
Cisco vPath technology for efficient and optimized integration of virtual network services
•
Virtual Extensible Local Area Network (VXLAN), supporting cloud networking
These capabilities help ensure that the virtual machine is a basic building block of the data center, with full switching capabilities and a variety of Layer 4 through 7 services in both dedicated and multitenant cloud Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
15
environments. With the introduction of VXLAN on the Nexus 1000V Series, network isolation among virtual machines can scale beyond traditional VLANs for cloud-scale networking. VMware vCenter Server 5 – Provides a scalable and extensible platform that forms the foundation for virtualization management for the VMware vSphere 5 cluster. All vSphere hosts and their virtual machines are managed from vCenter. VSI for VMware vSphere – EMC VSI for VMware vSphere is a plug-in to the vSphere client that provides storage management for EMC arrays directly from the client. VSI is highly customizable and helps provide a unified management interface. Microsoft SQL Server 2008 R2 – VMware vCenter Server requires a database service to store configuration and monitoring details. A Microsoft SQL 2008 R2 server is used for this purpose. IP Network – All network traffic is carried by standard Ethernet network with redundant cabling and switching. User and management traffic is carried over a public network while storage traffic is carried over a private, nonroutable subnet. Shared Infrastructure – DNS and authentication/authorization services like Microsoft Active Directory can be provided via existing infrastructure or set up as part of the new virtual infrastructure. EMC VNX5500 array – Provides storage by presenting NFS datastores to vSphere hosts.
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
16
Hardware resources
Table 2 lists the hardware used in this solution. Table 2.
Solution hardware
Hardware VMware vSphere servers
Configuration Memory: •
2 GB RAM per virtual machine (500 GB RAM across all servers)
•
2 GB RAM reservation per vSphere host
Notes Configured as a single vSphere cluster.
CPU: •
One vCPU per virtual machine
•
Four vCPUs per physical core
Network: Six 1 GbE NICs per server Note: To implement VMware vSphere High Availability (HA) functionality and to meet the listed minimums, the infrastructure should have one additional server. Cisco Nexus 5548 UP
Cisco Nexus 6248
Two Nexus 5548 UP switches each with: •
32 fixed Unified ports (supports 10 GbE, FCoE, FC)
•
1 x 1 GbE Management port
Two Nexus 6248 UP switches each with:
•
32 fixed Unified ports (supports 10 GbE, FCoE, FC)
•
1 x 1 GbE Management port
•
2Cluster link port
Redundant LAN configuration
Redundant LAN configuration
Avamar
EMC nextgeneration backup
•
One Gen4 utility node
•
One Gen4 3.9TB spare node
•
Five Gen4 3.9TB storage nodes
Data Domain •
One Data Domain DD670
•
Two ES30 15x1TB HDD
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
17
Avamar
EMC nextgeneration backup
Software resources
One Gen4 utility node
•
One Gen4 3.9TB spare node
•
Five Gen4 3.9TB storage nodes
Data Domain
EMC VNX5500
Cisco UCS B200 M3
•
•
One Data Domain DD670
•
Two ES30 15x1TB HDD
•
Two Data Movers (active / standby)
•
Four 1 GbE interfaces per data mover
•
One 1 GbE interface per control station for management
•
One hundred fifty 300 GB 15k rpm 3.5inch SAS disks
•
Six 300GB 15k 3.5-inch SAS disks as hot spares.
VNX shared storage
10 x B200 M3 Servers each with: •
2 x Intel Xeon E5-2650 processor
•
64 GB (8 x 8GB 1600 MHz) memory
•
1 x Cisco UCS VIC1240 40 GbE Dual port PCIe card (for 10 GbE Network)
The following table lists the software used in this solution. Table 3.
Solution software
Software
Configuration
Cisco UCS Server Cisco UCS B200 M3
2.0.2c.0.031320121609
Cisco UCS Manager
2.0(2m)
Network Cisco Nexus 5548UP
version 5.1(3)N1(1a)
Cisco UCS 6248 UP
NX OS: version 5.0(3)N2(2.02m)
Cisco Nexus 1000V Virtual Supervisor Module (VSM)
4.2(1) SV1(4a)
Virtual Ethernet Module (VEM)
v131-4.2.1.1.4.1.0-3.0.4
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
18
VNX5500 VNX OE for file
Release 7.0.50-2
VNX OE for block
Release 31 (05.31.000.5.704)
EMC VSI for VMware vSphere: Unified Storage Management
Version 5.1
EMC VSI for VMware vSphere: Storage Viewer
Version 5.1
VMware vSphere vSphere server
5.0
vCenter Server
5.0
Operating system for vCenter Server
Windows Server 2008 R2 Standard Edition
Microsoft SQL Server
Version 2008 R2 Standard Edition
Virtual Machines Base operating system
Microsoft Window Server 2008 R2
VDBench
5.02 Note: VDBench was used to validate this solution. However, it s not required in this configuration.
Next-generation backup Avamar
6.0.0-592
Data Domain OS
5.1.0.9-282511
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
19
Server and Network architecture Topology
The architecture diagram in this section shows the server configuration and network topology (Figure 2). Figure 2.
Server details
Network Layout
Server configuration and Network topology
The Server uses the following configuration: •
Cisco Unified Computing System for managing servers, storage and networking components within single management domain.
•
The VMs created on the server are stored in NFS shares provisioned on the VNX 5500 storage to enable vMotion.
•
Network adapters are teamed for fault tolerance and throughput.
The network layout is as follows: •
Management, Storage, vMotion and Production traffic are provisioned as separate VLANs using 10GE connectivity
•
Virtual port channels are implemented on two Nexus switches for redundant loop-free topologies.
•
Virtual switch for this environment is provisioned using Cisco Nexus 1000v Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
20
Storage architecture Storage layout
Figure 3 shows the physical disk layout.
Figure 3. Storage layout overview
Storage layout
The reference architecture uses the following configuration: •
One hundred fifty 300 GB SAS disks are allocated to ten storage pools as thirty 4+1 RAID 5 groups.
•
Six 300GB SAS disks are configured as hot spares. Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
21
•
Optionally you can configure up to 20 flash drives in the array FAST Cache. These drives are not considered to be a required part of the solution, and additional licensing may be required in order to use the FAST Suite.
•
EMC recommends that at least one hot spare disk is allocated for each 30 disks of a given type.
•
At least two NFS shares are allocated to the vSphere cluster from each storage pool to serve as datastores for the virtual servers.
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
22
High availability and failover Introduction
This VSPEX solution provides a highly available virtualized server and storage infrastructure. Each component is configured to provide a scalable, robust architecture for the host, connectivity, and storage layers.
Storage layer
The VNX family is designed for five 9s availability by using redundant components throughout the array. All of the array components are capable of continued operation in case of hardware failure. The RAID disk configuration on the array provides protection against data loss due to individual disk failures, and the available hot spare drives can be dynamically allocated to replace a failing disk.
Connectivity layer
The advanced networking features of the VNX family provide protection against network connection failures at the array. Each vSphere host has multiple connections to user and storage Ethernet networks to guard against link failures. These connections are spread across multiple Ethernet switches to guard against component failure in the network.
Host layer
The vSphere servers have redundant power supplies and network connections to reduce the impact of failure of these components. vSphere HA is configured on the vSphere cluster to help recover virtual servers quickly in case of a complete host failure.
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
23
Validated environment profile Profile characteristics
The solution was validated with the following environment profile. Table 4.
Profile characteristics
Profile characteristic
Value
Number of virtual machines
250
Virtual machine OS
Windows Server 2008 R2
Processors per virtual machine
1
Number of virtual processors per physical CPU core
4
RAM per virtual machine
2 GB
Average storage available for each virtual machine
100 GB
Average IOPS per virtual machine
25 IOPS
Number of datastores to store virtual machine disks
10
Number of virtual machines per datastore
25
Disk and RAID type for datastores
RAID 5, 300 GB, 15k rpm, 3.5inch SAS disks
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
24
Backup environment profile Backup characteristics
The solution was sized with the following application environment profile. Table 5.
Profile characteristics
Profile characteristic
Value
Number of users
2499
Number of virtual machines
250 (20% DB, 80% Unstructured)
Exchange data
2.5TB (1GB mail box per user)
SharePoint data
1.25 TB
SQL server
1.25 TB
User data
25 TB (10.0 GB per user)
Daily change rate for the applications Exchange data
10%
SharePoint data
2%
SQL server
5%
User data
2%
Retention per data types
Backup layout
All DB data
14 Dailies
User data
30 Dailies, 4 Weeklies, 1 Monthly
Avamar provides various deployment options depending on the specific use case and the recovery requirements. In this case, the solution is deployed with both Avamar and Data Domain managed as a single solution. This enables to backup the unstructured user data directly to the Avamar system for simple file level recovery. The database and virtual machine images are managed by the Avamar software, but will be directed to the Data Domain system with the embedded Boost client library. This backup solution unifies the backup process with industry leading deduplication backup software and storage, and achieves the highest levels of performance and efficiency.
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
25
Conclusion Cisco solution for VSPEX has been engineered to enable the small or medium business to simply, quickly, and reliably deploy a broad range of shared storage workloads. Setup wizards enable an IT generalist to provision VMware storage with a short series of mouse clicks, with recommended best practices applied automatically. In conjunction with VMware vCenter and vSphere, VNX provides a proven and economical path to server virtualization for the small or medium business. EMC nextgeneration backup enables efficient protection of this dynamic environment and allows for growth and flexibility. Table 6.
Solution benefits
Feature
Benefits
Cisco Unified Computing System
First truly unified platform that combines industrystandard, x86-architecture blade and rack servers with networking and storage access into a single system. The system uses a wire- once architecture with a self-aware, self-integrating, intelligent infrastructure delivering dramatic improvement in TCO.
Cisco Nexus series Switches
Designed for a broad range of physical, virtual, storage access, and high-performance computing environments, thus giving customers the flexibility to meet and scale their IT requirements in a gradual manner and at a pace that aligns with their business objectives.
EMC next-generation backup
Unifies the backup process with industry leading deduplication backup software and storage, and achieves the highest levels of performance and efficiency.
EMC VNX Series
Provides a robust, reliable, high-performance, common storage platform for many virtual servers. A single storage platform that is efficient, powerful, and built for the most demanding virtual environments.
VMware vSphere 5
VMware vSphere, including vCenter and vSphere, provides a well-established virtualization platform for server consolidation. vSphere 5 adds new storage features to an already rich set of capabilities to help increase and scale virtualized environments.
This reference architecture provides a blueprint of a validated server virtualization solution enabled by EMC VNX series, EMC next-generation backup, and the VMware vSphere 5 virtualization platform. Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
26
References Cisco documentation
EMC documentation
Other documentation
For documentation on •
Cisco Unified Computing System, please refer to the website http://www.cisco.com/go/unifiedcomputing
•
Cisco Nexus series Switches, please refer to the website http://www.cisco.com/go/nexus
•
Cisco Validated Designs, please refer to the website http://www.cisco.com/go/dcdesignzone
The following documents, available on the EMC Online support website, provide additional and relevant information. If you do not have access to a document, contact your EMC representative: •
EMC VSPEX Server Virtualization Solution VMware vSphere 5 for 50 Virtual Machines Enabled by VMware vSphere 5, EMC VNXe3100, and EMC Next-Generation Backup— Reference Architecture
•
EMC VSPEX Server Virtualization Solution VMware vSphere 5 for 100 Virtual Machines Enabled by VMware vSphere 5, EMC VNXe3300 , and EMC Next-Generation Backup — Reference Architecture
•
EMC VSPEX Server Virtualization Solution VMware vSphere 5 for 125 Virtual Machines Enabled by VMware vSphere 5, EMC VNX5300, and EMC Next-Generation Backup— Reference Architecture
•
EMC VSPEX Server Virtualization Solution VMware vSphere 5 for 250 Virtual Machines Enabled by VMware vSphere 5, EMC VNX5500 — Deployment Guide
•
EMC VSPEX Server Virtualization Solution VMware vSphere 5 for 101 250 Virtual Machines – Sizing Guide
•
EMC VSI for VMware vSphere: Storage Viewer — Product Guide
•
EMC VSI for VMware vSphere: Unified Storage Management— Product Guide
For VMware documentation, please refer to the VMware web site at http://www.vmware.com .
Cisco Solution for VSPEX Server Virtualization VMware vSphere 5 for 250 Virtual Machines
27
