Tivoli Storage Manager Sample Architecture

Transcript

Tivoli Storage Manager for Virtual Environments Sample Architecture Tivoli Storage Manager for Virtual Environments Sample Architecture Disk to Disk Virtual Machine Backup, TSM Deduplication on AIX, and XIV Storage 1.0 Authors: Linh Vo, Robert Riordan, Jason Basler Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 1 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture Document History Document Location This is a snapshot of an on-line document. Paper copies are valid only on the day they are printed. Revision History Revision Revision Number Date 1.0 11/16/12 Summary of Changes Changes marked Initial version About this document This document provides a sample architecture describing a deployment of Tivoli Storage Manager for Virtual Environments. This information is provided for reference purposes to provide guidance during the planning of other deployments of Tivoli Storage Manager for Virtual Environments. The information provided is based upon configurations used and test results derived from implementations in the test labs used by IBM Tivoli Storage Manager development. The architectures are not intended to be a suitable configuration for all situations, and do not replace the need to carefully plan and design the elements of your own implementation of Tivoli Storage Manager. Disclaimer This document contains measurements of Tivoli Storage Manager performance which are intended to be used for reference during planning implementations of Tivoli Storage Manager. Performance results reported in this document were measured by IBM under controlled test conditions. Performance results measured in other environments may vary from those reported herein, depending on factors such as system configuration, workload characteristics, and other environmental conditions. Accordingly, this data does not constitute a performance guarantee or warranty. The information contained in this document is distributed on an "as is" basis without any warranty either expressed or implied. This document has been made available as part of IBM developerWorks WIKI, and is hereby governed by the terms of use of the WIKI as defined at the following location: http://www.ibm.com/developerworks/tivoli/community/disclaimer.html Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 2 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture Contents Document History...................................................................................................................2 Document Location........................................................................................................................................2 Revision History.............................................................................................................................................2 About this document......................................................................................................................................2 Disclaimer......................................................................................................................................................2 Contents..................................................................................................................................3 1. Overview of the architecture...............................................................................................5 1.1 Overview of Data Protection for VMware Technologies...........................................................................5 1.1.1 Incremental forever............................................................................................................................5 1.1.2 Parallel sessions................................................................................................................................5 1.2 Deduplication technology.........................................................................................................................6 1.3 Architecture summary..............................................................................................................................6 1.4 Architecture details...................................................................................................................................7 1.4.1 Deduplicated primary pool.................................................................................................................8 2. Observed performance.......................................................................................................9 2.1 Data ingestion..........................................................................................................................................9 2.2 Protected data........................................................................................................................................10 2.3 Data movement......................................................................................................................................10 2.4 Sample backup statistics........................................................................................................................11 2.5 Database disk IOPS measurements......................................................................................................12 2.6 CPU and Memory usage on the vStorage backup server......................................................................12 3. Hardware Details..............................................................................................................14 3.1 VMware components.............................................................................................................................14 3.2 vStorage backup server.........................................................................................................................14 3.3 TSM server.............................................................................................................................................14 3.3.1 Server system..................................................................................................................................14 3.3.2 Software stack for the TSM server...................................................................................................15 3.3.3 Disk storage for the TSM server......................................................................................................15 3.4 Network overview...................................................................................................................................16 4. System configuration details.............................................................................................17 4.1 vStorage backup server configuration....................................................................................................17 4.1.1 Operating system details.................................................................................................................17 4.1.2 TSM software installed....................................................................................................................17 Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 3 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 4.1.3 TSM options file...............................................................................................................................18 4.1.4 Service configuration.......................................................................................................................19 4.1.5 Schedule configuration....................................................................................................................20 4.2 vCenter plugin server configuration........................................................................................................20 4.2.1 Machine details................................................................................................................................20 4.2.2 Node configuration...........................................................................................................................20 4.2.3 vmcliprofile options file.....................................................................................................................22 4.2.4 VMCLI setup....................................................................................................................................22 4.3 TSM Server Configuration......................................................................................................................23 4.3.1 Operating system tuning changes...................................................................................................23 4.3.2 DB2 configuration changes..............................................................................................................23 4.3.3 Volume manager and file system configuration...............................................................................24 4.3.4 TSM server processing options.......................................................................................................25 4.3.5 Device class creation.......................................................................................................................26 4.3.6 Storage pool creation.......................................................................................................................26 4.3.7 Policy settings..................................................................................................................................26 4.3.8 Schedules and macro definitions to control server maintenance tasks............................................27 Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 4 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 1. Overview of the architecture This document provides a sample architecture using Tivoli Storage Manager for Virtual Environments – Data Protection for VMware to protect a VMware environment using a deduplicated disk-based storage pool with a TSM server running on IBM Power hardware with AIX. A single physical vStorage Backup Server running Microsoft Windows is used as the data mover for transferring VMware data to the TSM server. The storage for the TSM server is provided using IBM XIV storage. This document does not describe the highest performing environment possible. Rather, it describes one example environment to serve as an aid to deployment. 1.1 Overview of Data Protection for VMware Technologies Tivoli Storage Manager for Virtual Environments (TSM for VE) provides off-host backup capabilities for VMware environments using the VMware vStorage APIs for Data Protection (VADP). A single backup can be used for both complete virtual machine recoveries, as well as file-level recovery. The capabilities of TSM for VE are further enhanced in the 6.4 release to provide additional scalability with new capabilities to allow for an incremental-forever backup model, and the ability to use multiple sessions to backup virtual machines in parallel. 1.1.1 Incremental forever A major new capability provided by TSM for VE 6.4 is incremental-forever backups, a new type of VM backup that replaces the TSM for VE 6.3 periodic-full backups. Incremental forever eliminates the need to schedule periodic full backups. Each incremental-forever backup is represented as a self-contained backup, similar to a synthetic full backup produced without movement of previously stored data. This means that a consistent restore can be performed even if all prior versions of the VM backup are deleted. For incremental-forever backups, expiration policy is applied on a per-backup basis. By contrast, for periodic-full backups, expiration policy is applied to each FULL backup and subsequent incremental backups. With incremental-forever backup, only one backup schedule is needed. An initial full backup is done automatically if TSM detects that the VM has not yet been backed up, and all subsequent backups are incremental backups. The technology also makes the transition from the periodic-full model (vis-a-vis TSMVE 6.3) to the new incremental-forever model as transparent as possible, by providing a simple migration path from implementations using periodic-full backups to the new incremental-forever model. 1.1.2 Parallel sessions TSM for VE 6.4 provides the ability to schedule backups of a large collection of VMware virtual machines, thereby providing an automated backup solution that allows for the automatic discovery of newly created virtual machines as well as providing parallel backup of multiple virtual machines. This is a significant enhancement over previous releases which offered what was essentially a serial VE backup solution, where parallelism could only be realized through the use of multiple instances of the backup-archive client. The use of parallel sessions optimizes backup throughput by load balancing across multiple threads from the entire domain of virtual machines to backup. The new TSM for VE option that determines the degree of parallelism of a VM backup is the vmmaxparallel option. In addition, there are options that allow the user to limit the impact of a parallel backup to individual ESX hosts and/or individual datastores, thereby providing a throttling mechanism across the various ESX hosts and datastores in the VMware environment. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 5 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture The combination of the new incremental-forever and multi-session backup capabilities provides a more scalable backup solution for protecting VMware environments, with fewer vStorage Backup Servers, less impact to the TSM server, and smaller backup windows. 1.2 Deduplication technology Deduplication technology provides an effective method of reducing the amount of data which TSM for VE needs to transfer over a network and store in storage pools. Detailed information describing the deduplication technology in TSM is available in the paper Effective Planning and Use of Tivoli Storage Manager V6 Deduplication. 1.3 Architecture summary The sample TSM for VE architecture provides a disk-based backup solution that retains data in a deduplicated storage pool for its entire life cycle. Data protection is provided for hundreds of virtual machines in the deduplicated storage pool. The following table summarizes some key points which are detailed in the remainder of the document: Key point Summary • • Protected clients • • • • TSM database size • TSM log sizes • • • • Deduplicated storage pool Networks Document: • • 340 virtual machines are protected with a daily scheduled backup. Backups take advantage of incremental-forever, multi-session backup, and client-side deduplication technologies available in TSM for VE 6.4 (these technologies are independent from the TSM server version, but benefit from enhancements provided in the TSM 6.3.3 server). 16TB of used VMware storage is being protected. All of the virtual machines are protected using a single vStorage Backup Server and a single TSM server. Server policy is set to retain each daily recovery point for 45 days in a deduplicated storage pool. 1 TB of storage allocated for the database with 565 GB currently used by the database. 2.7 million objects are tracked in the TSM server database to represent 45 days worth of recovery points for 340 virtual machines. 128 GB for the active log. 433 GB for the archive log. As much as 2.5TB of new data is ingested into the server each day (size before client-side deduplication) 27 TB of disk storage is allocated for the deduplicated storage pool. This storage is used to provide: o 675TB of data is protected across 45 backup versions (size before incremental) o 66TB of data is managed in a deduplicated storage pool (size after incremental, before deduplication) o 19TB is stored (size after incremental and deduplication) 1Gb LAN for client backup ingestion. 4Gb SAN for VMware, vStorage Backup server, and TSM server storage. Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 6 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 1.4 Architecture details The diagram above illustrates the architecture that includes the use of TSM for Virtual Environments to perform incremental-forever virtual machine backups. A physical machine is used for the vStorage backup server, and is able to take advantage of the SAN transport on the reads from the VMware datastores. Backups are sent to the TSM server across a LAN using a combination of client-side deduplication and compression to a deduplicated file storage pool. Data remains in the deduplicated file storage pool for its entire retention and is never allowed to migrate to another storage pool. The following summarizes key aspects of the architecture: • A physical machine acts as the data mover component of the vStorage backup server. This capability is implemented in the TSM backup-archive client portion of the Data Protection for VMware solution. • Data is read from VMware datastores across the SAN by the vStorage backup server using the VMware Virtual Disk Development Kit (VDDK) for SAN transport. • The virtual machine backups are ingested over the LAN to the TSM server using client-side deduplication and compression. • The TSM for Virtual Environments incremental-forever backup capability is used to keep the amount of data process to a minimum before other data reduction technologies are applied. • The best practices implementation of separating the activities of data ingestion and server data maintenance tasks into two distinct windows each day is used on the TSM server. This includes ordering the data maintenance tasks optimally to avoid resource contention. • Volumes on IBM XIV storage is used for both the TSM server database and file storage pool. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 7 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 1.4.1 Deduplicated primary pool The deduplicated primary storage pool retains backup objects for their entire retention. A single vStorage backup server ingest backups directly into this storage pool using the capability to use multiple sessions to backup more than one virtual machine simultaneously. Additional details are provided in the section on TSM server configuration related to configuring the deduplicated storage pool. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 8 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 2. Observed performance Observed performance results are presented in this section based on measurements taken on the sample architecture in the test lab. This information is provided to serve as a reference during planning a TSM implementation, and is not intended to be a guarantee of the performance to be expected in a different environment. 2.1 Data ingestion Data ingestion is performed over an eight hour backup window. Item/metric Variations Protected virtual machines Number of VMs Limit/Range 1 Notes 340 Typical number of 60 – 90 sessions on TSM server This range is due to a maximum of 30 parallel VM backups. Each parallel VM backup uses 3 sessions. Typical backup elapsed 7 - 9 hours time (incremental) New data ingested (daily) 1 Unplanned full backups 4.4 (per day) This is an average taken across 30 days, and includes full backups taken due to VMware CBT resets, and new virtual machines taking their first full backup. Number of objects 70,000 – 200,000 Virtual machines are stored on the TSM server as a group of smaller objects. Volume of data before dedup/compression 2 1.5 – 2.5 TB Volume of data after dedup/compression 300 - 600 GB Virtual machines range in size (used capacity) from 8GB to 500GB, with an average used capacity of 50GB. 2 Approximately 40 of the virtual machines have a virtual hardware level of 4 which prevents them from performing incremental backups. They take full backups every day, and account for approximately 1.1TB of the daily backup ingestion. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 9 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 2.2 Protected data Item/metric Variations Limit/Range TSM database (used) 565 GB Number of stored objects 3 2.7 million Total protected data 2.3 Notes Logical 66 TB Amount of protected data including all versions after incremental reductions, but before deduplication reductions. Stored 19 TB 47 TB eliminated by deduplication Data movement Item/metric Limit/Range Notes Objects deleted by expire inventory (daily) 200,000 – 500,000 The expire inventory processing typically completes in under one hour. Throughput of database backup 80 – 120 MB/sec Throughput of reclamation 20 - 25 MB/sec Throughput of backup ingestion 4 80 - 90 MB/sec 8 processes 3 TSM for VE stores virtual machine backups as a group of smaller objects. This number reflects the number of resulting objects tracked in the TSM server database for all virtual disks across all backup versions for the collection of virtual machines. 4 This is the aggregate throughput across 30 parallel backup sessions. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 10 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 2.4 Sample backup statistics Charts which follow are based on the following day’s backup reflected in the following summary statistics reported in the TSM backup-archive client schedule log. The negative compression value reflects the random nature of the data generated by the tools used to drive change activity within the virtual machine. Typically, compression will provide benefit in space savings. 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 10/16/2012 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 05:49:36 Total number of objects inspected: Total number of objects backed up: Total number of objects updated: Total number of objects rebound: Total number of objects deleted: Total number of objects expired: Total number of objects failed: Total objects deduplicated: Total number of bytes inspected: Total number of bytes processed: Total bytes before deduplication: Total bytes after deduplication: Total number of bytes transferred: Data transfer time: Network data transfer rate: Aggregate data transfer rate: Objects compressed by: Deduplication reduction: Total data reduction ratio: Elapsed processing time: Document: Tivoli Storage Manager for Virtual Environments Sample Architecture 340 328 0 0 0 0 12 325 16.61 TB 2.26 TB 2.26 TB 619.72 GB 653.70 GB 335,323.71 sec 7,248.03 KB/sec 86,340.28 KB/sec -5% 73.31% 96.16% 07:49:09 Date: 11/16/2012 Version: 1.0 Page 11 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 2.5 Database disk IOPS measurements The following chart shows database disk IOPS measurements taken throughout one day with the general time ranges of the various TSM server activities labeled. The data is taken from the output of the iostat command using a 5 second sampling interval. The data points represent the sum of the reported “transfers per second” for all four of the disks used for holding the TSM server database volumes. TSM server database volume IOPS 20000 max 20628 18000 16000 14000 avg 2251 backup ingestion reclamation stddev 1675 dbbackup 12000 expiration 10000 8000 6000 IOPS (all DB disks) 4000 2000 0 0:00:00 2.6 4:48:00 9:36:00 14:24:00 19:12:00 0:00:00 CPU and Memory usage on the vStorage backup server The following charts show the memory and CPU usage of the vStorage Backup Server during a backup that includes more than 300 virtual machines, using a vmmaxparallel option setting of 30. The data is taken from the Windows Performance Monitor using a 15 second sampling interval. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 12 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture The following chart shows memory usage of the TSM backup-archive scheduler process performing a backup of the VMware environment using 30 parallel VM backups. The following chart shows CPU usage of the TSM backup-archive scheduler process during the same backup. The scale is from 0% to 800%, with each 100% corresponding to one of the eight CPU cores. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 13 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 3. Hardware Details 3.1 VMware components The VMware environment consists of a Virtual Center managing 13 ESX/ESXi servers of different versions. • Virtual Center 5.1 running on a physical Microsoft Windows 2008R2 64bit server • VMware ESX 4.1 • VMware ESXi 5.1, 5.0, and 4.1 • Virtual machines are primarily using virtual hardware levels 7 and 8, with a small subset using a virtual hardware level 4. • The virtual machines are running a vast assortment of Microsoft Windows and Linux operating systems, and have the most recent version of VMware tools installed for the specific VMware host version they are running under. The VMware datastores are backed by local disks and a variety of storage subsystems including: • IBM XIV Storage System (2810-A14) • IBM System Storage DS4000 and DS5020 • IBM System Storage DS8100 3.2 3.3 vStorage backup server • IBM System x3550 M3 (7944-AC1) • 2 x 2.4GHz 4-core processors • 24 GB RAM • 2-port 4Gb fibre channel adapter • 1Gb Ethernet adapter TSM server 3.3.1 Server system • IBM Power 780 (9179-MHB) • LPAR configured with: o 4 x 3.86 GHz POWER7 processor cores in POWER7 compatibility mode. o 32 GB RAM o 2-port 4Gb fibre channel adapter Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 14 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture o Virtual 1Gb Ethernet adapter from virtual I/O server I/O adapters Feature code of installed adapter Slot# Slot type Description Slot 1 PCIe x8 Not used for this test Slot 2 PCIe x8 Not used for this test Slot 3 PCIe x8 Not used for this test Slot 4 PCIe x8 5774 2-port 4Gb Fibre channel adapter Slot 5 PCIe x8 5767 2-port 1Gb Ethernet adapter Slot 6 PCIe x8 Not used for this test Additional system details for the p780 are available here: http://www.redbooks.ibm.com/redpapers/pdfs/redp4639.pdf 3.3.2 Software stack for the TSM server • AIX 7.1 (7100-00-04-1140), 64bit kernel • TSM server for AIX Version 6.3 (6.3.3.0) o TSM server 6.3.3 includes enhancements to expiration and deduplication dereferencing that benefit VM backups to deduplicated storage pools. 3.3.3 Disk storage for the TSM server The TSM server database and storage pools are implemented on an IBM XIV Storage System. Separate storage volumes are allocated within the XIV for the database and storage pools as described in the section which follow. • IBM XIV Storage System (2810-A14) o 1TB SATA 7.2K drives o System software level 10.2.4.e o A single storage pool holds all of the volumes for the TSM server. o 2 x 4Gbps fibre channel connections to the SAN fabric 3.3.3.1 Disk for TSM database • 1 x 50GB volume for the TSM server instance directory • 1 x 137GB volume for the TSM server active log • 1 x 446GB volume for the TSM server archive log Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 15 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture • 4 x 257GB volumes for the TSM database 3.3.3.2 Disk for TSM storage pools • 3.4 7 x 4105GB volumes for the TSM storage pool Network overview VM backups involve two data paths. During backup, the data is transferred from the datastores to the vStorage Backup Server, then from the vStorage Backup Server to the TSM server. In this architecture, data from vStorage Backup Server to the TSM server is all LAN-based using a 1Gb connection. Data from the datastores to the vStorage Backup Server is transferred using different transports. The SAN transport is used for a majority of the virtual machines. The NBD (LAN) transport is used for a small subset of virtual machines which reside on local datastores. The storage network is a 4Gb fibre channel switched fabric. The TSM server has a total of two fibre channel ports which are zoned to see the IBM XIV disks. The vStorage Backup Server contains two fibre channel ports zoned in the SAN with access to the VMware storage. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 16 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 4. System configuration details The sections which follow provide detailed information on how the major components of the sample architecture were configured. 4.1 vStorage backup server configuration The vStorage Backup Server is configured to use multiple backup sessions, with limits to the number of sessions to use for each host and datastore. Both compression and client-side deduplication are enabled. One datamover node backs up VMware data to a datacenter node. The incremental-forever backup strategy is used, which requires a single daily backup schedule on the TSM server. The SAN fabric 4.1.1 Operating system details • Microsoft Windows 2008 R2 64 bit • QLogic Fibre Channel Adapter driver 9.1.9.48 • The SAN fabric has been zoned and disk subsystem volumes assignments made so that the vStorage backup server has access to the volumes holding VMware datastores. • Important: Windows automatic volume mounting has been disabled using the Windows diskpart command. • Multi-path software for the disk subsystems used for VMware storage: o IBM DS Storage Manager Host Software version 10.60.x5.17 o IBM XIV Host Attachment Kit 1.90.1114 o IBM Subsystem Device Driver 4.1.2 TSM software installed • Tivoli Storage Manager backup-archive client 6.4.0 including the VMware vStorage API runtime files component. • Tivoli Storage Manager for Virtual Environments Data Protection for VMware Enablement File feature. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 17 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 4.1.3 TSM options file ** TSM server connection options TCPSERVERADDRESS viossadec.mycompany.com TCPPORT 1500 PASSWORDACCESS generate NODENAME dm_immortal_640 ** Scheduling and logs SCHEDMODE prompted MANAGEDSERVICES SCHEDULE ERRORLOGNAME “c:\Program Files\tivoli\tsm\baclient\dsmerrorvm.log” SCHEDLOGNAME “c:\Program Files\tivoli\tsm\baclient\dsmschedvm.log” ** VMware connection and backup type VMCHOST overlord.mycompany.com VMCUSER administrator VMBACKUPTYPE full VMVSTORTRANSPORT san:nbd ** Backup scope DOMAIN.VMFULL "VMHOSTCLUSTER=SVT- Cluster; VMHOST=superman.mycompany.com,boxboro.mycompany.com,ordinals.mycompany.com,prism vm1.mycompany.com,delta.mycompany.com,dora.mycompany.com,drinks.mycompany.com,ir onthrone.mycompany.com,maximo5.mycompany.com,swiper.mycompany.com; -VM=FCMEXCVM,FCMSQLVM,batmanvm01_SQL,*test*" EXCLUDE.VMDISK "win2008x64 - medic" "Hard disk 3" EXCLUDE.VMDISK "win2008x64 - medic" "Hard disk 4" VMPROCESSVMWITHPRDM yes VMPROCESSVMWITHINDEPENDENT yes VMENABLETEMPLATEBACKUPS yes ** Backup optimization VMMAXPARALLEL 30 VMLIMITPERHOST 5 VMLIMITPERDATASTORE 5 ** Data reduction DEDUPLICATION DEDUPCACHESIZE COMPRESSION yes 2048 yes The following table describes the key options in the dsm_immortal.opt file. Option Setting Explanation compression yes Enables client compression deduplication yes Enabled client-side deduplication Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 18 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture **Varies depending on environment** Specifies the scope of the backup. Note: this can also be specified as part of the schedule definition on the TSM server. nodename dm_immortal_640 The datamover node used for authentication. Note: that the asnode to which backups are stored is specified as part of the schedule definition on the TSM server. vmbackuptype fullvm Backs up VMs as single entities, required for incremental forever backups vmchost overlord.mycompany.com Hostname of the Virtual Center vmcuser administrator VMware admin user name vmenabletemplatebackups yes Allow backup of VMware templates vmlimitperdatastore 5 Each datastore has, at most, 5 VMs being backed up in parallel vmlimitperhost 5 Each ESX or ESXi server has, at most, 5 VMs being backed up in parallel vmmaxparallel 30 VM backups uses up to 30 parallel sessions domain.vmfull vmprocessvmwithindependent yes Backup VMs that contain independent disk volumes. However, the independent disks are not included as part of the VM backup. vmprocessvmwithprdm yes Backup VMs that contain RDM volumes in physical compatibility mode. However, the pRDM volumes are not included as part of the VM backup. vmvstortransport san:nbd Attempts to use san transport first. If it does not work, then fall back to nbd (LAN) transport. 4.1.4 Service configuration The following commands create a CAD-managed scheduler service and remote client agent service. Change into the TSM client installation directory: set DSM_DIR=c:\progra~1\tivoli\tsm\baclient dsmcutil install scheduler /name:"TSM Sched immortal" /optfile:%DSM_DIR %\dsm_immortal.opt /autostart:no /password:password1 /errorlog:%DSM_DIR%\dsmerror_immortal.log /schedlog:%DSM_DIR %\dsmsched_immortal.log /startnow:no /node:dm_immortal_640 Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 19 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture dsmcutil install cad /name:"TSM CAD immortal" /cadschedname:"TSM Sched immortal" /optfile:%DSM_DIR%\dsm_immortal.opt /autostart:yes /password:password1 /startnow:no /node:dm_immortal_640 dsmcutil install remote /name:"TSM Agent immortal" /optfile:%DSM_DIR %\dsm_immortal.opt /partnername:"TSM CAD immortal" /password:password1 /startnow:no /node:dm_immortal_640 4.1.5 Schedule configuration The following commands on the TSM server creates a daily Incremental Forever backup schedule that starts at midnight. The ASNODE option directs the backup data into the datacenter node DC_CLIENTDEDUP. There is no requirement to create a separate schedule to perform full backups. The TSM client automatically performs the initial full backup for machines which have not been previously backed up. DEFINE SCHEDULE vmdomain VM_DAILY_MIDNIGHT ACTION=BACKUP SUBACTION=VM OPTIONS=’vmfulltype=vstor -vmbackuptype=full –mode=ifincr –asnodename=dc_clientdedup’ SCHEDSTYLE=ENHANCED STARTDATE=11/07/2012 STARTTIME=00:00 DEFINE ASSOCIATION vmdomain VM_DAILY_MIDNIGHT dm_immortal_640 4.2 vCenter plugin server configuration In an environment with one datacenter, at least four nodes are required for the vCenter plugin user interface: 1. Virtual Center node 2. VMCLI node 3. datacenter node (used to store the backups) 4. datamover node The configuration setup can be done simply by using the Configuration Wizard on the vCenter plugin user interface; however, this section describes the manual configuration steps. 4.2.1 Machine details • Microsoft Windows 2008 R2 64bit • IBM Tivoli Storage Manager for Virtual Environments 6.4.0 feature Data Protection for VMware vCenter plug-in 4.2.2 Node configuration Register VMCLI node register node vmcli_overlord password1 domain=vmdomain Register Virtual Center node register node vc_overlord password1 domain=vmdomain Register datacenter node register node dc_clientdedup password1 domain=vmdomain maxnummp=100 Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 20 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture Register datamover node register node dm_immortal_640 password1 domain=vmdomain Grant proxy authority grant proxy target=vc_overlord agent=dc_clientdedup,vmcli_overlord grant proxy target=dc_clientdedup agent=dm_immortal_640,vmcli_overlord Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 21 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 4.2.3 vmcliprofile options file >>> VMCLI VMCLI_TRACE NO VE_TSM_SERVER_NAME viossadec.mycompany.com VE_TSM_SERVER_PORT 1500 # -p VE_TSMCLI_NODE_NAME vmcli_overlord VE_VCENTER_NODE_NAME vc_overlord #VE_TRACE_FILE tsmcli.trace # -x tsmcli trace file #VE_TRACE_FLAGS api api_detail # -y trace flags VE_DATACENTER_NAME ARC Lab::dc_clientdedup VMCLI_TASK_EXPIRATION_TIME 864000 # in seconds, defaults to 864000s = 10 days VMCLI_RESTORE_TASK_EXPIRATION_TIME 2592000 # in seconds, defaults to 2592000s = 30 days VMCLI_GRACE_PERIOD 2592000 # in seconds, defaults to 2592000s = 30 days VMCLI_SCHEDULER_INTERVAL 60 # in seconds, defaults to 1s VMCLI_DB_HOST localhost VMCLI_DB_PORT 1527 VMCLI_CACHE_EXPIRATION_TIME 600 # in seconds, defaults to 600s = 10 min VMCLI_DB_NAME VMCLIDB VMCLI_RECON_INTERVAL_FCM 600 # backend specific recon interval setting in seconds default 600s = 10 min VMCLI_RECON_INTERVAL_TSM 1200 # backend specific recon interval setting in seconds default 1200s = 20 min VMCLI_DB_BACKUP AT 00:00 VMCLI_DB_BACKUP_VERSIONS 3 VMCLI_LOG_DIR logs DERBY_HOME C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\derby <<< 4.2.4 VMCLI setup Go to the scripts directory: C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\scripts echo pwd>pwd.txt vmcli –f set_password –I pwd.txt Note: The pwd.txt file will be deleted after the vmcli command completes. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 22 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 4.3 TSM Server Configuration The following sections summarize configuration changes that were made to software components after installation. 4.3.1 Operating system tuning changes The following AIX operating system settings were changed on the test system: • Enable I/O completion ports chdev -l iocp0 -P • Tune hard disks used for TSM database, logs, and storage pool volumes. The commands below were repeated for all disks used for the TSM database and storage pools. chdev -l hdisk2 -a max_transfer=0x100000 chdev -l hdisk2 -a queue_depth=32 • Several configuration choices were made relative to the volume manager and file systems. The following options are reflected in commands which follow in a later section: • Volume groups are created as the Big-type. • JFS2 file systems are used. • File system logs are not created as in-line. • File systems are mounted with the release-behind (rbrw) option for file systems used by the archive log and file storage pools. 4.3.2 DB2 configuration changes No additional configuration changes were made to DB2 beyond what the TSM installation and database formatting perform automatically. The following TSM command was used to format the database: dsmserv -o /tsminst1/dsmserv.opt -i /tsminst1 format dbdir=/tsmdb01,/tsmdb02,/tsmdb03,/tsmdb04 activelogsize=122880 activelogdir=/tsmactlog archlogdir=/tsmarchlog archfailoverlogdir=/tsmarchfail The following DB2 level is currently installed: DB2 v9.7.0.6", "s120516", "IP23321", and Fix Pack “6”. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 23 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 4.3.3 Volume manager and file system configuration 1. Create volume groups for the database, database logs, and storage pools (several additional volume groups are created for file storage pools which are not shown) mkvg mkvg mkvg mkvg -B -B -B -B -y -y -y -y tsmdb hdisk2 hdisk3 hdisk4 hdisk5 tsmactlog hdisk6 tsmarchlog hdisk7 tsmfile hdisk8 hdisk9 hdisk10 hdisk11 hdisk12 hdisk13 hdisk14 2. Create logical volumes and file systems. In some cases repeated commands have been omitted. Note the pp size counts used in your environment will vary depending on the physical volume size. The rbrw mount option has been shown to improve performance for storage pool volumes, archive log volumes, and file systems used for TSM database backups to disk. After file systems are created and mounted, be sure to assign ownership to the user id which will be used as the TSM database instance id. • DB volumes: mklv -y crfs -v < … > mklv -y crfs -v • tsmdb01 -t jfs2 -u 1 -x 950 tsmdb 950 hdisk2 jfs2 -d tsmdb01 -p rw -a agblksize=4096 -m /tsmdb01 -A yes tsmdb04 -t jfs2 -u 1 -x 950 tsmdb 950 hdisk5 jfs2 -d tsmdb04 -p rw -a agblksize=4096 -m /tsmdb04 -A yes Active log: mklv -y tsmactlog -t jfs2 -u 1 -x 511 tsmactlog 511 hdisk6 crfs -v jfs2 -d tsmactlog -p rw -a agblksize=4096 -m /tsmactlog -A yes • Archive log: mklv -y tsmarchlog -t jfs2 -u 1 -x 831 tsmarchlog 831 hdisk7 crfs -v jfs2 -d tsmarchlog -p rw -a options=rbrw -a agblksize=4096 -m /tsmarchlog -A yes • Sequential file storage pool: mklv -y crfs -v -A yes < … > mklv -y crfs -v -A yes Document: tsmfile1 -t jfs2 -u 1 -x 3823 tsmfile 3823 jfs2 -d tsmfile1 -p rw -a options=rbrw -a agblksize=4096 -m /tsmfile1 tsmfile7 -t jfs2 -u 1 -x 3823 tsmfile 3823 jfs2 -d tsmfile7 -p rw -a options=rbrw -a agblksize=4096 -m /tsmfile7 Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 24 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture 4.3.4 TSM server processing options The following table summarizes TSM server options which are set in dsmserv.opt, as well as, server-wide options which have been set using various set commands on the TSM server. Options (dsmserv.opt) Setting Explanation activelogsize 122880 Slightly below the maximum allowed active log size. allowreorgtable default: yes Allows on-line DB2 table reorg to run allowreorgindex yes Allows on-line DB2 index reorg to run. Default: 300 Limit objects using client-side deduplication to 300GB clientdeduptxnlimit commtimeout 10000 Certain TDP’s benefit from an increased setting for this timeout. dedupdeletionthreads 8 Increases the number of threads performing deletion of deduplicated chunks which are no longer referenced. deduprequiresbackup No A secondary copy is not being created in this implementation. deduptier2filesize Default: 100 Only files smaller than 100GB are processed in tier1. deduptier3filesize Default: 400 Files in the range 100 - 400GB will be processed in tier2. Files 400GB and larger will be processed in tier3. devconf /tsminst1/DEVCONF.txt Controls which file the server device configuration information is copied to. enableautodbbackup no A full database backup is only taken once a day and they are scheduled. expinterval 0 This disables automatic inventory expiration. Instead, this task is scheduled to run at a specific time every day. idletimeout 240 Allows idle client sessions to remain through the expected backup window duration. maxsessions 700 Clients which use client-side deduplication use one additional session. numopenvolsallowed 20 Document: This option controls the number of volumes that a process such as a reclamation process or client restore sessions can hold open at the same time. A small increase to Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 25 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture this option is recommended, and some trial and error may be needed. The device class mount limit parameter was also increased because of this change. serverdeduptxnlimit 500 Limit objects using server-side deduplication to 500GB txngroupmax Default: 4096 Allows more client objects per transaction. volhist /tsminst1/VOLHIST.txt Controls which file the server volume history is written to. Set options Command Notes Activity log retention set actlogreten 60 Summary record retention set summaryreten 45 Max schedule session set maxschedsess 90 Increases the percentage of allowed sessions which can be used for scheduled backups 4.3.5 Device class creation A device class must be created for the sequential file deduplication storage pool. Of particular importance is the mountlimit parameter which needs to be set to a large value to account for all of the simultaneous volumes mounts drive by multi-session deduplicated VMware backups. • sequential file > define devclass largefile1 devt=file mountlimit=650 maxcap=102400M dir=/tsmfile1,/tsmfile2,/tsmfile3,/tsmfile4,/tsmfile5,/tsmfile6,/tsmfile7 4.3.6 Storage pool creation Below is the command used to create the deduplicated file storage pool. The options controlling reclamation, migration, and duplicate identification are set to disable the automatic launching of these tasks. Instead, the tasks are scheduled to run at specified times as detailed in a later section. • deduplicated sequential file > define stgpool vmpool largefile1 maxscratch=297 deduplicate=yes identifyprocess=0 reclaim=100 reclaimprocess=8 collocate=group nextstgpool=”” 4.3.7 Policy settings The tested architecture includes a policy domain created for VMware backups which retains 45 days worth of backups assuming only one backup is taken per day. The verexists parameter is set to a value of nolimit so that ad-hoc backups taken will no roll off an earlier backup prior to 45 days elapsing. Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 26 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture • The vmdom domain provides 45 days of backup retention and uses the deduplicated storage pool target. Although it is defined, the archive copy group is not used. > define domain VMDOM > define policy VMDOM STANDARD > define mgmtclass VMDOM STANDARD STANDARD > assign defmgmt VMDOM STANDARD STANDARD > define copygroup VMDOM STANDARD STANDARD type=backup destination=VMPOOL VERE=nolimit VERD=1 RETE=45 RETO=120 > define copygroup VMDOM STANDARD STANDARD type=archive destination=VMPOOL RETV=1000 > activate pol VMDOM STANDARD 4.3.8 Schedules and macro definitions to control server maintenance tasks The following section shows how to configure TSM to schedule data maintenance tasks to follow the best practices of separating backup ingestion from data maintenance. The recommended ordering is explained below, along with examples of commands to implement these tasks through scheduling. Here is the implemented sequence of tasks: 1. 22:00 - 07:00. Client data ingestion. 2. 07:00 - 08:00. Remove objects that have exceeded their allowed retention using the EXPIRE INVENTORY command. 3. 08:00 - 12:00. Perform server-side duplicate identification by running the IDENTIFY DUPLICATES command. This processes data that was not already deduplicated on the clients which is typically not required. 4. 14:00 - 20:00. Reclaim unused space from storage pool volumes that has been released through deduplication and inventory expiration using the RECLAIM STGPOOL command. 5. 17:00 - 18:00. Create a DR copy of the TSM database by running the BACKUP DATABASE command. Following the completion of the database backup, the DELETE VOLHISTORY command is used to remove older versions of database backups which are no longer required. At completion of the backup, the volume history and device configuration are backed up using the BACKUP VOLHISTORY and BACKUP DEVCONFIG commands. 4.3.8.1 Define scripts which run each required maintenance task The following scripts are defined to perform the data maintenance tasks, and invoked via scheduled administrative commands. def script DEDUP "/* Run identify duplicate processes */" upd script DEDUP "identify duplicates FILEPOOL numprocess=4 duration=360" line=010 set dbrecovery DBBDEVC numstreams=1 define script DBBACKUP "/* Run DB backups */" update script DBBACKUP "backup db devclass=DBBDEVC type=full wait=yes" numstreams=1 line=010 Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 27 of 28 Tivoli Storage Manager for Virtual Environments Sample Architecture update script DBBACKUP "backup volhistory" line=020 update script DBBACKUP "backup devconfig" line=030 update script DBBACKUP "delete volhistory type=dbbackup todate=today-7 totime=now" line=040 define script RECLAIM "/* Run stg pool reclamation */" update script RECLAIM "reclaim stgpool VMPOOL threshold=40 wait=yes" line=010 define script EXPIRE "/* Run expiration processes. */" update script EXPIRE "expire inventory resource=8 wait=yes" line=010 4.3.8.2 Define schedules to run the data maintenance tasks The following commands define administrative schedules which execute the scripts which were created in the previous section. define schedule DEDUP type=admin cmd="run DEDUP" active=yes \ desc="Run indentify duplicates." startdate=today starttime=08:00:00 \ duration=15 durunits=minutes period=1 perunits=day define schedule DBBACKUP type=admin cmd="run DBBACKUP" active=yes \ desc="Run database backup." startdate=today starttime=17:00:00 \ duration=15 durunits=minutes period=1 perunits=day define schedule RECLAIM type=admin cmd="run RECLAIM" active=yes \ desc="Reclaim space from storage pools." startdate=today starttime=14:00 \ duration=15 durunits=minutes period=1 perunits=day define schedule EXPIRATION type=admin cmd="run expire" active=yes \ desc="Run expiration." startdate=today starttime=07:00:00 \ duration=15 durunits=minutes period=1 perunits=day Document: Tivoli Storage Manager for Virtual Environments Sample Architecture Date: 11/16/2012 Version: 1.0 Page 28 of 28