Transcript
IBM Elastic Storage Server (ESS) Architecture and Configuration Guide for SAP HANA Tailored Datacenter Integration Location of this document: http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102347
[email protected]
Version 1.2 – October 2015
IBM ESS Reference Architecture for SAP HANA TDI
Table of Contents Table of Contents ...................................................................................................................... 2 1
Preface ................................................................................................................................ 3
2
IBM storage system certified for SAP HANA TDI ....................................................... 3
3
IBM ESS at a glance ......................................................................................................... 4
4
IBM ESS sizing information for SAP HANA TDI ............................................................. 5
5
Required IP network......................................................................................................... 5
6
Spectrum Scale cluster concept ................................................................................. 6
7
Installing LINUX ................................................................................................................... 8
8
Setting up the Spectrum Scale clusters ..................................................................... 9
9
Installing SAP HANA.......................................................................................................... 9
10
SAP HANA TDI backup with IBM Tivoli Storage Manager for ERP ..................... 10
11
Resources.......................................................................................................................... 10
12
Change History................................................................................................................ 12
13
Trademarks ....................................................................................................................... 12
14
Disclaimers ........................................................................................................................ 13
Page - 2
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI
1 Preface This paper is intended as an architecture and configuration guide to setup IBM® Elastic Storage Server™ (ESS) and IBM Spectrum Scale™ (formerly known as GPFS) clients for SAP® HANA™ tailored datacenter integration (SAP HANA TDI). The SAP HANA TDI allows the SAP customer to use external storage to attach to the SAP HANA nodes. This document has been written for IT technical specialists and architects, with advanced skill levels on SUSE® Linux™ Enterprise Server or Red Hat® Enterprise LINUX ™ and IBM Spectrum Scale, with a focus on architecting and setting up the HANA nodes and IBM ESS. The recommendations in this guideline apply to both single node and scale out configurations for Intel™ server. As of today IBM Spectrum scale (GPFS or ESS) is not supported with SAP HANA on IBM POWER™. For more details on SAP specification for using external storage for SAP HANA TDI please read SAP document: http://www.saphana.com/docs/DOC-3633
2 IBM storage system certified for SAP HANA TDI All IBM Storage Systems are certified for SAP HANA TDI production:
IBM ESS GS2,4,6 & GL2,4,6 IBM Storwize family, model V7000, V5000, V3700, and IBM Spectrum Virtualize (IBM SAN Volume Controller / SVC) IBM FlashSystem family, model 820, 840, 900, V840, V9000 IBM XIV Gen 3 Storage System IBM DS8800 and DS8870
For a list of all IBM storage systems certified for SAP HANA production please visit: http://global.sap.com/community/ebook/2014-09-02-hana-hardware/enEN/enterprise-storage.html
Page - 3
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI
3 IBM ESS at a glance The IBM Elastic Storage Server (ESS) is a modern implementation of software defined storage built on the IBM Spectrum Scale. This technology combines the CPU and I/O capability of the IBM POWER8 architecture and matches it with 2U and 4U storage enclosures. This architecture permits the IBM Spectrum Scale Raid software capability to actively manage all RAID functionality formerly accomplished by a hardware disk controller. Newly developed RAID techniques from IBM use this CPU and I/O power to help overcome the limitations of current disk drive technology and simplify your transition to a multi-tier storage architecture employing solid state flash technology and robotic tape libraries. In addition to the technological advancements available with the Elastic Storage Server, it can also address other data issues found in many businesses. For example, as each department or division in your organization evolves their own storage needs, this can result in a costly duplication of hardware resources. The resulting islands of information may hold valuable insights that may not be accessible in such a disparate environment. By consolidating storage requirements across your organization onto the Elastic Storage Server, you can reduce inefficiency and acquisition costs while simplifying management and improving data protection. ESS is designed for performance. Storing petabytes of data is meaningless unless it can be accessed and analyzed quickly. Sustained streaming performance of data can reach 20 GB per second in each building block, growing as more blocks are added to a configuration. By combining the superior data movement capability of IBM Power Systems™ servers with the enhanced I/O subsystem introduced in the POWER8 processor as well as adding the disk management capability of the Power server driven Native RAID technology, a complete storage solution can be deployed without traditional storage controllers acting as a bottleneck to overall system performance. With support for multiple 10 Gb per second and 40 Gb per second Ethernet, as well as Infiniband speeds of up to 56Gb per second (FDR speed), Elastic Storage Servers have the architecture to deliver improved data throughput.
Page - 4
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI
4 IBM ESS sizing information for SAP HANA TDI The IBM ESS should be configured (sized) through the IBM tool SAPmagic; this tool is available to IBM Sales and IBM Business Partner on IBM TechDocs. Note: The performance sizing described in this chapter is only required for productive HANA nodes (HANA databases for the SAP production systems) and not for the non-production HANA Nodes (e.g. QA, Test, Dev, Sandbox Systems). For the non-production systems only a capacity sizing is required, DATA capacity is roughly three times the RAM, and LOG capacity is one time the RAM size of the HANA node -- please check with the latest SAP HANA installation guide. This table shows the maximum number of HANA nodes, depending on their RAM size, for the different IBM ESS models. It is recommended to use the SSD model if you choose a small scale GS device. IBM ESS models RAM per HANA NODE 1 TB 2 TB 4 TB 8 TB
GS2 SSD
GS4 SSD
8 8 4 2
16 16 8 4
GS6 SSD
GS2 HDD
GS4 HDD
GS6 HDD
GL2 HDD
16 2 4 6 4 16 1 2 3 4 8 1 1 2 4 1 maximum number of HANA nodes
GL4 HDD
GL6 HDD
8 8 4 2
16 16 8 4
5 Required IP network Figure 1 shows the principle network setup for HANA TDI with Spectrum Scale. It is required to have a separate network for the application and storage layer.
A 10G or 40G Ethernet Network to connect the SAP HANA nodes with the SAP application systems. This network will be used for management as well. A 10G, 40G Ethernet or Infiniband network connecting the HANA nodes with IBM ESS and the Backup Server, e.g. IBM Tivoli Storage Manager.
Page - 5
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI
Figure 1 Network overview for SAP HANA TDI with Spectrum Scale
6 Spectrum Scale cluster concept The IBM Spectrum Scale (formerly known as GPFS), is a cluster file system. This means that it provides concurrent access to a single file system or set of file systems from multiple nodes. These nodes can all be SAN attached or a mix of SAN and network attached. This enables high performance access to this common set of data to support a scale-out solution or provide a high availability platform. Spectrum Scale has many features beyond common data access including data replication, policy based storage management, and multi-site operations. You can create a cluster of AIX nodes, Linux nodes, Windows server nodes, or a mix of all three. It can run on virtualized instances providing common data access in environments, leverage logical partitioning, or other hypervisors. Multiple clusters can share data within a location or across wide area network (WAN) connections. Spectrum Scale provides a global namespace, shared file system access among multiple Spectrum Scale clusters, simultaneous file access from multiple nodes, high recoverability and data availability through replication, the ability to make changes while a file system is mounted, and simplified administration even in large environments.
Page - 6
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI Spectrum Scale provides storage management based on the definition and use of:
Storage pools Policies File sets
Storage pools A storage pool is a collection of disks or RAIDs with similar properties that are managed together as a group. Storage pools provide a method to partition storage within a file system. Policies Files are assigned to a storage pool based on defined policies. File placement policies are used to automatically place newly created files in a specific file system pool. File management policies are used to manage files during their lifecycle by moving them to another file system pool, moving them to near-line storage, copying them to archival storage, changing their replication status, or deleting them. Filesets Filesets provide a method for partitioning a file system and allow administrative operations at a finer granularity than the entire file system.
Figure 2 illustrates the recommended Spectrum Scale cluster concept. The IBM ESS is a cluster on its own, the Spectrum Scale storage cluster, each HANA system, or a group of HANA systems build up the Spectrum Scale application clusters. Each application cluster access the file system on the ESS (storage cluster) via Spectrum Scale cross cluster file system mount . This is a remote mount of a file system (in this figure mount point “/hana”) using NSD server access, through a virtual connection to the file system data through an NSD server, which means no direct, physical connection between the disks (NSD, network shared disk) and the application nodes.
Page - 7
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI
Figure 2 Spectrum Scale Cluster concept
7 Installing LINUX Install SUSE LINUX Enterprise Server™ 11, SUSE LINUX Enterprise Server for SAP™ 11 – Service Pack 3 or higher, or Red Hat Enterprise Linux for SAP HANA on local disks or on SAN attached storage. Follow the instructions and requirement according the following documentations: SAP Note 2001528 SAP Note 2009879 SAP Note 2013638 SAP Note 1944799
- Linux: SAP HANA Database SPS 08 revision 80 (or higher) on RHEL 6 or SLES 11 - SAP HANA Guidelines for Red Hat Enterprise Linux (RHEL) Operating System - SAP HANA DB: Recommended OS settings for RHEL 6.5 - SAP HANA Guidelines for SLES Operating System Installation
Setup ssh without prompt for password between all HANA/Spectrum Scale nodes and ESS. Ensure that proper DNS resolution is in place - via /etc/hosts and DNS. To ensure that the traffic inside the cluster of a multi-node configuration is piped through the internal HANA network in order to gain throughput, a route to each node must be set on each node.
Page - 8
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI
8 Setting up the Spectrum Scale clusters Install the IBM Spectrum Scale software; follow the instructions and requirement according the following documentations: SAP Note 1084263 - Cluster File System: Use of GPFS on Linux http://www.redbooks.ibm.com/abstracts/sg248086.html?Open: In-memory Computing with
SAP HANA on IBM eX5 and X6 Systems
http://scn.sap.com/docs/DOC-60254 - SAP HANA on RHEL on IBM xServer using GPFS
The minimum maintenance level of Spectrum Scale is 4.1 TL1 (4.1.0.5) In general, you need to download the packages required for running Spectrum Scale directly from IBM or install them from your local repository. 1. Create separate, dedicated, diskless SAP HANA Cluster Details can be found here: https://www.ibm.com/developerworks/community/wikis/home?lang=en# !/wiki/General%20Parallel%20File%20System%20%28GPFS%29/page/Linux 2. Mounting a file system owned and served by another Spectrum Scale cluster (Spectrum Scale Cross Cluster mount), details can be found here: http://www01.ibm.com/support/knowledgecenter/SSFKCN_4.1.0.4/com.ibm.cluster.g pfs.v4r104.gpfs200.doc/bl1adv_admrmsec.htm
Additional information on the same topic can be found in the Spectrum Scale Advanced Admin Guide: http://publib.boulder.ibm.com/epubs/pdf/c2370320.pdf
9 Installing SAP HANA Please follow the instruction in the SAP HANA Server Installation and Update Guide, current release: SAP HANA Platform SPS 09. No additional storage configuration is necessary. This is an example of a /hana/shared/global.ini file. The [storage] section will be empty:
Page - 9
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI [communication] listeninterface = .global [persistence] basepath_datavolumes = /hana/data/SID basepath_logvolumes = /hana/log/SID [storage] [trace]
10 SAP HANA TDI backup with IBM Tivoli Storage Manager for ERP IBM® Tivoli® Storage Manager for Enterprise Resource Planning protects your vital SAP system data. It provides automated data protection designed for SAP and SAP HANA environments. Now you can improve the availability of your SAP database servers and reduce your administration workload. Please see current product documentation how to install, configure, and run this integrated solution.
11 Resources For any product or documentation provided by SAP please contact SAP. For any product or documentation provided by SUSE please contact SUSE. For any product or documentation provided by Red Hat please contact Red Hat. For any product or documentation provided by IBM regarding SAP send an email to
[email protected].
Page - 10
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI IBM documentations: This document is ID WP102347 on www.ibm.com How to setup Spectrum Scale & ESS can be found here: http://www-01.ibm.com/support/knowledgecenter/ Select in the Table of Content Cluster software General Parallel File System GPFS storage server (ESS) Installation, configuration and usage of Tivoli Storage Manager for Enterprise Resource Planning V6.4 Data Protection for SAP HANA® http://www-01.ibm.com/support/docview.wss?uid=swg21608240
SAP documentation: Overview - SAP HANA tailored data center integration http://www.saphana.com/docs/DOC-3633
FAQ - SAP HANA tailored data center integration http://www.saphana.com/docs/DOC-3634
Introduction to High Availability for SAP HANA http://www.saphana.com/docs/DOC-2775
IBM/SAP Whitepaper "High-end customer IT Landscapes based on SAP HANA" http://www.saphana.com/docs/DOC-3211
Access to sapjam and documentation will be provided by SAP – the listed links only work after login: login first, and then access the links. SAP HANA reference architecture
https://jam4.sapjam.com/wiki/show/48495
SAP HANA Bill of Material
https://jam4.sapjam.com/wiki/show/48202
Novell / SUSE documentation: List of all SLES 11 documentations https://www.suse.com/documentation/sles11/
Highly Available NFS Storage with DRBD and Pacemaker with SUSE Linux Enterprise High Availability Extension 11 https://www.suse.com/documentation/sle_ha/singlehtml/book_sleha_techguides/book_sleha_techguides.html
Page - 11
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI Red Hat documentation: RED HAT ENTERPRISE LINUX FOR SAP HANA ON IBM: INSTALLATION GUIDE https://hcp.sap.com/content/dam/website/saphana/en_us/Technology%20Documents/SAP_HAN A_on_RHEL_on_IBM_xS erver_using_GPFS.pdf
12 Change History V1.1
Initial version
V1.2
changed support statement for HANA on POWER
13 Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: AIX®, BladeCenter®, DB2®, Global Business Services®, Global Technology Services®, GPFS™, Spectrum Scale™, IBM SmartCloud®, IBM®, Intelligent Cluster™, Passport Advantage®, POWER®, PureFlex™, RackSwitch™, Redbooks®, Redpaper™, Redbooks (logo) ®, System Storage®, System x®, System z®, Tivoli®, z/OS®. The following terms are trademarks of other companies: SAP, R/3, ABAP, BAPI, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA, the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany or an SAP affiliate company. Intel Xeon, Intel, Itanium, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Page - 12
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI Linux is a trademark of Linus Torvalds in the United States, other countries, or both. SUSE is a registered trademark of SUSE Germany and Novell USA, other countries, or both. SLES is a trademark of SUSE Germany and Novell USA, other countries, or both. Read Hat is a registered trademark of Red Hat United States and other countries. Other company, product, or service names may be trademarks or service marks of others.
14 Disclaimers This information was developed for products and services offered in Germany. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to ev aluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements Page - 13
© Copyright IBM Corporation 2015
IBM ESS Reference Architecture for SAP HANA TDI and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on developmentlevel systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.
Page - 14
© Copyright IBM Corporation 2015
