Zfs Hardware Primer - Disruptive Storage

Transcript

ZFS Hardware Primer April 2016 Disruptive Storage Workshop John Hopkins Bloomberg School of Public Health ZFS Hardware Choices Overview With Options 2 CONFIDENTIAL | Silicon Mechanics Session Goals • Overview of ZFS Cache • Evaluate ZFS Cache Hardware Choices – – – – ARC ZIL L2ARC SLOG • ZFS hardware choices • General ZFS hardware best practices 3 CONFIDENTIAL | Silicon Mechanics ZFS Cache How does ZFS utilize cache? • ZFS uses different layers of disk cache to speed up R/W operations – ARC, L2ARC, ZIL • Adding additional cache devices can be used to increase both Read and Write performance Separate ZFS Intent Log (ZIL) ARC Main Pool Write Optimized Device (SSD) Read Optimized Device (SSD) HDD 4 CONFIDENTIAL | Silicon Mechanics Level 2 ARC (L2ARC) ARC Adaptive Replacement Cache • Exists in the RAM • MRU – Most Recently Used blocks from the file system • MFU – Most Frequently Used blocks from the file system What happens to evicted data? 5 CONFIDENTIAL | Silicon Mechanics L2ARC Level 2 Adaptive Replacement Cache • Evicted data from the ARC is sent to the L2ARC (if present) – Data is copied • Cache directory remains in the ARC • Not needed if doing archival storage • Can be added or removed to a pool at any point in time 6 CONFIDENTIAL | Silicon Mechanics L2ARC Benefits • Improve read performance • Read cache VDEV has a lower latency than the pool • More IOPS are available for writes since fewer are being used for reads Drawbacks • It is volatile - read performance can decrease in case of L2ARC failure or head node failover – Note that data is not lost 7 CONFIDENTIAL | Silicon Mechanics L2ARC Sizing and Hardware • Sizing the L2ARC • Requires ARC space to operate so it must be sized accordingly – As a rule of thumb it is about 1GB of ARC for 5GB of L2ARC • Popular hardware choices? – HGST Ultrastar MM MLC • 12Gb/s, 10 DWPD http://storagegaga.com/arc-reactor-also-caches/ 8 CONFIDENTIAL | Silicon Mechanics ZFS Intent Log (ZIL) • Similar to a journal in ext3 or ext4 • Will flush data to the zpool every 5sec (tunable) in a transactional group commit • Function is to replay a lost transaction in the event of a failure. – When the system recovers the ZIL is read and the transactions are replayed and commited to the pool • Stored on disk (unless you have a SLOG) 9 CONFIDENTIAL | Silicon Mechanics SLOG The Separate Intent Log https://pthree.org/2012/12/06/zfs-administration-part-iii-the-zfs-intent-log/ 10 CONFIDENTIAL | Silicon Mechanics SLOG The Separate Intent Log What makes a good SLOG? • Low latency • Battery backed RAM (ZeusRAM) • SSD – Be sure to overprovision Calculating SLOG usage • 1Gb connection (0.125 GB/s) • 5 second SLOG flush • 5 sec x 0.125 GB/s = 0.625 GB 11 CONFIDENTIAL | Silicon Mechanics Hard Drives SATA vs SAS Things to consider • Physical layout of drives • Expected duty cycle • Error rates on writes/reads • MTBF • Need for full duplex? • Multipath I/O • Capacity options • smartmontools 12 CONFIDENTIAL | Silicon Mechanics HBA vs RAID Controllers RAID controller • Redundant Array of Independent Disks • Hardware device that can present drives as a logical unit • Might have other features such as cache, encryption HBA • Host Bus Adapter • Provides I/O processing and physical connectivity between a host and storage • Connect large number of devices to enable large-scale storage 13 CONFIDENTIAL | Silicon Mechanics 14 CONFIDENTIAL | Silicon Mechanics General Hardware Necessary To Build A Solution Pros/Cons to different hardware selection/addition/omission Reference Architecture Framework for Guidance 15 CONFIDENTIAL | Silicon Mechanics Minimum Requirements • The minimum amount of disk space required for a storage pool is 64 MB. • The minimum disk size is 128 MB. • The minimum amount of memory needed to install a Solaris system is 768 MB. – However, for good ZFS performance, use at least one GB or more of memory. https://docs.oracle.com/cd/E18752_01/html/819-5461/gbgxg.html 16 CONFIDENTIAL | Silicon Mechanics Basic Requirements The suggested hardware requirements • ECC memory – This isn't really a requirement, but it's highly recommended • At least 8-16GB to perform well – It's perfectly possible to run with less than that (and people do). – But you'll need more if using deduplication. http://zfsonlinux.org/faq.html#BasicRequirements 17 CONFIDENTIAL | Silicon Mechanics Basic General Purpose System 18 CONFIDENTIAL | Silicon Mechanics Large Stack 19 CONFIDENTIAL | Silicon Mechanics 20 CONFIDENTIAL | Silicon Mechanics HA Stack 21 CONFIDENTIAL | Silicon Mechanics JBOD Cable Options Not Optimal 22 CONFIDENTIAL | Silicon Mechanics ZFS Best Practices Generalities ZFS defaults are generally good Turn compression on Use a mirrored SLOG Use a L2ARC Understand your IOP requirements, workload and working set size 23 CONFIDENTIAL | Silicon Mechanics Thank you for your time. Silicon Mechanics and the Silicon Mechanics logo are trademarks and/or registered trademarks of Silicon Mechanics, Inc. in the U.S. and/or other countries. Copyright © 2015 Silicon Mechanics, Inc. Silicon Mechanics and the Silicon Mechanics logo are trademarks and/or registered trademarks of Silicon Mechanics, Inc. in the U.S. and/or other countries. Copyright © 2015 Silicon Mechanics, Inc.