Transcript
W H I T E PA P E R
STORNEXT STORAGE PLATFORM FOR MEDIA WORKFLOWS Part 2: StorNext Storage Manager
CONTENTS Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 StorNext Platform for Data Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 The Value of Tiered Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Solving the Paradox: Keeping Data on Different Tiers Visible and Available . . . . . . . . . . . . . . . . . . . . 4 Different Media Types Provide Different Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Policies Supporting Data Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Policies for Migrating Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Truncation Removes File Content From Primary Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Managing Retrieval Order From Multiple Tiers and Copy Expiration . . . . . . . . . . . . . . . . . . . . . . . . . 7 Efficient Media File Retrieval With Partial File Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 StorNext API Enables Integration With Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Ensuring Fast and Efficient Data Movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Integrating with LTO Archives to Manage Long-Term Retention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Leveraging Public and Private Clouds as a Storage Tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Advanced Administration Functions Provide Protection and Flexibility . . . . . . . . . . . . . . . . . . . . . . . . 11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Appendix: Advanced StorNext Storage Manager Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2 WHITE PAPER | StorNext Storage Platform for Media Workflows: StorNext Storage Manager
INTRODUCTION The challenges of managing media workflows all are ultimately linked to the unprecedented increase in the creation, distribution, and storage of video content—a phenomenon that was unimaginable 20 years ago. Advances in camera and screen resolutions, and satellite and sensor technologies are creating a massive influx of data that needs to be captured, processed, and preserved. Video entertainment content providers are challenged with 4K production on one end and more viewing devices that require more formats in more places on the other end. Government agencies need shared access to video data, so they can deliver increased situational awareness. Corporations are increasing their use of video to promote products and train employees, stressing traditional IT data center infrastructure. The fact that video content is being kept longer to be repurposed in the future—combined with data growth and budget pressures—leads to a need for high-performance, scalable storage that is also cost-effective.
STORNEXT PLATFORM FOR DATA MANAGEMENT For almost two decades, Quantum’s StorNext® engineers have been developing media-specific storage technologies in lockstep with industry requirements. StorNext was the first widely adopted collaborative storage area network (SAN) environment to deliver predictable real-time performance for the massive data storage and management challenges of uncompressed video and satellite image analysis. As industry demands on storage continued to grow, StorNext evolved—providing seamless, automated data moving and archiving capabilities from high-speed primary production storage to nearline disk, tape, object storage, and cloud services. It expanded support for different platforms, and today StorNext presents data as a single managed volume to the hundreds of Windows-, Mac-, and Linux-based applications used in content creation. The latest in the technology’s evolution is StorNext 6, designed to support the demands associated with 4K/UHD, 8K, and virtual reality (VR) by taking advantage of the latest multi-core processing architectures, high-capacity system memory, and solid-state storage technologies. StorNext 6 also added multi-protocol NAS support, enabling users to connect to StorNext environments over more ubiquitous Ethernet. The Quantum StorNext platform provides data management services through two distinct, but integrated components: • StorNext Parallel File System: Provides high-speed shared data access to primary storage from SAN and local-area network (LAN) connections. For more information, read Part 1 of this white paper. • StorNext Storage Manager: Provides automated storage tiering, archiving, and data protection across a wide range of storage types. Read this white paper, Part 2, to learn about StorNext Storage Manager.
THE VALUE OF TIERED STORAGE It has long been recognized that different tasks and workflow stages benefit from using different kinds of storage technologies. In early incarnations, data workers would work on files locally using direct-attached disks, and then move the content to a removable storage device for customer delivery or keep it available for later use. A different kind of strategy is required to cope with the rapidly increasing volume of data associated with higher-definition resolutions and other large data sets, collaborative workflows, and the need to store and actively reuse digital assets in future projects. These requirements drive the need to support more kinds of storage types, to move data between them in a controlled yet automated way, and to keep the data readily available to the applications and users who need access to it.
StorNext Storage Platform for Media Workflows: StorNext Storage Manager | WHITE PAPER 3
There are three primary reasons to deploy a tiered storage approach for media workflows: • Cost reduction: As data volumes increase, the cost of keeping data on high-performance primary storage becomes untenable. The costs to keep data on high-performance disk or solid-state media are both direct (buying and maintaining suitable media) and indirect (powering systems and keeping them cool)—and it is especially wasteful for files not actively being worked on. • Scalability: RAID systems have inherent capacity limitations that the massive data volumes of new media projects can easily exceed. When RAID systems become too big, data becomes vulnerable and rebuild times after a disk failure become so long that they can negatively impact production schedules. • Data protection: Having all of the data on one storage system makes files vulnerable to loss through a variety of causes: human error, viruses, and natural disasters. Keeping duplicate copies of files on different media types and in different locations is a key element in providing protection. Quantum StorNext Storage Manager Disaster Recovery
Ingest, Edit, Distribute
STORNEXT FILE SYSTEM
Applications
SHARE Users
Nearline Disk
Object Storage
PRESERVE Tape
STORNEXT STORAGE MANAGER
Cloud
Data Protection
StorNext Storage Manager provides tiered storage management and archive using policy-based data copying and migration between different storage tiers. It also reduces costs, provides automated protection, and provides long-term retention and archiving.
SOLVING THE PARADOX: KEEPING DATA ON DIFFERENT TIERS VISIBLE AND AVAILABLE The StorNext platform uses its Storage Manager component to solve the inherent problem that is always associated with storing data on different systems and different locations: how to manage data movement and how to find the assets after they have been moved. With StorNext, a single namespace is created using the StorNext File System (see Part 1 of this white paper for details) that lets users identify and access any asset by its folder and file location, either directly through the file system or indirectly through a Media Asset Manager (MAM) that is integrated with StorNext. StorNext Storage Manager moves files to different storage tiers based on policies set by users—but no matter where the file is ultimately located, users can always find it by accessing the last folder location. With StorNext, workflows benefit from storing data on different media types at different stages; the data movement is automated; and data can be accessed directly through a single namespace regardless of where it is located.
4 WHITE PAPER | StorNext Storage Platform for Media Workflows: StorNext Storage Manager
DIFFERENT MEDIA TYPES PROVIDE DIFFERENT VALUE The decision about which kind of storage is appropriate at different points in a workflow depends on several factors, including capacity, cost, access times, and the kind of protection offered. Typical media workflows usually rely on up to five different storage tiers, all of which are supported by StorNext. • Performance disk: High-speed disk or flash arrays provide the fastest access for ingest, editing, finishing, and playing back content, and are often accessed via Fibre Channel. • Value disk: Workflow stages that function adequately with lower performance and slightly longer access times often use a tier of lower-cost, higher-capacity disk. These nearline disk tiers can use conventional RAID arrays built with less costly hard-drive technology. These arrays can be part of the StorNext File System environment or NAS appliances connected through Ethernet connections. • LTO tape libraries: Tape libraries provide a very high-capacity storage layer that offers the lowest per-gigabyte cost of any tier, as well as the advantages of portability and isolation from dangers like viruses that can affect data on spinning disk. When the tape tier is a StorNext AEL Archive, it also offers advanced features for protecting data and retaining it safely for indefinitely long periods. • Object storage: This storage technology, powered by Quantum’s Lattus™ solution, provides an online, disk-based storage tier that spreads data over many different disk resources to provide massive capacities, better protection than conventional RAID, geo-distribution options, and flexible scalability. Access times for data stored in object storage are slower than for local SANbased disk, but faster than archives using removable media. • Cloud: Storing assets in the cloud, either public or private, has become common for content that is no longer an active work-in-progress—but until now, managing the cloud as part of tiered storage has been complex and largely manual. StorNext Storage Manager, through its cloud access capabilities, now provides automated management for data in the cloud. Support includes Amazon Web Services (AWS) and Microsoft Azure, as well several other private cloud solutions.
POLICIES SUPPORTING DATA PROTECTION Since media files are generally irreplaceable assets that represent high value to content creation companies and their customers, protecting them throughout their life cycle is a critical task. That job is very complex, however, because file sizes are often too large and file content changes too rapidly for conventional backup protocols to work effectively. StorNext Storage Manager provides an alternative data protection system designed specifically to support media workflows. One common data protection tool is using StorNext to automatically create copies of assets as they are ingested and as changes occur during work. This policy is usually set to create a copy of all files as soon as they are ingested into a designated file system or project folder while keeping the original copy in its primary location. The second tier could be a value disk resource, a tape archive, object storage, or the cloud, depending on policies. Copies of files can also be automatically created periodically as work is being carried out, with StorNext keeping copies of different file versions. Versioning protects work in progress and allows users to go back to previous work stages to correct errors or make other adjustments. In one common protection scheme that also enables long-term retention, StorNext creates two copies of work in two different archive locations, often in one or more tape libraries, or in a tape library and in the cloud. One copy is retained in an accessible location where it will later become StorNext Storage Platform for Media Workflows: StorNext Storage Manager | WHITE PAPER 5
an archive version of the file when the original is truncated (removed from primary storage), while the second is moved off site to provide an additional layer of disaster-recovery protection. StorNext Storage Manager keeps track of files in all locations, even those in off-site vaults, giving administrators easy access to all copies through the file system or a MAM interface.
StorNext Storage Manager Policy Functions PROTECT CONTENT
MANAGE DISK SPACE
1. Store/Archive Copy Content to Nearline/Archive Storage
2. Truncate Remove Data Blocks from Tier 1 TIER 1
Copy 2
File Metadata
Copy 3
File Data
Copy 4
{
Copy 1
Watermarks
Truncate File Candidates (Remove File Data Blocks)
OBJECT STORAGE
TAPE LIBRARY ARCHIVE
PUBLIC CLOUD
File Metadata
{
TIER 2 DISK
Watermarks
StorNext Storage Manager automatically creates multiple copies of files to protect data. It also moves files to higher-capacity, lower-cost tiers to reduce costs and preserve assets as part of an archive and long-term retention strategy.
POLICIES FOR MIGRATING DATA StorNext Storage Manager manages content across storage tiers based on administrator-defined policies. Different policies can be applied to different assets based on a number of factors, including where they are located (which storage asset or which file system) and which jobs or projects they are associated with as defined through the user’s MAM. StorNext migration starts by creating a duplicate copy or copies of the targeted file in the secondary media, and then removing or truncating the primary copy. When the secondary copy is created and when the primary copy is truncated depends on administrative policies. It is common to create the secondary copy or copies as soon as the primary copy is created or changed. It is also common to leave the primary copy in its original location until space is needed on the primary resource. StorNext administrators can use watermarks to manage capacity, leaving primary copies in place until a specific volume level is reached on the primary storage element, and then truncating them when that level is reached. Files can be designated or prioritized for truncation using several factors—including location in a particular folder, age, and last access date. Files can also be exempted from truncation—in which case all data will always be kept in the original location.
6 WHITE PAPER | StorNext Storage Platform for Media Workflows: StorNext Storage Manager
StorNext Storage Manager Watermarks Watermark Reached
File System Fill Level
High Mark Low Mark
Time
StorNext Storage Manager uses administrator-defined watermarks to manage files on disk resources. When files are truncated, StorNext leaves an inode or stub in the file system, allowing it to be accessed by users directly through the file system or through integrated applications, including Media Asset Managers.
TRUNCATION REMOVES FILE CONTENT FROM PRIMARY STORAGE When StorNext Storage Manager truncates a file, what is left behind is either a pointer to the new location (an inode), or a portion of the original file (a stub file). Inodes take up little space, but require that the whole file, or part of it, be moved back to the original primary disk location before it can be accessed. Stub files allow a user to immediately begin accessing the file from the primary tier while the remainder of the file is being retrieved from the secondary tier, a feature that can accelerate editing or playback. The size of the file stub can be defined by the administrator, up to 2 TB, and different stub sizes can be applied by policy to different file types, directories, or projects. Recent modifications to StorNext 6 extend this offline file visibility and access to Mac users through easyto-access context menus. This new functionality eliminates the query-and-wait behavior inherent in Mac client systems where there are access pointers or stub files in archive environments.
MANAGING RETRIEVAL ORDER FROM MULTIPLE TIERS AND COPY EXPIRATION StorNext 6 provides additional control for environments with multiple secondary tiers. First, a copy expiration feature lets administrators set a retention time on a per-file, per-tier basis with automated file removal after expiration. Copy expiration is ideal for staging secondary copies on a moderate cost/performance tier (e.g., nearline disk) for a moderate period of time while keeping copies long term on more economical storage, such as LTO tape. At the same time, a selectable retrieve function allows users to dictate the order of retrieval of the range of copies in secondary tiers. This functionality ensures that the file is retrieved from the most appropriate storage tier based on business, cost, or organizational needs.
EFFICIENT MEDIA FILE RETRIEVAL WITH PARTIAL FILE RETRIEVAL When files are recovered, the entire file can be retrieved. However, StorNext also supports partial file recovery in which only relevant sections of a file, defined by time codes, are retrieved. Partial file recovery gives users the advantages of tiered storage while minimizing the use of higherperformance, more expensive storage tiers and reducing network traffic. When data is copied or moved, administrators have the option of applying a data-integrity check that automatically verifies that the new file matches the original.
StorNext Storage Platform for Media Workflows: StorNext Storage Manager | WHITE PAPER 7
STORNEXT API ENABLES INTEGRATION WITH APPLICATIONS Coordination between the StorNext platform and media applications, including MAMs, is enabled by the StorNext API (SNAPI), a well-defined interface that allows StorNext to work with thirdparty applications. This coordination allows StorNext activities—including copying, migration, and truncation—to be carried out based on project-specific information that is held in the MAM. It also provides a mechanism for giving users information about the status and location of assets. For example, it can let editors know through their MAM or application that a particular file is now located in the cloud or in a tape library. SNAPI has also been used by media production and finishing companies to create their own proprietary MAMs and integrate them with StorNext policies.
ENSURING FAST AND EFFICIENT DATA MOVEMENT In an active media workflow environment, managing the movement of data for maximum speed and efficiency is an important consideration for overall productivity. StorNext provides several features to ensure maximum effectiveness. One is allowing administrators to define resources that will be used to move data. For example, data can be written to or retrieved from a tape library using multiple high-speed drives in parallel to make transfers as fast as possible. Distributed Data Movers (DDMs) are another important StorNext feature. DDMs use dedicated processor cores or nodes in a load-balanced configuration to carry out multi-stream movement of data between storage tiers. DDMs keep file transfer performance high without slowing down other ingest, editing, or finishing operations. As demands on a system change, faster transfer performance can be enabled by adding additional DDMs.
Distributed Data Movers (DDM) STORNEXT FILE SYSTEM
STORAGE/ARCHIVE TIERS
PRIMARY TIER
RETRIEVE
Relocate
RETRIEVE
RETRIEVE
COMMAND/CONTROL
RETRIEVE SECONDARY TIER
MDC
STORE
DDM Server
STORE
DDM Server
STORE
DDM Server
STORE
DISK
OBJECT STORAGE
TAPE
CLOUD
StorNext Storage Manager automatically creates multiple copies of files to protect data. It also moves files to higher-capacity, lower-cost tiers to reduce costs and preserve assets as part of an archive and long-term retention strategy.
8 WHITE PAPER | StorNext Storage Platform for Media Workflows: StorNext Storage Manager
INTEGRATING WITH LTO ARCHIVES TO MANAGE LONG-TERM RETENTION One of the important challenges facing IT departments that manage media assets is preserving valuable media-based content for future use and re-monetization. StorNext Storage Manager is built into StorNext AEL Archive libraries to provide advanced tools to preserve archived data safely, securely, and inexpensively for extended periods of time while still providing easy access. StorNext AEL Archives use the latest LTO technology for performance, capacity, and reliability. They are available in a variety of sizes to match the capacity needs of different organizations. The largest library models hold petabytes of data and provide cost-effective growth though capacityon-demand scalability. Extended Data Life Management (EDLM). StorNext AEL Archives offer an automated process for monitoring the status of data stored on removable media. Periodically the library loads media into dedicated drives that scan the tapes; count reads, writes, and mounts for each cartridge; detect the presence of recoverable errors; and issue reports to administrators. When usage levels or error rates reach specific thresholds, files are rewritten to new media, even upgrading to a new LTO level if appropriate. StorNext Storage Manager integrates with EDLM functions to control the monitoring and migration of data to automate data movement, preserve data for decades or longer, and ensure that all applications connected to the StorNext platform know the location of the data. Extended Data Life Management (EDLM) 1. System scans tapes based on user-defined policies. 2. When degraded tape conditions are identified, Scalar® communicates with StorNext to move files to a new tape.
3. StorNext moves files to a new tape, and updates the metadata.
ACTIVE TAPE ARCHIVE
StorNext Storage Platform for Media Workflows: StorNext Storage Manager | WHITE PAPER 9
Active Vault: Users often keep multiple copies of inactive files, vaulting some off site and others on site where they remain under the direct control of administrators and can be recovered more quickly. With its Active Vault feature, LTO-based StorNext AEL Archives offer users a safer and more secure alternative to on-site vaulting. Active Vault provides low-cost offline cartridge slots inside an AEL archive where tapes can be held safely offline, but remain secure inside the library. StorNext Storage Manager is aware of tapes and content in the Active Vault, and cartridges stored there can be scanned and monitored using the EDLM feature. Tape consolidation: As data is written to tape cartridges, deleted, and moved, tapes become fragmented and their capacity is reduced. StorNext Storage Manager includes an archiving feature that automatically consolidates tape volumes to reclaim space that is no longer being used to store tracked data. It tracks the content’s new location and updates the metadata, allowing applications to access the files through the normal interface. Alternate file formats: For writing data to tape with StorNext Storage Manager, users can choose either the StorNext File System format (ANTF)—used and recognized by any StorNext-based system or application—or the Linear Tape File System (LTFS). LTFS is an open file format that allows any user or application to access files and data stored on tape directly, as if it were written on a hard drive. LTFS makes it easy to share data with users and applications outside the StorNext platform. StorNext Storage Manager supports making multiple copies of the same files in different formats for maximum flexibility.
LTFS Tape Format Support
StorNext Storage Manager can write data to tape archives in StorNext format (ANTF) and the LTO tape file system (LTFS). LTFS is an open system file format that lets users access material on tape through a file system interface as if it were on disk.
LEVERAGING PUBLIC AND PRIVATE CLOUDS AS A STORAGE TIER Large-scale storage in the cloud has become an increasingly important feature for a large number of organizations that store, preserve, and reuse very large data sets. Almost all of the cloud storage implementations leverage the concept of object storage, a methodology designed not only to keep data on disk, but to manage much higher volumes than RAID storage and at much higher levels of data integrity. RAID protection is designed to protect against failures on a specific disk or disks. The size of the storage pool it can support is limited, and performance becomes worse as the size of disks in the system increase. When individual disks store multiple terabytes, rebuild times often take days, during which time data is more vulnerable and system performance may be slowed substantially. Object storage provides an alternative to RAID with higher levels of reliability and essentially unlimited scale. Object storage uses erasure coding, a forward error correction methodology, first developed more than 50 years ago. It was used by NASA and other organizations that needed to 10 WHITE PAPER | StorNext Storage Platform for Media Workflows: StorNext Storage Manager
ensure the highest levels of data integrity in situations where errors were very likely to occur. Erasure coding breaks data into many discrete units, includes parity information with each, and then spreads the data, with redundancy, over many different storage nodes. If errors occur, or if the disk that data is stored on fails, the data is automatically recreated from the other locations. In erasure code-based systems, disks can be replaced without requiring rebuilds; there is virtually no downtime when disks fail; and the integrity of the data is much higher than with conventional RAID. Using Lattus Object Storage as a StorNext Tier StorNext Storage Manager includes direct support for Quantum’s Lattus object storage, allowing a private cloud Lattus tier to be defined by administrators through StorNext policies. Lattus distributes data across multiple storage nodes automatically, and it supports geographical spreading where redundancy includes multiple different physical locations. Unlike many other object storage systems, Lattus provides immediately consistent data, even for files spread over multiple locations. That means that all reads, no matter when they are performed, provide correct data at all times. Lattus offers unlimited scaling. New nodes can be added—or failed nodes replaced—transparently, without any negative impact on system performance. Lattus allows newer drive technology to be added to the storage pool and directly incorporated into the existing pool—giving administrators a cost-effective, non-disruptive way to take advantage of technology advances and preserve data for extended periods of time. Supporting Third-Party Clouds With FlexTier FlexTier™ is a StorNext feature that enables customers to include destinations that are not provided or maintained by Quantum as StorNext Storage Manager–supported tiers. These destinations include customer-owned cloud storage accounts and third-party object storage systems. FlexTier optimizes performance for all supported providers by transferring data using multiple threads and dividing uploads into multiple parts to take maximum advantage of the available uplink and to move content as quickly as possible. • FlexTier for public cloud: Provides support for customers who maintain public cloud accounts through cloud service providers, including Amazon and Microsoft, allowing them to automatically manage content in and out of those accounts through StorNext. Compatible with Xcellis™ Workflow Director and Artico™ archive appliances. • FlexTier for private cloud: Provides automated tiering of data through StorNext for customers who use a thirdparty object storage system to create their own private cloud. As of June 2017, FlexTier for private cloud supports NetApp Webscale, IBM Cleversafe, and Scality RING as archive targets. Compatible with Xcellis Workflow Director.
ADVANCED ADMINISTRATION FUNCTIONS PROVIDE PROTECTION AND FLEXIBILITY Although StorNext is designed to create a highly automated managed storage environment, the administrative interface also provides tools to allow manual control of many operations. Truncation, normally an automated process carried out according to policies, can be initiated manually for any file using the administrator’s GUI, and specific files may be designated to be exempt from truncation through the same interface. Administrators can also manually control where truncated files are restored. Usually, they are restored to their original location, but the administrator can manually restore an asset to a different location—an important feature for keeping data available after a system failure or damage to equipment or a site. From the StorNext GUI, administrators can manually recover files or directories that have been deleted, as long as the space where the files are located has not yet been reclaimed. Large-scale recovery after a major outage is simplified in a StorNext environment because files that have been copied or truncated can be made available immediately, without having to perform a system-wide restore operations.
StorNext Storage Platform for Media Workflows: StorNext Storage Manager | WHITE PAPER 11
SUMMARY Media workflows have always been among the most demanding applications for storage. As new advances in resolution, color depth, and frame rates have emerged, they have placed even more demands on performance and scalability. The StorNext File System meets performance demands by enabling teams to directly access the files simultaneously at high speed from clients running multiple operating systems. Its companion for data management, StorNext Storage Manager, solves the problem of how to store, protect, and repurpose very large content pools for extended periods by automatically moving data between different storage tiers based on policies. The results are massive scalability, increased data protection, and low total ownership costs. StorNext Storage Manager supports a variety of different storage tiers: performance disk, value disk, tape archives, object storage, and the cloud. It allows users to direct the storage of assets on different storage tiers at different points in the life cycle of the data to meet varying performance, access time, and cost requirements. With StorNext Storage Manager, access to all assets in all locations and tiers is provided transparently through a single namespace—the last location of the file in the StorNext File System. StorNext Storage Manager provides maximum flexibility and power. It supports a wide range of access time requirements through the use of stub files and partial file retrieval. StorNext Storage Manager supports the creation of multiple copies on different media types for automated protection, and it can build effectively unlimited archives on tape or in the cloud. StorNext Storage Manager and its storage tiers feature advanced data integrity capabilities to ensure the safety and security of data during extended retention, to allow newer technologies to be easily incorporated in the future, and to provide flexible management options to give users maximum control. With the StorNext File System, Storage Manager provides an essential technology for the storage, protection, and reuse of even the largest and most critical sets of assets.
APPENDIX: ADVANCED STORNEXT STORAGE MANAGER FEATURES Migration Policy Definition Features • Assigning policy to folders: When you create a policy in StorNext, you assign the policy to one or more directories to control the protection and migration of the data in each directory. • Scheduling a policy: There are many cases when moving or copying files during work hours may not be ideal. StorNext can be scheduled to process files outside of business hours, allowing users to work on data without disruption during their workday. • Store exclusions: When defining policies, many users have files that they do not want to protect or they always want to keep online. This feature allows you to define the files StorNext should never store or truncate. Migration Process Optimizations • Advanced storing for small files through batch store: Storing small files to tape over time can cause “tape thrashing.” In some cases, a tape may be mounted to store a small file, and then dismounted only to try to store another small file later. By defining a threshold for the size of the small files or a time window, StorNext optimizes storing small files to tape sequentially, avoiding this issue. • Queue management: Under heavy load when many retrievals might be queued, it may be desirable to make sure certain assets are available sooner than others. StorNext allows you to prioritize the retrieval of the most important assets. Secondary Storage Location Features • Assigning buckets and media to policies: StorNext allows you to assign media to a policy. This can be a group of LTO tapes or even specific object storage buckets, or in the case of SDISK, particular NAS shares. This allows you to control what files are stored on what media.
12 WHITE PAPER | StorNext Storage Platform for Media Workflows: StorNext Storage Manager
• Steering: Based on policies, StorNext allows you to create multiple copies of the files to different targets as well as within a copy by file size to move data to specific media by size. For example, all small files would be steered to the cloud, and all large files of copy 1 would be moved to tape. • Drive pools for two libraries with option of dedicated reads/writes: StorNext allows you to define drive pools for LTO media to control where data flows and where to access data. You can define drive pools to protect data by creating copy 1 in the first tape library and copy 2 in the second library. This feature also allows tape drives to be defined to just read or write, ensuring that you can both store new data and access truncated data. • Alternate store location (alternate retrieve location): You can define another StorNext instance to store files as a copy (ASL). This feature allows any files in site A to be protected in site B. It does not propagate deletes or renames. When defining multiple copies, if StorNext systems are mirrored by ASL or other methods, StorNext can also retrieve files from another StorNext location if desired. • Disk-to-disk relocation: A StorNext File System can be created with multiple tiers of disk targets, including flash, hybrid, 10K, 7.2K, and 5.4K drives. These can be defined in two distinctive stripe groups within the file system. Then you can create a policy to first protect the copy to one archive target, move data between the two tiers after that, and then finally truncate the data. • Auto vaulting: StorNext can automate vaulting of media based on the copy of the media, fill rate of the media, or time. This works extremely well with Active Vault and eliminates administration by moving media to the vault based on time, fill rate, and the specified copy of the media. Data Protection Features • Point-in-time recovery: With StorNext Storage Manager versioning enabled, you can restore an entire directory to its state at a previous point in time, provided the version count was not exceeded during this time. • Cleaning old files (default time for files): When files are changed or deleted, StorNext keeps the older versions as well as the deleted data by default if it was stored on the target media. This allows for the recovery of deleted files or earlier versions of the files. StorNext will keep these files for seven years unless otherwise specified. • Retrieve to another place: Users may want to retrieve data from another location and not overwrite the files in the original location. StorNext can retrieve or recover deleted files to an alternate location. • Batch retrieves: StorNext is optimized to retrieve files in sequential order or all from the same tape. • Remove data protection copy on delete: In some cases, it may be preferable to remove all copies and versions of files when removing the original data. You can define policies to delete files on the target media (LTO, cloud, and SDISK) immediately when the file is deleted—saving space and ensuring files cannot be accessed after files are deleted. Truncation Features • Truncate exclusions: You can use this feature to define file types that should never be truncated. • Disable truncation: Completely disables truncation on files that are stored to a tier. • Truncate immediately: StorNext allows for immediate truncation of data to protect the data without taking up the primary disk space. • Emergency truncation: In cases where the StorNext File System is full, StorNext will automatically attempt to keep space free for new files by initiating emergency truncation of the oldest files. • Stub files: StorNext allows you to retain between 1 KB and 2 GB of a file when it is truncated. This allows indexing systems to read the first part of the file, and applications or users to access the stub while StorNext retrieves the remainder of the file. Data Integrity Features • Checksum (integrity and cloud): Checksum data integrity is available for file store and retrieve. For cloud targets, the checksum is validated when both storing and retrieving files. For tape and SDISK targets, the file checksum is compared on retrieval.
StorNext Storage Platform for Media Workflows: StorNext Storage Manager | WHITE PAPER 13
ABOUT QUANTUM Quantum is a leading expert in scale-out tiered storage, archive, and data protection. The company’s StorNext® platform powers modern high-performance workflows, enabling seamless, real-time collaboration and keeping content readily accessible for future use and re-monetization. More than 100,000 customers have trusted Quantum to address their most demanding content workflow needs, including top studios, major broadcasters and cutting-edge content creators. With Quantum, customers have the end-to-end storage platform they need to manage assets from ingest through finishing and into delivery and long-term preservation. See how at www.stornext.com.
www.quantum.com • 1-800-677-6268 ©2017 Quantum Corporation. All rights reserved.
WP00232A-v01 June 2017 StorNext Storage Platform for Media Workflows: StorNext Storage Manager | WHITE PAPER 14
