Gallery Sienna And Dell Emc Isilon Clustered Storage

Transcript

GALLERY SIENNA AND DELL EMC ISILON CLUSTERED STORAGE Clustered storage for automated media workflow ABSTRACT This white paper explains how Dell EMC Isilon can help eliminate silos of storage that introduce bottlenecks and interruptions into the digital media workflow. As both the sheer volume and strategic business value of this content continues to increase, broadcast organizations are looking for ways to streamline the end-to-end process of ingesting, editing and bringing content to air while maintaining the highest level of program quality. January 2017 WHITE PAPER The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any software described in this publication requires an applicable software license. Copyright © 2016 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners. Published in the USA. 01/17 White Paper H12270 Dell EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice. 2 TABLE OF CONTENTS ABSTRACT ................................................................................................................................1 EXECUTIVE SUMMARY ...........................................................................................................4 WORKFLOW OVERVIEW .........................................................................................................4 INGEST – SIENNA PICTUREREADY AND INGEST CONTROL .............................................5 MEDIA ASSET MANAGEMENT – SIENNA MEDIA SERVICES ..............................................6 ENPS MOS INTEGRATION – SIENNA ORIGINONE ...............................................................7 FINAL CUT PRO INTEGRATION – SIENNA STORYCUT .......................................................8 PLAYOUT AUTOMATION LIVE ASSIST – SIENNA AUTOMATIONX ....................................9 PLAY TO AIR – SIENNA VIRTUAL VTR ............................................................................... 10 SUMMARY .............................................................................................................................. 11 APPENDIX: SAMPLE DEPLOYMENTS ................................................................................ 12 Test Configurations ......................................................................................................................... 12 Test Notes ....................................................................................................................................... 12 Test Conclusion............................................................................................................................... 12 ABOUT DELL EMC ISILON SYSTEMS ................................................................................. 13 3 EXECUTIVE SUMMARY The broadcast industry is facing an unprecedented influx of digital content driven by the incredible surge in sales of high-definition flat screen displays, and the increasing ubiquity of high-speed Internet access across TVs, PCs, and mobile devices. Not only are broadcasters managing significantly larger digital files but in an age of breaking news and high-definition programming, production teams cannot afford to wait for production servers to ingest data before editing raw footage. Today, broadcast programming is being created, managed, delivered and archived digitally, end-to-end, using silos of storage that introduce bottlenecks and interruptions into the workflow. As both the sheer volume and strategic business value of this content continues to increase, broadcast organizations are looking for ways to streamline the end-to-end process of ingesting, editing, and bringing content to air while maintaining the highest level of program quality. Isilon has joined with Gallery to build a collaborated solution using the Sienna software suite, integrated with ENPS News System and Final Cut Pro editing software, to deliver a single media broadcast workflow that is cross-platform, highly scalable, reliable, and easy to manage. In this solution, as content is being captured directly into the Scale-out NAS storage cluster, it is immediately streamlined into the broadcast workflow using Sienna software, allowing producers and journalists to associate media assets with story scripts as new story run-downs. Subsequently, Final Cut Pro editors can start editing the run-downs on their stations and send those back as finished run-downs to the playout automation system—all while new content is still being captured. During the NAB 2008 conference this end-toend solution was presented as a live demonstration using multiple ingest, edit, and playout streams of DVCProHD content stored on an Isilon 3000 4-node cluster. Workflow Overview A typical Isilon/Sienna broadcast workflow is comprised of multiple modules and services tightly integrated within a gigabit Ethernet network, accessing data directly off a single pool of Isilon storage. All Sienna software modules are installed on Apple Xserve hardware equipped with AJA KONA cards for AV I/O processing. The workflow begins with the Sienna IngestControl application which controls multiple Sienna PictureReady ingest servers to record live content to QuickTime compatible formats. Recorded content is stored directly on the Isilon cluster and promptly made available throughout the production network. During ingest PictureReady generates video proxies that can be searched and reviewed by other users on the networks such as the ENPS clients and the Final Cut Pro editors. The ENPS users can link news stories with the QuickTime movie through the OriginOne MOS gateway while Final Cut Pro editors can edit the video in place directly off the Isilon cluster. Once the story is complete it is delivered through the OriginOne gateway back to the ENPS system, where the list of run-downs is automatically updated, and to the Sienna AutomationX server from which playout automation controls multiple Sienna VirtualVTR playout servers. The complete program is then played out directly off the Isilon cluster. 4 Figure 1. Isilon/Sienna Deployment Ingest – Sienna PictureReady and Ingest Control Ingest is often the most taxing and sensitive part of a broadcast workflow. Content is either recorded live from satellite or other live feeds during which any recording interruption may result in loss of the entire content or poor quality content. Recording content using Sienna PictureReady directly to the Isilon cluster provides the following features and benefits: • • • • • • PictureReady can record in SD/HD QuickTime® compatible formats—from either baseband or tapeless feeds—leveraging the potential of QuickTime as a real-time record and edit architecture. PictureReady employs a large buffer ensuring that any network interruptions do not cause recording errors or interruptions such as dropped frames. Sienna IngestControl application allows the user to manage and schedule up to 15 simultaneous PictureReady ingest servers. Broadcast managers do not need staff on site to record feeds during off hours. As additional bandwidth is required, more PictureReady ingest servers and more Isilon nodes can be added to meet the performance demand. Isilon SmartConnect zones can be used to create access zones for ingest only activity to ensure the right amount of bandwidth is available exclusively for ingest. While content is being ingested, proxies are created to review the recorded content. Proxy content trails behind the recorded content in as little as 10 frames. While content is being recorded the rest of the workflow can continue practically in real-time. 5 Multiple IngestControl stations can reside on the same network to scale the media infrastructure, while all content is recorded directly onto a single Isilon cluster by configuring the appropriate amount of nodes to sustain the aggregate bandwidth and storage requirements. Figure 2. Picture Ready Ingest Control Media Asset Management – Sienna Media Services Once initial content is recorded, it is immediately registered as media assets in the Sienna MediaServices content management system. Sienna MediaServices provides web-based access to search, review, log, and archive recorded content. MediaServices utilizes the Apache web service on the Isilon nodes to serve the content. Immediately upon start of lower-resolution ingest proxy of the content can be reviewed using the following methods: • • • Using any standard browser pointed to the MediaServices URL; An embedded browser within the StoryCut plugin for Final Cut Pro; An embedded browser within ENPS client application As a media asset is being reviewed it can be marked with in/out tags and additional meta-data. Assets can be archived and restored from other tiers of storage, or distributed across the network to local or remote distribution points using customized droplets. The tight integration of Media Services with Isilon NAS provides the following features and benefits: • • • • • Both low-resolution proxy and high-resolution media assets are referenced and served from a single pool of storage. Content is accessible across multiple platforms using any standard web browser. As more nodes are added for performance/capacity more Apache web services are available for accessing the content. SmartConnect zones may be created to provide web access failover, limit content management activities to specific nodes and isolate browsing traffic from other performance sensitive activities. MediaServices is a critical component for streamlining the process of review, mark and edit content, providing news story producers and editors a tightly integrated communication channel within their specific environments. Multiple Isilon clusters can be deployed across media distribution points. With a MOS Redirection compatible New System, such as ENPS, Sienna MediaServices will automatically redirect content from one storage cluster to another, on demand, to serve different stations based on proximity to the distribution point. 6 Figure 3. Web Based Media Search and Review ENPS MOS Integration – Sienna OriginOne While content is being recorded the process of news production takes place. ENPS from the Associated Press is the world leading software for news production management and programming, controlling over half the market share. Using industry standard MOS protocol, Sienna provides a MOS gateway through the Sienna OriginOne server to mirror the MOS information in the News System. Sienna delivers MOS media objects from other OriginOne clients such as the PictureReady, StoryCut, and VirtualVTR. OriginOne provides the following features and benefits: • • • • Embed QuickTime proxy links within MOS objects that are carried across to all other clients. Auto-creation of MOS object within the ENPS run-down entries to prepare “dummy” objects early in the process and share with other news collaborators such is incoming video feeds or delivery points for Final Cut Pro editors using StoryCut. Single point of configuration—such as video file source and proxy location, destination delivery points and delivered proxy format, all packaged within the MOS object and delivered to all news collaborators. At any point changes to these settings can be done centrally and automatically distributed to all the OriginOne clients. The OriginOne database is used for the MediaServices QuickTime asset management service. All high-resolution and proxy media assets are stored on the Isilon cluster in one single shared pool of storage. 7 Figure 4. ENPS MOS integration with Sienna OriginOne Final Cut Pro Integration – Sienna StoryCut In traditional environments, editors have to wait until the recording of content is complete and copies are made for editing on the local edit stations or dedicate SAN. Existing automation techniques would enhance the process by creating chunks of the recorded stream and deliver copies to editors. This process still involved multiple copies of the content, delivered to different pools of storage. The IsilonSienna solution streamlines this process by introducing a single media workflow extending from the ingest servers and news system to the craft editing station using the Sienna StoryCut plug-in. StoryCut adds MOS functionality along with seamless Sienna QuickTime media integration providing the following features and benefits: • • • • • • Video content can be dragged directly onto the editing timeline, preserving all previously marked log entries and meta-data. The editor can craft the high-resolution video in place directly off the Isilon cluster that is used as scratch disk space. Using QuickTime, edit changes are contained within a separate proxy QuickTime file generated upon delivery back to the OriginOne and the configured delivery points. As the editor is working on the content the MOS news text is available for reference, allowing the editor to synchronize the video sequence with the text based on individual speech rates. StoryCut automatically updates the editor when the story is changed in the New System, or when new content is recorded from PictureReady, or any media asset changes are done through MediaServices. StoryCut presents a simple dialog for delivering the package back to OriginOne. If a MOS placeholder was previously defined, a proxy will be generated to populate that placeholder in the New System run-down. StoryCut automatically generates media proxies for each of the delivery points defined in OriginOne, avoiding the need for the user to know anything about the file storage location, delivery points or format. 8 Figure 5. StoryCut Integration with Final Cut Pro Once delivered from StoryCut to Sienna, OriginOne automatically updates the ENPS MOS auto-create object with the proxy created by the editor and the News System run-down is available for final approval by the news system client. Figure 6. ENPS auto-object and run-down updates Playout Automation Live Assist – Sienna AutomationX Once a news story or a show is ready for air, it is automatically delivered to AutomationX server through OriginOne to be loaded into any one of the VirtualVTR play out servers available on the IP network. Sienna AutomationX provides a scalable scheduling and dispatching service to assist in locating MOS media assets and playing them to air. AutomationX provides the following features and benefits: • • • • • • Automatically detect available playout channels using Bonjour Apple protocol Preload media assets based on running order and channel availability Flexible playout control using Pause, Chain, Loop or Scheduled modes MOS News System events can be grouped into contextual stories Status feedback from switchers and routers trigger external events Multiple playout systems can exist on the same network all delivering media assets off the same Isilon storage. The storage cluster can scale performance and capacity to meet the playout storage and bandwidth requirements. SmartConnect zones can be created to exclusively provide playout bandwidth requirements. 9 Figure 7. ENPS and AutomationX live assist Play to Air – Sienna Virtual VTR Sienna VirtualVTR is a robust and scalable play to air service. VVTR support playout of all SD/HD QuickTime compatible formats. VVTR can be controlled by many standard controllers and APIs, all potentially active simultaneously: • • • • Sony P2 protocol with Odetics Extensions TCP/IP using Sienna AutomationX or VVTR Remote Control application MMC, USB Keyboard or Jog Shuttle controller GPIs VirtualVTR provides the following feature and benefits: • • • • Pairs of VVTRs can be couple in buddy configuration for redundancy. The secondary VVTR will continuously chase and monitor the primary VVTR and will take over in complete synchronization should the primary VVTR fail. VVTR supports long-GOP formats such as HDV and XDCam HD. VVTR can also be used for Sienna Media Objects, review and mark clips, and deliver back to OriginOne with additional meta-data. All content is delivered directly off the Isilon cluster. A separate SmartConnect zone can be created to guarantee the appropriate bandwidth is available for playout. 10 Figure 8. Play to Air control Summary The integration between Sienna software suite and Dell EMC Isilon is seamless and straightforward. By leveraging both products’ scalable architecture, a robust and flexible media workflow can be designed to meet a wide variety of performance and feature requirements. Isilon storage provides a single pool of storage for the entire Sienna media workflow, allowing all users from program producers to Final Cut Pro editors to interact in real-time and deliver the highest level of quality programming in record time. This solution highlights the Isilon clustered storage unique value in managing unstructured data in an environment of multiple media formats and distribution channels. 11 Appendix: Sample Deployments Our tests covered a range of ingest/edit/playout workflows using different load and video formats. Sienna PictureReady module from the Sienna software suite was mostly used for ingest testing while some ingests tests were conducted directly off of Final Cut Pro. Final Cut Pro was used for editing, and Sienna VirtualVTR software module for playout. All hardware was Apple Xserve and Mac Pro. Test Configurations The tests were conducted at 2 separate locations: 1. 1. The Isilon UK representatives joined representatives from Gallery the maker of Sienna software suite and Root6 integrator. They tested with six dual-socket/quad-core Mac Pro systems and 12 quad-core XServers, all running Leopard (Specifically, 10.5.2), all with Kona capture cards. Cluster was 4 x IQ1920X running 4.7.4.22. All hosts and cluster on the network were configured to run over NFS with jumbo frames over gigabit Ethernet connected to Cisco 3750 switch. 2. 2. Switch was Cisco 3750. Isilon US engineers visited Tekserve in NYC, a large Apple reseller for video/audio production houses and broadcast networks (ABC, NBC, Mojo, etc). They tested with six dual-socket Intel Duo Xserve systems with 4GB RAM, Kona LHe cards, running Leopard 10.5.1. They used XDCAM HD tape deck for ingest source. Cluster was a 5 x IQ3000x running OneFS 4.7.5.10. All hosts and cluster on the network were configured to run over NFS with jumbo frames over gigabit Ethernet connected to Cisco 3750 switch. Test Notes • • • • Designing the storage system is just like doing any spec for bandwidth, apart from the fact that you want to leave some room on the CPU/bandwidth limits leaving about 25% of room for CPU load. For ingest, Gallery’s PictureReady showed far superior results compared to the native FCP ingest feature, and from our point of view, it buffers an extremely large number of frames (over 400), so it’s a lot more forgiving to storage network delay. FCP doesn’t forgive at all—one frame drop and it’s over. The integrators at both sites independently, who are XSAN super-gurus, all said that we were performing much better, and with higher scalability, than XSAN. XSAN is notorious for dropping frames as soon as one of your colleagues suddenly decides to start doing a render or starts scrubbing the timeline. One of the founders of Gallery was with us, and exclaimed that he’d never seen so many simultaneous streams of ProRes being driven simultaneously on any single storage product. The results are summarized in the table below: ENCODER # STREAMS PER NODE ACTION DV25 4-5 Ingest DV50 8 Ingest DVCPRO100 6 Playback DVCPRO100 4-5 Ingest DVCPR100 3 Edit 3 Ingest 2 Ingest/ Playback ProRes SD (160 Mbit) ProRes HD (220 Mbit) COMMENTS Cluster experienced only light load and could handle over 3-4x this number of streams per node with no dropped frames. 8 DV50 ingest streams to one node using FCP, with simultaneously 10 other XServes pounding the other three nodes with multiple DV50 streams from PictureReady, both read and write. The Isilon had plenty of breathing room CPU was 50% idle on average. This is a 6-layer composite from one Isilon node to one Mac Pro, each layer being streamed in at DVCPRO100 (100 Mbit + audio, we observed about 120 Mbit) validating HD workflows. Ran 18 streams of DVCPRO100 against 4 nodes. Isilon had plenty of breathing room. Edited small frame sequences while ingest was still coming in. Were able to edit clips as short as 2-5 frames long with no dropped frames. Similarly we were able to run 2 ingest and playback stream of HD per node. Test Conclusion Isilon X-series displayed a high-level of performance for all but the uncompressed HD formats, which we are targeting for our next generation platform. This round of testing strengthened our and our partners’ confidence that the X-series offers a great storage solution to Final Cut Pro centric video ingest, editing and playback workflows. 12 About Dell EMC Isilon Systems Dell EMC Isilon's product and market leadership in scale-out storage enables enterprises worldwide to manage massive amounts of file-based data, regardless of how fast or how unpredictably that data grows. Whether it's customers across vertical industries or large enterprises trying to manage rapidly growing home directories and content archives, Isilon's unique and powerful simplicity at scale is proving to be the industry's preferred solution for scale-out NAS storage. 13