Transcript
Caution: The Sonexion rack has multiple input power connectors. Disconnect all supply power for complete isolation prior to installation.
3000 Quick Start Guide © 2016 Cray Inc. All Rights Reserved. This manual or parts thereof may not be reproduced in any form unless permitted by contract or by written permission of Cray Inc. The information in this document is subject to change without notice. While every effort has been made to ensure that all information in this document is accurate, Cray accepts no liability for any errors that may arise. No part of this document may be transmitted or copied in any form, or by any means, for any purpose, without the written permission of Cray.
Caution: You are responsible for ensuring that the floor will support the static and dynamic load rating of the rack. This is especially important for installations that involve raised flooring. Caution: The rack side panels can potentially be a hazard when they become unlatched. It is recommended that the side panels remain locked during normal operations.
Important: You must perform a ground (earth) continuity and insulation test after completing all assemblies and prior to powering on any system.
2 Preparation and Rack Setup
1.1 Safe Handling
2.1 Facilities
Caution: Given the weight and size of the Sonexion rack, it is possible for the rack to topple over while it is being moved. Do not tip the rack more than 10 degrees from a level surface or when rolling down an incline or ramp.
• (Qty 1) 13mm combination wrench and (Qty 2) 19mm combination wrenches • T-10 and T-20 Torx screwdrivers • 61mm/24 inch spirit level • Pozidriv (or Phillips) screwdriver
1.2 Precautions The following safety requirements must be considered:
Complete this prescribed user checklist prior to each use of the ramp (See Section 2.3). CHECK LIST • Visually inspect both the ramp and the pallet brackets. • Visually inspect the ramp sections for any damage, particularly to the folded edge, which acts as a barrier to stop the castors falling off the side of the ramp. • Inspect for physical tightness of the bolts holding the sections of the ramp together. • Inspect that the ramp support bridge is present and correctly installed. See safety notices in section 2.3. • Visually inspect the support bridge for welds/damage. • The area near the pallet MUST BE clear of any trip hazards, people, or obstacles while lowering the rack. • If any inspection fails, do not continue. Contact your service representative.
Caution: After removing the packaging and after moving the Sonexion rack from the crate pallet, the bottom two SSUs must be fully populated with disk drives before the rack is moved to its final location.
1 Introduction
Tools Required:
2.2 Unpacking the Rack
Caution: When handling disk drives or components, avoid touching the printed circuit boards. It is recommended that you observe all conventional ESD precautions.
Caution: To avoid the possibility of facilities circuit breakers tripping in the event of a fault within the Sonexion rack, careful consideration must be taken when selecting the type of facilities circuit breaker installed. Refer to the chart in Section 5.
0
Two people are needed to safely unpack the Sonexion system and four people are needed to bring the rack off the pallet and move it into position.
1 Remove the plastic interlock clips from the front and rear panels. The panels are each secured with seven interlock clips, three on each side and one at the top (shown in Figure 2). The three interlock clips on each side are also used to secure the side panels. 2 Remove the front and rear panels. With the clips removed, lift the front and rear panels from the pallet. Each panel weighs approximately 5 kg (11 lbs). 3 Remove the plastic interlock clips from the side panels. Each panel is secured with 10 interlock clips, three on each side, two at the top, and two at the bottom (shown in Figure 2). Six of these clips were removed in Step 1 (to remove the front and rear panels), so only four remaining clips (top and bottom) need to be removed. 4 Remove the side panels. With the clips removed, lift the side panels from the pallet. Each panel weighs approximately 5 kg (11 lbs). 5 Lift off the top panel. With the front, rear, and side panels removed, the top panel is unattached on top of the rack. The top panel weighs approximately 5 kg (11 lbs).
is one set of bolts at both the front and rear of the rack (shown in Figure 3). 7 Using a 13mm combination wrench, lower the levelling feet, which will raise the rack up off the hold-down brackets. 8 Using a 13mm combination wrench, remove the remaining bolts securing the hold-down brackets to the pallet. Set them aside. 9 Using two 19mm combination wrenches, unbolt and remove the ramp mounting bracket from the shipping pallet. 10 Raise the levelling feet to lower the rack back down onto the castors. The levelling feet must be raised before the rack is unloaded. 11 With the levelling feet raised, remove the transport blocks that are positioned under the feet. The transport blocks go under the levelling feet when the rack is on the pallet. They allow the levelling feet to be lowered down to support the corners of the rack during shipment so that it does not sag. The transport block has a recessed area in the top where the levelling feet reside. Raise the levelling feet high enough to remove the transport blocks (shown in Figure 3).
Figure 2: Removing the Interlock Clips
6 Using a 13mm combination wrench, remove the bolts that secure the rack to the hold-down shipping brackets. There
2.3 Installing the Crate Ramp and Outriggers
• The Sonexion rack design is capable of supporting the total weight of the installed enclosures and incorporates stabilizing features suitable to prevent the rack from tipping or being pushed over during normal use.
IMPORTANT: This procedure requires four people, one to ensure that the castors are properly aligned before the Sonexion rack is moved and three people to move the rack down the ramp.
• When loading the rack, fill from the bottom up and empty from the top down.
1 The Ramp & Outriggers are a separate orderable part. Unpack the loading ramp parts and the outriggers from their packaging. 2 Using the M8 bolts, washers, and nuts (provided), bolt the top and bottom sections of one ramp together (shown
• Do not slide more than one drawer out of the enclosure at a time to avoid the danger of the rack toppling over.
in Figure 4). Using a 13mm combination wrench, securely tighten the fasteners. Repeat for the other side ramp. 3 Set the assembled ramps and outriggers aside. 4 Install the ramp mounting bracket. Using two 19mm wrenches, remove the
Figure 3: Removing the Side Panels & Rack Hold-Down Bracket Bolts
• This equipment is designed to be installed on a dedicated circuit.
7 mounting bracket from its shipping position (shown in Figure 5), position it to the front edge of the pallet, and then use four nuts and bolts to secure it (shown in Figure 4).
• The dedicated circuit must have circuit breaker or fuse protection. The Power Distribution Units (PDUs) provide handletype circuit breakers. Adequate protection must be provided for the dedicated power circuit. Protection of capacity equal to the current rating of the distribution unit must be provided and must meet all applicable codes and regulations. • The plugs on the PDUs should be installed near the equipment and should be easily accessible.
Figure 5: Ramp Mounting Bracket Location
• The rack also has a dynamic load rating of 953 kg (2,100 lbs).
HR-6140-0| August 2016 |
a. Lower the ramps into the slots on the ramp mounting bracket and install the supplied retaining bolts (one under each ramp) using a 13mm combination wrench (shown in Figure 6). These bolts prevent the ramps from lifting out of the ramp mounting bracket. b. Position the support bridge under the two ramps.
• The rack has a static load rating of 1,361 kg (3,000 lbs).
• Frame load ratings are not dependent on side panels, doors or other components for structural support.
5 Position the assembled ramps and support bridge (shown in Figure 4).
Figure 1: Rack Service Area Clearance
Sonexion 3000 Quick Start Guide
Figure 4: Installing the Crate Ramp
WARNING: The mounting bracket must be properly installed before moving the rack off the shipping pallet
6 Ensure that the ramp support bridge is properly positioned. The center of the ramp support bridge must be placed withiin the zone indicated by the yellow labels on the ramps (see Figure 4). CONTINUED >>> Page 1 of 6
2.3 Installing the Crate Ramp & Outriggers (continued) NOTE: Due to variations in the weight of configured Sonexion systems, the pallet/ ramp geometry will differ from system to system. It may be necessary to adjust the lower ramp angle to ensure that the ramp’s leading edge is flush with the floor. To adjust the ramp floor contact point you are permitted to move the ramp support bridge within the 50mm (2”) working zone on the pallet side of the joint.
WARNING: The ramp bridge must be located within the designated area shown by the yellow labels near the ramp’s center joint on the pallet side of the joint.
9 Position one outrigger on the front of the rack enclosure, as indicated in Figure 7, and secure the outrigger to the rack base using a Pozidriv (or Phillips) screwdriver and the two bolts provided. Repeat this, fitting the second outrigger to the rear of the rack. 10 Move the rack to as close as possible to its final location. Do not push sideways on the rack, as indicated in Figure 8. Once installed, the outrigger bracket limits the rack tilt to less than 1.5° (shown in Figure 9), preventing the rack from tipping over.
Figure 6: Installation of the Retaining Bolt IMPORTANT: The person checking that the castors are properly aligned MUST exercise care in ensuring that the ramp support is correctly positioned before the rack is moved down the ramp. The operators MUST also be aware of the surroundings when following this procedure. Only trained operators should be permitted to carry out the unloading of a rack system. 7 Once the ramps are secured to the crate pallet (Figure 4), the rack can be lowered from the shipping pallet by TRAINED PERSONNEL FOLLOWING THE DESIGNATED UNLOADING PROCEDURE. Do not allow the rack to turn sideways. It must slowly roll off in a straight forward direction.
NOTE: With the outrigger installed, the rack may only be moved up and down a maximum incline / decline of 5 degrees. 11 Remove the outriggers and store them with the ramps and all associated bolts in a safe location for future use. 12 Position the rack to its final location and lower the levelling feet using a 13mm wrench. Use a spirit level to ensure the rack is level.
IMPORTANT: Do not push sideways on the rack.
15a and 15b for drive slot locations. 5 Pull the locking pin to release the locking arm, and remove the Speedloader from the drives.
Figure 9: Outrigger Tilt Limits
3 Disk Drives IMPORTANT: Drives are labeled and must be installed in the exact order and the exact slots in which they were installed at the factory.
There are two methods to consider. If a Speedloader is available, it can be used to install up to seven drives at a time to significantly reduce the loading time. Alternatively, drives may be installed individually by hand. Caution: To ensure rack stability, only one drawer in a 5U84 enclosure may be open at a time. Caution: Observe all conventional ESD precautions when handling disk drives and avoid touching the printed circuit boards.
Beginning with the lowest drawer, populate the drives from the rear drive slots to the front drive slots. Close that drawer, open the upper drawer of the lowest 5U84 enclosure, and populate it with disk drives from the rear slots to the front slots. Repeat the process for the remaining 5U84 enclosures, working upwards.
8 Once the rack is clear of the ramp, stop its movement.
This procedure describes how to use the Speedloader to load multiple disk drives at a time into a 5U84 (SSU) enclosure, thereby significantly reducing the time required to fully populate the enclosure. Refer to Section 3.1.2 for the manual drive installation process. 1 Using a T-20 Torx screwdriver, unlock the two drawer latches on the front of each drawer for each 5U84 enclosure in the rack. 2 Refer to the packaging documentation to locate the marked carton containing drives for the target 5U84 enclosure.
The drives will be installed using the Speedloader in groups of seven. Lower drawer Drive slots 83-77 Drive slots 76-70 Drive slots 69-63 Drive slots 62-56 Drive slots 55-49 Drive slots 48-42
Note the position of the interface card, as the drives are installed with the card facing the front of the drawer (shown in Figures 13 & 14). 7 Repeat steps 3-6, until both the drawers have been populated as desired.
Figure 10: Speedloader Figure 11: Speedloader Attached to Drives in the Box
Figure 15a: 5U84 Drive Slot Location: Upper Drawer
8 Close the drawers and secure the latches using a T-20 Torx screwdriver.
3.1 Installing Disk Drives in 5U84 SSUs
3.1.1 Using the Speedloader Figure 8: No Side Push Label
6 Lock each drive into its slot by pressing down while sliding the latch to the locked position, towards the rear of the enclosure (shown in Figures 13 and 14). Repeat for each disk drive.
Upper drawer Drive slots 41-35 Drive slots 34-28 Drive slots 27-21 Drive slots 20-14 Drive slots 13-7 Drive slots 6-0
3 Capture the disk drives in the box to install in the enclosure. The Speedloader attaches to the drives by pulling out on the locking pin, which releases the locking arm side (shown in Figure 10). Position the rigid side of the Speedloader to one side of the drives (using a slight angle) and rotate the Speedloader into position on the other side of the drives, and then close the locking arm (shown in Figure 11). Ensure the drives are in alignment between the fingers on the underside of the Speedloader. Do not force the locking arm to close. If necessary, apply slight pressure to the drive(s) to align them, and then close the locking arm. 4 Once the drives are secured in the Speedloader, carefully lift them clear of the packaging, and position them into the 5U84’s drawer at the correct location (shown in Figure 12). Refer to Figures Caution: The Speedloader with drives weighs up to 8 kg (17.6lbs). Handle with care to avoid the risk of back injury.
9 Continue installing disk drives into the remaining 5U84 enclosures, working from the bottom upwards, repeating steps 2 to 8 for each enclosure.
3.1.2 Manually Installing Drives 1 Perform Steps 1 and 2 from Section 3.1.1. 2 Remove the correct disk drive from the box of drives. Ensure that the slide lock is in the unlock position. If it is not, move it to the unlocked position, as shown in Figure 13.
Figure 12: Speedloader Installing Drives in the Drawer
Figure 15b: 5U84 Drive Slot Location: Lower Drawer
3 With a drawer open, position the drive in the correct slot with the interface card side facing the front of the drawer. Refer to Figures 15a and 15b for slot locations. Starting with the lowest 5U84 enclosure and with its lowest drawer, begin populating the drives from rear to the front, starting with the slot labeled “83” and working towards slot “42.” Repeat the process for the upper drawer, starting with slot “41” and working towards slot “0.” 4 Carefully slide the drive down into position and press down while sliding the latch to the locked position, towards the rear of the enclosure. Move the lock button to the left to ensure it reset (shown in Figures 13 and 14). 5 Repeat steps 2-4 until both drawers have been populated. 6 Close the drawer and secure the latches with a T-20 Torx screwdriver, rotating until the arrow points to the lock icon. 7 Continue installing the drives into the remaining 5U84s, repeating steps 2-6 for each enclosure.
Figure 13: Disk Drive in Drive Carrier
Figure 14: Inserting Disk Drive into Chassis Drawer
Figure 7: Installing the Outrigger
CONTINUED >>> HR-6140-0| August 2016 |
Sonexion 3000 Quick Start Guide
Page 2 of 6
4.4 Operator Control Panel - 5U84
4.2 Embedded Application Controller (EAC)
3 Disk Drives (continued)
The front of the 5U84 enclosure incorporates an operator’s control panel (OPS) with five LEDs and the Unit ID LCD display. The OPS panel provides the user with a consolidated view of the operational status of the enclosure components and the enclosure’s assigned ID. In addition, an alarm mute button is provided on the OPS panel.
3.2 Installing Disk Drives (2U24) Normally, the disk drives in the 2U24 enclosure are pre-installed at the factory. As a convenience, the procedure to install those drives is provided below.
Display
Unit Identification Display The unit identification display is a dual seven segment display that can be used to provide feedback to the user. Its primary usage is to display an enclosure unit identification number to assist in setting up and maintaining multiple enclosure systems. The unit identification display is configured via a VPD option. By default the display will be OFF, and the dual seven segment display will be OFF. If the VPD selects use of the display, the seven segment display will be ON and display the number stored in VPD.
Warning: All drive slots must have a Drive Carrier module or Dummy Carrier module installed to maintain a balanced air flow.
Figure 19: Embedded Application Controller (EAC) – SMU
Caution: Observe all conventional ESD precautions when handling disk drives and avoid touching the printed circuit boards.
1 Remove the disk drive from its anti-static protected package.
Unlocked
Locked Indicator Aperture
3 Insert the drive carrier module into the desired empty drive slot in the enclosure. Drive slots are number 0-23, left to right.
Lock Socket
4 Seat the drive carrier by pressing with your fingers on the handle latch rotating it to a close position (see Figure 16). You will hear a “click” as the latch engages and holds the handle closed. 5 Continue populating all the drives for your configuration. Any drive bay not occupied by a populated drive carrier must have a dummy drive carrier installed.
Figure 17: Drive Carrier Locked and Unlocked (2U24)
6 Engage the drive carrier locks (see Figure 17). Rotate the T-10 Torx screwdriver in a clockwise direction until the “red” indicator is visible in the aperture above the key.
4 Component Descriptions
A
1
2
4
B
1
2
3
4
Description
Ethernet
Right Side - Link/Activity: (Steady) Link active. (Flashing) Network activity. Left Side - Baud rate/network speed: (Yellow) 1000Mb, (Green) 100Mb, (Off) 10Mb.
SAS Lane
ON - Ready, no traffic. FLASHING - Active, I/O traffic.
ID LED
Identification.
Fault LED
Module fault condition.
OK LED
Module is operating normally.
C
1
2
3
4
Drawer Fault
The activity bar graph is a six segment LED graph that shows activity of the SAS interface to the sideplane. For zero activity, no segments are lit. Increasing activity is indicated by segments being lit from the bottom segment upwards. When the interface is transferring data at full capacity, all segments are illuminated. Drawer Fault LED (Amber)
Activity Bar Graph (Green)
Sideplane Card OK/Power Good
On
Off
Off
Off
-
Sideplane Card Fault
Off
On
-
-
Off
Drive Fault
On
-
-
On
Off
Cable Fault
Off
-
On
-
Off
Drive Activity
On
Off
Off
Off
On
Status
OK LED
ON - Module is operating normally.
Activity Bar Graph
Mute Button
Power On/ Standby
Drawer 2 Fault
Cable Fault LED (Amber)
ON - A module fault condition.
Drive, Cable, or Sideplane fault.
4.3 Drawer Status LEDs - 5U84
Fault LED (Amber)
Fault LED
ALL FLASHING - Fault condition or rebooting. OFF - Not ready, no power.
Drive, Cable, or Sideplane fault.
Drawer 2 Fault (Amber)
Mute/Input Switch
Power LED (Green)
ON - Ready, no traffic. FLASHING - Active, I/O traffic.
Drawer 1 Fault (Amber)
Logical Status
SAS Lane LEDs
Description
The logical status LED provides indication of a change of status or fault from something other than the enclosure management system. This may be from an internal or external source and communicated to the enclosure (normally via SES). It is usually associated with a disk drive and LEDs at each disk drive position that helps the user to identify the affected drive.
Diagnostic These eight diagnostic LEDs provide a dual digital hex code. During the boot process, the LEDs BIOS executes several platform configuration processes and each process is assigned a specific hex POST code.
Figure 18: EBOD I/O Module SAS Lane LED
Logical Fault LED (Amber)
Operation
208V
415V
208V
208V
Plug Type
Hubbell CS8365C
IEC60309 32A 3PY
NEMA L21-30P
IEC60309 60A 3P
Input Phase
208-240V 3Φ-Delta
380-415 3Φ Wye
190-208Φ Delta
208-240V 3Φ-Delta
Output
208-240V
220-240V
190-208V
208-240V
Outlet Type
C19
C19
C19
C19
Form Factor
0U
0U
0U
0U
Circuit Breaker (Qty)
20A, 2-pole UL489 (12)
16A, 1-pole UL489, VDE (12)
20A, 2-pole UL489 (9)
20A, 2-pole LEGBX66-20 (6)
Rated Current
40A/Phase
32A/Phase
24A/Phase
48A/Phase
Power Capacity
28.8kVA
44.3kVA
25.9kVA
17.3kVA at 208V, 20.0kVA at 240V
2.66” (67.5mm)
1.6” (40.6mm)
Module Fault
Unit ID Display
Figure 23: Operator’s Panel - 2U24
The front of the 2U24 enclosure incorporates an operator’s control panel (OPS) with three LEDs and the Unit ID LCD display. The OPS panel provides the user with a consolidated view of the operational status of the enclosure components and the enclosure’s assigned ID. Please refer to table below for details of the LED status conditions. State
System Power Steady Green Module Fault
Steady Amber
Logical Fault
Steady Amber
2.70 (86.6 mm)
NOTE: Depending on the installation, the PDU Mains power cords will either be routed out of the top of the rack cabinet or out of the bottom into the sub-floor system and connected to the appropriate power systems.
Resetting the Handle-Type Circuit Breaker
4.5 Operator Control Panel - 2U24
LEDs
Figure 24: PDU Models
Logical Fault
Drawer 1 Fault
Figure 22: Operator’s Panel - 5U84
Image
System Power
Module Fault
Cable Fault
LED
This LED shows amber when standby power ONLY is available. It is green when system power is available.
ALL FLASHING - Fault condition or rebooting. OFF - Not ready, no power.
Logical Fault
Fault LED
Power On/Standby LED (Green/Amber)
Important: You must perform a ground (earth) continuity and insulation test after completing all assemblies and prior to powering on any system.
Unit ID Display
LED
Sideplane OK/ Power Good
6G SAS (X3)
3
Used as above in the setting of the unit identification display. It is also used to transition alarm states.
NOTE: The ratings labels provide the 20% de-rated factor values as required for operations in the USA. See Figure 25.
Figure 20: Embedded Application Controller (EAC) – SSU/MMU
4.1 EBOD I/O Module Descriptions Health LED Maintenance Port
Mute/Input Switch
Power connections within the system should be carefully considered, as the loads are large. Where possible, a balanced load across the 3 phases should be maintained. Refer to the charts below for the proper connections.
Module Fault LED (PSU/ The module fault LED is illuminated when there is a system hardware fault. It Cooling Fan/SBB Status) may be associated with a fault LED on a PSU or I/O module which helps the user to identify the component that has a fault. (Amber)
Figure 16: Securing the Drive Carrier (2U24)
2 Ensure the drive carrier handle is released and in the open position. It may require the use of a T-10 Torx screwdriver to unlock the drive carrier.
Description
5 PDU Mains Power and Power Cords
1 Lift the hinged cover over the breaker. 2 Check if the colorful rectangle or triangle below the operating handle is GREEN, indicating the breaker has tripped. 3 Determine the cause that resulted in the overload or short circuit. 4 Pull up the operating handle until the colorful rectangle or triangle turns RED. Voltage: 400V (3P+N+ Current: 32A Frequency: 50/60HZ
)
415V Operations
Voltage: 280V (3P+N+ Current: 30A Frequency: 50/60HZ
)
208V Operations
Voltage: 240V (3P+N+ Current: 50A Frequency: 50/60HZ
)
208V Operations
Voltage: 240V (3P+N+ Current: 60A Frequency: 50/60HZ
)
208V Operations
Figure 25: Power Rating Labels
Description AC Power is applied to the enclosure. • PCM fault • EBOD module fault • ESM fault • Over / under temp. fault Refer to individual module fault LEDs. Failure of a disk drive.
Figure 21: Drawer Status LEDs
CONTINUED >>> HR-6140-0| August 2016 |
Sonexion 3000 Quick Start Guide
Page 3 of 6
5 PDU Mains Power and Power Cords (continued) PX2-5100X2-V2 - Base Rack 4U MMU (Standard Orientation) Left PDU
Connector
Input Line
IBS1,SW1,SMU IBS0, SW0, MMU N/C
1
A
2
A
3
A
N/C
4
N/C
Phase
PX2-5100X2-V2 - Expansion Rack 4U MMU (Standard Orientation)
Right PDU
Left PDU
Connector
Input Line
Phase
L1L2
IBS1,SW1,SMU
L2L3
IBS0, SW0, MMU
IBS1,SW1,(Optional Additional MMU) IBS0, SW0
1
A
L1L2
2
A
L3L1
N/C
SSU6
3
B
L1L2
N/C
N/C
5
B
L2L3
N/C
N/C
6
B
L3L1
SSU3
7
A
SSU5
8
SSU4
PX2-5100X2-V2 - Base Rack 4U MMU (Inverted Orientation)
Right PDU
Left PDU SSU1
1
L2L3
IBS1,SW1,(Optional Additional MMU) IBS0, SW0
SSU0
2
A
L3L1
SSU6
SSU2
4
B
L1L2
N/C
N/C
5
B
L2L3
N/C
N/C
N/C
6
B
L3L1
N/C
L1L2
SSU3
SSU3
7
A
L1L2
SSU3
A
L2L3
SSU5
SSU5
8
A
L2L3
SSU5
9
A
L3L1
SSU4
SSU4
9
A
L3L1
SSU4
SSU2
10
B
L1L2
SSU2
SSU2
10
B
L1L2
SSU2
SSU0
11
B
L2L3
SSU0
SSU0
11
B
L2L3
SSU0
SSU1
12
B
L3L1
SSU1
SSU1
12
B
L3L1
SSU1
PX2-5104X2-V2 - Base Rack 4U MMU (Standard Orientation)
PX2-5104X2-V2 - Expansion Rack 4U MMU (Standard Orientation)
Left PDU
Connector
Input Line
Phase
Right PDU
Left PDU
IBS1,SW1,SMU
1
A
L1
IBS1,SW1,(Optional Additional MMU) IBS0, SW0 SSU6
L1L2
N/C
N/C
7
A
L1L2
N/C
N/C
8
A
L2L3
N/C
N/C
8
A
L2L3
N/C
N/C
9
A
L3L1
N/C
N/C
9
A
L3L1
N/C
N/C
10
B
L1L2
N/C
SSU6
10
B
L1L2
SSU6
IBS0,SW0,MMU
11
B
L2L3
IBS0,SW0,MMU
B
L2L3
12
B
L3L1
IBS1,SW1,SMU
IBS0,SW0 IBS1,SW1,(Optional Additional MMU)
11
IBS1,SW1,SMU
12
B
L3L1
IBS0,SW0 IBS1,SW1,(Optional Additional MMU)
PX2-5104X2-V2 - Base Rack 4U MMU (Inverted Orientation)
PX2-5104X2-V2 - Expansion Rack 4U MMU (Inverted Orientation)
SSU3
SSU5
SSU5
5
B
L2
SSU5
SSU4
6
B
L3
SSU4
SSU4
6
B
L3
SSU4
N/C
7
A
L1
N/C
N/C
7
A
L1
N/C
N/C
8
A
L2
N/C
N/C
8
A
L2
N/C N/C
A
L2
SSU5
SSU5
8
A
L2
SSU5
A
L3
SSU3
SSU3
9
A
L3
SSU3
SSU2
10
B
L1
SSU2
SSU2
10
B
L1
SSU2
SSU0
11
B
L2
SSU0
SSU0
11
B
L2
SSU0
SSU1
12
B
L3
SSU1
SSU1
12
B
L3
SSU1
HR-6140-0| August 2016 |
A
L2
8
* Core = Core Components
7
B
9
SSU3
SSU3
N/C
5
SSU5
L3L1
L3L1
SSU5
SSU3
C
B
SSU2
SSU4
9
6
L1
L1
SSU3
SSU3
L3
A
SSU0
SSU3
A
7
L2L3
L3L1
B
SSU4
C
B
3
SSU4
8
6
4
L1
SSU0
SSU5
SSU3
SSU2
A
SSU1
L2L3
SSU3
7
L1L2
B
SSU3
SSU4
C
5
SSU2
N/C
7
SSU5
L1
L3
SSU1
SSU5
L3
B
SSU4
L2L3
A
6
SSU5
B
B
N/C
L3L1
5
4
N/C
L2L3
SSU4
SSU5
3
L3
B
L1L2
SSU3
B
B
B
SSU2
6
6
4
SSU6
A
N/C
5
SSU4
L3
A
3
N/C
SSU4
SSU4
L2
2
L2
SSU5
L1L2
SSU1
B
SSU3
B
SSU0
5
L1L2
4
L1
N/C
B
SSU2
SSU4
L2
N/C
4
L3L1
A
L2
SSU3
A
A
B
N/C
3
1
5
L3L1
SSU2
2
N/C
A
SSU2
SSU1
N/C
3
L3L1
SSU0
L1
N/C
A
SSU0
B
PX2-5965X3-V2 - Expansion Rack 4U MMU (Standard Orientation)
IBS1,SW1,(Optional Additional MMU) IBS0, SW0
SSU0
3
SSU1
4
IBS1,SW1,SMU
SSU1
L2L3
L2
N/C
IBS0, SW0, MMU
L1L2
A
L1
N/C
L2L3
A
2
A
L1
L1L2
1
SSU0
A
B
A
SSU1
SSU0
2
4
A
SSU1
L2L3
1
N/C
2
L1L2
A
SSU0
L1
L3
1
A
SSU1
A
A
IBS1,SW1,SMU
Right PDU
IBS1,SW1,(Optional Additional MMU) IBS0, SW0
1
3
IBS0, SW0, MMU
Phase
Right PDU
N/C
Left PDU
Input Line
Phase
L2
Right PDU
Connector
Input Line
A
Phase
Left PDU
Connector
Phase
2
Connector Input Line
Right PDU
Left PDU
Input Line
IBS0, SW0, MMU
Left PDU
Phase
Right PDU
Connector
IBS1,SW1,SMU IBS0, SW0, MMU N/C
PX2-5965X3-V2 - Base Rack 4U MMU (Standard Orientation)
Connector Input Line
PX2-5100X2-V2 - Expansion Rack 4U MMU (Inverted Orientation)
Connector
Input Line
Phase
1
A
L1L2
Left PDU
Connector
Input Line
Phase
Right PDU
N/C
9
A
L3
N/C
N/C
9
A
L3
N/C
10
B
L1
N/C
N/C
10
B
L1
N/C
IBS0,SW0,MMU
11
B
L2
IBS0,SW0,MMU
11
B
L2
IB1,SW1,SMU
12
B
L3
IB1,SW1,SMU
IBS0,SW0 IB1,SW1,(Optional Additional MMU)
12
B
L3
IBS0,SW0 IB1,SW1,(Optional Additional MMU)
PX2-5965X3-V2 - Base Rack 4U MMU (Inverted Orientation) Input Line
Left PDU
Connector
Input Line
Phase
Right PDU
Right PDU
Left PDU SSU2
1
A
L1L2
SSU2
SSU2
1
A
L1L2
SSU2
SSU0
2
A
L2L3
SSU0
SSU0
2
A
L2L3
SSU0 SSU1
2
A
L2L3
IBS1,SW1,(Optional Additional MMU) IBS0, SW0
SSU6
3
A
L3L1
SSU6
SSU3
4
B
L1L2
SSU3
SSU5
5
B
L2L3
SSU5
SSU4
6
B
L3L1
SSU4
SSU1
7
C
L1L2
SSU1
SSU0
8
C
L2L3
SSU0
SSU3
9
C
L3L1
SSU3
Connector
PX2-5965X3-V2 - Expansion Rack 4U MMU (Inverted Orientation)
Phase
Right PDU
SSU1
3
A
L3L1
SSU1
SSU1
3
A
L3L1
SSU4
4
B
L1L2
SSU4
SSU4
4
B
L1L2
SSU4
SSU5
5
B
L2L3
SSU5
SSU5
5
B
L2L3
SSU5
SSU3
6
B
L3L1
SSU3
SSU3
6
B
L3L1
SSU3
N/C
7
C
L1L2
N/C
SSU6
7
C
L1L2
SSU6
IBS0,SW0,MMU
8
C
L2L3
IBS0,SW0,MMU
C
L2L3
9
C
L3L1
IBS1,SW1,SMU
IBS0,SW0 IBS1,SW1,(Optional Additional MMU)
8
IBS1,SW1,SMU
9
C
L3L1
IBS0,SW0 IBS1,SW1,(Optional Additional MMU)
CONTINUED >>> Sonexion 3000 Quick Start Guide
Page 4 of 6
5 PDU Mains Power and Power Cords (continued) PX2-5551-N4V2 - Base Rack 4U MMU (Standard Orientation) Left PDU
Connector
Input Line
Phase
PX2-5551-N4V2 - Expansion Rack 4U MMU (Standard Orientation)
Right PDU
Left PDU
Connector
PX2-5551-N4V2 - Base Rack 4U MMU (Inverted Orientation)
Input Line
Phase
Right PDU
Left PDU
Connector
Input Line
Phase
Right PDU
PX2-5551-N4V2 - Expansion Rack 4U MMU (Inverted Orientation) Left PDU
Connector
Input Line
Phase
Right PDU
IBS1
1
A
L1L2
IBS1
IBS1
1
A
L1L2
IBS1
N/C
1
A
L1L2
N/C
N/C
1
A
L1L2
N/C
IBS0
2
A
L1L2
IBS0
IBS0
2
A
L1L2
IBS0
N/C
2
A
L1L2
N/C
N/C
2
A
L1L2
N/C
N/C
3
A
L1L2
N/C
N/C
3
A
L1L2
N/C
SSU0
3
A
L1L2
SSU0
SSU0
3
A
L1L2
SSU0
SSU5
4
A
L1L2
SSU5
SSU6
4
A
L1L2
SSU6
N/C
4
A
L1L2
N/C
SSU1
4
A
L1L2
SSU1
SW1
5
A
L1L2
SW1
SW1
5
A
L1L2
SW1
N/C
5
A
L1L2
N/C
N/C
5
A
L1L2
N/C
SW0
6
A
L1L2
SW0
SW0
6
A
L1L2
SW0
N/C
6
A
L1L2
N/C
N/C
6
A
L1L2
N/C SSU2
N/C
7
A
L1L2
N/C
N/C
7
A
L1L2
N/C
SSU1
7
A
L1L2
SSU1
SSU2
7
A
L1L2
SSU4
8
A
L1L2
SSU4
8
A
L1L2
8
A
L1L2
N/C
N/C
8
A
L1L2
N/C
9
A
L2L3
SMU
N/C
9
A
L2L3
N/C
N/C
9
A
L2L3
N/C
MMU
10
A
L2L3
MMU
N/C
11
A
L2L3
N/C
SSU5 (Optional Additional MMU) N/C
N/C
SMU
SSU5 (Optional Additional MMU) N/C
SSU3
12
A
L2L3
SSU3
N/C
13
A
L2L3
N/C
N/C
14
A
L2L3
N/C
N/C
15
A
L2L3
N/C
SSU2
16
A
L2L3
SSU2
N/C
17
A
L3L1
N/C
N/C
18
A
L3L1
N/C
N/C
19
A
L3L1
N/C
SSU1
20
A
L3L1
SSU1
N/C
21
A
L3L1
N/C
N/C
22
A
L3L1
N/C
N/C
23
A
L3L1
N/C
SSU0
24
A
L3L1
SSU0
9
A
L2L3
10
A
L2L3
N/C
11
A
L2L3
N/C
SSU4
12
A
L2L3
SSU4
N/C
13
A
L2L3
N/C
N/C
14
A
L2L3
N/C
N/C
15
A
L2L3
N/C
SSU3
16
A
L2L3
SSU3
N/C
17
A
L3L1
N/C
N/C
18
A
L3L1
N/C
N/C
19
A
L3L1
N/C
SSU2
20
A
L3L1
SSU2
N/C
21
A
L3L1
N/C
N/C
22
A
L3L1
N/C
SSU1
23
A
L3L1
SSU1
SSU0
24
A
L3L1
SSU0
N/C
10
A
L2L3
N/C
N/C
10
A
L2L3
N/C
SSU2
11
A
L2L3
SSU2
SSU3
11
A
L2L3
SSU3
N/C
12
A
L2L3
N/C
N/C
12
A
L2L3
N/C
MMU
13
A
L2L3
MMU
N/C (Optional Additional MMU) SSU4
13
A
L2L3
14
A
L2L3
15
A
L2L3
N/C (Optional Additional MMU) SSU4
N/C
16
A
L2L3
N/C
SW0
17
A
L3L1
SW0
SW1
18
A
L3L1
SW1
SSU5
19
A
L3L1
SSU5
SMU
14
A
L2L3
SMU
SSU3
15
A
L2L3
SSU3
N/C
16
A
L2L3
N/C
SW0
17
A
L3L1
SW0
SW1
18
A
L3L1
SW1
SSU4
19
A
L3L1
SSU4
N/C
20
A
L3L1
N/C
IBS0
21
A
L3L1
IBS0
IBS1
22
A
L3L1
IBS1
SSU5
23
A
L3L1
SSU5
N/C
24
A
L3L1
N/C
N/C
20
A
L3L1
N/C
IBS0
21
A
L3L1
IBS0
IBS1
22
A
L3L1
IBS1
SSU6
23
A
L3L1
SSU6
N/C
24
A
L3L1
N/C
50/60 Hz Medium Delay Curve
CONTINUED >>> HR-6140-0| August 2016 |
Sonexion 3000 Quick Start Guide
Page 5 of 6
6 Standard Rack Configurations and Data Cabling Figures 26 and 27 show the component positions installed in the base rack and expansion rack. Note: in the examples below a 36-port InfiniBand switch is used to illustrate where the Data Switch is located. 4 2
LMN Switch 1
4 2
4 1
LMN Switch 0
4 1
4 0
LDN Switch 1
4 0
3 9
LDN Switch 0
3 9
3 8
KVM/RAS Ports
3 8
3 7
3 6
3 5
3 4
SMU MMU
3 7
3 6
3 5
3 4
3 3
3 3
3 2
3 2
3 1
3 1
3 0
3 0
2 9
2 9
2 8
SSU n+5
2 8
2 7
2 7
2 6
2 6
2 5
2 5
2 4
2 4
2 3
SSU n+4
2 3
2 2
2 2
2 1
2 1
2 0
2 0
1 9
1 9
1 8
SSU n+3
1 8
1 7
1 7
1 6
1 6
1 5
1 5
1 4
1 4
1 3
SSU n+2
1 3
1 2
1 2
1 1
1 1
1 0
1 0
9
9
8
SSU n+1
8
7
7
6
6
5
5
4
4
3
SSU n+0
3
2
2
1
1
Figure 26: 42U Base Rack Layout
4 2
LMN Switch 1
4 2
4 1
LMN Switch 0
4 1
4 0
LDN Switch 1
4 0
3 9
LDN Switch 0
3 9
3 8
KVM/RAS Ports
3 8
3 7
3 6
Optional Additional MMU
3 4
3 4
SSU n+6
3 3
3 2
3 2
3 1
3 1
3 0
3 0
2 9
2 9
2 8
SSU n+5
2 8
2 7
2 7
2 6
2 6
2 5
2 5
2 4
2 4
2 3
SSU n+4
2 3
2 2
2 2
2 1
2 1
2 0
2 0
1 9
1 9
1 8
SSU n+3
1 8
1 7
1 7
1 6
1 6
1 5
1 5
1 4
1 4
1 3
SSU n+2
1 3
1 2
1 2
1 1
1 1
1 0
1 0
9
9
8
SSU n+1
8
7
7
6
6
5
5
4
4
3
SSU n+0
The subsections below provide the most common configuration connections to interface the system to the customer environment. For inter-rack Ethernet 24-port switch cabling, ports 5, 7, 17, and 19 in the base rack are available uplink ports. For inter-rack Ethernet 48-port switch cabling, ports 1 through 24, and 29, 31, 41, and 43 are available uplink ports. The base rack and expansion rack upper switch and lower switch port assignments in the configuration figures are provided for example only. Refer to the lists in each section for the specific inter-rack InfiniBand port switch cabling and inter-rack Ethernet switch cabling for the upper switch and lower switch available uplink ports.
3 6
3 5
3 5
3 3
3 7
6.1 Connecting the Data Cables
6.2 SMU and InfiniBand 36-Port Configuration: Dual 24-Port LAN Switches For inter-rack InfiniBand 36-port switch cabling, the available uplink ports are: - Upper switch 1 (IBS1) - ports 1 through 21, 23, 25, 26, 29, 31, 33, and 35 - Lower switch 0 (IBS0) - ports 1, 3, 5, 7, 9, 11, 13, 15, 17 through 36 For inter-rack Ethernet switch cabling, the available uplink ports are: - Upper switch 1 (SW1) - 5, 7, 17, and 19 - Lower switch 0 (SW0) - 5, 7, 17, and 19 1 In the base rack and each expansion rack, connect a QSFP+ data cable to the lower InfiniBand switch (IBS0) (available ports listed above), and then to the customer core switch.
6.3 SMU and InfiniBand 36-Port Configuration: Dual 48-Port LAN Switches For inter-rack InfiniBand 36-port switch cabling, the available uplink ports are: - Upper switch 1 (IBS1) - ports 1 through 21, 23, 25, 26, 29, 31, 33, and 35 - Lower switch 0 (IBS0) - ports 1, 3, 5, 7, 9, 11, 13, 15, 17 through 36 For inter-rack Ethernet switch cabling, the available uplink ports are: - Upper switch 1 (SW1) - ports 1 through 24, 29, 31, 41, and 43 - Lower switch 0 (SW0) - ports 1 through 24, 29, 31, 41, and 43 1 In the base rack and each expansion rack, connect a QSFP+ data cable to the lower InfiniBand switch (IBS0) (available ports listed above), and then to the customer core switch. 2 In the base rack and each expansion rack, connect a QSFP+ data cable to the upper InfiniBand switch (IBS1) (available ports listed above), and then to the customer core switch. 3 In each expansion rack of a multi-rack system, connect a Cat 5/6 Ethernet cable to the lower Ethernet switch (SW0) (available ports listed above). Connect the other end of the cable to the lower Ethernet switch (SW0) in the base rack. 4 In each expansion rack of a multi-rack system, connect a Cat 5/6 Ethernet cable to the upper Ethernet switch (SW1) (available ports listed above). Connect the other end of the cable to the upper Ethernet switch (SW1) in the base rack. 5 In the base rack, connect a Cat 5/6 Ethernet cable from the NIC card, SMU Node 0, Port 4 to the local network. 6 In the base rack, connect another Cat 5/6 Ethernet cable from the NIC card, SMU Node 1, Port 4 to the local network.
6.4 Optional Additional MMU Unit The Lustre Distributed Namespace (DNE) feature allows multiple MDTs operating through multiple MDS nodes to be configured and operate as part of a single file system. The optional hardware MMU component adds the additional MDS/MDT capability to a file system cluster. The optional additional MMU is deployed as a single 2U24 enclosure containing: - 2 x optional additional MMU EACs (Embedded Application Controllers) - 22 x 10K HDDs - 2 x standard 2U24 power supply / fan modules Each optional additional MMU provides two Lustre Metadata Servers (MDSs), one running on each of the 2 optional additional MMU EACs and operating together as an HA failover pair. Normally, each MDS owns and operates 1 of the 2 MDTs provided by the optional additional MMU; during failover the surviving MDS owns and operates both of the 2 MDTs. Each MDT is provisioned as a single RAID 10 (5+5) array, using a total of 20 HDDs with the remaining 2 HDDs configured as global hot spares. A single Sonexion file system cluster may have from 0 to 8 optional additional MMUs installed. One optional additional MMU per expansion rack (2U of rack space) can be installed, and includes the following cables and connections: - 2 x power cable, 1 to each PDU - 2 x LMN cable to each node = 4 total - 1 x LDN cable to each node = 2 total The following switch port connections are allocated: - 2 ports on each LMN switch = 4 total - 1 port on each LDN switch = 2 total The optional additional MMU occupies the same position in the expansion rack as the upper MMU component in the base rack, immediately below the four Network switches.
2 In the base rack and each expansion rack, connect a QSFP+ data cable to the upper InfiniBand switch (IBS1) (available ports listed above), and then to the customer core switch. 3 In each expansion rack of a multi-rack system, connect a Cat 5/6 Ethernet cable to the lower Ethernet switch (SW0) (available ports listed above). Connect the other end of the cable to the lower Ethernet switch (SW0) in the base rack. 4 In each expansion rack of a multi-rack system, connect a Cat 5/6 Ethernet cable to the upper Ethernet switch (SW1) (available ports listed above). Connect the other end of the cable to the upper Ethernet switch (SW1) in the base rack. 5 In the base rack, connect a Cat 5/6 Ethernet cable from the NIC card, SMU Node 0, Port 4 to the local network. 6 In the base rack, connect another Cat 5/6 Ethernet cable from the NIC card, SMU Node 1, Port 4 to the local network. Figure 30: 36-Port IB Base Rack with 48-Port LAN Using Core Switch
Figure 32: Optional Additional MMU Cabling
3
2
2
1
1
Figure 27: 42U Expansion Rack Layout
Figure 28: 36-Port IB Base Rack with 24-Port LAN Using Core Switch
Figure 31: 36-Port IB Expansion Rack with 48-Port LAN Using Core Switch
Figure 29: 36-Port IB Expansion Rack with 24-Port LAN Using Core Switch HR-6140-0| August 2016 |
Sonexion 3000 Quick Start Guide
Page 6 of 6
