Array Networking And Communication Report

Transcript

EISCAT Scientific Association      EISCAT_3D    Work Package 12  Networking and Reference Time and  Frequency      Deliverable D12.2  Array Networking and Communication  Report          Peter Bergqvist  EISCAT Scientific Association  P.O. Box 812, SE‐981 28 Kiruna, Sweden  2009‐04‐06 17.06    EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  Table of Contents  1  Introduction ....................................................................................................... 4  2  Performance requirements ................................................................................ 4  3  High‐speed serial I/O ......................................................................................... 5  3.1  Serializer/Deserializer (SerDes) ..................................................................... 5  3.2  Interconnect standards .................................................................................. 6  3.3  ASIC ................................................................................................................ 9  3.4  FPGA ............................................................................................................. 10  3.5  Clock distribution ......................................................................................... 11  4  Optical devices ................................................................................................. 11  4.1  Fiber cables .................................................................................................. 11  4.2  Fiber connectors .......................................................................................... 12  4.3  Optical transceivers ..................................................................................... 13  4.4  Differential serial electrical interface .......................................................... 15  5  Demonstrator Array ......................................................................................... 17  6  Glossary ........................................................................................................... 19  Peter Bergqvist  2009‐04‐06 17.06  3 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency  1          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  Introduction  EISCAT_3D is a design study of a new European next generation VHF Incoherent Scatter (IS) radar and interferometry receiver system based on phased-array antennas located in the northern part of Norway, Sweden and Finland. The antenna array internal network topology should be a high-speed serial I/O supporting fiber cable and perhaps in some parts copper cable. The serial link transfer rate should be in the Gbit/s range for the data stream from the receiving antenna elements and lower for the control data to the antenna elements, but today’s interconnect standards are only in full duplex mode. It should also be possibility to increase the transfer rate if needed. 2  Performance requirements  The VHF incoherent scatter radar requires more than 16000 antenna elements, with a total diameter of 120 meters, to approach the full 3D array target perfomance specification. The phased-array antenna will be physically divided into close packed hexagonal subgroups of 49 antenna elements, each of the subgroups are composed of 7 hexagonal cells with 7 antenna elements. All electronics for the subgroup will be housed in a common equipment container and the antenna elements are connected to the container by low loss coaxial cable. The subgroups of 49 antenna elements is also a very suitable size for the interferometry receiver system. The distance requirement for the long baseline (pair of subgroups) interferometry is up to 1.5 km. The serial link transfer rate requirement depends on what kind of encoding method will be used e.g. the 8B/10B encoding translate a 8-bit data to 10-bit symbol (20% overhead) and for the 64B/66B encoding a 64-bit data is translate to a 66-bit symbol (~3% overhead) Peter Bergqvist  2009‐04‐06 17.06  4 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  The total serial link transfer rate requirement for antenna elements with two polarisations, 16-bit data at 80 MHz sampling rate and 8B/10B encoding are as following. • One antenna element. The symbol(or data) rate should be at least 2 polarisations x 16-bit x 80 MHz x 10/8(or 1) encoding factor = 3.2(or 2.56) Gbit/s. • 49 antenna elements. The symbol(data) rate is 49 antennas x 3.2(2.56) Gbit/s = 156.8(125.44) Gbit/s. • Subgroup of 49 antenna element, 49-input FPGA based beamformer and five (N=5) simultaneously beams, i.e. the total serial link transfer rate requirement. By installing a 49-input FPGA based beamformer in each equipment container i.e. subgroup, the transfer rate can be reduced by a factor of 49/N, where N is the number of simultaneous beams. The total serial link symbol(data) rate requirement is 5/49 beamformer factor x 156.8(125.44) Gbit/s = 16.0(12.8) Gbit/s for each subgroup. 3  High­speed serial I/O  3.1  Serializer/Deserializer (SerDes)  When designing a high-speed serial I/O one basic device is the SerDes, also known as Multi-Gigabit Transceiver MGT or RocketIO Transceiver. The basic building blocks of a SerDes are differential transmitter/receiver, serializer/deserializer, parallel 8B/10B or 64B/66B encoder/decoder and transmitter/receiver buffers (FIFO). SerDes converts data from-parallel-to-differential serial and from-differential serial-to-parallel up to 10+ Gbit/s (full duplex) and was initially used in box-tobox communication, but is nowadays also used in chip-to-chip communication. The purpose of encoding is to guarantee enough transitions in the serial datastream for the transmitter/receiver to create/extract the embedded clock. Another reason for encoding is to generate a DC-balanced serial datastream which will reduce ISI (InterSymbol Interference), e.g. 8B/10B encoding scheme uses a lookup table to translate a 8-bit data to 10-bit symbol to achieve a DC-balanced serial datastream with enough transitions for the CDR (Clock Data Recovery) to recovery the clocksignal. Peter Bergqvist  2009‐04‐06 17.06  5 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency  3.2          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  Interconnect standards  There are several industry standards for high-speed serial I/O who uses SerDes. Some of the standards are using the full duplex parallel electrical interface XAUI (10-Gigabit Attachment Unit Interface), using four parallel lanes of selfclocked serial differential links in each direction with 8B/10B encoding to create/extract the embedded clock and the newer full duplex serial electrical interface XFI (10-Gigabit Serial Electrical Interface), using a single lane of self-clocked serial differential links in each direction with 64B/66B encoding to create/extract the embedded clock. A lane is a set of differential pairs, one pair for transmit TX and one pair for receive RX. A link is a communication path between two devices. An xN link is composed of N lanes i.e. x4 link is composed of 4 lanes. • 1 GbE (Gigabit Ethernet), Switched Point-to-Point interconnection. 1000BASE-CX (copper cable), 1000BASE-SX (multimode fiber) and 1000BASE-LX (both multi- and singlemode fiber) are using 8B/10B encoding with a symbol(data) rate of 1.25(1.0) Gbit/s. Gigabit Ethernet involves only the bottom layers 1 and 2 of the OSI model. 2. Data Link layer. 1. Physical layer. • 10 GbE (10 Gigabit Ethernet), Switched Point-to-Point interconnection. 10GBASE-CX4 (copper cable) and 10GBASE-LX4 (both multi- and singlemode fiber) are using XAUI and 8B/10B encoding with a symbol(data) rate of 3.125(2.5) Gbit/s per lane and with 4 lanes it gives a total symbol(data) rate of 12.5(10.0) Gbit/s per link. 10GBASE-SR (multimode fiber), 10GBASE-LR/ER (singlemode fiber) are using XFI and 64B/66B encoding with a symbol(data) rate of 10.3125(10.0) Gbit/s. 10 Gigabit Ethernet involves only the bottom layers 1 and 2 of the OSI model. 2. Data Link layer. 1. Physical layer. • PCIe or PCI-E (PCI Express), Switched Point-to-Point interconnection for short distance use. PCIe is a replacement of the older and very different parallel bus architecture PCI-X (PCI eXtended). Peter Bergqvist  2009‐04‐06 17.06  6 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  PCIe 1.x and 2.0 are using 8B/10B encoding (20% overhead), but PCIe 3.0 removes the requirement for 8b/10b encoding and uses scrambling only instead (0% overhead). It gives for version 1.x, 2.0 and 3.0 symbol(data) rate of 2.5(2.0), 5.0(4.0) respective 8.0(8.0) Gbit/s per lane. The specification defines 1 (x1), 2 (x2), 4 (x4), 8 (x8), 12 (x12), 16 (x16) and 32 (x32) lanes. For PCIe 2.0 it gives a total symbol(data) rate of up to 160.0(128.0) Gbit/s per link (20% overhead), and for PCIe 3.0 0 a total symbol(data) rate of up to 256.0(256.0) Gbit/s per link (0% overhead) The PCI Express standard is defined in three layers. 3. Transaction layer. 2. Data Link layer. 1. Physical layer. • Serial RapidIO using XAUI, Switched Fabric Point-to-Point interconnection for short distance use. Serial RapidIO (SRIO) is a serial version of the older Parallel RapidIO. SRIO is using 8B/10B encoding with a symbol(data) rate of 1.25(1.0), 2.5(2.0), 3.125(2.5) or 6.25(5.0) Gbit/s per lane. The specification defines 1, 2, 4, 8 and 16 lanes which gives a total symbol(data) rate of up to 100.0(80.0) Gbit/s per link. The Serial/Parallel RapidIO standard is defined in three layers. 3. Logical layer. 2. Transport layer. 1 Physical layer. • InfiniBand, Switched Fabric Point-to-Point interconnection. InfiniBand is using the 8B/10B encoding with a symbol(data) rate of 2.5(2.0), 5.0(4.0) or 10.0(8.0) Gbit/s per lane (SDR Single Data Rate, DDR Double Data Rate respective QDR Quad Data Rate). The specification defines 1 (1X), 4 (4X), and 12 (12X) lanes which gives a total symbol(data) rate of up to 120.0(96.0) Gbit/s per link. Peter Bergqvist  2009‐04‐06 17.06  7 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  InfiniBand standard is defined in five layers where each layer operates independently of one another. 5. Upper layer. 4. Transport layer. 3. Network layer. 2. Link layer. 1. Physical layer. • GFC (Gigabit Fibre Channel), Have three major topologies Point-to-Point (FC-P2P), Arbitrated Loop (FC-AL) and Switched Fabric (FC-SW). 1 GFC, 2 GFC, 4 GFC and 8 GFC are using 8B/10B encoding with a symbol(data) rate of 1.0625(0.8), 2.125(1.6), 4.25(3.2) respective 8.5(6.4) Gbit/s (> 20% overhead includes 8B/10B encoding, frame headers and other overhead). 10 GFC and 20 GFC are instead using the 64B/66B encoding with a symbol(data) rate of 10.51875(8.0) respective 10.52(16.0) Gbit/s. Fibre Channel standard is defined in five layers. FC4. Protocol Mapping layer. FC3. Common Services layer. FC2. Network layer. FC1. Data Link layer. FC0. Physical layer. • Aurora (developed by Xilinx), Lightweight Serial Point-to-Point interconnection for short distance chip-to-chip or board-to-board use. Aurora have both 8B/10B and 64B/66B encoding with a symbol rate up to 3.75 Gbit/s. Aurora involves only the bottom layers 1 and 2 of the OSI model. 2. Data Link layer. 1. Physical layer. • SerialLite II (developed by Altera), Lightweight Serial Point-to-Point interconnection for short distance chip-to-chip or board-to-board use. SerialLite II is only using 8B/10B encoding. SerialLite II involves only the bottom layers 1 and 2 of the OSI model. 2. Data Link layer. 1. Physical layer. Peter Bergqvist  2009‐04‐06 17.06  8 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  In the following table the application are divided into four categories (chip-tochip, backplane, copper cable and fiber cable), which also indicated the distance between communication devices. Chip-to-Chip Backplane Board-to-Board High-speed serial I/O standards 10 Gb Ethernet with XAUI 10 Gb Ethernet with XAUI Serial RapidIO with XAUI Serial RapidIO with XAUI PCI Express PCI Express Aurora Aurora SerialLite II SerialLite II InfiniBand 3.3  Copper cable Fiber cable Box-to-Box Box-to-Box 1 Gb Ethernet (1000BASE-CX) 1 Gb Ethernet (1000BASE-SX/LX) 10 Gb Ethernet (10GBASE-CX4) 10 Gb Ethernet (10GBASE-SR/LX4) Fibre Channel Fibre Channel InfiniBand InfiniBand ASIC  There are several ASIC manufacturer with SerDes transceivers, one is Texas Instrument with a wide range of SerDes devices. • General Purpose – TLK1501/TLK2501/TLK3101 .. • Gigabit Ethernet/Fibre Channel – TLK1201/TLK1221/TLK2201 .. • 10 Gigabit Ethernet (XAUI) – TLK3118,TLK3138 .. and the new four-channel multi-rate/function/protocol SerDes transceiver TLK3134 with integrated reference clock jitter cleaner. • LVDS SerDes - SN65LV1023/SN75LVDT1422 .. Peter Bergqvist  2009‐04‐06 17.06  9 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  An alternative is to use Altera HardCopy IV GX ASIC, a process which offers transition of the FPGA design to a ASIC design. The prototype design work is done with the Stratix IV GX FPGA and then seamlessly migrate to HardCopy IV GX ASIC with the same Altera Quartus II design software. 3.4  FPGA  There are three major FPGA manufacturers that have integrated high-speed serial transceivers, also known as SerDes, Multi-Gigabit Transceiver MGT, RocketIO or RocketIO X/MGT/GTP/GTX. • Altera Arria GX, Arria II GX, Stratix GX, Stratix II GX, Stratix IV GX/GT and HardCopy IV GX ASIC (A process which offers transition of the FPGA design to a ASIC design). • Xilinx Spartan-6 LXT, Virtex-II Pro/Pro X, Virtex-4 FX, Virtex-5 LXT/SXT/FXT/TXT and Virtex-6 LXT/SXT. • Lattice LatticeSC/SCM, LatticeECP2M/ECP3, ORCA ORT82G5/42G5, ORCA ORSO82G5/42G5 and ORCA ORSPI4. Altera and the FPGA founder Xilinx are the leading manufacturers concerning high-speed serial I/O with transfer rates between 155 Mbit/s to 11.3 Gbit/s and theirs transceivers are rather complex because they support several interconnect standards e.g. 1 and 10 Gigabit Ethernet. PCI Express 1.x and 2.0. Serial RapidIO. InfiniBand 1, 2, 4 and 8 Gigabit Fibre Channel. Aurora. SerialLite II. They also provides other preverified and predefined Intellectual Property (IP) core functions like e.g encoders/decoders, counters, Finite Impulse Response FIR-filter, Digital Signal Processor DSP or Fast Fourier Transform FFT. Peter Bergqvist  2009‐04‐06 17.06  10 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency  3.5          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  Clock distribution  Clock Distribution Circuits (CDC) are divided into three main groups. • Clock synthesizers - Basically consist of an external crystal (X or XTAL) and an internal oscillator circuit. The crystal provides a stable and accurate reference frequency (typical 8-32 MHz) to the oscillator circuit. Additional Phase-Locked Loops (PLLs) are used to allow easy multiplication/division of the oscillator frequency. • Clock buffers – Distribute and copy a clock signal from one input to several outputs. This circuit can handle the conversion between different supply voltage levels e.g. 3.3V to 1.8V or between different signaling standards e.g. single ended to differential ended. Clock buffers often have a built-in PLL (also known as Zero Delay Buffer) that allows the phase of the output clocks to adjusted with reference to the input clock i.e. minimize the phase displacement between the output clocks (skew). • Jitter cleaners – Basically consist of an external high performance crystal oscillator e.g. Voltage Controlled Crystal Oscillator VCXO and a built-in PLL. Jitter is an unwanted phase displacement of the real clock in reference to an ideal clock. If a clock signal is sent across a Printed Circuit Board (PCB) noise is added, which creates jitter compared to the clock source (clock synthesizer). System components like Analog-to-Digital Converters (ADCs), Serializer/Deserializer (SerDes) or Digital Signal Processors (DSPs) require a very accurate clock signal with almost no jitter (e.g. peakto-peak jitter < 40 ps). If such components are located a long way from the clock source or if the clock source is not clean enough then a jitter cleaner can be used to remove the phase displacement. 4  Optical devices  4.1  Fiber cables  Multi- and singlemode fiber MMF respective SMF are the two types of fiber in common use. Both fibers has a 125 µm outside cladding diameter, but multimode fiber has two standard core diameter 50 µm and 62.5 µm and for the singlemode fiber the core diameter is between 8 µm to 10 µm. Peter Bergqvist  2009‐04‐06 17.06  11 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  Multimode fiber support data rates from 10 Mbit/s up to 10 Gbit/s for distances up to 600 meters, with standards in development to support data rates up to 100 Gbit/s. Multimode fiber are classified in groups by the International Organization for Standardization and the International Electrotechnical Commission ISO/IEC 11801, which is based on the OverFilled Launch OFL bandwidth, in practice it refer to the 50 µm and 62.5 µm core diameter. • OM1 - 62.5/125 µm multimode fiber, found in legacy systems. • OM2 - Standard 50/125 µm multimode fiber. • OM3 - LOMMF Laser optimized 50/125 µm multimode fiber is designed to use with 850 nm Vertical Cavity Surface Emitting Laser VCSEL, and recommends by the Fiber Optics LAN Section FOLS. Singlemode fiber support data rates from 10 Mbit/s up to 10 Gbit/s for distances over 60 kilometers with e.g. XENPAK optical transceivers. Singlemode fiber is more expensive to use than multimode fiber because it use more expensive devices e.g. laser emitting diode in the transmitters. Singlemode fiber has also extremely low signal attenuation, but it requires more care and expertise to avoid signal loss in the connectors. 4.2  Fiber connectors  There are various types of fiber connectors available on the market e.g. the almost Swedish industry standard SC and LC connectors. • ST/BFOC (Straight Tip/Bayonet Fiber Optic Connector), a old bayonet type connector. ST connectors are among the most commonly used connectors in networking applications, second choice in Swedish Standards Institute SS-EN50173. Can be used with multi- and singlemode fiber and available in both simplex and duplex configurations. • SC (Subscriber Connector), a push-pull type connector. SC connectors are used for newer network applications, first choice in Swedish Standards Institute SS-EN50173. Can be used with multi- and singlemode fiber and available in both simplex and duplex configurations. • MU (also known as mini-SC), a push-pull Small Form-Factor SC connector, but half the size. MU connectors are popular in Japan. Can be used with multi- and singlemode fiber and available in both simplex and duplex configurations. Peter Bergqvist  2009‐04‐06 17.06  12 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  • MT-RJ (Mechanical Transfer-Registerad Jack), a snap Small Form-Factor duplex connector, half the size of the SC connector. Can be used with multi- and singlemode fiber, but only available in duplex configuration. • LC (Lucent Connector), a snap Small Form-Factor connector, half the size of the SC or FC connectors. LC connectors are often used in SFP optical transceivers. Can be used with multi- and singlemode fiber and available in both simplex and duplex configurations. • E-2000 (also known as LSH), a snap type connector like a LC connector but with a protective cover over the end of the fiber. The cover automatically opens and closes as the connector is connected and disconnected. Can be used with multi- and singlemode fiber and available in both simplex and duplex configurations. • LX-5, a snap Small Form-Factor connector like E-2000, but half the size of the SC or E-2000 connectors. Can be used with multi- and singlemode fiber and available in both simplex and duplex configurations. • FC (Ferrule Connector), a old screw thread connector with high mechanical resistance and were designed for use in high-vibration enviroments, but are nowadays replaced by SC and LC connectors. Can be used with multi- and singlemode fiber and available in both simplex and duplex configurations. 4.3  Optical transceivers  A optical transceiver module is a transmitter/receiver device for multimode fiber (MMF) or singlemode fiber (SMF). The module is pluggable and hot swappable and using different electrical interface e.g. XAUI, XFI. Many GBIC/SFP/SFF/XFP modules often include a powerful digital diagnostics tool (according to SFF-8472 standard) for monitoring the optical interface and to set alarm and warning thresholds for e.g. TX/RX power, voltage and temperature. • GBIC (GigaBit Interface Converter), bidirectional optical serial link with signaling rates up to 1.25 Gbit/s. Hot swappable edge-card electrical interface. Support following interconnect standards. 1 GbE at 1.25 Gbit/s. 1 GFC at 1.0625 Gbit/s. Peter Bergqvist  2009‐04‐06 17.06  13 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  • SFP (Small Form-factor Pluggable), bidirectional optical serial link with signaling rates up to 4.25 Gbit/s. SFP is a upgraded GBIC with smaller size and for that reason also known as mini-GBIC. The SFP is also upgraded to SFP+ which support signaling rates up to 12.5 Gbit/s. Hot swappable edge-card electrical interface e.g. parallel XAUI and serial CML (Current Mode Logic). SFP support following interconnect standards. SONET/SDH. 1 GbE at 1.25 Gbit/s. 1, 2 and 4 GFC at 1.0625, 2.125 respective 4.25 Gbit/s. SFP+ support following interconnect standards. SONET/SDH. 10 GbE at 12.5 Gbit/s. 8 GFC at 8.5 Gbit/s. • SFF (Small Form-Factor), similar to the SFP but is soldered to the board as a pin through-hole device, instead of using an edge-card socket. • QSFP (Quad Small Form-factor Pluggable), four bidirectional optical serial lane with signaling rates up to 10.3125 Gbit/s per lane which give an aggregate bandwith of 41.25 Gbit/s per link. SFP-like electrical interface. Support following interconnect standards. SONET/SDH. 2, 4, 8 and 10 GFC at 2.125, 4.25, 8.5 respective 10.51875 Gbit/s. 10 GbE with XAUI at 4x3.125 Gbit/s. 4X SDR/DDR/QDR InfiniBand at 4x2.5, 4x5.0 respective 4x10.0 Gbit/s. x4 PCIe at 4x8.0 Gbit/s. • XENPAK, bidirectional optical serial link with signaling rates up to 10.3125 Gbit/s. Parallel electrical interface XAUI. Support following interconnect standards. 10 GbE at 10.3125 Gbit/s. • X2, bidirectional optical serial link with signaling rates up to 10.51875 Gbit/s. X2 is based on XENPAK, but is smaller in size. Parallel electrical interface XAUI and OIF SFI-4 phase2. Support following interconnect standards. SONET/SDH with OIF SFI-4 phase2. 10 GbE at 10,3125 Gbit/s. 10 GFC at 10,51875 Gbit/s (exactly 2% higher clock rate than 10 GbE). Peter Bergqvist  2009‐04‐06 17.06  14 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  • XPAK, bidirectional optical serial link with signaling rates up to 10.51875 Gbit/s. Parallel electrical interface XAUI and OIF SFI-4 phase2. Support following interconnect standards. SONET/SDH with OIF SFI-4 phase2. 10 GbE at 10,3125 Gbit/s. 10 GFC at 10,51875 Gbit/s. • XFP (10-Gigabit Small Form-factor Pluggable), bidirectional optical serial link with signaling rates up to 11,09 Gbit/s. XFP is designed after XENPAK, X2, XPAK (and also SFP) and is the next generation of 10Gigabit optical transceivers. Serial electrical interface XFI. Support following interconnect standards. SONET/SDH. 10 GbE at 10,3125 Gbit/s. 10 GFC at 10,51875 Gbit/s. 4.4  Differential serial electrical interface  Common types of differential serial electrical interface for the above mentioned optical transceivers. • LVPECL (Low Voltage Positive Emitter Coupled Logic), High-speed differential interface, designed for low jitter, skew and signaling rates between 600 Mbit/s to 3.125 Gbit/s. Both output and input require termination resistors. • CML ( Current Mode Logic), High-speed differential interface, support signaling rates between 600 Mbit/s to 2.5 Gbit/s. The NMOS output transistor can only drive falling edges, require pullup resistors to drive rising edges and termination at the input. • VML (Voltage Mode Logic), High-speed differential interface, support signaling rates between 500 Mbit/s to 3.125 Gbit/s. The PMOS and NMOS output transistors can drive both rising and falling edges, no requirement for pullup resistors only termination at the input. • LVDS (Low Voltage Differential Signaling). Low power, support signaling rates between 100 Mbit/s to 1.4 Gbit/s. Peter Bergqvist  2009‐04‐06 17.06  15 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  • XAUI (10-Gigabit Attachment Unit Interface), XAUI is a full duplex parallel electrical interface, using four parallel lanes of self-clocked serial differential links in each direction with 8B/10B encoding to create/extract the embedded clock. Each lane operates at 3,125 Gbit/s for 10 Gigabit Ethernet or 3,1875 Gbit/s for 10 Gigabit Fibre Channel. • XFI (10-Gigabit Serial Electrical Interface), XFI is a full duplex serial electrical interface, using a single lane of self-clocked serial differential links in each direction with 64B/66B encoding to create/extract the embedded clock. The single lane operates at 10.3125 Gbit/s for 10 Gigabit Ethernet, but operates up to 11,09 Gbit/s for other standards. Peter Bergqvist  2009‐04‐06 17.06  16 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency  5          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  Demonstrator Array  The demonstrator array is divided into three subarrays of 4x4 antenna elements, on the next page you can see a overview of one subarray. The 1.5 Gbit/s full duplex optical serial links, one for each polarisations, is based on standard 50/125 µm multimode fiber (OM2) with a equal length of 58.0 meters. Both the SFF and SFP optical transceivers supports high-speed serial links over LC connectors and 50/125 µm multimode fiber at signaling rates up to 2.125 Gbit/s and link length of up to 300 meters. The Remote VHF RX Front End and the Digital Down Converter DDC are controlled and configured via USB interface. An USB extender is used to handle the long distance, because the maximum length of standard USB is only 5.0 meters. The USB extender supports full speed 12 Mbit/s USB 1.1 over MTRJ connectors and 50/125 µm multimode fiber. When the new European next generation VHF Incoherent Scatter (IS) radar and interferometry receiver system are build following items should be taken under consideration. • Cable labeling – Use colour coaxial cables and connectors for the antenna elements, due to the large numbers of antenna elements and the dual polarisations. Furthermore is neccessary to have a good general cable tagging. • Fiber connectors – Avoid non standard fiber connectors like the MT-RJ connector, due to manufacturing and delivery problems. • Clock distribution – Sensitive components like ADC, SerDes or DSP require a very accurate clock signal with almost no jitter e.g. the SerDes TLK1501 require an input reference clock with maximum peak-to-peak jitter of 40 ps and duty cycle between 40% to 60%, for that reason a highperformance Clock Distribution Circuits (CDCs) are necessary. • System requirement – Oversize the design, in regard to the numbers of fiber cables and also to the serial link signaling rate requirement, to prevent major infrastructure changes at last-minute design changes. Peter Bergqvist  2009‐04‐06 17.06  17 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  4x4 antenna elements with  2 polarisations  Remote VHF RX Front End   LNA ‐ BP filter ‐ 2nd AMP ‐ ADC  16‐bit electrical parallels 12 Mbit/s USB 1.1 Digital Down Converter DDC   Intersil ISL5416 ASIC  16‐bit electrical parallels Frame Count Logic   Altera MAX II CPLD  USB extender   Icron USB Ranger 422i  Remote EXtender REX  (4 x USB type A)  16‐ bit electrical parallels SerDes with  8B/10B encoding    Texas Instruments TLK1501 ASIC    SFF Optical Transceiver  50/125 µm MMF (OM2) with  MT‐RJ connectors   Avago AFBR‐5921ALZ  1.5 Gbit/s optical serial links 50/125 µm MMF (OM2)  with  LC connectors  SFP Optical Transceiver   Avago AFBR‐57M5APZ  USB extender   RocketIO GTP Transceiver  Digital Beamformer  Icron USB Ranger 422i  Local EXtender LEX  (1 x USB type B)     Xilinx Virtex‐5 SXT FPGA Peter Bergqvist  2009‐04‐06 17.06  18 (19)  EISCAT_3D ‐ Work Package 12  Networking and  Reference Time and  Frequency  6          Deliverable D12.2  Array Networking and  Communication Report  D12.2_Report_090406.docx  Glossary  SMF MMF LOMMF Singlemode fiber, typical 9/125 µm. 62.5/125 µm or standard 50/125 µm multimode fiber. Laser optimized 50/125 µm multimode fiber for use with 850 nm Vertical Cavity Surface Emitting Laser VCSEL. Lane Set of differential pairs, one pair for transmit TX and one pair for receive RX. Communication path between two devices. An xN link is composed of N lanes i.e. x4 link is composed of 4 lanes. Link XAUI 10-Gigabit Attachment Unit Interface. XAUI is a full duplex parallel electrical interface, using four parallel lanes of self-clocked serial differential links in each direction with 8B/10B encoding to create/extract the embedded clock. Each lane operates at 3,125 Gbit/s for 10 Gigabit Ethernet or 3,1875 Gbit/s for 10 Gigabit Fibre Channel. XFI 10-Gigabit Serial Electrical Interface. XFI is a full duplex serial electrical interface, using a single lane of self-clocked serial differential links in each direction with 64B/66B encoding to create/extract the embedded clock. The single lane operates at 10.3125 Gbit/s for 10 Gigabit Ethernet and operates up to 11,09 Gbit/s for other standards. OIF SONET SDH Optical Internetworking Forum. Synchronous Optical Network, is used in USA. Synchronous Digital Hierarchy, is used in Europe. OIF SFI-4 phase 2 SerDes Framer Interface Level 4 phase 2, defines four parallel lanes of serial differential links in each direction. Each differential link operates at 2,488 Gbit/s for SONET/SDH. GbE GFC Gigabit Ethernet. Gigabit Fibre Channel. Peter Bergqvist  2009‐04‐06 17.06  19 (19)