100 Tb/s Aggregate Capacity Router Using An Optical Switching Core

Transcript

100 Tb/s Aggregate Capacity Router using an Optical Switching Core Prof. Mohammed N. Islam Department of Electrical Engineering and Computer Science University of Michigan, Ann Arbor 1 OUTLINE I. II. III. IV. V. Motivation and Router Background Broadcast & Select Optical Switching Core Sub-system Issues with B&S Architecture Key Enabling Technologies (Components) Summary 2 100000% Relative preformance increase DWDM Link speed x2/8 months 10000% 1000% Internet x2/yr Router capacity x2.2/18 months Moore’s law x2/18 m DRAM access rate x1.1/18 m 100% 1996 1998 2000 2002 3 Generic Router Architecture Data Hdr Header Processing Lookup IP Address Update Header Header Processing Lookup IP Address Update Header Address Address Table Table Data Hdr Buffer Data MemoryHdr Memory Header Processing Lookup IP Address Buffer Manager Buffer Address Address Table Table Data Hdr Data Hdr Buffer Buffer Memory Memory Address Address Table Table Data Hdr Buffer Manager Update Header Buffer Manager Buffer Buffer Memory Memory 4 First Generation Routers Shared Backplane Li CP n U In e te rf ac M e em or y CPU Route Table Buffer Memory Line Interface Line Interface Line Interface MAC MAC MAC Typically <0.5Gb/s aggregate capacity Limitation: buffer memory 5 Second Generation Routers CPU Route Table Buffer Memory Line Card Line Card Line Card Buffer Memory Buffer Memory Fwding Cache Buffer Memory Fwding Cache MAC MAC Fwding Cache MAC Typically <5Gb/s aggregate capacity Limitation: bus interconnection 6 Third Generation Routers Switched Backplane Li CPIn ne Ute rf ac M em e or y Line Card CPU Card Line Card Local Buffer Memory Routing Table Local Buffer Memory Fwding Table Fwding Table MAC MAC Typically <160Gb/s aggregate capacity Limitation: number of LC’s in a rack 7 4th Generation Routers/Switches Optics inside a router for the first time Optical links 100s of metres Switch Core Linecards 0.3 - 10Tb/s routers in development 8 Fundamental Problems • We are designing for 100Tb/s next generation • Fundamental Problems – Switching Core » Electronic switch requires a rack for 2Tb/s switching core » Size, power dissipation and cost prohibitive for 100Tb/s – Scheduler Algorithm » Algorithm typically grows as some power of N, as much as N^3 » Takes more and more time to compute algorithm as N gets larger 9 100Tb/s Routers using DWDM? • Reasonable bit rate: 40Gb/s • Spectral efficiency: 4 b/s/Hz (using sophisticated coding, super-FEC, good filtering) • Telecom window: 1400-1650nm (250nm) • 100Tb/s = 40Gb/s x 2500 channels • For channel spacing 12.5GHz (0.1nm), telecom window has 2500 channels • Bottom line: doable, but tough 10 Goals of Design • Low cost, low power dissipation, small size – Lowest cost using tunable filters – Capacitive devices – Packaging or array of devices • Combine interconnect with switching fabric – Fiber and star couplers • Functionality of OXC and Router – Same DWDM on outside and inside • Minimal impact on line cards – Tunable filters in the common bay equipment • Hide switching time of devices – Use architectural tricks to use slower, cheaper devices 11 5th Generation Routers/Switches Switching Fabric and Fiber Interconnection Combined Star Coupler Fiber Interconnect No Intermediate O/E/O for Interconnect Linecards 12 Broadcast & Select Switching Fabric Line Line Card Line Card Card 1 l1 2 Line Card 1 l2 2 Star Tunable Optical Filter (TOF) 1 ln Line Core Card 2 Line Card Tuning Schedule TOF 1 Tune TOF 1 Tune TOF 1 Tx TOF 2 Tune TOF 2 Tx TOF 1 Tx TOF 2 Tune TOF 2 Tx 13 Multi-Channel Fast Tunable Filters 0.55 Filter Array 0.55 14 Filters in Common Bay Equipment Optical Switch Power Supplies Midplane Control Boards 15 Advantages of B&S Switching Fabric • • • • • • Hitless reconfiguration and hide switching time Multicast & Broadcast traffic without copies OXC/Router functionality combined Passive, transparent switching fabric scalable LC’s minimally impacted because tunable filters and circuits in common bay equipment RU Low-cost approach – Power splitters/taps cheaper than WDM – Transmitters can be low-cost EAM/DFB lasers – All but fast tunable filters are off-the-shelf 16 Comparing with Tunable Lasers • • • • Significant space on Line Card! Switching time ~30-100nsec minimum External modulators required (~3-4GHz bw) Bottom Line: tunable filters much cheaper [ Lucent, IEEE PTL, July 2001 ] 17 Issues for B&S Switching Core • Scalability using amplifiers – Overcoming 1/N loss using WDM and amplifiers • Emulating fast switching time using inexpensive devices – Speed-up, aggregation, ping-ponging between filters • Simplifying scheduler for large N – Two stage switch using inexpensive switching fabric • Continuum source for >100 channels of 40Gb/s – One expensive laser and SC copies to >100nm wavelengths 18 Scalability Issues • Inherently have a 1/N loss with Star Coupler • For 2500 channels, loss of 34dB • If use multiple filters per LC, then 3-6dB additional loss • By way of reference, a typical optical link will have a budget of about 30dB (~22dB loss budget and another ~8dB for OADM/DC, etc) 19 Broadcast & Select Switching Fabric Line 1 Card Line Card Line l1 Card 2 Line 1 l2 ln 2 W D M Tunable Optical Filter (TOF) 1 Line Card Tuning Schedule Line 2 By-pass traffic Card By-pass traffic Core TOF 1 Tune Card TOF 1 Tune TOF 1 Tx TOF 2 Tune TOF 2 Tx TOF 1 Tx TOF 2 Tune TOF 2 Tx 20 Emulating fast switching using low-cost components • 50-100ns devices cost << than sub-ns devices • Speed-up • Aggregation and superframes • Ping-pong between different filters 50 ns 50 ns 25 ns 1 ms … Rx Rx 21 Simplified Scheduler with 2Stages • Simple round-robin scheduler in two sections • Middle stage may have additional memory to avoid mis-sequencing of packets • 2x switching fabric, so fabric must be low cost! • Questions to scheduler experts: – Does the switching fabric need to reconfigure every cell cycle (in which case our approach is important) – Are LC’s needed in all three locations, or can we get away with two (in which case fabric is simplified) 22 Two-Stage Switch External Inputs Internal Inputs External Outputs 1 1 N N 1 N Load Balancing First Round-Robin Second Round-Robin Switch gives 100% throughput for non-uniform, bursty traffic, without a scheduler or speedup! [Nick McKeown, Stanford University, Opticomm 2001] 23 Exemplary 2-stage Switching Fabrics LC#1 LC#1 LC#2 LC#2 LC#2 LC#N Power LC#N LC#1 LC#1 LC#2 LC#2 LC#N Star … Power combiner LC#N … … splitter Star … Star … … Input fibers LC#1 LC#N 24 Cost Estimate for 1Tb/s Core (2004) Tx Tx • • • • • • • • • Tx l1 l2 l3 W Assume 100 ch @ OC-192 D Transmitter: $1.7k x 100 = 170K M WDM: $0.1Kx100 = 10K Amplifier: = 50K lN 1xN splitter: $0.05Kx100 = 5K Tx Tunable filter: $0.1k x 100 =10K Total: $245K Connectors, electronics, cases, software, extra Cost < $200K if we use broadcast & select without amplifier 25 Laser Sources in Large Router • Many LD’s become expensive • If channel spacing is close, then stabilizing wavelengths and maintaining channel spacing difficult and expensive • For 40Gb/s (or 160Gb/s) per channel, light source can be expensive • For many LD wavelengths, many part #’s and have to match LD wavelength per LC • With 40Gb/s sources and stabilization circuits, significant space on LC will be used 26 Supercontinuum (SC) Source • • • • • One modelocked source and a common SC set-up Channel spacing set by passive WDM demux, which is used to carve out channels from SC For 40Gb/s (or muxed to 160Gb/s), only one expensive ML source required. SC copies to many wavelengths Each LC can have a modulator, but individual LD’s are not required. If modulator broadband, few part #’s ML laser and SC set-up will be in common bay equipment. Only modulator placed on LC BOTTOM LINE: SC less expensive for #l’s >100 and 40Gb/s per channel or higher 27 Exemplary System ML 40 Gb/s source TDM Continuum MUX Fiber WDM Common bay equipment • • • LC#1 Mod l1 LC#2 Mod l2 Mod lN … SC source can all be placed in common equipment bay Modulator placed on line card WDM can be replaced by power splitters and fixed or tunable filters LC#N 28 SC Experimental Setup Diagnostics Mode Locked Ring Cavity EDFL 15 dB EDFA Dt = 400 fs SC Generation P0 Pulse Chirping (2m SMF-28) Optical Spectrum Analyzer L (D) Autocorrelator L: only 2 meters long 29 0 Experimental Parameters L lo -10 = 2 [m] = 1539 [nm] Pavg = 1100 [W] D = 1.13 [ps/nm-km] -20 dB Bandwidth: 211 nm 1450 1500 1550 1600 -20 Optical Power [dB] Exemplary SC Spectrum 1650 Wavelength [nm] 30 SC Spectral Flatness ±0.5 dB maximum power fluctuation over 61 nm 2.5 ± 0.1 dB power fluctuation over 35 nm 0.0 35 nm ± 0.1 dB -2.5 1485 1500 1515 -5.0 61 nm -7.5 ± 0.5 dB -10.0 1450 1475 1500 1525 1550 31 Coherence of Carved SC Spectrum Autocorrelation SC Fiber Compensation Fiber 2 1 3 Intensity [AU] STD Fiber 1.0 0.8 0.6 0.4 0.2 Laser EDFA OBF 0.0 -4 0 2 4 Time [psec] Spectrum -10 Optical Power [dBm] Filtered Pulse t ~ 1 ps Compensated Pulse t ~ 0.5 ps -2 -20 -30 • Pulse width tFWHM £ 500 fsec can be carved from SC for high-speed TDM applications -40 -50 -60 -70 1450 1500 1550 1600 1650 Wavelength [nm] 1700 32 Timing Jitter of Source and Filtered SC Output Carved Spectrum Laser Output Time Jitter [sec] Timing Jitter [psec] 1E-10 1E-11 10 1E-12 1.0 Measurement Parameters 5 KHz Span 30 Hz Resolution 430 th Harmonic of laser fundamental 1E-13 0.1 1E-14 0.01 1E-15 10 100 1000 Frequency [Hz] • Pulses carved from flat section of continuum have same timing jitter as the source • Short SC fiber length minimizes additional timing jitter 33 Key Enabling Technologies (Components) • Broadband Amplifiers • Tunable Filters • Surface Normal Modulators 34 All-Raman Broadband Amps Line fiber A B In • • • • Gain fiber GFF Distributed Pump Module Lumped Pump Module Gain fiber Lumped Pump Module C Out Very low MPI level Dispersion and slope compensation provided by the gain fiber Large gain & low NF over a 100nm continuous spectral window Demonstrated the transmission feasibility of 240 OC-192 channels over > 1500km SSMF 35 +24dBm Launch Power Flat input spectrum Output spectrum after 8 spans (DSE output) Line amplifier flat operation over 100nm demonstrated at +24dBm output power (corresponds to 0dBm/ch for 240 channels) 36 Electro-optic Tunable Filter Filter X-Section • Optical cavity with electro-optic material Cavities Mirrors – Tune filter by voltage induced index changes of EO material Substrate Filter characteristics 1-Cavity Filter Transmission (%) 100 80 3-Cavity Filter -1 dB BW 25GHz 25 GHZ -30 dB BW 625 GHz 100 GHz In-Band Ripple <0.25 dB < 0.25 dB 60 40 20 0 1549.4 1549.6 1549.8 1550.0 1550.2 1550.4 1550.6 Wavelength (nm) 37 Pass-band shape vs. number of cavities Filter Type Fabry Perot Narrow Band Wide Band Filter Shape Lorentzian Square Square # Channels 1 channel 1 channel 4-16 channels 1 cavity 3 cavities 8-10 cavities Channel Monitor OADM, MUX/DeMux OADM # Cavities 100 80 60 100 40 Transmission (%) 80 20 60 0 1549.5 1550.0 Wavelength (nm) 1550.5 40 0 20 0 1549.4 1549.6 1549.8 1550.0 1550.2 1550.4 1550.6 Wavelength (nm) Transmission (dB) Transmission (%) Application -10 -20 -30 -40 -50 -60 1548 1550 1552 1554 Wavelength (nm) 1556 38 1558 3-cavity filter tuning characteristics 0.00 0 Transmission (dB) Transmission (dB) 0.5 MV/cm 0.0 MV/cm -0.5 MV/cm -1.0 MV/cm -0.03 ~15 nm -20 -0.06 -40 1535 1540 1545 1550 1555 Wavelength (nm) • • 1560 1565 1560.00 1560.08 1560.16 Wavelength (nm) A tuning range of 15 nm is obtained with electric fields of –1 MV/cm to 0.5 MV/cm across each cavity The pass-band ripple increases near the extreme ends of the tuning range – – The transmission ripple remains less than 0.1 dB in the 15 nm tuning range The group delay ripple increases by ~ 1 ps 39 Alternate Approach– 1D MEMS simple piston up-down motion • Combine optical functionality with well-known electro-static actuators • Challenge: combining MEMS actuator growth with optical coating technology 40 FIMS– Fast Interferometric MEMS Switch • High speed achieved because – Interferometer: maximum displacement l/4 – Stress: make a tight guitar string – 1D: simple 1D motion with simple control • High reliability expected – Small motion without hinges – Electro-static actuators well-proven technology • Low cost – Simple packaging – Standard processing steps with many devices on a wafer 41 Predicted Performance of FPI High speed while maintaining optical performance and low voltage Parameter Value Tuning Speed 60 ns mechanical, 100 ns electrical Tuning voltage 40 V max -3 dB bandwidth -30 dB bandwidth 0.1 nm 1.5 nm Channel Selectivity 12.5 Ghz (0.1nm) Tuning range 100 nm Finesse 6300 Insertion loss 1-3 dB PDL < 0.1 dB 42 Surface Normal Modulator +DV -DV FSR Transmission (dB) Source array -20 -40 1549.2 1549.6 1550.0 1550.4 Surface-normal modulator array 0 1550.8 Data p encoder Tunable filter functions as a surface normal modulator Wavelength (nm) • – Operates on single channel/frequency to encode data on cw light » Filter is tuned in and out of channel frequency band to create high and low signal states – Sharp transition from high to low transmission » Does not require p phase shift for high contrast – High-speed » Fast EO response + low capacitance • Surface normal configuration is advantageous for building transmitter array – Source array: Laser diode array, LED array or broadband light source with optical filter array 43 Summary • Routers expected to be bottleneck in future systems • 100Tb/s router project currently at UM – Strawman system design to understand limiting technologies – Limitations from switching fabric and scheduler – Broadcast & Select architecture using DWDM technology • Sub-system Issues of B&S Architecture – – – – Scalability using Broadband Amplifiers & WDM Hide switching time by ping-ponging between filters Two-stage switch for simplified scheduler Broadband continuum source to simplify transmitters • Key Enabling Technologies – Broadband Amplifiers are commercially available – Fast tunable filter » Two approaches: Electro-optic thin films and 1-D MEMS – Surface Normal Modulators can be made with fast filters 44