Transcript
FPGAs Customize Your Video Solutions Franco Leung
Business Manager, Asia Pacific Programmable Solutions Group (PSG), Intel Corporation
Copyright © Intel Corporation. All rights reserved. Intel Corporation, 2200 Mission College Blvd., Santa Clara, CA 95052-8119, USA
Video Industry Trends
4K Ultra HD HDR
8K?
Altera video IP technology is deployed in 6 market segments today!
2
Per-market FPGA Video Use Cases
FPGA Automotive IVI Video Applications
Atom + Cyclone V synergy LCD
Scale + Select
B
A
Interface Bridge & Timing Controller (LVDS & miniLVDS)
B R R
LCD
Video MUX with Scaling
ISP IO HPS
MAC Decode
CPU LCD
Early Video / Off-loaded (AVB, BT656) CAN
Gesture Recognition (RGB, CAN) 5
B
DES Interleave TCON A
Video MUX & Format Conversion (YUV & eDP)
Navi
A
miniLVDS, DDR
Switch + Format
Dual LVDS 4D+C
CPU
Professional-AV Example Applications
6
Arria 10: UHD & beyond Cyclone V: legacy
Some Atom and Core synergy opps.
Video Conferencing / Unified Communications
Display and Projection
AV Networking
Media Processing
Professional-AV Example Applications
7
Arria 10: UHD & beyond Cyclone V: legacy
Some Atom and Core synergy opps.
Studio
Use Cases
8
Arria 10 today
Stratix 10 next
Synergy with Xeon for server/cloud compute
Acquisition
Distribution & Contribution
Production
Consumption
Studio
Use Cases
9
Arria 10 today
Stratix 10 next
Synergy with Xeon for server/cloud compute
Medical Apply range of FPGA families to meet scalable requirements
10
Diagnostic Imaging (CT, MRI, PET…)
AV Networking (similar to requirement in Pro-AV)
Life Sciences (e.g. Endoscopy)
Hardware Acceleration (part video processing)
Industrial Low-end to mid-range FPGA families
11
Atom synergy potential
Video Surveillance (including WDR, vision…)
Single-chip Camera SoC (vision continued...)
Machine Vision (image processing, frame store…)
Internet of Things (many emerging exciting IOT video applications for FPGA)
Visit Altera Business Specific Landing Pages for More Insights… • https://www.altera.com/solutions/industry.html
12
Altera Video IP Solutions
Altera Video Framework
“PLUG & PLAY SYSTEM” Connectivity Support
DDR4
VGA
DeInterlacer
Custom
Scaler FIR
H.265 SMPTE 2022
Custom
14
TICO
Custom
Gamma Corrector
TR-03 Chroma Re-sampler
VGA
PCIe
H.264
HDMI / DVI
CVO
Switch
HDMI / DVI
JPEG2K
Alpha Blending Mixer
DP
DP
Color plane Sequencer
Video over IP Support
Design Modularity
CVI
SDI
DDR
• DisplayPort 1.3 / 12G SDI • HDMI 2.0a / DVI • PCIe, DDR, GbE • And much more…
• Ethernet MAC/PHY • 1588 timing (incl. SMPTE 2059) • SMPTE 2022, TR-03, eAVB • Other protocols available
Video Compression • H.265, H.264 • JPEG2000 • TICO
Video Processing Suite • 28 Highly Configurable Building Block IP for Fast time to Market • 4K ready
Differentiate your Product • Qsys Productivity Tool • Create custom IP blocks
Altera Video Framework
“PLUG & PLAY SYSTEM” Connectivity Support
DDR4
VGA
DeInterlacer
Custom
Scaler FIR
H.265 SMPTE 2022
Custom
15
TICO
Custom
Gamma Corrector
TR-03 Chroma Re-sampler
VGA
PCIe
H.264
HDMI / DVI
CVO
Switch
HDMI / DVI
JPEG2K
Alpha Blending Mixer
DP
DP
Color plane Sequencer
Video over IP Support
Design Modularity
CVI
SDI
DDR
• DisplayPort 1.3 / 12G SDI • HDMI 2.0a / DVI • PCIe, DDR, GbE • And much more…
• Ethernet MAC/PHY • 1588 timing (incl. SMPTE 2059) • SMPTE 2022, TR-03, eAVB • Other protocols available
Video Compression • H.265, H.264 • JPEG2000 • TICO
Video Processing Suite • 28 Highly Configurable Building Block IP for Fast time to Market • 4K ready
Differentiate your Product • Qsys Productivity Tool • Create custom IP blocks
Connectivity IP
Connectivity IP
Enabling 4K Video Processing and Beyond
Avalon ST Video
1080p60 ~ 3G
4Kp60 ~ 12G
8kp60 ~ 48G
1 pixel x clk at 148.5 MHz
4 pixels clk at 148.5 MHz 2 pixels clk at 297 MHz 1 pixel at 594 MHz
4 pixels clk at 594 MHz
Scalability 16
FPGA Value Proposition vs ASSP: Video Connectivity
FPGA
ASSP
Flexible lane number configuration, color depth, data mapping and video resolutions
• Dual mode not available
Adaption between video interface standards Proven high speed transceiver technology
I/O banks support range of I/O standards
Fast adaption of newer features and standards
Integration with Altera’s VIP suite and other IO protocols Multiple source / sink devices integration in single FPGA with single or dual mode 17
• High volume, short lifespan consumer devices • Fixed link rate, lane count, color depth and video resolution
Video Processing
Video and Image Processing IP Suite Collection of highly configurable MegaCore® functions that you can use to facilitate the development of custom video and image processing designs suitable for a wide range of applications.
19
Scalingin480i Picture Picture 1080p60 +toBackground
Background
Altera Video IP Suite Functions • Suite of 27 video processing Megacores
Developed by in-house system solution team
• Highly parameterisable IPs
Video resolutions from SD to UHD • Optimum resource count (bus width adapted to suite maximum resolution)
Performance vs. size trade-off, for example: • Bi-linear vs. Polyphase edge sharpened scaling • Bob/weave vs. motion adaptive de-interlacing
Run-time control (e.g. custom coefficients)
• Clocked video IPs support any video I/O
DVI, DL-DVI, DP etc.
Embedded or external sync signals
Gen-lock for lowest system latency/no DDR
• Easy system creation/control 20
Using Qsys system builder tool
VIP Suite Hardware Verification • Trace monitor – Wireshark-like video packet inspection Includes pixel viewer for live subsampled video stream image
21
Video IP Suite Landing Page • Getting Started Videos New! 3 linked tutorials on VIP suite from Altera training partner ALSE
• Including building video design (in 21 minutes)
• New! links to latest reference designs Now available for immediate download form Altera design store
1) Basic complexity: using ALSE AVDB board
22
More details at: https://www.altera.com/vip
2) Medium complexity: using Altera Cyclone V development kit (previous favourite on Cyclone III)
DisplayPort and HDMI
DisplayPort vs HDMI
HDMI • • • •
Video + Audio on same cable Full HD, 4k60fps and 3D support Popular on consumer devices Offers Media content protection
• Users pay royalty • Controlled by HDMI consortium
27
Display Port • • • • •
Video + Audio on same cable Full HD, 4k60fps and 3D support Appearing newer Laptops/PCs Apple big proponent Offers Media content protection
• No royalty • Controlled by VESA
DisplayPort Standard
• Next generation video display interface technology Defined by VESA (Video Electronics Standards Association).
NO ROYALTY
AC coupling and low EMI 3 m cable lengths (15m for lower resolutions) and latchable DP 1.2 with HDCP 1.3 optional
28
Altera Advantage: DP Interop Testing • Interop with Unigraf, Dell, Samsung, NVidia, AMD • DP Interop with 16 different devices (by Altera) DP1.2 RX Certification passed by 2 customers on AVGX DP1.2 TX Certification approved by VESA on AVGX
DP1.1a
29
DP1.2
DP 1.3
Sink
• • •
Unigraf DPR-100 Unigraf DPA-400 (MST) Dell U2410
• • • • • • • •
Unigraf DPR-120 (MST) Unigraf DPA-400 (MST) Dell U2713H, U2913WM (2K) Dell P2715Q (4K60) Samsung U28D590 (4K60) Quantum Data 980 Philips 288P6 (4K60) Viewsonic VP2780-4K
•
Plugfest early testing
Source
•
Unigraf DPT-200
• • • • • • •
Unigraf DPT-200 Nvidia GTX660 (4K60) Nvidia GTX960 (4K60) Nvidia GTX970 (4K60) AMD HD7750 (4K60) AMD HD7700 (4K60) Quantum Data 980
•
Plugfest early testing
Altera Advantage: DP Interop Testing Multiple frame rates, resolutions, and fast switching Resolutions. RBG, YCbCr 4:2:2, 4:4:4, Unigraf source Sink: Unigraf, 4:2:0 capability Samsung, Dell 640x480 30,60 30,60 720x480 30,60,75,85 30,60 720x576 30,60 30,60 800x600 30,60 30,60 1024x768 30,60 30,60 1152x864 60 60 1280x720 30,60 30 1280x768 30,60 30,60 1280x800 30,60,75,85 30 1280x960 30,60,75 30 1280x1024 30,60,75 30 1360x768 30,60 30,60 1360x1024 30,60 30,60
30
Resolutions. RBG, YCbCr 4:2:2, 4:4:4, 4:2:0 1366x768 1440x900 1600x900 1600x1200 1680x1050 1792x1344 1800x1440 1856x1392 1920x1080 1920x1200 2048x1152 2048x1536 3840x2160
Unigraf source capability 30,60 30,60 30,60 30,60 30,60 30,60 30,60 30,60 30,60 30,60 30 30,60 30,60
Sink: Unigraf, Samsung, Dell 30,60 30,60 30 30,60 30,60 30,60 30,60 30,60 30,60 30,60 30,60 30,60
HDMI Standard • HDMI is the “go-to” for most consumer electronic vendors • Consumer usage driving other markets… New Market
HDMI 2.0 also supports… Controlled by HDMI consortium 31
• Users pay royalty 3D support HDMI 2.0: x3 Lanes at 6Gbps -> 4K60p. RGB and YCbCr modes HDCP 2.2 optional Video + Audio on same cable Scrambling mode
Deep Color
76% of Visible Colors!
Altera HDMI Solution • HDMI 2.0 with 4k60p supported on ArriaV, StratixV and Arria10 • 2 to 8 audio channels
• 16 bit deep color support • 1, 2 or 4 symbols per clock. Scalable Core across multiple families
32
Altera Advantage: HDMI Interop Testing HDMI Interop (by Altera and customers) with 27 different devices • Plugtest participation since 2012
HDMI 2.0 Sink
• • • • • • • • • •
QD980 (4k60) Dell U2713H (2K) Philips (4K30) Samsung U28D590 (4K30) Samsung UN50JS7000 (4K60) UK Samsung UE40HU6900U (4K60) Panasonic TC-58AX800U (4Kp60) UK Panasonic TX-40CX400B (4K60) Viewsonic VP2780-4K (4K60) Sony XBR-55X900C (4K60)
Source
• • • • •
QD980 (4k60) Nvidia GTX660 (4K30) Nvidia 970 (4K60) AMD HD7750 (4K30) AMD HD7700 (4K30)
• Plugtest in Q2’15 with HDMI 2.0 on AV
• Lead customers passed certification for HDMI 1.4
33
Altera Advantage: HDMI Interop Testing • Multiple frame rates, resolutions, and fast switching Resolutions. RBG, YCbCr QD source 4:2:2, 4:4:4, 4:2:0 capability
Sinks: QD, Samsung, Dell
Resolutions. RBG, YCbCr 4:2:2, 4:4:4, 4:2:0 QD source capability
1440x240 2880x240 1440x288 2880x288
60 60 50 50
720x480
60, 120, 240
1440x480
60
2880x480
60
1920x1080
1440x240
30, 60, 120
2880x240
30
34
720x576 1440x576 2880x576 1440x288
50, 100, 200 50 50 25, 50, 100
60
2880x288
60
1280x720
1920x540
25 24, 25, 30, 50, 60, 100, 120 24, 25, 30, 50, 60, 100, 120 24, 25, 30, 50, 60, 100, 120
3840x2160
24, 25, 30, 50, 60
Sinks: QD, Samsung, Dell 50 50
30,60 24,30,60
30,60
Altera Connectivity Landing Pages • Per function landing pages: SDI: HERE HDMI: HERE
DP: HERE
• Includes: Design examples: now including +VIP Suite
Getting started videos, documentation and tutorials Quality metrics, features and device support And much more…
35
The next chapter…
41
http://www.streetinsider.com/Press+Releases/eBrisk+Video+to+Demonstrate+its+eBLive+A5000+H.265HEVC+Video+Encoder+at+NAB+2016/11498480.html
Low Latency H.265 Codec on FPGA
HD Low Latency H.265 SoC • Single chip CODEC – no external CPU required for algorithm processing • User interface flexibility Video I/O – 3G SDI, HDMI,
DisplayPort…
PCI-e Gen 2 or 3 Integrate VIP components Integrate customer specific
pre-analysis
• Host CPU SW – Linux target Linux drivers provided Elementary stream
to Host over PCI-e
User code – Audio, AV sync,
Transport Stream, scrambling, etc.
43
Low Latency H.265 Configurations Available Today:
• Ultra low latency (I&P) Encoders: 1080p60, 8bit, 4.2.0
Arria10-480
1080p60, 10bit, 4.2.2
Arria10-1150
4kp30, 8bit 4.2.0
Arria10-1150
• Decoders: 1080p60, 12bit, 4.2.2
Arria10-270
4kp30, 12bit, 4.2.2
Arria10-320
4kp60, 12bit, 4.2.2
Arria10-480
• Encode & Decode – single chip 1080p60 8bit, 4.2.0 44
StratixV-D8
FPGA Decoders
Single chip FPGA – Arria 10 - Decoders • Full roadmap for H.265 decoders
HEVC 4:2:0 8 Bit Main Profile 4.1 1080p60 + 4Kp15 Jan 2015
HEVC 4Kp60
HEVC 4:2:0 10 Bit Main Profile 4.1 1080p60 + 4Kp15
Mar 2015
May 2015
HEVC 4:2:2 10/12/16 Bit Main Profile 4.1 1080p60 + 4Kp15
Nov 2015
• 3rd party decoders also available (MPEG2, H,264 J2K etc). Contact Altera for further details
[email protected]
46
Other CODEC standards
3rd Party Video Codec IP: Legacy standards Company (Location)
Codecs
Altera Family
IBEX (Japan)
MPEG2 Main/High 1080i Codec, H.264, & AVC-I. Encoders / Decoders
Arria, Stratix
Adaptive Microware (US)
MPEG2, H.264 Codecs
Cyclone, Arria, Stratix
SoC Technologies (Canada)
MPEG2, H.264 Codecs (Low Latency)
Cyclone, Arria, Stratix
A2E Technologies (US)
H.264 Codec
Cyclone, Arria SoC
Digital Design Corp (US)
MPEG2, H.264 Codecs (part of design services)
Cyclone, Arria, Stratix
JointWave (China)
H.264 Baseline, Main, High Codec
Cyclone, Arria, Stratix
Teklium (US / Japan)
H.264 Baseline, Main, High
Cyclone, Arria, Stratix
Tata Elxsi (India)
MPEG2, H.264 Main, High
Arria, Stratix
CAST (US / EMEA)
H.264 IP Codec
Cyclone, Arria, Stratix
Coreel (India)
MPEG2 / H.264 Main/High 422 / 10 bit decoders, AVC-I Encoder
Cyclone, Arria, Stratix
Omnitek (UK)
MPEG2 Decoder
Cyclone, Arria, Stratix
IntoPIX (Belgium)
JPEG2000, TICO
Arria, Stratix
Korusys (UK)
H.265 4k 4.2.2 12bit decoder & H.264 HD 4.2.2 10bit decoder.
Arria
Fraunhofer Institute (Germany)
H.264 low latency video encoder
Arria
Please contact 3rd parties direct for more information
Altera Video System Design Examples
UDX10: 4K Video Processing with HDMI 2.0
Video source 4K60
10-bit 4:4:4 UDX pipeline Demonstrating: video scaling, format convert, alpha blending mixing, logo insertion, more…
HDMI 2.0
Provided as solution demonstrator (or IP design example) Video available at https://www.altera.com/solutions/industry/broadcast/learn.html Includes key functions to support first 4K/UHD processing requirements Plug and play replacement of HDMI with DP, Ethernet or other interface HDMI 2.0
• • • •
UHD Panel 4K60
API SW control (from PC command line)
Arria 10
Tablet control
50
Cyclone V Video Design Example
Download from Altera.com/vip
Bitec Quad Video Card
Bitec DVI Card
“Demo in a box”
DVI Input
Composite Input Monitor/Display (DVI TX output)
51
Cyclone V GT Development Kit
Conclusions
Conclusions • Video: most ubiquitous technology across Altera market segments • Altera video framework: best in class to support your needs today and tomorrow
• Utilize the full range (Max 10 to Stratix 10) of Altera product families to get the right fit for your application • Exciting potential synergy options with Intel processors across market segments … … join us for the future! 53
Thank You
Copyright © Intel Corporation. All rights reserved. Intel Corporation, 2200 Mission College Blvd., Santa Clara, CA 95052-8119, USA
