Transcript
SMARTER ROBOTS AND AUTONOMOUS MACHINES Barrett Williams | Robotics | Technical Marketing
[email protected]
IEEE–CNSV Talk December 11, 2015
1
JETSON AT THE LEADING EDGE Research Humanoids, Intelligent Appliances, Interactive Caretakers Spacecraft, Drones, Self-Driving Vehicles
2
THE COMING ERA OF
AUTONOMOUS MACHINES New use cases demand autonomy
Deep learning enables autonomy
GPUs deliver the best performance
x1
x2
x3
x4
3
REAL-LIFE ROBOTS — IN RECENT HISTORY
4
DARPA ROBOTICS CHALLENGE FINALS Robots Have Come a Long Way, And Have a Long Way to Go • Research trails Hollywood, Pop Culture • Many subsystems still hard-coded • Still needs work: • Path-planning, pose estimation • Computer vision, object recognition • Force-based control, simulation
• Humanoids are highly complex machines (so are humans) *Homework: https://youtu.be/g0TaYhjpOfo
5
AUTONOMOUS DRIVING
Spurring on Research in Sensors, Vision, Operating Systems • Stanford à Google self-driving cars • Automotive OEMs, e.g. Audi • Add-on/retrofit companies such as Cruise Automotive (SF-based) • New depth-imaging systems: • LIDAR from Velodyne, SICK, Hokuyo, numerous startups • Stereo imaging systems, cost reduction
6
THE BOTS JETSON POWERS (prototypes with Jetson TK1)
• Household appliances • Jibo (Cynthia Breazeal of MIT Media Lab) • June Intelligent Oven
• Drones (consumer and industrial) • DJI Manifold for Matrice M100 • Prototype: Kespry drones for outdoor asset tracking • Prototype: Parrot and stereo-based SLAM 7
ACADEMIC RESEARCH
Computer Vision, Autonomous Systems • MIT RACECAR (RSS Course) • Open source autonomous scale vehicle: 200 laps without a collision
• UCLA ReMap OpenPTrack • Person/skeleton tracking to replace proprietary parts of OpenNI
• ROS Lab at Oregon State University • Maintain, host, teach with various ports of ROS (e.g., ARM) 8
MIT VIDEO
9
(DEEP LEARNING + COMPUTER VISION) == BETTER ROBOTS
10
GPUS — A FASTER PLATFORM FOR MACHINE LEARNING Image Recognition Challenge
120
1.2M training images • 1000 object categories
80
GPU Entries
100
110
60
60
40
Hosted by
20
4
0 2010 person car
bird
helmet
frog
motorcycle
2011
2012
dog chair
2014
Classification Error Rates 30%
28%
25%
26%
20% 16%
15% person
2013
person hammer flower pot power drill
12%
10%
7%
5% 0% 2010
2011
2012
2013
2014 11
HOW FAR HAS AUTONOMY COME? ImageNet classification accuracy
95.1% Human:94.9% 88%
93%
84%
72%
2010
74%
2011
2012
2013
2014
Deep Learning on GPUs
2015
12
MACHINE LEARNING USING
DEEP NEURAL NETWORKS
Today’s Largest Networks:
Input
Result
• • • • •
~10 layers 1B parameters 10M images ~30 Exaflops ~30 GPU days
Human brain has trillions of parameters – 1,000 more. 13
TRAIN, THEN DEPLOY
YOUR NEURAL NETWORKS Programming Approaches
The
The
Training: Server, GRID, DIGITS
Classified Object!
Inference: Embedded (Jetson)
Solver Network Data Scientist
Trained Deep Neural Net Model
Camera Inputs
14
CUDNN CUDA-accelerated Deep Learning library
Your Application
(dog breed detector, for example) Frameworks Supports industry-standard frameworks:
Out-of-the-box speedups of neural networks:
cuDNN CUDA
For both Inference and Training:
Jetson TX1 // Tesla/Titan/GRID 15
Object Classification Segmentation
Collision Avoidance 3D Reconstruction Localization/Mapping 16
KESPRY VIDEO
17
KEEP CALM AND
BUILD SAFER, SMARTER
ROBOTS
18
ROBOT OPERATING SYSTEM (ROS) What is it?
• Standardized package manager • ROS packages run on Ubuntu (by default) • Standardizes message-protocols for: • Pose estimation • Localization • Navigation • Computer vision
• Incorporates the latest contributions from academia (bleeding-edge research)
Andrew Symington, UCL Gazebo and rviz with several quadrotors, ROSCon 2014 19
ROBOT OPERATING SYSTEM (ROS) What’s next? • Message-passing interface now designed to work with lossy protocols and embedded microcontrollers • Focus on ‘robust’ components • and also ‘experimental’ components
• Code quality control and vetting process
2D SLAM using Kinect depth data in rviz for autonomous navigation
David Lu, UCL Layered costmaps in rviz, ROSCon 2014 20
VISIONWORKS™ CUDA-accelerated Computer Vision Toolkit •
Full OpenVX 1.1 implementation
•
Custom extensions
•
Easy integration with existing CV pipelines
Applications
Robotics
Jetson TX1
Drones
Object Tracking
Dense Optical Flow
Example Applications Feature Tracking
VisionWorks CUDA
Augmented Reality
Structure from Motion
VisionWorks™ API + FrameWorks
VisionWorks™ Toolkit
IMAGE ARITHMETIC
Absolute Difference Accumulate Image Accumulate Squared Accumulate Weighted Add / Subtract / Multiply Channel Combine Channel Extract
GEOMETRIC TRANSFORMS
Affine Warp + Perspective Warp Flip Image Gaussian Pyramid Remap Scale Image
Features
Canny Edge Detector Fast Corners + Fast Track Harris Corners + Harris Track Hough Circles Hough Lines
21
JETSON LINUX SDK Deep Learning and Computer Vision
GPU Compute
Developer Tools
NVTX
NVIDIA Tools eXtension
Debugger | Profiler | System Trace 22
THE PERIPHERALS WE CONNECT WITH including Community Contributions
• Infrared devices: • SICK LIDAR (LMS 200); Hokuyo • Asus Xtion Pro Live (PrimeSense) • Intel RealSense (mult. generations)
• Stereo and color cameras: • StereoLabs Zed (consumer-oriented) • Point Grey Research USB3 and GigE • e-con Systems CSI-MIPI Cameras with external ISP 23
JETSON TX1 GPU
1 TFLOP/s 256-core Maxwell
CPU
64-bit ARM A57 CPUs
Memory
4 GB LPDDR4 | 25.6 GB/s
Storage
16 GB eMMC
Wifi/BT
802.11 2x2 ac/BT Ready
Networking
1 Gigabit Ethernet
Size
50mm x 87mm
Interface
400 pin board-to-board connector
24
Jetson TX1 Developer Kit Jetson TX1 Developer Board 5MP Camera
25
10X ENERGY EFFICIENCY FOR MACHINE LEARNING Alexnet 50
Efficiency Images/sec/Watt
45 40 35 30 25 20 15 10 5 0
Intel core i7-6700K (Skylake)
Jetson TX1
26
JETSON MONTAGE VIDEO
27
WHAT’S NEXT?
Neural Nets Enable Science Fiction? • Cubesats: low-cost stratosphere imaging, transmission is expensive • Drones: sense-and-avoid; smart tracking • Submersibles: on and under the sea, data is expensive • Advance humanity: Please, hack your heart out.
28
THE TOOLS — TO BUILD IT ALL
29
Jetson TX1 Developer Kit $599 retail $299 educational discount Started Shipping Nov 16 (US) Int’l to follow
30
Jetson TX1 Module $299 Available 1Q16 Distributors Worldwide
31
(1000 unit QTY)
ONE ARCHITECTURE — END-TO-END AI PC GAMING
Tesla
for Cloud
Titan X for PC
DRIVE PX for Auto
Jetson
for Embedded
32
THANK YOU! Q&A: WHAT CAN I HELP YOU BUILD?
Special thanks to: 33
JETSON SDK Vertically Vertically Integrated Packages Integrated Packaages Machine Learning
OpenGL Tools
Vision Source code editor cuFFT
Compute
CUDA
Debugger
CUDA Math Library
cuBLAS NPP
cuSPARSE
cuRAND Thrust
cuSolver
Graphics
Linux for Tegra
Profiler System Trace V4L2 libjpeg
NVTX
NVIDIA Tools eXtension
Jetson TX1 34
JETSON TX1
Jetson TX1
GPU
1 TFLOP/s 256-core Maxwell
CPU
64-bit ARM A57 CPUs
Memory
4 GB LPDDR4 | 25.6 GB/s
Video decode
4K 60Hz
Video encode
4K 30Hz
CSI
Up to 6 cameras | 1400 Mpix/s
Display
2x DSI, 1x eDP 1.4, 1x DP 1.2/HDMI
Wi-Fi
802.11 2x2 ac
Networking
1 Gigabit Ethernet
PCI-E
Gen 2 1x1 + 1x4
Storage
16 GB eMMC, SDIO, SATA
Other
3x UART, 3x SPI, 4x I2C, 4x I2S, GPIOs
35
VISIONWORKS™ PRIMITIVES IMAGE ARITHMETIC
All OpenVX Primitives
NVIDIA extensions
Absolute Difference Accumulate Image Accumulate Squared Accumulate Weighted Add / Subtract / Multiply Channel Combine Channel Extract Color Convert + CopyImage Convert Depth Magnitude Not / Or / And / Xor Phase Table Lookup Threshold FLOW & DEPTH Median Flow Optical Flow (LK) Semi-Global Matching Stereo Block Matching
GEOMETRIC TRANSFORMS Affine Warp + Warp Perspective + Flip Image Remap Scale Image +
FILTERS
BoxFilter Convolution Dilation Filter Erosion Filter Gaussian Filter Gaussian Pyramid Laplacian3x3 Median Filter Scharr3x3 Sobel 3x3
FEATURES
Canny Edge Detector Fast Corners + Fast Track Harris Corners + Harris Track Hough Circles Hough Lines
ANALYSIS
Histogram Histogram Equalization Integral Image Mean Std Deviation Min Max Locations
+ type/mode extension by NVIDIA NVIDIA extension primitives
36
