Transcript
Bluecoin - Voice and Music Over an Embedded BLE Platform Central Labs – AST Robotics
Voice communication: a key driver of innovation since 1800’s
Central Labs – AST Robotics
2
Voice Automation “in the things”: Voice Internet Assistant Call me a taxi E-gadgets, Wearables
Play Music Control Lighting, heating, … News, sport, traffic, weather, … Answer questions, create to-do lists, shopping lists, … Place orders online, use other online services: taxi, pizza, … Central Labs – AST Robotics
Presentation Title
3
BlueCoin: the Robotic Ear Augmented hearing and motion sensing Motion, Activity and Balance
Acoustic Beamforming
Bluetooth Low Energy
Embedded Processing
Sound Localization
Central Labs – AST Robotics
BlueCoin – Augmented hearing & Sensing Battery Expansion
• Microphone array processing • Wide band audio over BLE (BlueVoice) • Sensor fusion • Complete development kit
Main components: STM32F446 •
ARM Cortex-M4F@180MHz - 128KB RAM
u4 Microphone Array (4x MP23DB01MM) Bluetooth-Low-Energy radio (BlueNRG-MS) • Core System
Central Labs – AST Robotics
Bluetooth 4.1, multiple role simultaneously
6+3 axis inertial module (LSM6DS3+LIS3MDL) Absolute pressure sensor (LPS22HB)
5
Full Embedded Sensing Software Development Kit
4 x MP23DB01MM
Central Labs – AST Robotics
LSM6DS3
LIS3MDL
LPS22HB
BLUENRG-MS
BALF-NRG-01D3
MEMS
Voice Terminal
Voice to Cloud: and Audio System quality
Voice & data Gateway
Cloud Central Labs – AST Robotics
7
Digital MEMS Microphones • Ultra compact, low power, omnidirectional • System-in-package combines: • capacitive sensing MEMS element • ASIC interface
Sensor ASIC
Bottom port
A/D and Digital i/f PDM (Pulse Density Modulation) interface: • 1 to 3 MHz • 1-bit resolution • Fully digital Central Labs – AST Robotics
Top port
Sound Inlet
Sensing
• Capacitive membrane • Omnidirectional • Analog output
8
Omnidirectional MEMS microphone Omnidirectional microphones Sensitivity is the same in all directions Power of captured sound is independent from its Direction of Arrival. 3-D Polar Pattern
2-D Polar Pattern 90
1.5 60
120 1
30
150 0.5
180
0
210
330
240
300 270
Central Labs – AST Robotics
9
Software Building Blocks: MEMS-microphones.augmented
Source Localization
Acoustic Echo-Cancellation Beamforming
Room far end
Time delay of arrival (TDOA) estimation
h(t)
• • • Central Labs – AST Robotics
Delay and Sum Endfire / Broadband Advanced GSC
• •
GCC_PATH X-Correlation
10
Acoustic Beamforming algorithms Sound Source Adaptive Filtering
𝑓𝑓1 .
𝑓𝑓2 . Y
Ʃ
HW/SW System noise
ƞ1
ƞ2 ƞ3
𝑓𝑓3 . ƞN 𝑓𝑓𝑁𝑁 .
Beamformer
Microphone Array Environmental Noise
Central Labs – AST Robotics
11 11
First Order Directional Beam Patterns • Directionality is achieved by combining two microphones and by processing their synchronized audio output. Example: “Figure of 8” beamforming is achieved by subtraction of 2 microphones signals
3-D Polar Pattern
2-D Polar Pattern 90
1
120
60 0.8 0.6
150
30 0.4 0.2
180
0
330
210
300
240 270
Central Labs – AST Robotics
12
Cardioid beamforming • Cardioid beam pattern implements single-sided Audio capture m2
+
Cardioid Beamforming Basic Scheme
m1 Delay = ∆
3-D Polar Pattern
2-D Polar Pattern
m1
Central Labs – AST Robotics
m2
13
libBeamforming100: Strong Cardioid 2 DMA cardioids in “back to back” configuration
µ4 array: MEMS microphone side by side; the smallest array you can build
4 x MP23DB01MM
Central Labs – AST Robotics
Back
Front
14
Polar pattern tests Test setup: • • • •
Microphone Array mounted on a rotating support Inter-microphone distance: 4mm Rotation in steps of 10 degrees Gaussian White Noise played by high quality loudspeaker
• Resulting beampattern • Blue: omnidirectional microphone • Red: «Basic cardioid» mode • Green: «Strong» mode
Central Labs – AST Robotics
15
Beamforming: ASR Test Test setup:
Male and female spoken words - at 0°
Inputs WORDS
Output
4 synchronous output channels : • Omnidirectional microphone • Basic Cardioid • ASR Ready • Strong Cardioid
Gaussian White Noise - at 90°
NOISE
Recorded words are sent to Google ASR and recognition data are collected
Central Labs – AST Robotics
16
libBeamforming100: ASR test results
ASR confidence
omnidirectional cardioid ASR strong
Signal to Noise ratio Central Labs – AST Robotics
17
BlueVoice: augmenting Bluetooth Smart with Audio
Vendor Specific Profile over a Standard BlueTooth 4.x protocol stack
Advanced Audio Compression and streaming architecture
Bluetooth Smart Low Power protocol design
Central Labs – AST Robotics
BlueVoice mapping over standard Bluetooth 4.0 protocol stack BlueVoice Vendor Specific Profile
Audio processing
Application: BlueVoice Profile (vendor specific)
GAP configuration Central Unit (Master)
Peripheral Unit (Slave)
Generic Attribute Profile (GATT)
Generic Access Profile (GAP)
GATT configuration
Attribute Protocol Client
Security Manager
Server
Logical Link Control and Adaptation Protocol Service
Audio exported as a Service
Characteristic
Characteristic
Host-Controller Interface Link Layer
Direct Test Mode
Descriptor
Physical Layer Bluetooth Low Energy Stack Central Labs – AST Robotics
19
BlueVoice: embedded-to-Cloud communication Augmented Audio Terminal
Platform
Signals
Comm Interface
smart
Central Labs – AST Robotics
Mobile Device
Cloud-based Services
20
Sensors to Cloud architecture Integrated Terminal
Gateway
3rd Party Platforms
Signals Communication
Central Labs – AST Robotics
21
Cloud-based Services
ST Development Kits for IoT: Integrated & Modular Design environment
Central Labs – AST Robotics
Evaluation, tracking & monitoring
Fast Prototyping
Reference Design
System Software Development
22
Central Labs – AST Robotics
