Transcript
Department of Electrical and Computer Engineering
Current Efforts in Applying Technology and Research for Homeland Security Richard A. Messner, Ph.D.
[email protected]
2
Credits Whole Project54 Team
Advanced Research Team z PhD Students – Dragan Vidacic – Pavlo Melnyk z MS Students – Robert Bogucki – Barrett Stetson – Todd Crowley z Undergraduate Student – Katelyn Palmer
Department of Electrical and Computer Engineering
3
Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The un-tethered emergency responder – Mobile video surveillance for emergency response – Some ongoing basic research at UNH
Department of Electrical and Computer Engineering
1
4
The Security Problem z What does security mean to you? z Assume that you are in charge of homeland security for the United States. – What one thing would you like to have in order to ensure your success in making the United States safe and secure
INFORMATION! Department of Electrical and Computer Engineering
5
Information z In order to be useful – Must be accurate – Must not be “stale” – Must be able to be conveyed to those who need it and in the timeframe they can use it Each of these requires both current as well as future technologies
Why both current AND future technologies? Department of Electrical and Computer Engineering
6
How to best prepare and respond z Gaps exist in our understanding of how best to prepare and respond to attacks against our homeland – Depending upon who you ask you will get many opinions z What is definitely necessary is: – Applications of current and developing technology – Research and development of new technologies and subsequent application of those technologies – Education +++ Education +++ Education • Must integrate education and research that involves postsecondary students and scholars and develop innovative educational approaches including multidisciplinary and interdisciplinary efforts that would enhance research and development in critical areas
Department of Electrical and Computer Engineering
2
7
Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The Un-tethered Emergency Responder – Mobile Video Surveillance for Emergency Response – Some ongoing basic research at UNH
Department of Electrical and Computer Engineering
8
Project54 – A Collaborative Effort Integrated Voice-Controlled System zConsolidated Advanced Technologies Laboratory (CATLab) zUniversity of New Hampshire zNew Hampshire Department of Safety zUnited States Department of Justice
zInitial funding began in 1999
Department of Electrical and Computer Engineering
9
Goals of Project54 1.
Unified control interface
2.
Hands-off and eyes-off operation
3.
In-car storage of MV records
4.
State-wide real-time information network
5.
Standard off-the-shelf components
6.
Modular software
7.
Promote industry standards
Department of Electrical and Computer Engineering
3
10
Project54 – Inside the Cruiser
Department of Electrical and Computer Engineering
11
REMOTE FROM VEHICLE
Department of Electrical and Computer Engineering
12
Lights and Siren Scenario The officer is dispatched to an emergency situation and must respond in “Code 3” Typical Cruiser 1. Take hand off wheel 2. Find control head 3. Switch on strobes 4. Switch on wigwags 5. Switch on siren
P54 Equipped Cruiser 1. Say “Lights and Siren”
Department of Electrical and Computer Engineering
4
13
Radio Control Scenario The officer needs to switch from the Troop A frequency to the Coast Guard frequency Typical Cruiser 1. Take hand off wheel 2. Find control head 3. Determine current channel 4. Recall location of Coast Guard channel 5. Press zone up/down X times 6. Press channel up/down X times
P54 Equipped Cruiser 1. Say “Coast Guard”
Department of Electrical and Computer Engineering
14
Records Check Scenario The officer needs to run a check on a car with NH plate 123456 Typical Cruiser 1. Take hand off wheel 2. Find radio mic 3. Request check from dispatcher 4. Wait for dispatcher to be available 5. Relay plate information to dispatcher 6. Wait for dispatcher to retrieve info 7. Listen as dispatcher reads info
P54 Equipped Cruiser 1. Say “Vehicle by plate” 2. Say “123456” 3. Say “Check Record” 4. Listen as P54 system reads back, displays and stores info
Department of Electrical and Computer Engineering
15
High-Speed Pursuit Scenario The officer needs to pursue a suspect at high speeds Typical Cruiser 1. Take hand off wheel 2. Find control head 3. Turn on lights 4. Turn on siren 5. Start video recorder 6. Determine location 7. Pick up radio mic 8. Alert other officers 9. Update location
P54 Equipped Cruiser 1. Say “Pursuit”
Department of Electrical and Computer Engineering
5
16
Lee, NH Dodge Charger
Department of Electrical and Computer Engineering
17
Greenland, NH Motorcycle
Department of Electrical and Computer Engineering
18
NH Local Agencies P54 Equipped 648 in state vehicles z z z z z z z z z z z z z z z z z z z z
Allenstown PD, Alstead PD, Alton PD Barrington PD, Bartlett PD, Bedford PD, Berlin PD, Boscawen PD, Bridgewater PD Campton PD, Canaan PD, Center Harbor PD, Cheshire County PD, Cheshire Cty PD, Colebrook PD, Concord PD, Conway PD, Coos County Sheriff Dalton PD, Deerfield PD, Dublin PD, Durham PD East Kingston PD, Epping PD, Epsom PD, Exeter PD Farmington PD, Fire-Durham, Fire-Exeter, Fire-Portsmouth, Franklin PD, Fremont PD Gilford PD, Grafton County Sheriff, Greenland PD, Groton PD Hampstead PD, Hampton Falls PD, Hampton PD, Hancock PD, Harrisville PD, Hebron PD, Henniker PD, Hill PD, Holderness, Hooksett PD, Hudson PD Jaffrey PD Kensington PD, Kingston PD Laconia PD, Lancaster PD, Lee PD, Lincoln PD, Lyndeborough PD Madbury PD, Madison PD, Marlborough PD, Mason PD, Meredith PD, Middleton PD, Milton PD, Moultonboro PD New Castle PD, New Durham PD, New Hampton PD, New Ipswich PD, Newfields PD, Newington PD, Newmarket, Newton PD, North Hampton PD, Northumberland PD, Northwood PD, Nottingham PD Ossipee PD Pelham PD, Pittsfield PD, Plymouth PD, Plymouth University PD, Portsmouth PD Raymond PD, Rindge PD, Rochester PD, Rollinsford PD, Rye Fire PD, Rye Police PD Salem PD, Seabrook PD, Somersworth PD, South Hampton PD, Strafford County DA, Strafford County Sheriff, Strafford PD, Stratham PD, Sullivan County Sheriff, Swanzey PD Thornton Police PD, Tilton PD, Tuftonboro PD UNH PD Wakefield PD, Waterville Valley, Webster PD, Wilton PD, Windham PD, Wolfeboro PD, Woodstock PD
Department of Electrical and Computer Engineering
6
19
National Agencies P54 Equipped 139 out of state vehicles z z z z z z z z z z z z
Berkeley, Ca PD Boston, Ma PD Burlington, Ma PD Carlsbad, CA PD Colorado State Police Coronado, Ca PD Glastonbury, Ct PD Maryland State Police Minnesota State Patrol North Dakota State Police San Diego, Ca FD Scarborough, Me
Department of Electrical and Computer Engineering
20
The Un-tethered Emergency Responder
Department of Electrical and Computer Engineering
21
Need More Technology
Department of Electrical and Computer Engineering
7
22
Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The Un-tethered Emergency Responder – Mobile Video Surveillance for Emergency Response – Some ongoing basic research at UNH
Department of Electrical and Computer Engineering
23
First Responders
HAZMAT … FBI
Fire Fighters
Police
Paramedics Department of Electrical and Computer Engineering
24
Interoperability? ?
$()$!
?
z Multiple agencies and jurisdictions can be involved – Local – State – Federal – Citizen based (i.e., Amateur Radio, Civic) z Interoperability issues exist – communication – collection, storage, and dissemination of time critical information z Problem with organizing information being acquired from multiple sources (e.g. keeping information current – real time) z Problems with data management, processing and analysis
Department of Electrical and Computer Engineering
8
25
Emergency Response z Need: – Standardized organizational structures, processes, and procedures z Need: – Standards for planning, training and exercising, and personnel qualification standards z Need: – Equipment acquisition and certification standards z Need: – Interoperable communications processes, procedures and systems z Need: – Information management systems z Need: – Supporting technologies – voice and data communications systems, information systems, data display systems and specialized technologies Department of Electrical and Computer Engineering
26
National Incident Management System Incident Command System z Combines up to 7 units z Facilitates information exchange – Multiple radio channels – Wireless networking – Audio and Video over wireless data channels z Local data processing and storage z Reports to higher hierarchical control node only relevant information
Control
Department of Electrical and Computer Engineering
27
Emergency Response Control
Control
Control
Control
Department of Electrical and Computer Engineering
9
28
Communications and data exchange HQ Communication (narrow band): • radio or cell phone network
Control
Control Data Status, Requests, Images, etc.
Vehicle-level LAN
Control
Wireless LAN (broad band) Video/Audio/Data streaming to Mobile Control Unit
Department of Electrical and Computer Engineering
29
Development approach z Leverage off of Project54 and prior expertise in software development z Implement a modular system – Integration and development of Off-the shelf components z Interact with emergency personnel in order to fully understand the system requirements z Use speaker independent and inexpensive speech engine
Department of Electrical and Computer Engineering
30
Our Test Bed
Department of Electrical and Computer Engineering
10
31
Basic Test Bed Architecture
Department of Electrical and Computer Engineering
32
Test Bed Software Architecture
Department of Electrical and Computer Engineering
33
Test Bed Software Architecture - PDA PROJECT54 FRAMEWORK GUI INCIDENTS APPLICATION
RECORDS APPLICATION
WINDOWS REGISTRY ACCESS SPEECH APPLICATION
INTER – APPLICATION MESSAGING
VIDEO APPLICATION
FINGERPRINT APPLICATION
CAPWIN APPLICATION
UOAPP APPLICATION
UTILITIES PUSH-TO-TALK
WINCLIENTP54
Department of Electrical and Computer Engineering
11
34
Test Bed Functionality z Microsoft SAPI5 based Text-to-Speech and Speechto-Text functionality on PDA z Interfaced Wireless communications to mobile data terminal via speech or direct PDA touch screen invocation – Access to data stored in remote location when officer leaves vessel control cabin/police vehicle (via both speech and touch screen) – Allows for remote control of devices while the officer is not in the vessel control cabin/vehicle (via both speech and touch screen) z Application allows the login, query, and incident reporting to a mock-up database server Department of Electrical and Computer Engineering
35
PDA device used by the officer
Department of Electrical and Computer Engineering
36
System Login – Main UO Application z Remote server login settings z Activates other UTO applications
Department of Electrical and Computer Engineering
12
37
Remote Data Access z Access to remote database z Login and initial connection to the remote server z Data display
Department of Electrical and Computer Engineering
38
Incident Report z Incident report right on the incident scene z Limited speech grammar used for vocabulary
Department of Electrical and Computer Engineering
39
Records Query z Records query check from remote location z Speech interface z Pre-configured grammar for speech engine
Department of Electrical and Computer Engineering
13
40
Biometrics Validation z Fingerprint reader attached to the PDA z Live fingerprint capture followed by image transmission to the remote PC z Validation by scanning the biometrically encoded DL (AAMVA DL/ID-2000)
Department of Electrical and Computer Engineering
41
Biometric capture/transmission/validation Fingerprint Imager
WiFi 802.11b
Department of Electrical and Computer Engineering
42
Fingerprint Received and Displayed
Department of Electrical and Computer Engineering
14
43
DL scan and data display
Department of Electrical and Computer Engineering
44
Fingerprint validation - result
Department of Electrical and Computer Engineering
45
Proposed use of this technology to Port and waterway security efforts z Vessel inspection and data entry on remote location – the officer performs the report while inspecting the vessel z Wireless WiFi link realized between the PDA and the computer in the patrol boat z Incident report available to all security personnel instantly z Biometrics technology provides additional security features (restricted access to certain areas, collection and verification of fingerprint data from distant location)
Department of Electrical and Computer Engineering
15
46
One possible scenario: vessel inspection at the remote location
WiFi (802.11b) To command center
Department of Electrical and Computer Engineering
47
Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The Un-tethered Emergency Responder – Mobile Video Surveillance for Emergency Response – Some ongoing basic surveillance research at UNH
Department of Electrical and Computer Engineering
48
Need More Technology
Department of Electrical and Computer Engineering
16
49
Mobile Video System Configuration
Department of Electrical and Computer Engineering
50
Mobile Digital Video System
Department of Electrical and Computer Engineering
51
Video System Overview z Digital Video Recording in MPEG2 format (DVD quality) z Hot-swap digital storage z Voice control z Network connectivity z Remote control from PocketPC z Video streaming z Configurable set of cameras
Department of Electrical and Computer Engineering
17
52
DVR54 – digital video recorder z Real-time MPEG2 video CODEC z Web and Video Streaming Server z Removable USB/FireWire storage media z Windows XP Embedded z Small OS footprint z Flash memory instead of HDD
Department of Electrical and Computer Engineering
53
Camera Setup z Front view camera with tilt/pan and zooming capabilities z Color bullet cameras for front and rear view from cruiser and wireless on-officer z Wireless A/V transceiver z Video switch or quad processor
Department of Electrical and Computer Engineering
54
P54 Video Control Screen z DVR54 Recording and Playback Controls z Broadcasting live video z Switching between up to 4 cameras z Tilt/Pan and Zooming controls to Panasonic network camera
(
z Voice activated as well as hands on
Department of Electrical and Computer Engineering
18
55
PocketPC Remote Control z Voice control z Touch screen GUI z Streaming video back to PDA
Department of Electrical and Computer Engineering
56
Video Database Remote Access
z Use Internet Explorer to access DVR54 video database and view stored video files Department of Electrical and Computer Engineering
57
Video Database Remote Access
z Access DVR54 using Internet Explorer z View streaming video with Windows Media Player
Department of Electrical and Computer Engineering
19
58
Video Scenarios: Access to Stored Video z Remote DVR54 Video Database Access – Use Internet Explorer to access DVR54 video database and view stored video files with Media Player z Possible scenario: – In emergency situations it may be critical to gain control over the DVR54 and view recent video in order to analyze the current situation
Department of Electrical and Computer Engineering
59
Migration of digital video system from police land vehicles to watercrafts
Department of Electrical and Computer Engineering
60
Project54 system migration
z Project54 system can be installed to a watercraft as is with minor modifications z Directed microphone array has excellent performance in high-noise environments z All related periphery devices: Radio, Lightbar, Siren, GPS and Video System are the same as in police cruisers z Watercraft specific equipment can be integrated into Project54 system to have computer and voice control
Microphone Array
Project54 console Department of Electrical and Computer Engineering
20
61
Video System in Watercraft Rear Camera
Mobile Camera Front Camera
DVR54
z z z z z
Digital Video Recorder DVR54 Color bullet cameras for front and rear view from cruiser Optional front view camera with tilt/pan and zooming capabilities On-officer wireless mobile camera Wireless A/V transceiver (currently a 1.2Ghz ISM baseband A/V transmitter)
Department of Electrical and Computer Engineering
62
Need Research to Fuel Technology
Department of Electrical and Computer Engineering
63
Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The Un-tethered Emergency Responder – Mobile Video Surveillance for Emergency Response – Some ongoing basic surveillance research at UNH
Department of Electrical and Computer Engineering
21
64
The Human Visual System
Department of Electrical and Computer Engineering
65
Model of the Human Visual System
Department of Electrical and Computer Engineering
66
The Falcon Visual System
Department of Electrical and Computer Engineering
22
67
New Concept for Composite Sensor
Tilt-pan drive
Camera control
Visual or textual output
Visual attention system
Video capture Digital video recording
HDD
Operator interacts with application integrated into Project54
Department of Electrical and Computer Engineering
68
Engineering Prototype Block Diagram Video Processing Central Visual attention system Video recording Wide View Camera
Vibration tolerant feature extraction
License plate recognition
Narrow View Camera
Composite sensor Tilt, pan, zoom mechanics
Target registration and classification
Vehicle verification
Target detection and tracking
Camera position control
Department of Electrical and Computer Engineering
69
Wide/Narrow View Camera Pair peripheral foveal
xp
yp
xf
yf
Wide-view camera (peripheral)
Narrow-view camera (foveal)
Department of Electrical and Computer Engineering
23
70
System View using P54 Driving Simulator
Wide-view camera (periphery)
Narrow-view camera (fovea)
Department of Electrical and Computer Engineering
71
First Prototype – June 2006
Department of Electrical and Computer Engineering
72
Raw Test Output from Prototype
(a)
(b)
Department of Electrical and Computer Engineering
24
73
Calibration of Camera Pair Uncalibrated
Calibrated
(a)
(b)
Note Lines Department of Electrical and Computer Engineering
74
Car in Periphery of View
(a)
(b)
Department of Electrical and Computer Engineering
75
Recap of Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The un-tethered emergency responder – Mobile video surveillance for emergency response – Some ongoing basic research at UNH
Many other technology projects are currently underway at UNH Department of Electrical and Computer Engineering
25
76
Questions? Department of Electrical and Computer Engineering
26