Current Efforts In Applying Technology And Research For

Transcript

Department of Electrical and Computer Engineering Current Efforts in Applying Technology and Research for Homeland Security Richard A. Messner, Ph.D. [email protected] 2 Credits Whole Project54 Team Advanced Research Team z PhD Students – Dragan Vidacic – Pavlo Melnyk z MS Students – Robert Bogucki – Barrett Stetson – Todd Crowley z Undergraduate Student – Katelyn Palmer Department of Electrical and Computer Engineering 3 Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The un-tethered emergency responder – Mobile video surveillance for emergency response – Some ongoing basic research at UNH Department of Electrical and Computer Engineering 1 4 The Security Problem z What does security mean to you? z Assume that you are in charge of homeland security for the United States. – What one thing would you like to have in order to ensure your success in making the United States safe and secure INFORMATION! Department of Electrical and Computer Engineering 5 Information z In order to be useful – Must be accurate – Must not be “stale” – Must be able to be conveyed to those who need it and in the timeframe they can use it Each of these requires both current as well as future technologies Why both current AND future technologies? Department of Electrical and Computer Engineering 6 How to best prepare and respond z Gaps exist in our understanding of how best to prepare and respond to attacks against our homeland – Depending upon who you ask you will get many opinions z What is definitely necessary is: – Applications of current and developing technology – Research and development of new technologies and subsequent application of those technologies – Education +++ Education +++ Education • Must integrate education and research that involves postsecondary students and scholars and develop innovative educational approaches including multidisciplinary and interdisciplinary efforts that would enhance research and development in critical areas Department of Electrical and Computer Engineering 2 7 Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The Un-tethered Emergency Responder – Mobile Video Surveillance for Emergency Response – Some ongoing basic research at UNH Department of Electrical and Computer Engineering 8 Project54 – A Collaborative Effort Integrated Voice-Controlled System zConsolidated Advanced Technologies Laboratory (CATLab) zUniversity of New Hampshire zNew Hampshire Department of Safety zUnited States Department of Justice zInitial funding began in 1999 Department of Electrical and Computer Engineering 9 Goals of Project54 1. Unified control interface 2. Hands-off and eyes-off operation 3. In-car storage of MV records 4. State-wide real-time information network 5. Standard off-the-shelf components 6. Modular software 7. Promote industry standards Department of Electrical and Computer Engineering 3 10 Project54 – Inside the Cruiser Department of Electrical and Computer Engineering 11 REMOTE FROM VEHICLE Department of Electrical and Computer Engineering 12 Lights and Siren Scenario The officer is dispatched to an emergency situation and must respond in “Code 3” Typical Cruiser 1. Take hand off wheel 2. Find control head 3. Switch on strobes 4. Switch on wigwags 5. Switch on siren P54 Equipped Cruiser 1. Say “Lights and Siren” Department of Electrical and Computer Engineering 4 13 Radio Control Scenario The officer needs to switch from the Troop A frequency to the Coast Guard frequency Typical Cruiser 1. Take hand off wheel 2. Find control head 3. Determine current channel 4. Recall location of Coast Guard channel 5. Press zone up/down X times 6. Press channel up/down X times P54 Equipped Cruiser 1. Say “Coast Guard” Department of Electrical and Computer Engineering 14 Records Check Scenario The officer needs to run a check on a car with NH plate 123456 Typical Cruiser 1. Take hand off wheel 2. Find radio mic 3. Request check from dispatcher 4. Wait for dispatcher to be available 5. Relay plate information to dispatcher 6. Wait for dispatcher to retrieve info 7. Listen as dispatcher reads info P54 Equipped Cruiser 1. Say “Vehicle by plate” 2. Say “123456” 3. Say “Check Record” 4. Listen as P54 system reads back, displays and stores info Department of Electrical and Computer Engineering 15 High-Speed Pursuit Scenario The officer needs to pursue a suspect at high speeds Typical Cruiser 1. Take hand off wheel 2. Find control head 3. Turn on lights 4. Turn on siren 5. Start video recorder 6. Determine location 7. Pick up radio mic 8. Alert other officers 9. Update location P54 Equipped Cruiser 1. Say “Pursuit” Department of Electrical and Computer Engineering 5 16 Lee, NH Dodge Charger Department of Electrical and Computer Engineering 17 Greenland, NH Motorcycle Department of Electrical and Computer Engineering 18 NH Local Agencies P54 Equipped 648 in state vehicles z z z z z z z z z z z z z z z z z z z z Allenstown PD, Alstead PD, Alton PD Barrington PD, Bartlett PD, Bedford PD, Berlin PD, Boscawen PD, Bridgewater PD Campton PD, Canaan PD, Center Harbor PD, Cheshire County PD, Cheshire Cty PD, Colebrook PD, Concord PD, Conway PD, Coos County Sheriff Dalton PD, Deerfield PD, Dublin PD, Durham PD East Kingston PD, Epping PD, Epsom PD, Exeter PD Farmington PD, Fire-Durham, Fire-Exeter, Fire-Portsmouth, Franklin PD, Fremont PD Gilford PD, Grafton County Sheriff, Greenland PD, Groton PD Hampstead PD, Hampton Falls PD, Hampton PD, Hancock PD, Harrisville PD, Hebron PD, Henniker PD, Hill PD, Holderness, Hooksett PD, Hudson PD Jaffrey PD Kensington PD, Kingston PD Laconia PD, Lancaster PD, Lee PD, Lincoln PD, Lyndeborough PD Madbury PD, Madison PD, Marlborough PD, Mason PD, Meredith PD, Middleton PD, Milton PD, Moultonboro PD New Castle PD, New Durham PD, New Hampton PD, New Ipswich PD, Newfields PD, Newington PD, Newmarket, Newton PD, North Hampton PD, Northumberland PD, Northwood PD, Nottingham PD Ossipee PD Pelham PD, Pittsfield PD, Plymouth PD, Plymouth University PD, Portsmouth PD Raymond PD, Rindge PD, Rochester PD, Rollinsford PD, Rye Fire PD, Rye Police PD Salem PD, Seabrook PD, Somersworth PD, South Hampton PD, Strafford County DA, Strafford County Sheriff, Strafford PD, Stratham PD, Sullivan County Sheriff, Swanzey PD Thornton Police PD, Tilton PD, Tuftonboro PD UNH PD Wakefield PD, Waterville Valley, Webster PD, Wilton PD, Windham PD, Wolfeboro PD, Woodstock PD Department of Electrical and Computer Engineering 6 19 National Agencies P54 Equipped 139 out of state vehicles z z z z z z z z z z z z Berkeley, Ca PD Boston, Ma PD Burlington, Ma PD Carlsbad, CA PD Colorado State Police Coronado, Ca PD Glastonbury, Ct PD Maryland State Police Minnesota State Patrol North Dakota State Police San Diego, Ca FD Scarborough, Me Department of Electrical and Computer Engineering 20 The Un-tethered Emergency Responder Department of Electrical and Computer Engineering 21 Need More Technology Department of Electrical and Computer Engineering 7 22 Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The Un-tethered Emergency Responder – Mobile Video Surveillance for Emergency Response – Some ongoing basic research at UNH Department of Electrical and Computer Engineering 23 First Responders HAZMAT … FBI Fire Fighters Police Paramedics Department of Electrical and Computer Engineering 24 Interoperability? ? $()$! ? z Multiple agencies and jurisdictions can be involved – Local – State – Federal – Citizen based (i.e., Amateur Radio, Civic) z Interoperability issues exist – communication – collection, storage, and dissemination of time critical information z Problem with organizing information being acquired from multiple sources (e.g. keeping information current – real time) z Problems with data management, processing and analysis Department of Electrical and Computer Engineering 8 25 Emergency Response z Need: – Standardized organizational structures, processes, and procedures z Need: – Standards for planning, training and exercising, and personnel qualification standards z Need: – Equipment acquisition and certification standards z Need: – Interoperable communications processes, procedures and systems z Need: – Information management systems z Need: – Supporting technologies – voice and data communications systems, information systems, data display systems and specialized technologies Department of Electrical and Computer Engineering 26 National Incident Management System Incident Command System z Combines up to 7 units z Facilitates information exchange – Multiple radio channels – Wireless networking – Audio and Video over wireless data channels z Local data processing and storage z Reports to higher hierarchical control node only relevant information Control Department of Electrical and Computer Engineering 27 Emergency Response Control Control Control Control Department of Electrical and Computer Engineering 9 28 Communications and data exchange HQ Communication (narrow band): • radio or cell phone network Control Control Data Status, Requests, Images, etc. Vehicle-level LAN Control Wireless LAN (broad band) Video/Audio/Data streaming to Mobile Control Unit Department of Electrical and Computer Engineering 29 Development approach z Leverage off of Project54 and prior expertise in software development z Implement a modular system – Integration and development of Off-the shelf components z Interact with emergency personnel in order to fully understand the system requirements z Use speaker independent and inexpensive speech engine Department of Electrical and Computer Engineering 30 Our Test Bed Department of Electrical and Computer Engineering 10 31 Basic Test Bed Architecture Department of Electrical and Computer Engineering 32 Test Bed Software Architecture Department of Electrical and Computer Engineering 33 Test Bed Software Architecture - PDA PROJECT54 FRAMEWORK GUI INCIDENTS APPLICATION RECORDS APPLICATION WINDOWS REGISTRY ACCESS SPEECH APPLICATION INTER – APPLICATION MESSAGING VIDEO APPLICATION FINGERPRINT APPLICATION CAPWIN APPLICATION UOAPP APPLICATION UTILITIES PUSH-TO-TALK WINCLIENTP54 Department of Electrical and Computer Engineering 11 34 Test Bed Functionality z Microsoft SAPI5 based Text-to-Speech and Speechto-Text functionality on PDA z Interfaced Wireless communications to mobile data terminal via speech or direct PDA touch screen invocation – Access to data stored in remote location when officer leaves vessel control cabin/police vehicle (via both speech and touch screen) – Allows for remote control of devices while the officer is not in the vessel control cabin/vehicle (via both speech and touch screen) z Application allows the login, query, and incident reporting to a mock-up database server Department of Electrical and Computer Engineering 35 PDA device used by the officer Department of Electrical and Computer Engineering 36 System Login – Main UO Application z Remote server login settings z Activates other UTO applications Department of Electrical and Computer Engineering 12 37 Remote Data Access z Access to remote database z Login and initial connection to the remote server z Data display Department of Electrical and Computer Engineering 38 Incident Report z Incident report right on the incident scene z Limited speech grammar used for vocabulary Department of Electrical and Computer Engineering 39 Records Query z Records query check from remote location z Speech interface z Pre-configured grammar for speech engine Department of Electrical and Computer Engineering 13 40 Biometrics Validation z Fingerprint reader attached to the PDA z Live fingerprint capture followed by image transmission to the remote PC z Validation by scanning the biometrically encoded DL (AAMVA DL/ID-2000) Department of Electrical and Computer Engineering 41 Biometric capture/transmission/validation Fingerprint Imager WiFi 802.11b Department of Electrical and Computer Engineering 42 Fingerprint Received and Displayed Department of Electrical and Computer Engineering 14 43 DL scan and data display Department of Electrical and Computer Engineering 44 Fingerprint validation - result Department of Electrical and Computer Engineering 45 Proposed use of this technology to Port and waterway security efforts z Vessel inspection and data entry on remote location – the officer performs the report while inspecting the vessel z Wireless WiFi link realized between the PDA and the computer in the patrol boat z Incident report available to all security personnel instantly z Biometrics technology provides additional security features (restricted access to certain areas, collection and verification of fingerprint data from distant location) Department of Electrical and Computer Engineering 15 46 One possible scenario: vessel inspection at the remote location WiFi (802.11b) To command center Department of Electrical and Computer Engineering 47 Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The Un-tethered Emergency Responder – Mobile Video Surveillance for Emergency Response – Some ongoing basic surveillance research at UNH Department of Electrical and Computer Engineering 48 Need More Technology Department of Electrical and Computer Engineering 16 49 Mobile Video System Configuration Department of Electrical and Computer Engineering 50 Mobile Digital Video System Department of Electrical and Computer Engineering 51 Video System Overview z Digital Video Recording in MPEG2 format (DVD quality) z Hot-swap digital storage z Voice control z Network connectivity z Remote control from PocketPC z Video streaming z Configurable set of cameras Department of Electrical and Computer Engineering 17 52 DVR54 – digital video recorder z Real-time MPEG2 video CODEC z Web and Video Streaming Server z Removable USB/FireWire storage media z Windows XP Embedded z Small OS footprint z Flash memory instead of HDD Department of Electrical and Computer Engineering 53 Camera Setup z Front view camera with tilt/pan and zooming capabilities z Color bullet cameras for front and rear view from cruiser and wireless on-officer z Wireless A/V transceiver z Video switch or quad processor Department of Electrical and Computer Engineering 54 P54 Video Control Screen z DVR54 Recording and Playback Controls z Broadcasting live video z Switching between up to 4 cameras z Tilt/Pan and Zooming controls to Panasonic network camera ( z Voice activated as well as hands on Department of Electrical and Computer Engineering 18 55 PocketPC Remote Control z Voice control z Touch screen GUI z Streaming video back to PDA Department of Electrical and Computer Engineering 56 Video Database Remote Access z Use Internet Explorer to access DVR54 video database and view stored video files Department of Electrical and Computer Engineering 57 Video Database Remote Access z Access DVR54 using Internet Explorer z View streaming video with Windows Media Player Department of Electrical and Computer Engineering 19 58 Video Scenarios: Access to Stored Video z Remote DVR54 Video Database Access – Use Internet Explorer to access DVR54 video database and view stored video files with Media Player z Possible scenario: – In emergency situations it may be critical to gain control over the DVR54 and view recent video in order to analyze the current situation Department of Electrical and Computer Engineering 59 Migration of digital video system from police land vehicles to watercrafts Department of Electrical and Computer Engineering 60 Project54 system migration z Project54 system can be installed to a watercraft as is with minor modifications z Directed microphone array has excellent performance in high-noise environments z All related periphery devices: Radio, Lightbar, Siren, GPS and Video System are the same as in police cruisers z Watercraft specific equipment can be integrated into Project54 system to have computer and voice control Microphone Array Project54 console Department of Electrical and Computer Engineering 20 61 Video System in Watercraft Rear Camera Mobile Camera Front Camera DVR54 z z z z z Digital Video Recorder DVR54 Color bullet cameras for front and rear view from cruiser Optional front view camera with tilt/pan and zooming capabilities On-officer wireless mobile camera Wireless A/V transceiver (currently a 1.2Ghz ISM baseband A/V transmitter) Department of Electrical and Computer Engineering 62 Need Research to Fuel Technology Department of Electrical and Computer Engineering 63 Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The Un-tethered Emergency Responder – Mobile Video Surveillance for Emergency Response – Some ongoing basic surveillance research at UNH Department of Electrical and Computer Engineering 21 64 The Human Visual System Department of Electrical and Computer Engineering 65 Model of the Human Visual System Department of Electrical and Computer Engineering 66 The Falcon Visual System Department of Electrical and Computer Engineering 22 67 New Concept for Composite Sensor Tilt-pan drive Camera control Visual or textual output Visual attention system Video capture Digital video recording HDD Operator interacts with application integrated into Project54 Department of Electrical and Computer Engineering 68 Engineering Prototype Block Diagram Video Processing Central Visual attention system Video recording Wide View Camera Vibration tolerant feature extraction License plate recognition Narrow View Camera Composite sensor Tilt, pan, zoom mechanics Target registration and classification Vehicle verification Target detection and tracking Camera position control Department of Electrical and Computer Engineering 69 Wide/Narrow View Camera Pair peripheral foveal xp yp xf yf Wide-view camera (peripheral) Narrow-view camera (foveal) Department of Electrical and Computer Engineering 23 70 System View using P54 Driving Simulator Wide-view camera (periphery) Narrow-view camera (fovea) Department of Electrical and Computer Engineering 71 First Prototype – June 2006 Department of Electrical and Computer Engineering 72 Raw Test Output from Prototype (a) (b) Department of Electrical and Computer Engineering 24 73 Calibration of Camera Pair Uncalibrated Calibrated (a) (b) Note Lines Department of Electrical and Computer Engineering 74 Car in Periphery of View (a) (b) Department of Electrical and Computer Engineering 75 Recap of Outline z The security problem and why technology is critical for security z Applied technology and basic research efforts at UNH – CATLab - Consolidated Advanced Technologies Laboratory • Project54 – The un-tethered emergency responder – Mobile video surveillance for emergency response – Some ongoing basic research at UNH Many other technology projects are currently underway at UNH Department of Electrical and Computer Engineering 25 76 Questions? Department of Electrical and Computer Engineering 26