Transcript
CS 414 – Multimedia Systems Design Lecture 20 – Multimedia Session Protocols
Klara Nahrstedt Spring 2010 CS 414 - Spring 2010
Administrative
HW1 deadline March 5, today, midnight 11:59pm Individual
effort!!! (take it as a midterm preparation)
Midterm, March 8 11-11:50am in class
All topics/lectures until February 26, i.e., the material covered this week March 1-8 will not be on the midterm You can bring calculator and 1 page of cheat-sheet (otherwise, exam is closed book, closed notes)
Class on Friday, March 8 - discussion of midterm topics CS 414 - Spring 2010
Outline
Multimedia Protocols – Standards RTP/UDP/IP
– Transmission Protocol RTCP Control/Negotiation Protocol to RTP
CS 414 - Spring 2010
Internet Multimedia Protocol Stack APPLICATION
Media encaps (H.264, MPEG-4) RTSP
SIP
RSVP
Layer 5 (Session)
RTCP RTP
KERNEL
TCP
UDP IP Version 4, IP Version 6
AAL3/4
AAL5
MPLS
CS 414 - Spring 2010
Layer 3 (Network)
Layer 2 (Link/MAC)
Ethernet/WiFi
ATM/Fiber Optics
Layer 4 (Transport)
Relation between RTP and RTCP Application Decoding
Coding
RTP
RTCP
Application Coding
RTCP
UDP/IP
UDP/IP
CS 414 - Spring 2010
Decoding
RTP
RTCP: Control and Management
Out-of-band control information for RTP flow. Monitors
QoS for RTP in the delivery and packaging of multimedia data Used periodically to transmit control packets to participants in a streaming multimedia session. Provides feedback on the quality of service being provided by RTP. Gathers statistics on media connection
Bytes sent, packets sent, lost packets, jitter, feedback and round trip delay. Application may use this information to increase the quality of service, perhaps by limiting flow or using a different codec. CS 414 - Spring 2010
RTCP Functions
There are several type of RTCP packets: Sender
report packet, Receiver report packet, Source Description RTCP Packet, Goodbye RTCP Packet and Application Specific RTCP packets.
RTCP itself does not provide any flow encryption or authentication means. SRTCP protocol can be used for that purpose. CS 414 - Spring 2010
RTP Services
Payload Type Identification Determination
of media coding Source identification RTP works with Profiles
Profile defines a set of payload type codes and their mappings to payload formats
Sequence numbering Error
Time-stamping Time
detection monitoring, synchronization, jitter calculation
Delivery monitoring CS 414 - Spring 2010
RTP Message MAC header IP header UDP header RTP message 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 Ver
P X CC
M PT
Sequence Number
Timestamp SSRC CSRC [0..15] :::
Ver – Version 2 P – Padding X – Extension, if set, the fixed head is followed by exactly one header extension CC – CSRC count M – Marker – intended to allow significant events such as frame boundaries to be marked (defined by profile) PT – Payload type SSRS – synchronization source, CSRC – contribution source CS 414 - Spring 2010
RTP Services – Support of Heterogeneity
Mixer service Allows
for resynchronization of incoming audio packets Reconstructs constant 20 ms spacing generated by sender Mixes reconstructed audio streams into single stream Translates audio encoding to lower bandwidth Forwards lower bandwidth packet streams
Translator service Allows
for translation between IP and other high speed protocols May change encoding data CS 414 - Spring 2010
Difference between Mixers and Translators S3 S1
M1
T
S2
S4
CS 414 - Spring 2010
M2
R1
Payload Formats
Static Payload formats Established
in RTP Profile Payload type 0 := µ-law audio codec
Dynamic Payload formats Applications
agree per session on payload
format H.263, JPEG, MPEG
CS 414 - Spring 2010
Session Management (Layer 5)
Important part of multimedia communication Separates control aspects from transport aspects SESSION MANAGER Participant Management
Conference control
Session Control Configuration control
Session Control Protocol (e.g., RTSP)
Media control
Presentation data communication whiteboard
Continuous data communication
video
Continuous data communication audio CS 414 - Spring 2010
Session Manager
Tasks: Membership
control Monitoring of shared workspace Coordination of Media control management Exchange of QoS parameters Conference control management – establishment, modification, termination
CS 414 - Spring 2010
Session Control
Session Described by Session
state
Name of session, start, valid policies
Session management – two steps for state processing Establishment
of session Modification of session
CS 414 - Spring 2010
Session Control
Conference Control Centralized
or distributed approach
Media Control Synchronization
Configuration Control Negotiation
of QoS parameters, admission control and reservation/allocation of resources
Membership Control Invitation
of users; registration of users, change of membership CS 414 - Spring 2010
Conclusion
RTP usage – in several application audio and video tools (vat, vic) RTP follows the principle of application level framing and integrated layer processing RTP/UDP/IP is being used by the current streaming session protocols such as RTSP Session protocols are actually negotiation/session establishment protocols that assist multimedia applications Multimedia applications such as QuickTime, Real Player and others use them CS 414 - Spring 2010
Midterm March 8 (Monday), 11-11:50am, 1302 SC Closed Book, Closed Notes You can bring calculator and 1 page cheat sheet
CS 414 - Spring 2010
Covered Material Class Notes (Lectures 1-17) Book Chapters to read/study:
Media
Coding and Content processing book
Chapter 2, Chapter 3.1-3.2, 3.8, Chapter 4.1-4.2.2.1, Chapter 4.3 (as discussed in lecture) Chapter 5, chapter 7.1-7.5, 7.7
Multimedia
Systems book
Chapter 2.1-2.4, (not 2.4.4 – we have not covered QoS routing ), CS 414 - Spring 2010
Material
Media Characteristics Synchronous,
Isochronous, Asynchronous Regular, irregular Weakly and strongly periodic streams
Audio Characteristics Samples,
frequency, Perception, psychoacoustic effects, loudness, pitch, decibel, intensity Sampling rate, quantization CS 414 - Spring 2010
Material
Audio Characteristics PCM,
DPCM, ADPCM, signal-to-noise ratio
Image Characteristics Sampling,
quantization, pixels Image properties: color, texture, edges Simple edge detection process
CS 414 - Spring 2010
Material
Video technology Color
perception: hue, brightness, saturation, Visual representation: horizontal and vertical resolution , aspect ratio; depth perception, luminance, temporal resolution and motion Flicker effect Color coding: YUV, YIQ, RGB NTSC vs HDTV formats CS 414 - Spring 2010
Material
Basic Coding schemes Run-length
coding Statistical coding Huffman coding Arithmetic coding
Hybrid codes JPEG:
image preparation, DCT transformation, Quantization, entropy coding, JPEG-2000 characteristics CS 414 - Spring 2010
Material
Hybrid Coding Video
MPEG: image preparation, I, P, B frames characteristics, quantization, display vs processing/transmission order of frames Audio MPEG: role of psychoacoustic effect, masking, steps of audio compression MPEG-4: differences to MPEG-2/MPEG-1
Audio-visual objects, layering CS 414 - Spring 2010
Material
Quality of Service concepts Service
classes, QoS specification – deterministic, predictive, best effort, QoS classification – application, system, network QoS, relation between QoS and resources QoS operations: translation, negotiation of QoS parameters
CS 414 - Spring 2010
Material
Resource Management concepts QoS
and resources, establishment phase and transmission/enforcement phase Admission control of resources, reservation and allocation of resources LBAP arrival model
CS 414 - Spring 2010
Sample Problems Consider the following alphabet {C,S,4,1}, with probabilities: P(C) = 0.3, P(S) = 0.2, P(4)= 0.25, P(1) = 0.25. Encode the word CS414 using
Huffman
coding and arithmetic coding Compare which encoding requires less bits
CS 414 - Spring 2010
Sample Problems
Describe briefly each step in MPEG-1 audio encoding. Specify the functionality, which is performed in each step. You don’t have to provide equations, only a clear explanation of the functionality that is performed inside each step.
CS 414 - Spring 2010
Sample Problems What is flicker effect and how to remove it? Explain difference between synchronous and isochronous transmission stream modes Provide five differences between MPEG-4 video encoding standard and the previous MPEG video encoding standards
CS 414 - Spring 2010
Sample Problems
Consider voice conversational application (like Skype). What multimedia-sensitive algorithms at the setup phase would you deploy to make sure that you start with a good voice transmission ? Specify clear design of order of algorithms/protocols to be used
CS 414 - Spring 2010
