Real-Time Protocols

RTP/RTCP/RTSP

Presented by1. A. H. M. Kamal (0411054002) 2. KAZY NOOR E- ALAM SIDDIQUEE (0411052009) 3. MASUD 4.Mahmudul Hasan Razib (0411052060)

Overview
Streaming performance requirements Protocol stack for multimedia services Real-time transport protocol (RTP) RTP control protocol (RTCP) Real-time streaming protocol (RTSP)

Real-time multimedia streaming

Real-time multimedia applications Video teleconferencing Internet Telephony (VoIP) Internet audio, video streaming (A-PDUs)

Streaming performance requirements

Sequencing to report PDU loss to report PDU reordering to perform out-of-order decoding Time stamping and Buffering for play out for jitter and delay calculation Payload type identification for media interpretation Error concealment covers up errors from lost PDU by using redundancy in most-adjacent-frame

Quality of Service (QoS) feedback from receiver to sender for operation adjustment
Rate control sender reduces sending rate adaptively to network congestion

Ideal Timing no jitter

00.00.00
00.00.10 00.00.20 00.00.30

application
00.00.11 00.00.21

00.00.31

Send time Play time

Reality jitter

00.00.00 00.00.10 00.00.20 00.00.30 00.00.40

00.00.35 00.00.37 00.00.41 00.00.47 00.00.51 00.00.11 00.00.21 00.00.25

delay

Send time

Play time

Jitter

(contd.)

00.00.00 00.00.10 00.00.20 00.00.30 00.00.40

00.00.35 00.00.37 00.00.41 00.00.47 00.00.51 00.00.11 00.00.21 00.00.25

00.00.18

00.00.28

00.00.38 00.00.48 00.00.58

Send time

Play time

Jitter
Playback buffer

(contd.)

At time 00:00:18

At time 00:00:28

At time 00:00:38

How does Sequence number and Timestamp help ? Audio silence example:
Consider audio data What should the sender do during silence? Not send anything Why might this cause problems? silence

Receiver cannot distinguish between loss

and silence

Solution:
After receiving no PDUs for a while, next PDU received at the receiver will reflect a big jump in timestamp, but have the correct next seq. no. Thus, receiver knows what happened.

Streaming performance requirements

Sequencing to report PDU loss to report PDU reordering to perform out-of-order decoding Time stamping and Buffering for play out for jitter and delay calculation Payload type identification for media interpretation Error concealment covers up errors from lost PDU by using redundancy in most-adjacent-frame

Quality of Service (QoS) feedback from receiver to sender for operation adjustment
Rate control sender reduces sending rate adaptively to network congestion

Support from transport layers

TCP is not used because:
TCP does retransmissions unbounded delays No provision for time stamping TCP does not support multicast TCP congestion control (slow-start) unsuitable for real-time transport

RTP + UDP usually used for multimedia services

Real time interactive audio video

Not like streaming stored audio and video support longer delay?
Video conference Internet Telephony

Protocol stack for multimedia services

RTSP

RTP

RTCP
TCP
(till now)

RTP: Introduction
Provides end-to-end transport functions for real-time applications Supports different payload types All RTP and RTCP PDUs are sent to same multicast group (by all participants) All RTP PDUs sent to an even-numbered UDP port, 2p All RTCP PDUs sent to UDP port 2p+1
Transport layer

Application RTP RTCP

UDP
IP Data Link

Does NOT provide timely delivery or other QoS guarantees Relies on other protocols like RTCP and lower layers Does NOT assume the underlying network is reliable and delivers PDUs in sequence Uses sequence number

Physical

RTP Session

For an RTP session Single multicast address

RTP Session

RTP session is sending and receiving of RTP data by a group of participants For each participant, a session is a pair of transport addresses used to communicate with the group If multiple media types are communicated by the group, the transmission of each medium constitutes a session.

RTP Synchronization Source

synchronization source - each source of RTP PDUs Identified by a unique,randomly chosen 32-bit ID (the SSRC) A host generating multiple streams within a single RTP must use a different SSRC per stream

RTP Basics of Data Transmission

RTP PDUs

RTP Packet
RTP Header Audio/Video Chunk

RTP Packet
RTP Header Audio/Video Chunk

RTP Header

RTP PDU Header

Sampling instant of first data octet multiple PDUs can have same timestamp not necessarily monotonic used to synchronize different Payload type media streams Incremented by one for each RTP PDU: PDU loss detection Restore PDU sequence

Identifies synchronization source Identifies contributing sources

(used by mixers)

RTP Payload-Type
Audio Payload Type
Payload-Type Number 0 1 3 7 9 14 15 Audio Format PCM u-low 1016 GSM LPC G.722 MPEG Audio G.728 Sampling Rate (kHz) 8 8 8 8 16 90 8 Rate (kbps) 64 4.8 13 2.4 48-64 16

Video Payload Type

Payload Type Number 26 31 32 33 Video Format Motion JPEG H.261 MPEG 1 video MPEG 2 video

Mixer
RTP mixer - an intermediate system that receives & combines RTP PDUs of one or more RTP sessions into a new RTP PDU

Stream may be transcoded, special effects may be performed.

A mixer will typically have to define synchronization relationships between streams.Thus
Sources that are mixed together become contributing sources (CSRC) Mixer itself appears as a new source having a new SSRC

Translator
An intermediate system that Connects two or more networks Multicasting through a firewall
Modifies stream encoding, changing the streams timing Transparent to participants SSRCs remain intact

end system 1
from ES1: SSRC=6 from ES2: SSRC=23

from ES1: SSRC=6 from ES2: SSRC=23

transl.1

transl.2

end system 2

authorized tunnel firewall

from ES1: SSRC=6 from ES2: SSRC=23

RTP: Developing Software App. with RTP

Application RTP
Transport layer

Application RTP RTCP UDP IP Data Link Physical

Socket
UDP IP Data Link Physical

Do by hand Add RTP header to frame

Use JAVA Class/ C RTP Library Send a chunk of media, Payload type no., SSRC, timestamp to interface

Pass to UDP socket

Approach One

Alternative Approach

RTP Security
Confidentiality Allows encription to RTP Packets Believes to lower levels to ensure authentication. Integrity by sanity checking descriptor header: protocol version no, pkt length, payload type.

Authentication and Integrity

Key Management

Conference type App:

Combination of SIP, SAP, SDP

RTP Control Protocol (RTCP)

RTCP specifies report PDUs exchanged between sources and

destinations of multimedia information

receiver reception report sender report source description report Reports contain statistics such as the number of RTP-PDUs sent,

number of RTP-PDUs lost, inter-arrival jitter

Feedback Information: modify sender transmission rates and for

diagnostics purposes-whether problems are local, regional, global

Synchronization of streams using RTCP

RTP audio RTCP audio RTP video RTP video

Internetwork

Timestamps in RTP PDUs are tied to the individual video and audio sampling clocks timestamps are not tied to the wall-clock time, or each other!

Each RTCP sender-report PDU contains (for most recently generated PDU in
associated RTP stream): The timestamp of RTP PDU The wall-clock time for when PDU was created Receivers can use this association to synchronize the playout of audio and video

RTCP bandwidth scaling

Problem What happens when there is one sender and many receivers? RTCP reports scale linearly with the number of participants and would match or exceed the amount of RTP data! More overhead than useful data! Solution RTCP attempts to limit its traffic to 5% of the session bandwidth to ensure it can scale! RTCP gives 75% of this rate to the receivers; and the remaining 25% to the sender. Example Suppose one sender, sending video at a rate of 2 Mbps. Then RTCP attempts to limit its traffic to 100 Kbps. The 75 kbps is equally shared among receivers: With R receivers, each receiver gets to send RTCP traffic at 75/R kbps. Sender gets to send RTCP traffic at 25 kbps.

RTCP bandwidth scaling

Sender Can Send T= # of Senders Avg. RTP packet size .25x.05xSession BW

Receiver Can Send

T=

# of Senders Avg. RTP packet size .75x.05xSession BW

Real-Time Streaming Protocol (RTSP)

Application layer protocol (default port 554) Usually runs on RTP for stream & TCP for control Provides the control channel to transmit Uses out-of-band signaling Usable for Live broadcasts / multicast

Also known as Network remote control for multi-media servers.

RTSP Overview
Reference to several continuous media file Directive for synchronization Web Server

web browser
Presentation descriptor

HTTP
presentation descriptor

Web Server/Media server

RTSP
pres. desc,streaming commands

media player

RTP/RTCP
audio/video content

RTSP Session

Default port 554

RTSP server
get UDP port

RTSP SETUP RTSP OK RTSP PLAY RTSP OK RTSP TEARDOWN RTSP OK

TCP

RTSP client
choose UDP port

data source media server

RTP VIDEO RTP AUDIO RTCP

UDP

AV subsystem

media player

Example:Media on demand (Unicast)

Media server A audio.example.com Media server V video.example.com

Client C

Web server W -holds the media descriptors

RTSP Message sequence

C -> W : GET/Twister.sdp HTTP/1.1 Host: www.example.com Accept: application/sdp W-> C : HTTP/1.0 200 OK Content-Type: application/sdp

W V C

C-> A : SETUP rtsp://audio.example.com/twister/audio.en RTSP/1.0 Cseq:1 Transport : RTP/AVP/UDP;unicast;client_port=3056-3057 A-> C : RTSP/1.0 200 OK Cseq:1 Session: 12345678 Transport : RTP/AVP/UDP;unicast;client_port=3056-3057 server_port=5000-5001 C->V : SETUP rtsp://video.example.com/twister/video.en RTSP/1.0 Cseq:1 Transport : RTP/AVP/UDP;unicast;client_port=3058-3059 A-> C : RTSP/1.0 200 OK Cseq:1 Session: 23456789 Transport : RTP/AVP/UDP;unicast;client_port=3058-3059 server_port=5002-5003

RTSP Message sequence

C->V: PLAY rtsp://video.example.com/twister/video RTSP/1.0 Cseq: 2 Session: 23456789

(contd.)

W
V->C: RTSP/1.0 200 OK Cseq: 2 Session: 23456789 RTP-Info: url=rtsp://video.example.com/twister/video; seq=12312232;

V C

C->A: PLAY rtsp://audio.example.com/twister/audio.en RTSP/1.0 Cseq: 2 Session: 12345678 A->C: RTSP/1.0 200 OK Cseq: 2 Session: 12345678 RTP-Info: url=rtsp://audio.example.com/twister/audio.en; seq=876655;

RTSP Message sequence

C->A: TEARDOWN rtsp://audio.example.com/twister/audio.en RTSP/1.0 Cseq: 3 Session: 12345678 A->C: RTSP/1.0 200 OK Cseq: 3

(contd.)

W V

C->V: TEARDOWN rtsp://video.example.com/twister/video RTSP/1.0 Cseq: 3 Session: 23456789 V->C: RTSP/1.0 200 OK Cseq: 3

RTCP compound PDU

RTCP PDU 1
SSRC SSRC

RTCP PDU 2

SR

sender report

SSRC

receiver report source 2

receiver report source 3

SDES

SSRC

CNAME PHONE

compound PDU (single UDP datagram)

RTCP processing in Translators

SR sender information : Does not generate their own sender information(most of the times), but forwards the SR PDUs received from one side to other RR reception report blocks : Does not generate their own RR reports (most of the times), but forwards RR reports received from one side to another. SSRC are left intact SDES : Forwards without changing the SDES info. but may filter non CNAME SDES, if bandwidth is limited BYE : Forwards BYE PDU unchanged. A translator about to cease forwarding, send a BYE PDU to each connected nodes

RTCP processing in Mixers

SR sender information : Generates its own SR info. Because the characteristics of source stream is lost in the mix. The SR info is sent in same direction as the mixed stream RR reception report blocks : Generates its own reports for sources in each cloud and sends them only to same cloud

SDES : Forwards without changing the SDES info. but may filter non CNAME SDES, if bandwidth is limited
BYE : Forwards BYE PDU unchanged. A mixer about to cease forwarding, send a BYE PDU to each connected nodes

Source description PDUs

May contain:
a CNAME item (canonical identifier/name) a NAME item (real user name) an EMAIL item a PHONE item a LOC item (geographic location) a TOOL item (application name) a NOTE item (transient msg, e.g. for status) a PRIV item (private extension) Value 1 2 3 4 5 6 7 8

CNAME=1

length

user and domain name

Here is the end of my presentation on RTP

=Thanks to all=