CU9255 internetworking Multimedia

CU9255
INTERNETWORKING
MULTIMEDIA
Adri Jovin J J

SYLLABUS FOR AFFILIATED INSTITUTIONS
ANNA UNIVERSITY, CHENNAI
REGULATIONS – 2009
CU9255 INTERNETWORKING MULTIMEDIA L T P C
3 0 0 3
UNIT I INTRODUCTION 9
Digital sound, video and graphics, basic multimedia networking, multimedia characteristics, evolution
of Internet services model, network requirements for audio/video transform, multimedia coding and
compression for text, image, audio and video. Multimedia Communication in Wireless Network.
UNIT II SUBNETWORK TECHNOLOGY 9
Broadband services, ATM and IP, IPV6, High speed switching, resource reservation, Buffer
management, traffic shaping, caching, scheduling and policing, throughput, delay and jitter
performance.
UNIT III MULTICAST AND TRANSPORT PROTOCOL 9
Multicast over shared media network, multicast routing and addressing, scoping multicast and NBMA
networks, Reliable transport protocols, TCP adaptation algorithm, RTP, RTCP.
UNIT IV MEDIA - ON – DEMAND 9
Storage and media servers, voice and video over IP, MPEG-2 over ATM/IP, indexing synchronization
of requests, recording and remote control.
UNIT V APPLICATIONS 9
MIME, Peer-to-peer computing, shared application, video conferencing, centralized and distributed
conference control, distributed virtual reality, light weight session philosophy.
TOTAL:45 PERIODS
REFERENCES:
1. Jon Crowcroft, Mark Handley, Ian Wakeman. “Internetworking Multimedia”, Harcourt Asia Pvt.
Ltd. Singapore, 1998.
2. B.O. Szuprowicz, “Multimedia Networking”, McGraw Hill, NewYork. 1995
3. Tay Vaughan,Multimedia making it to work, 4ed,Tata McGrawHill, NewDelhi,2000.
4. Ellen kayata wesel, Ellen Khayata, “Wireless Multimedia Communication: Networking Video,
Voice and Data”, Addison Wesley Longman Publication, USA, 1998.

UNIT I INTERNETWORKING MULTIMEDIA
Adri Jovin J.J.
Page | 1
UNIT I
INTRODUCTION
Multimedia is, as described previously, a woven combination of digitally manipulated text,
photographs, graphic art, sound, animation, and video elements. When you allow an end user—
also known as the viewer of a multimedia project—to control what and when the elements are
delivered, it is called interactive multimedia. When you provide a structure of linked elements
through which the user can navigate, interactive multimedia becomes hypermedia.
Although the definition of multimedia is a simple one, making it work can be complicated. Not
only do you need to understand how to make each multimedia element stand up and dance, but
you also need to know how to use multimedia computer tools and technologies to weave them
together. The people who weave multimedia into meaningful tapestries are called multimedia
developers.
The software vehicle, the messages, and the content presented on a computer, television screen,
PDA (personal digital assistant), or mobile phone together constitute a multimedia project. If the
project is to be shipped or sold to consumers or end users, typically delivered as a download on
the Internet but also on a CD-ROM or DVD in a box or sleeve, with or without instructions, it is
a multimedia title. Your project may also be a page or site on the World Wide Web, where you
can weave the elements of multimedia into documents with HTML (Hypertext Markup
Language) or DHTML (Dynamic Hypertext Markup Language) or XML (eXtensible Markup
Language) and play rich media files created in such programs as Adobe’s Flash, LiveMotion, or
Apple’s QuickTime by installing plug-ins into a browser application such as Internet Explorer,
Safari, Google Chrome, or Firefox. Browsers are software programs or tools for viewing content
on the Web. See Chapter 12 for more about plug-ins, multimedia, and the Web. A multimedia
project need not be interactive to be called multimedia: users can sit back and watch it just as
they do a movie or the television. In such cases a project is linear, or starting at the beginning
and running through to the end. When users are given navigational control and can wander
through the content at will, multimedia becomes nonlinear and user interactive, and is a powerful
personal gateway to information.
Determining how a user will interact with and navigate through the content of a project requires
great attention to the message, the scripting or storyboarding, the artwork, and the programming.
You can break an entire project with a badly designed interface. You can also lose the message
in a project with inadequate or inaccurate content. Multimedia elements are typically sewn
together into a project using authoring tools. These software tools are designed to manage
individual multimedia elements and provide user interaction. Integrated multimedia is the
“weaving” part of the multimedia definition, where source documents such as montages,
graphics, video cuts, and sounds merge into a final presentation. In addition to providing a
method for users to interact with the project, most authoring tools also offer facilities for creating
and editing text and images and controls for playing back separate audio and video files that have
been created with editing tools designed for these media. The sum of what gets played back and

Adri Jovin J.J.
Page | 2
how it is presented to the viewer on a monitor is the graphical user interface, or GUI
(pronounced “gooey”).
The GUI is more than just the actual graphics on the screen—it also often provides the rules or
structure for the user’s input. The hardware and software that govern the limits of what can
happen here are the multimedia platform or environment.
History
Throughout the 1960s, 1970s 1980s and 1990s, Computers have been restricted to dealing with
two main types of data - words and numbers, text and arithmetic processing, through Word
Processing and Spreadsheets etc. etc. Codes for numbers (binary, BCD, Fixed point etc., IEEE
floating point), are fairly well standardized. Codes for text (ASCII, EBCDIC, but also fonts,
Kangi, ppt etc.) are also reasonably well understood. Higher level ``codes'' - links, indexes,
references, and so on are the subject of such standards as the ubiquitous Hyper-Text Markup
Language, HTML.
Now computers, disks and networks are fast enough to process, store and transmit audio and
video and computer generated visualization material as well as text and graphics and data: hence
the multimedia revolution
One thing about multimedia that cannot be overstated: It is big, like space in the Hitchhiker's
Guide to the Universe, it is much bigger than you can imagine. Of course, we are not talking
about the hype here, we are talking about the storage transmission and processing requirements!
To paraphrase Maurice Zapp, from David Lodge's novel, A Small World: ``Every Encoding is a
Decoding''. The idea behind this glib quote is that each time we invent a new way of representing
and transmitting information, we also have to teach ourselves to receive and comprehend that
new type of representation. In the rest of this section, we take a look at some aspects of
representation that need to be understood in regard to multimedia.
Numbers and letters have standard encodings: ASCII and IEEE Floating Point are the most
widespread now (at least for common English language text processing, and for numeric
programming) - in the past there have been a plethora of other encodings, even for simple
Roman alphabet text. As multi-lingual support has become common, we have seeing a brief
increase in the number of encodings, and then as the problems become better understood, a
standard set of character sets are emerging. Digital Multimedia Encodings in the form of audio
and video are still at a very early stage in terms of standards, and there are many, partly because
of the range of possible processing, storage and transmission performance capacities available on
computers and networks, where some systems are right at the limits of their abilities to do any
useful work at all!
Each new medium needs to be coded and we need to have common representations for objects in
the medium; there are many choices. For example, speech can be coded as a sequence of
samples, a sequence of phonemes, a string of text with a voice synthesizer setting, and so on,
requiring more or less intelligence or processing at the sender and receiver, and providing more

Adri Jovin J.J.
Page | 3
or less structural information (and as a result, typically allowing more compression). Similarly,
video can be coded as a sequence of bitmaps, or else can be broken down into some description
of scenes, objects within scenes, motion of objects and so on.
The codings now involve possible relationships with time and between different media. When
we read a block of text, it is usually up to the reader to choose how quickly to read it. Hypertext
to some extent breaks this rule, at least by relating text non linearly with other text. When we
listen to speech, or have a conversation with another autonomous being, we do not control the
rate of arrival of information so obviously. When we combine media, sound and vision, for
example, we typically expect the combined media on a recording (or seen remotely) to maintain
the temporal relationship that they had at source. This is what really defines data as being
multimedia. Hypermedia is multimedia that is arranged with non-linear relations between sub-
sequences.
Compression, and Hierarchical encoding are also needed. Multimedia data is typically much
more bulky than text or numeric data. A typical simple-minded sampled audio sequence might
take 8K bytes per second. This compares badly with 8K bytes of text: Assume we had 10
characters per word, then this would constitute 800 words, and might take a quick speaker
something like a minute to read aloud. In other words, the speech requires at least two orders of
magnitude more bytes than text. Video is far worse still, although clearly, comparisons are more
difficult, since the value of typical information content is quite different.
All of this means that we need to consider compression techniques, to save storage and
transmission capacity. Luckily, much audio and video is redundant (contains effectively repeated
or less useful data) and is often far more amenable to compression than text.
Meta-languages (codes for codings) are required. Typically, while we are still evolving a wide
range of codings and compression techniques, we need protocols for exchange of media between
different systems. We also need protocols to relate the different media (for synchronisation and
for hypermedia).
Next, lets look at some audio and video input forms and digital encodings.
Audio and Video all start life in the ``Analog Domain''. (Domain is used in this context just to
mean before or after some particular conversion). It is important to understand the basic
requirements of the media in time and space. The analog domain is usually best understood in
terms of the range of frequencies in use for a particular quality. For sound, this means how low
and high a note/sound is allowed. For video, this translates into the number of distinguishable
colours. For video, we also have to consider the frame rate. Video is similar to film in that it
consists of a number of discrete frames. You may recall seeing old films which were shot at a
lower frame rate than is used nowadays, and flicker is visible. To refine this point further, we
should distinguish between the rate at which a scene is sampled, and the rate at which a frame on
a screen is displayed. For many moving image systems, these may be different. For example,
films may show the same frame more than once to reduce flicker. Although Cathode Ray Tubes
have significant persistence, video systems may refresh different parts of the screen at different
rates - interlacing is used in many systems where alternate lines of the screen are refreshed in

Adri Jovin J.J.
Page | 4
alternate cycles. This is motivated by the possible reduction in bandwidth, both in the analog and
digital domain.
Both sound and image can be broken down at any instant into a set of basic frequencies. This is
the so-called ``waveform''. We can record all of the frequencies present at anyone time, or we
can choose to record only the ``important'' ones. If we choose to record less than all frequencies,
we get less ``fidelity'' in our recording, so that the playback is less like the original. However, the
less we record, the less tape/recording media we need.
Audio and Video start as waves, a sequence of compression and rare-faction of air, or the
fluctuation of an electric and magnetic field, with time.
Waves need to be captured, by some device, and then sampled digitally. Typically, a ``sample-
and-hold'' technique is used: An electro-mechanical or light sensitive device responds to the
sound or light, and produces an analogue electrical signal. This can be averaged over a sample
period by providing some discrete clock signal, and an averaging circuit. The value during this
sample period can then be converted to a digital value of a given accuracy (``quantized'').
We can do this sampling ``perfectly'' by sampling twice as often digitally as the highest analog
frequency, or we can take advantage of human frailty and reduce the quality by decreasing the
sample frequency (the clock rate above, and/or the quantization (number of bits used per
sample).
The term ``bandwidth'' is used by electrical engineers to refer to the frequency range of an analog
signal. Often, especially in the Internet Community, the term is used loosely to refer to channel
capacity, or the bit rate of a link. Note that because of sampling, quantization, and compression,
the bit rate needed for a given bandwidth analog signal is potentially many times (orders of
magnitude even) less than the perfect sample signal requirement would imply.
Analog Audio for humans is roughly in the range 50Hz to 20KHz. Human speech is intelligible,
typically even when restricted to the range 1-3KHz, and the telephone networks have taken
advantage of this since very early days by providing only limited quality lines. This has meant
that they can use low quality speakers and microphones in the handset - the quality is similar to
AM radio.
It is not entirely a coincidence, therefore that the copper wires used for transmission in the
telephone system were principally chosen for the ability to carry a baseband signal that could
convey speech (``toll'') quality audio.
In most systems luckily, phone wires are over-engineered. They are capable of carrying a signal
at up to 16 times the ``bandwidth'', of that used by pure analog phones from the home to the
exchange over a kilometre, and 300 times this bandwidth up to 100 meters. For the moment,
though, the ``last mile'' or customer subscriber-loop circuits have boxes at the ends that limit this
to what is guaranteed for ordinary audio telephony, while the rest of the frequencies are used for
engineering work.

Adri Jovin J.J.
Page | 5
Video signals, on the other hand, occupy a much wider frequency range. Analog TV, which
defined for about 60 years the input, output and transmission standards, has several different
standards, but is typically amplitude modulated on a 3.58MHz carrier. The signal that is
conveyed on this is a sequence of ``scanlines'', each making up a screen. The scanline is
essentially a sample of the brightness and colors across a horizontal line as detected in the
camera, and as used to control the electron gun in the TV monitor.
In the CCIR 601 standard, a digital version of this is defined, which two samples of 8 bits each
of colour (chrominance) and one of brightness (luminance) at 13/5MHz. The resulting data rate
is around 166Mbps per second.
It is not entirely a co-incidence that old cable TV networks are capable of transmitting these data
rates; however modern hybrid fiber-coax networks are targeted at carrying a much larger number
of compressed digital channels.
The purpose of talking about the media encoding and channel capacity requirements is to show
the relationship between particular media, and the transmission technology associated with them.
The two largest networks in the world in terms of terminals are the phone network and the TV
network. Each addresses a particular capacity and pattern of communication. If a data network
such as the Internet is to carry these media, and if the ``terminals'', or workstations and PCs of
the Internet are to be able to capture, store, transmit, receive, and display such media, then the
network and end systems have to deal with these types of data, one way or another. If we
compress the data, and decompress at a receiver, then the rate/capacity is till required outside of
the compressed domain, and the compression/decompression engines need to be able to cope
with this, even though they may spare the network (or storage device).
The word ``encoding'' is often used as a noun as well as a verb when talking about multimedia.
Nowadays, there is a vast range of encodings currently in use or development. There are a
variety of reasons for this: Codes for audio, video depend on the quality of audio or video
required. A very simple example of this is the difference between digital audio for ISDN
telephones (64Kbps PCM see later) and for CD (1.4Mbps 16 bit etc.) 1.1; another reason for the
range of encodings is that some encodings include linkages to other media for reasons of
synchronization (e.g. between voice and lips); yet another reason is to provide future proofing
against any new media (holograms?); finally, because of the range of performance of different
computers, it may be necessary to have a ``meta-protocol'' to negotiate what is used between
encoder and decoder. This permits programs to encode a stream of media according to whatever
is convenient to them, while a decoder can then decode it according to their capabilities. For
example, some HDTV (High Definition Television Standards) are actually a superset of current
standard TV encoding so that a ``rougher'' picture can be extracted by existing TV receivers from
new HDTV transmissions (or from paying back new HDTV videotapes). This principle is quite
general.
Digital Sound
Digital audio is created when you represent the characteristics of a sound wave using numbers—
a process referred to as digitizing. You can digitize sound from a microphone, a synthesizer,

Adri Jovin J.J.
Page | 6
existing recordings, live radio and television broadcasts, and popular CD and DVDs. In fact, you
can digitize sounds from any natural or prerecorded source.
Digitized sound is sampled sound. Every nth fraction of a second, a sample of sound is taken and
stored as digital information in bits and bytes. The quality of this digital recording depends upon
how often the samples are taken (sampling rate or frequency, measured in kilohertz, or thousands
of samples per second) and how many numbers are used to represent the value of each sample
(bit depth, sample size, resolution, or dynamic range). The more often you take a sample and the
more data you store about that sample, the finer the resolution and quality of the captured sound
when it is played back. Since the quality of your audio is based on the quality of your recording
and not the device on which your end user will play the audio, digital audio is said to be device
independent.
The three sampling rates most often used in multimedia are 44.1 kHz (CD-quality), 22.05 kHz,
and 11.025 kHz. Sample sizes are either 8 bits or 16 bits. The larger the sample size, the more
accurately the data will describe the recorded sound. An 8-bit sample size provides 256 equal
measurement units to describe the level and frequency of the sound in that slice of time. A 16-bit
sample size, on the other hand, provides a staggering 65,536 equal units to describe the sound in
that same slice of time. As you can see in Figure 4-1, slices of analog waveforms are sampled at
various frequencies, and each discrete sample is then stored either as 8 bits or 16 bits (or more)
of data.
Figure 4-1 It is impossible to reconstruct the original waveform if the sampling frequency is too
low.
The value of each sample is rounded off to the nearest integer (quantization), and if the
amplitude is greater than the intervals available, clipping of the top and bottom of the wave
occurs (see Figure 4-2). Quantization can produce an unwanted background hissing noise, and
clipping may severely distort the sound.

Adri Jovin J.J.
Page | 7
Figure 4-2 Examples of quantizing and clipping
Making Digital Audio Files
Making digital audio files is fairly straightforward on most computers. Plug a microphone into
the microphone jack of your computer. If you want to digitize archived analog source
materials—music or sound effects that you have saved on videotape, for example—simply plug
the “Line-Out” or “Headphone” jack of the device into the “Line-In” jack on your computer.
Then use audio digitizing software such as Audacity (see Figure 4-3), to do the work.
You should focus on two crucial aspects of preparing digital audio files:
• Balancing the need for sound quality against file size. Higher quality usually means
larger files, requiring longer download times on the Internet and more storage space on a
CD or DVD.
• Setting proper recording levels to get a good, clean recording.
Digital Video
In digital systems, the output of the CCD is digitized by the camera into a sequence of single
frames, and the video and audio data are compressed before being written to a tape (see Figure 6-
2) or digitally stored to disc or flash memory in one of several proprietary and competing
formats. Digital video data formats, especially the codec used for compressing and
decompressing video (and audio) data, are important.
Figure 6-2 Diagram of tape path across the video head for digital recording

Adri Jovin J.J.
Page | 8
In 1995, Apple’s FireWire technology was standardized as IEEE 1394, and Sony quickly
adopted it for much of its digital camera line under the name i.Link. FireWire and i.Link (and
USB 2) cable connections allow a completely digital process, from the camera’s CCD to the hard
disk of a computer; and camcorders store the video and sound data on an onboard digital tape,
writable mini-DVD, mini–hard disk, or flash memory.
HDTV
What started as the High Definition Television (HDTV) initiative of the Federal
Communications Commission in the 1980s changed first to the Advanced Television (ATV)
initiative and then finished as the Digital Television (DTV) initiative by the time the FCC
announced the change in 1996. This standard, which was slightly modified from both the Digital
Television Standard (ATSC Doc. A/53) and the Digital Audio Compression Standard (ATSC
Doc. A/52), moved U.S. television from an analog to a digital standard. It also provided TV
stations with sufficient bandwidth to present four or five Standard Television (STV, providing
the NTSC’s resolution of 525 lines with a 3:4 aspect ratio, but in a digital signal) signals or one
HDTV signal (providing 1,080 lines of resolution with a movie screen’s 16:9 aspect ratio).
HDTV provides high resolution in a 16:9 aspect ratio (see Figure 6-3). This aspect ratio allows
the viewing of Cinemascope and Panavision movies. There was contention between the
broadcast and computer industries about whether to use interlacing or progressive-scan
technologies. The broadcast industry promulgated an ultra-high-resolution, 1920 × 1080
interlaced format (1080i) to become the cornerstone of the new generation of high-end
entertainment centers, but the computer industry wanted a 1280 × 720 progressive-scan system
(720p) for HDTV. While the 1920 × 1080 format provides more pixels than the 1280 × 720
standard, the refresh rates are quite different. The higher resolution interlaced format delivers
only half the picture every 1/60 of a second, and because of the interlacing, on highly detailed
images there is a great deal of screen flicker at 30 Hz. The computer people argue that the picture
quality at 1280 × 720 is superior and steady. Both formats have been included in the HDTV
standard by the Advanced Television Systems Committee (ATSC), found at www.atsc.org.
Figure 6-3 Here you can see the difference between VGA and HDTV aspect ratios.

Adri Jovin J.J.
Page | 9
Graphics
Still images may be small or large, or even full screen. They may be colored, placed at random
on the screen, evenly geometric, or oddly shaped. Still images may be a single tree on a wintry
hillside; stacked boxes of text against a gray, tartan, or Italian marble background; an
engineering drawing; a snapshot of your department manager’s new BMW. Whatever their form,
still images are generated by the computer in two ways: as bitmaps (or paint graphics) and as
vector-drawn (or just plain “drawn”) graphics. Bitmaps may also be called “raster” images.
Likewise, bitmap editors are sometimes called “painting” programs. And vector editors are
sometimes called “drawing” programs.
Bitmaps are used for photo-realistic images and for complex drawings requiring fine detail.
Vector-drawn objects are used for lines, boxes, circles, polygons, and other graphic shapes that
can be mathematically expressed in angles, coordinates, and distances. A drawn object can be
filled with color and patterns, and you can select it as a single object. The appearance of both
types of images depends on the display resolution and capabilities of your computer’s graphics
hardware and monitor. Both types of images are stored in various file formats and can be
translated from one application to another or from one computer platform to another. Typically,
image files are compressed to save memory and disk space; many bitmap image file formats
already use compression within the file itself—for example, GIF, JPEG, and PNG.
Still images may be the most important element of your multimedia project or web site. If you
are designing multimedia by yourself, put yourself in the role of graphic artist and layout
designer. Take the time necessary to discover all the tricks you can learn about your drawing
software. Competent, computer-literate skills in graphic art and design are vital to the success of
your project. Remember—more than anything else, the user’s judgment of your work will be
heavily influenced by the work’s visual impact.
Bitmaps
A bit is the simplest element in the digital world, an electronic digit that is either on or off, black
or white, or true (1) or false (0). This is referred to as binary, since only two states (on or off) are
available. A map is a two dimensional matrix of these bits. A bitmap, then, is a simple matrix of
the tiny dots that form an image and are displayed on a computer screen or printed.
A one-dimensional matrix (1-bit depth) is used to display monochrome images—a bitmap where
each bit is most commonly set to black or white. Depending upon your software, any two colors
that represent the on and off (1 or 0) states may be used. More information is required to describe
shades of gray or the more than 16 million colors that each picture element might have in a color
image, as illustrated in Figure 3-1. These picture elements (known as pels or, more commonly,
pixels) can be either on or off, as in the 1-bit bitmap, or, by using more bits to describe them, can
represent varying shades of color (4 bits for 16 colors; 8 bits for 256 colors; 15 bits for 32,768
colors; 16 bits for 65,536 colors; 24 bits for 16,772,216 colors). Thus, with 2 bits, for example,
the available zeros and ones can be combined in only four possible ways and can, then, describe
only four possible colors:

Adri Jovin J.J.
Page | 10
Figure 3-1 A bitmap is a data matrix that describes the characteristics of all the pixels making up
an image. Here, each cube represents the data required to display a 4 × 4–pixel image (the face of
the cube) at various color depths (with each cube extending behind the face indicating the
number of bits— zeros or ones—used to represent the color for that pixel).
Together, the state of all the pixels on a computer screen make up the image seen by the viewer,
whether in combinations of black and white or colored pixels in a line of text, a photograph-like
picture, or a simple background pattern.
Vector Drawing
Most multimedia authoring systems provide for use of vector-drawn objects such as lines,
rectangles, ovals, polygons, complex drawings created from those objects, and text.
• Computer-aided design (CAD) programs have traditionally used vector-drawn object
systems for creating the highly complex and geometric renderings needed by architects
and engineers.
• Graphic artists designing for print media use vector-drawn objects because the same
mathematics that put a rectangle on your screen can also place that rectangle (or the fancy
curves of a good line-art illustration) on paper without jaggies. This requires the higher
resolution of the printer, using a page description format such as Portable Document
Format (PDF).
• Programs for 3-D animation also use vector-drawn graphics. For example, the various
changes of position, rotation, and shading of light required to spin an extruded corporate
logo must be calculated mathematically.
A vector is a line that is described by the location of its two endpoints.

Adri Jovin J.J.
Page | 11
Vector drawing uses Cartesian coordinates where a pair of numbers describes a point in two-
dimensional space as the intersection of horizontal and vertical lines (the x and y axes). The
numbers are always listed in the order x,y. In three-dimensional space, a third dimension—
depth— is described by a z axis (x,y,z). This coordinate system is named for the French
philosopher and mathematician, René Descartes. So a line might be simply
<line x1="0" y1="0" x2="200" y2="100">
where x1 and y1 define the starting point (in the upper-left corner of the viewing box) and x2 and
y2 define the end point. A simple rectangle is computed from starting point and size: your
software will draw a rectangle (rect) starting at the upper-left corner of your viewing area (0,0)
and going 200 pixels horizontally to the right and 100 pixels downward to mark the opposite
corner. Add color information like
<rect x="0" y="0" width="200" height="100" fill="#FFFFFF" stroke="#FF0000"/>
and your software will draw the rectangle with a red boundary line and fill it with the color
white. You can, of course, add other parameters to describe a fill pattern or the width of the
boundary line. Circles are defined by a location and a radius:
<circle cx="50" cy="50" r="10" fill="none" stroke="#000000" />
Type the following code into a text editor and save it as plain text with a .svg extension. This is a
Scalable Vector Graphics file. Open it in an HTML5-capable browser (File:Open File…) and
you will see:
<svg xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
width="200"
height="200"
viewBox="-100 -100 300 300">
<rect x="0" y="0" fill="yellow" stroke="red" width="200" height="100"/>
<text transform="matrix(1 0 0 1 60 60)" font-family="'TimesNewRomanPS-BoldMT'" font-
size="36">SVG</text>
</svg>
Because these SVG files can be saved in a small amount of memory and because they are
scalable without distortion (try changing the width and height of the view box in the preceding
code), SVG (Tiny) is supported by browsers on most mobile phones and PDAs. The SVG
specification also includes timebased changes or animations that can be embedded within the
image code (see www.w3.org/TR/SVG11/animate.html#AnimationElements). Figure 3-8 shows
Adobe Illustrator saving a file in SVG format. Vector drawing tools use Bézier curves or paths to
mathematically represent a curve. In practical terms, editing software shows you points on the
path, each point having a “handle.” Changing the location of the handle changes the shape of the
curve. Mastering Bézier curves is an important skill: these curves not only create graphic shapes
but represent motion paths when creating animations.

Adri Jovin J.J.
Page | 12
Figure 3-8 Drawing software such as Adobe Illustrator can save vector graphics in SVG format.
Vector-Drawn Objects vs. Bitmaps
Vector-drawn objects are described and drawn to the computer screen using a fraction of the
memory space required to describe and store the same object in bitmap form. The file containing
the vector-drawn colored rectangle described in the preceding section is less than 698 bytes of
alphanumeric data (even less—468 bytes—when the description is tokenized or compressed as
.svgz). On the other hand, the same rectangle saved as a .gif image with a 64-color palette takes
1,100 bytes. Because of this file size advantage, web pages that use vector graphics as SVG files
or in plug-ins such as Flash download faster and, when used for animation, draw faster than
pages displaying bitmaps. It is only when you draw many hundreds of objects on your screen
that you may experience a slowdown while you wait for the screen to be refreshed—the size,
location, and other properties for each of the objects must be computed. Thus, a single image
made up of 500 individual line and rectangle objects, for example, may take longer for the
computer to process and place on the screen than an image consisting of just a few drawn circle
objects. A vector-drawn object is created “on the fly,” that is, the computer draws the image
from the instructions it has been given, rather than displaying a precreated image. This means
that vector objects are easily scalable without loss of resolution or image quality. A large drawn
image can be shrunk to the size of a postage stamp, and while it may not look good on a
computer monitor at 72 dpi, it may look great when printed at 300 dpi to a color printer. Resizing
a bitmapped image requires either duplicating pixels (creating a blocky, jagged look called
pixelation) or throwing pixels away (eliminating details). Because vector images are drawn from
instructions on the fly, a rescaled image retains the quality of the original.
Basic Multimedia Networking
The Internet began as a research network funded by the Advanced Research Projects Agency
(ARPA) of the U.S. Defense Department, when the first node of the ARPANET was installed at
the University of California at Los Angeles in September 1969. By the mid-1970s, the
ARPANET “inter-network” embraced more than 30 universities, military sites, and government
contractors, and its user base expanded to include the larger computer science research

Adri Jovin J.J.
Page | 13
community. By 1983, the network still consisted of merely several hundred computers on only a
few local area networks.
In 1985, the National Science Foundation (NSF) aligned with ARPA to support a collaboration
of supercomputing centers and computer science researchers across the ARPANET. The NSF
also funded a program for improving the backbone of the ARPANET, by increasing its
bandwidth from 56 Kbps to T1 and then T3 (see “Connections” a little later in the chapter for
more information) and branching out with links to international sites in Europe and the Far East.
In 1989, responsibility and management for the ARPANET was officially passed from military
interests to the academically oriented NSF, and research organizations and universities
(professors and students alike) became increasingly heavy users of this ever-growing “Internet.”
Much of the Internet’s etiquette and rules for behavior (such as for sending e-mail and posting to
newsgroups) was established during this time. More and more private companies and
organizations linked up to the Internet, and by the mid-1990s, the Internet included connections
to more than 60 countries and more than 2 million host computers with more than 15 million
users worldwide. Commercial and business use of the Internet was not permitted until 1992, but
businesses have since become its driving force. By 2001 there were 109,574,429 domain hosts
and 407.1 million users of the Internet, representing 6.71 percent of the world’s population. By
the beginning of 2010 (see Table 12-1), about one out of every four people around the world
(26.6 percent) had access to the Internet, and more than 51 million domain names had been
registered as “dot coms.”
Protocols
Protocols are the rules needed for communication. In human and computer communication,
standard protocols eliminate confusion and wasted time misunderstanding one and other.
In computer communication, protocols comprise three main components: interfaces, which
define the rules for using a protocol, and provide a service to the software that uses the interface;
packet formats, which define the it syntax for the exchange of messages between local and
remote systems; procedures, which define the operational rules concerning which packets can be
exchanged when.
Communication systems are built up out of multiple layered protocols. The concept of layering is
twofold: firstly, common services can be built in all devices or subsystems, and specialised
services built out of these for those devices or subsystems that need them; secondly, the details
of operation of local, or technology specific features of a protocol can be hidden by one layer
from the layer it above.
In this book, we illustrate these different aspects protocols in several different ways: when we
want to show the layering of protocols, we make use of a stack diagram; when we want to show
the operational rules of a protocol, we sometimes use a time-sequence diagram such as 1.3,
which portraits a particular instance of exchange of packets; in some places, we document the
layout of the packets themselves, to show what control functions are conveyed in each
transmission, reception or exchange.

Adri Jovin J.J.
Page | 14
In the rest of this chapter, we introduce these aspects of the Internet multimedia protocols, as
well as the more subtle question of performance. Later chapters in the book cover these sections
in more detail.
First of all, we look at names, addresses and routes.
Names, Addresses and Routes
The user is usually very aware of the names of objects in the Internet. We frequently see
reference to World Wide Web sites and pages (e.g. http://www.cs.ucl.ac.uk/staff/jon, which
incorporate Domain Name System names for a system or service in the network. A name
effectively tells the user what something is.
Names are useful to human users for this very reason, but are unwieldy and inefficient for
computer systems. Typically, they are mapped into addresses, using a directory or nameservice.
Addresses then tell a piece of software (or the network operator or manager) where something is.
Each host in the network has a unique address of (currently) 32 bits (128 bits in IPv6.) More
accurately, each interface on each host in the Internet has a unique address, which is made up of
two components:
Host Part ; Network Part
Table 1.1: The Structured Internet Address
All IP packets carry both a source and a destination address. Typically, the network part is
indicated by applying a mask of some number of bits to the address, leaving the bits for the host
part. Common divisions are 24 bits for network, and 8 bits for host, and 16 bits for network part
and 16 for the host part..
The network part of the destination address is used by routers effectively to index routing tables
to figure out where a packet must be delivered to. In fact, a ``longest match'' lookup of the prefix
or network part must be done which can be difficult to optimise. Recent work[#!Rout!#] has
achieved very good lookup times and table sizes.
All packets in the Internet Protocol layer carry these addresses, so that they can be delivered to
the right destination, and so that we can determine where they came from. As we look at other
layers in the Internet Multimedia protocol stack, we will see what other control information is
carried in packets to carry out additional functions.
Internet Multimedia Protocols
The overall protocol architecture that makes up the capability to deliver multimedia over what
was originally a pure data network, is surprisingly not so very different from the original Internet
Architecture.

Adri Jovin J.J.
Page | 15
Figure 1.1: The Internet Multimedia Protocol Stack
The protocol stacks for Internet multimedia are show in figure 1.1 above. Most of the protocols
are not deeply layered unlike many other protocol stacks, but rather are used alongside each
other to produce a complete session. It is possible to use multimedia applications over the
Internet without some (or all) of the additional protocols (e.g. omission of RSVP, or omission of
SDP, and so on) depending on the performance or functionality required. Later chapters will
show how each new protocol adds value to the basic IP packet delivery model.
In the next section, we discuss the underlying unicast and multicast delivery model, which must
be added within the IP layer to give maximum benefit to the network.
Packet Switched Data networking adds value to circuit switched networks by sharing the
capacity amongst multiple users in time. The data stream from each user is broken into chunks to
form packets, which only require capacity when each packet is being sent. The capacity required
is thus the sum of the average bandwidth, rather than the total peak bandwidth. This ``statistical
multiplexing gain'' comes at the expense of having sometimes to queue packets for access to the
transmission channel.
The statistical multiplexing gain is quite large for traditional bursty data applications such as
WWW access. For multimedia traffic, this gain is harder to achieve (depending on the
compression algorithms used, as discussed in later chapters, it can even become even harder) and
yet we can get a spatial gain in use of the network by using group communication carefully. A
simple way to send data to multiple recipients is to send it multiple times from the source. Such
``multiple unicasts'' make very poor use of the links near the sender, and potentially incur a lot of
delay before a single transmission is completed. The Internet offers a mechanism to avoid these
overheads called IP multicast.
Multicast is essential in the context of Internet technologies that may replace television
(streaming services, as discussed in chapter nine), but it is also highly relevant in telephony
services, especially in the business community: it is extremely common for one phone call to
spawn another - the ability to teleconference is not widespread in the traditional plain old

Adri Jovin J.J.
Page | 16
telephone system except at some commercial cost, or in a restricted subset of telephones. The
Internet ability to provide this service is quite powerful.
We discuss some of the limitations of multicast as a service in chapter three. Some of the
problems that vendors have had in supporting large scale multicast as well as service guarantees
in Router products have prompted the design of a new generation of routers coming on to the
market as we write. Similarly, the understanding of service guarantees and multicast has been
slow to diffuse into the commercial Internet Service Provider community until relatively
recently, but this is changing rapidly.
Multimedia Characteristics
A multimedia system has four basic characteristics:
• Computer is an intrinsic part of the multimedia system. As a result, multimedia has
become interactive. In multimedia, computer allows the user to interact with the media
and thus manipulate it by controlling what is to be communicated and when. Multimedia
has resulted in the creation of many new possibilities—(1) the computational power of
computer is utilized for multimedia applications, (2) the telecommunication network
(Internet, WWW) along with the computer enables transmission and distribution of
information, and, (3) the use of computer facilitates design and creation of a variety of
new applications.
• The different elements of multimedia are combined and integrated into a single
multimedia system. Special software is required for the integration of different media
element files.
• The use of computer in multimedia requires all elements of multimedia to be in digital
format. In a digital multimedia system, the media streams are digital and are produced,
processed, stored, represented and transmitted using computers. The digital nature of
multimedia requires special treatment of the multimedia elements. The hardware and
software are needed to convert multimedia elements from analog to digital format and
vice versa. There is a need to decide about the resolution versus quality of output
required, during storing of data in the computer. Storing multimedia files on computer
hard disk takes large amount of disk space, so compression technologies and file formats
for storing the different media elements is required. Moreover, special programs are
required to play the compressed files. Similarly, special software is required to edit the
different media element files, and to combine and integrate the different elements of the
multimedia into a single multimedia system.
• Multimedia system is interactive. The user is active and can manipulate whatever is being
communicated. Multimedia allows two-way communication. The user can use devices
like keyboard, trackball or joystick to interact with the multimedia system. Interactive
multimedia is non-linear. The user is able to follow the links and jump from one part of
the document to the other. Hypermedia enables a user to gain or provide access to text,
audio and video, and computer graphics using links in a non-linear way, using computers.

Adri Jovin J.J.
Page | 17
World Wide Web (WWW) is an example of hypermedia application. The user is able to
respond and control what to see or hear and when to do it.
Evolution of Internet Service Models
Traditionally the Internet has provided best-effort delivery of datagram traffic from senders to
receivers. No guarantees are made regarding when or if a datagram will be delivered to a
receiver. However datagrams are normally only dropped when a router exceeds a queue size
limit due to congestion. The best-effort Internet service model does not assume first-in-first-out
(FIFO, also known as first-come-first-served) queueing, although many routers have
implemented this. The effect is to provide rather unfair distribution of resources.
With best-effort service, if a link is not congested, queues will not build at routers, datagrams
will not be discarded in routers, and delays will consist of serialisation delays at each hop plus
propagation delays. With sufficiently fast link speeds, serialisation delays are insignificant
compared to propagation delays. However, if a link is congested, with best-effort service
queueing delays will start to influence end-to-end delays, and packets will start to be lost as
queue size limits are exceeded.
Non-best effort service
Real-time internet traffic is defined as datagrams that are delay sensitive. ``Real-time'' is an oft-
misused term, and we are guilty here too. In process control systems, telemetry monitoring and
so on, real-time really refers to systems with drop dead deadlines, after which information is
irretrievably lost, or catastrophic consequences ensue if deadlines are missed. In multimedia
systems, while we might have data with real time delivery requirements, we are at liberty to lose
it without necessarily losing much information. We are also at liberty to relax schedules for
delivery, since humans are tolerant creatures compared with machines. It could be argued that all
datagrams are delay sensitive to some extent, but for these purposes we refer only to datagrams
where exceeding an end-to-end delay bound of a few hundred milliseconds renders the
datagrams useless for the purpose they were intended. For the purposes of this definition, TCP
traffic is normally not considered to be real-time traffic, although there may be exceptions to this
rule.
On congested links, best-effort service queueing delays will adversely affect real-time traffic.
This does not mean that best-effort service cannot support real-time traffic - merely that
congested best-effort links seriously degrade the service provided. For such congested links, a
better-that-best-effort service is desirable.
To achieve this, the service model of the routers can be modified. At a minimum, FIFO queueing
can be replaced by packet forwarding strategies that discriminate different ``flows'' of traffic.
The idea of a flow is very general. A flow might consist of `àll marketing site web traffic'', or
`àll file server traffic to and from teller machines'' or `àll traffic from the CEO's laptop
wherever it is''. On the other hand, a flow might consist of a particular sequence of packets from

Adri Jovin J.J.
Page | 18
an application in a particular machine to a peer application in another particular machine
between specific times of a specific day.
Flows are typically identifiable in the Internet by the tuple: source machine, destination machine,
source port, destination port, protocol any of which could be ``ANY'' (wild-carded).
In the multicast case, the destination is the group, and can be used to provide efficient
aggregation.
Flow identification is called classification and a class (which can contain one or more flows) has
an associated service model applied. This can default to best effort.
Through network management, we can imagine establishing classes of long lived flows -
enterprise networks (``Intranets'') often enforce traffic policies that distinguish priorities which
can be used to discriminate in favour of more important traffic in the event of overload (though
in an under-loaded network, the effect of such policies will be invisible, and may incur no
load/work in routers).
The router service model to provide such classes with different treatment can be as simple as a
priority queueing system, or it can be more elaborate.
Although best-effort services can support real-time traffic, classifying real-time traffic separately
from non-real-time traffic and giving real-time traffic priority treatment ensures that real-time
traffic sees minimum delays. Non-real-time TCP traffic tends to be elastic in its bandwidth
requirements, and will then tend to fill any remaining bandwidth.
We could imagine a future Internet with sufficient capacity to carry all of the world's telephony
traffic. Since this is a relatively modest capacity requirement, it might be simpler to establish
``POTS'' (Plain Old Telephone System) as a static class which is given some fraction of the
capacity overall, and then no individual call need be given an allocation (i.e. we would no longer
need the call setup/tear down that was needed in the legacy POTS which was only present due to
under-provisioning of trunks, and to allow the trunk exchanges the option of call blocking). The
vision is of a network that is engineered with capacity for all of the average load sources to send
all the time.
Reservations
For flows that may take a significant fraction of the network (i.e. are ``special'') , we need a more
dynamic way of establishing these classifications. In the short term, this applies to any
multimedia calls since the Internet is largely under-provisioned at time of writing.
The Resource Reservation Protocol, RSVP is being standardised for just this purpose. It provides
flow identification and classification. Hosts and applications are modified to speak RSVP client
language, and routers speak RSVP.

Adri Jovin J.J.
Page | 19
Since most traffic requiring reservations is delivered to groups (e.g. TV), it is natural for the
receiver to make the request for a reservation for a flow. This has the added advantage that
different receivers can make heterogeneous requests for capacity from the same source. Again
the routers conspire to deliver the right flows to the right locations. RSVP accommodates the
wild-carding noted above.
Admission Control
If a network is provisioned such that it has excess capacity for all the real-time flows using it, a
simple priority classification ensures that real-time traffic is minimally delayed. However, if a
network is insufficiently provisioned for the traffic in a real-time traffic class, then real-time
traffic will be queued, and delays and packet loss will result. Thus in an under-provisioned
network, either all real-time flows will suffer, or some of them must be given priority.
RSVP provides a mechanism by which an admission control request can be made, and if
sufficient capacity remains in the requested traffic class, then a reservation for that capacity can
be put in place.
If insufficient capacity remains, the admission request will be refused, but the traffic will still be
forwarded with the default service for that traffic's traffic class. In many cases even an admission
request that failed at one or more routers can still supply acceptable quality as it may have
succeeded in installing a reservation in all the routers that were suffering congestion. This is
because other reservations may not be fully utilising their reserved capacity.
Accounting
If a reservation involves setting aside resources for a flow, this will tie up resources so that other
reservations may not succeed, and depending on whether the flow fills the reservation, other
traffic is prevented from using the network. Clearly some negative feedback is required in order
to prevent pointless reservations from denying service to other users. This feedback is typically
in the form of billing. For real-time non-best effort traffic that is not reserved, this negative
feedback is provided in the form of loss due to congestion of a traffic class, and it is not clear
that usage based billing is required.
Billing requires that the user making the reservation is properly authenticated so that the correct
user can be charged. Billing for reservations introduces a level of complexity to the Internet that
has not typically been experienced with non-reserved traffic, and requires network providers to
have reciprocal usage-based billing arrangements for traffic carried between them. It also
requires mechanisms whereby some fraction of the bill for a link reservation can be charged to
each of the downstream multicast receivers.
Recent work on charging[#!kelly!#] has proposed quite simple models of billing associated with
multimedia traffic. A generalised model for pricing bursty connections (or flows in our context)
was proposed in [#!burst!#]:
a * V + b * T + c

Adri Jovin J.J.
Page | 20
where V is the traffic volume at the minimum requested rate (can be zero) and T is the time at
the average (measured) rate. The parameters a, b and c depend on the tarrifing scheme; e.g. peak
rate, or IP subscriber's line rate, plus equipment rental. A minimum rate (e.g. MCR or controlled
load) gives a volume related charge (constant also factors in providers' dimensioning) and a
mean rate (e.g. for VBR, or guaranteed) gives a time related charge; mixes are allowed. Delay
bound can be simply a boolean in the QoS API, which is easily implemented as the ToS delay
preference bit in IP header; in most cases, this just selects a priority queue, although sometimes it
selects a terrestrial over satellite route. For multimedia applications, which will probably initially
use a service that approximates to a lightly loaded network, we get a similar charge model as for
TCP, which may be surprising to people, but it has lots of nice practical properties: for example,
a typical network that permits ``premium service'' TCP and Internet telephony, might implement
the interface for the user as a single bit to select between normal TCP service (MCR==0), and
controlled load TCP as well as telephony. The option for how to actually bill could be based on
measured volume over time (the interval of an RTT is probably reasonable for both telephony
and TCP based traffic), or might simply relate to the fact that most domestic users access the net
over a dial up link, so the time related charge could be folded in there trivially, and the volume
related charge based on measured mean rate - conveniently, the access line polices the peak rate
trivially, since it is typically a modem+phone line or digital ISDN line, with a fixed line speed
anyhow.
Network Requirements for Audio/Video Transform
If your computer is connected to an existing network at an office or school, it is likely you are
already connected to the Internet. If you are an individual working from home, you will need a
telephone dial-up account or broadband cable, Digital Subscriber Line (DSL), or wireless
equipment to connect to the backbone (the ultra-high-bandwidth underlying network operated by
MCI, AT&T, Sprint, and other telecommunications companies) of the Internet through an
Internet service provider (ISP).
Bandwidth is how much data, expressed in bits per second (bps), you can send from one
computer to another in a given amount of time. The faster your transmissions (or the greater the
bandwidth of your connection), the less time you will spend waiting for text, images, sounds, and
animated illustrations to upload or download from computer to computer, and the more
satisfaction you will have with your Internet experience. To think in bytes per second, divide the
rate by eight. Table 12-2 lists the bandwidth of some common data transfer methods.
Table 12-2 Bandwidth of Typical Internet and Computer Connections

Adri Jovin J.J.
Page | 21
Table 12-2 Bandwidth of Typical Internet and Computer Connections
The bottleneck at a typical user’s low-bandwidth modem connection is the most serious
impediment to sending multimedia across the Internet. At low bandwidth, a page of text (3,000
bytes) can take less than a second to send, but an uncompressed 640 × 480, 8-bit/256-color
image (about 300,000 bytes) can take a few minutes; an uncompressed 640 × 480, 24-bit/16
million-color image (about 900,000 bytes) can take many minutes to send. Occasionally also,
even though you may have a high-speed connection, the server delivering your requested file or
content may be “throttled down” while it manages many requests at once, and yours must wait
its turn.
To work within the constraints of bandwidth bottlenecks, multimedia developers on the Internet
have but a few options:
• Compress data as tightly as possible (into ZIP or SIT or TAR files) before transmitting.
• Require users to download data only once; then store the data in a local hard disk cache
(this is automatically managed by most browsers).
• Design each multimedia element to be efficiently compact—don’t use a greater color
depth than is absolutely necessary or leave extra space around the edges.
• Design alternate low-bandwidth and high-bandwidth navigation paths to accommodate
all users.

Adri Jovin J.J.
Page | 22
• Implement streaming methods that allow data to be transferred and displayed
incrementally as it comes in (without waiting for the entire data file to arrive).
Multimedia Coding and Compression for Text, Image, Audio and Video
A key problem with multimedia is the sheer quantities of data that result from naive digitisation
of audio, image or video sources. Other problems involve quality, representation of meta data
such as timing and relationshgips between different media and so on.
There are a variety of compression techniques commonly used in the Internet and other systems
to alleviate the storage, processing and transmission (and reception) costs for such data.
We start by building a framework for understanding the systems requirements and components
in dealing with multimedia flows - to start with, we look at the nature of the information and its
use, leading to discussion of general principles of loss free and lossy compression. We look at
simple lossless schemes such as Run Length Encoding and systems based on the statistics of
frequency of occurrences of codewords such as Huffman codes. We look at substitutional or
dictionary based schemes such as the Lemple-Ziv family of algorithms. Then we look at
transform based schemes, and the way in which controlled loss of quality can be achieved using
these.
We contrast data, audio, still image and moving image, covering the ideas of redundancy in
images, sound and motion, We look at the cycles within the data that lead to the signal
processing models used by engineers, including those in computer generated and naturally
occurring data, leading to model based coding and compression, including future schemes such
as wavelet, vector quantization, fractal and hierarchical use of lossy schemes.
We look at Audio Compression. Audio-philes often use the term compression in another way - to
refer to the reduction in dynamic range of an audio signal - for example, some noise reduction
systems use compression and expansion devices so that the noise w.r.t signal at low power levels
(quiet bits) of a piece of music are less noticeable - this is quite different from compression of the
amount of data needed to represent a signal at all. We look at the effect of the network on the
design of coding and compression schemes - loss of synchronisation, data, re-ordering, and
duplication all lead to the need for recovery ponts in the data stream, and place limits on the
time-frames that compression (and decompression) can operate over at sender and receiver ends
for interactive applications.
We then discuss the main different current approaches and standards for multiplexing of audio
and video between sender and recipient. Finally we cover the performance of some example
systems.
Roadmap
Figure 4.1 illustrates the components of a system to capture, code, compress, transmit,
decompress, decode and display multimedia data. The rest of this chapter describes each of the

Adri Jovin J.J.
Page | 23
components in this diagram moving approximately from left to right across the picture. We cover
some example cases of coding and compression schemes along the way.
Figure 4.1: Road Map of Chapter Four
System Components
All data is eventually turned into some (typically repetitive) digital code, and the statistics of the
digital code are of great interest when we want to think about compressing a set of such data.
The statistics are important at several levels of granularity.
Even text input from a given language has several levels of interest, such as characters, words,
and grammatical structure (sentences, paragraphs etc).
In a similar way, speech or music signals have repetitive structure which shows correlation at
several levels of detail. Images may have some structure although natural images this tends to be
very subtle (fractal). Moving images clearly have at least two timescales of interest, partly due to
the nature of the input and display devices (and the human eye), the scanline and the frame.
Thus coding and compression go hand in hand. We choose some levels of granularities at which
to code an input signal or message - this determines the initial input buffer size over which we
run our code. Thiswhich, for real time applications, determines the CODEC delays. This also
determines the number of quanta or different values for each "letter of the alphabet" in the input
buffer.
The selection of these two (or more) timescales may be determined long in advance for some
data types. For example, for text in a given language and alphabet there are large sets of samples
that are amenable to analysis , so we can find nearly optimal digital representations of the data in
terms of storage. However, there may be other factors that affect our design of a code. For
example a variable length code for the english alphabet could be devised that used less bits for
the average block of text than a fixed length codeword such as 7 bit ASCII. On the other hand, it
may be more efficient in computing terms to trade off a small overhead (even as much as 50%)
in storage for speed of processing and choose a fixed length code - which is what has been done
in practice for text.

Adri Jovin J.J.
Page | 24
For audio, while a speech input signal is composed of a streams of phonemes, words and
sentences, it is relatively hard to build an intermediate representation of this, so typically we start
with a set of fixed length samples of short time spans of the signal and work from there.
Similarly with a still (or single image from a moving) image, we choose to sample the input
scene at typically fixed horizontal and vertical intervals giving a 2D image of a given resolution.
We make a design decision when we choose the number of levels (quanitisation) of the samples
(the familiar 8-bit versus 24-bit color display is such a decision - 1 or 3 bytes of storage per pixel
of the image).
Nature of the Signal
We must distinguish between raw data and information, but such a distinction is quite a subtle
business. ``Information'' refers to that part of a signal that constitutes useful information for some
user.
Thus depending on the user, some part of a signal may be regarded as less useful. This means
that there may be redundancy in the data. In some cases, the redundancy is unambiguous - e.g. in
the easy case of simple repetition - where data is coded in some grossly inefficient manner.
Depending on the source, and on the form of the signal, we may know something about the
statistics of the contents in advance, or we may have to do some sort of online analysis if we are
to remove redundancy. The performance of online analysis will depend on the range and
accuracy over which the signal repeats itself - in other words the blocksize.
How much data we can store in a compression algorithm that does onlien analysis will be
affected by how much delay we are allowed to incur (over and above the delay `` budget'' for
transmission and reception), and the CPU load incurred processign larger chunks of the signal.
Finally, redundancy is in the eye of the beholder - we are rarely obliged to keep the original
signal with 100% integrity since human frailty would mean that even without an Internet path
between light or sound source and a person, it is likely that the receiver would miss some parts of
a signal in any case. This latter point is extremely task dependent.
Analog to Digital Conversion: Sampling
An input signal is converted from some continuosly varying physical value (e.g. pressure in air,
or frequency or wavelength of light), by some electro-mechanical device into a continuously
varying electrical signal. This signal has a range of amplitude, and a range of frequencies that
can present. This continuously varying electrical signal can then be converted to a sequence of
digital values, called samples, by some analog to digital conversion circuit. Figure 4.2 illustrates
this process.

Adri Jovin J.J.
Page | 25
Figure 4.2: Sampling a Continuous Signal
There are two factors which determine the accuracy with which the digital sequence of values
captures the original continuous signal: the maximum rate at which we sample, and the number
of bits used in each sample. This latter value is known as the quanisation level, and is illustrated
in figure 4.3.
Figure 4.3: Quantisation of Samples
The raw (un-compressed) digital data rate associated with a signal then is simply the sample rate
times the number of bits per sample. To capture all possible frequencies in the original signal,
Nyquist's theorem shows that the digital rate must be twice the highest frequency component in
the continuous signal. However, it is often not necessary to capture all frequencies in the original
signal - for example, voice is comprehensible with a much smaller range of frequencies than we
can actually hear. When the sample rate is much lower than the highest frequency in the
continuosu signal, a band-pass filter which only allows frequencies in the range actually needed,
is usally put before the sampling circuit. This avoids possible ambiguous samples (``aliases'').

Adri Jovin J.J.
Page | 26
Constructing a Signal out of Components
One view of the input signal illustrated above in figure 4.2, is that it is made up of a number of
contributing signals - mathematically, we can consider any reasonable set of orthogonal signals
as components, but the easiest ones to use are sine functions.
One extreme case often used to illustrate this is the square wave signal. In figure 4.4, we show a
square wave. If this was made out of a number of sine waves with different frequencies, the
contribution of each frequency would be as illustrated in figure 4.5.
Figure 4.4: Square Wave
Figure 4.5: Spectrum of a Square Wave
The way then that we build up the square wave constructively out of a set of sine waves of
different frequencies can be seen in the progression of figures,
Figure 4.6: Square from One Sine Wave
Figure 4.7: Square from Two Sine Waves

Adri Jovin J.J.
Page | 27
Figure 4.8: Square from Three Sine Waves
Figure 4.9: Square from Four Sine Waves
It may seem odd that a simple ``on-off'' signal takes a lot of contributions, but then the point is
that this method of representing the continuous signal is general, and can represent any input
signal.
Input data can be transformed in a number of ways to make it easier to apply certain compression
techniques. The most common transform in current techniques is the Discrete Cosine Transform.
This is a variant of the Discrete Fourier Transform, which is in turn, the digital (discrete) version
of the Continuous Fourier Transform.
As described earlier, any signal (whether a video or audio signal) can be considered a periodic
wave. If we think of a sequence of sounds, they are a modulation of an audio wave; similarly, the
scan over an image or scene carried out by a camera conveys a wave which has periodic features
in time (in the time frame of the scene, as well as over multiple video frames in the moving
picture). It is possible to convert from the original signal as a function of time, to the fourier
series, which is the sum of a set of terms, each being a particular frequency (or wavelength). You
can think of these terms or coefficients as being the contribution of a set of base pure ``sine-
wave'' frequencies (also known as the spectral density), that together make up the actual signal.
and
You can imagine these sweeping through a typical audio signal as shown in figure 4.10, and
``pulling out'' a spectrum (see figure 4.11, or set of coefficients that represent the contribution of
each frequency to that part of the signal strength.

Adri Jovin J.J.
Page | 28
Lossless Data Compression
There is a huge range of data compression techniques - these are of some interest to the
multimedia systems designer, but there are many good books on them already. Suffice it to say
that three common techniques used are runlength encoding (removing repetitions of values and
replacing them with a counter and single value), Huffman coding, and dictionary techniques such
as the Lempel-Ziv family of substitutional compression algorithms.
Run Length Compression
Run length coding is fairly simple to implement, and with all lossless schemes, its performance
depends heavily on the input data statistics. Computer generated binary files are often very
amenable to this type of compression, for example with a codeword size of 1 bit or a byte or a
word often leading to elimination of many all 1s or all 0s successive fields. The nice feature of
this scheme is that it incurs very little delay at sender or receiver. Note that this and other
schemes do incur a variable output bit/symbol rate.
Huffman Compression
Huffman coding is the most widespread way of replacing a set of values of fixed size code words
with an optimal set of different sized code words based on the statistics of the input data. The
way a huffman code is constructed involves constructing a frequecny distribution of the symbols.
This is then used to decide the new compressed representation for each symbol. The easiest way
to do this is to consider the case for compressing alphabetic text, with symbols drawn from
characters in an alphabet with 256 letters. If these are all equally likely to occur, then it is hard to
compress the data. However, if there is a severe skew in the frequcny distribution in typical data
(texts in this alphabet) then we can use more less bits to represent the most frequently occurring
characters/codeword values, and more bits for the less commonly occurring ones, with some
compression gain. So, how to build this new coding scheme for the alphabet? The classic scheme
is to construct a tree from the frequency distribution, taking pairs of characters/codeword values
at a time from the least frequently occurring values in the frequency chart, and adding a bit
position to the string representation for them, with value 1 for one, 0 for the other, then moving
to the next less commonly occurring; eventually, the most commonly occuring two values take
two bits, one to say which is which, and one to say that it is one of these two values, or its one of
the 254 other values. And so on...
This scheme is optimal in only one case, which is when the probability of occurrences of
codeword values is distributed as a set of inverse powers of 1/2, i.e. 1/2, 1/4, 1/8, 1/6 etc.
Otherwise the scheme is less and less good.
A generalization of huffman coding that avoids this latter problem is arithmetic codeing, which
uses binary fraction representations in building the coding tree. In practice, this can be
computationally expensive.
If one is transmitting huffman (or arithmetic) compressed data, one must also share the same
codebook at sender and receiver: the list of codes and their compressed representation must be

Adri Jovin J.J.
Page | 29
the same at both ends. This can be done on a case by case basis, but is usually based in long term
statistics of the data (e.g. the frequency of occurence of the letter `ee'' in the written English
language is a well known example of this sort of statistic).
Dictionary Approaches to Compression
A completely different approach is to look at the data as it arrives and form a dictionary on the
fly. As the dictionary is formed, it can be used to look up new input, dynamically, and if the new
input existed earlier in the stream, the dictionary position can be transmitted instead of the new
input codes. These schemes are known as ``substitutional'' compression algorithms, and there are
two patented families of schemes invented by J Ziv and A Lempel in the 1970s that cover a
broad class of the ideas here.
Essentially, the dictionary is constructed as a data structure that holds strings of symbols that are
found in the input data, together with short bitstring entry numbers. Whenever an entry is not
found, it is added to the dictionary in a new position, and the new position and string sent. This
means that the dictionary is constructed at the receiver dynamically, so that there is no need to
carry out statistics or share a table separately.
A second family of Lempel-Ziv dictionary based compression schems is based on the idea of a
sliding window over the input text. The compression algorithm consists of searching for
substrings ahead in the text, in the current window. This approach constrains the size of the
dictionary, which could otherwise grow in an unbounded way.
Continuous Data: Sample Rates and Quantisation
In the next two sections, we look at audio and then video coding and compression. Here, one is
concerned with the initial fidelity of the signal which is tied up with the sampling mechanism -
the number of samples per second, and the number of bits per sample (quanitisation levels) - in
other words, one has to choose an accuracy right from the beginning, and this represents an
opportunity for compression even before we have got to the digital domain! After this, there are
a number of other techniques, including the lossless ones just described, which are applicable.
Audio
Devices that encode and decode audio and video, as well compress and decompress are called
CODECs or CODer DECoders. Sometimes, these terms are used for audio, but mainly they are
for video devices.
Figure 4.10: The Author Saying ``smith''

Adri Jovin J.J.
Page | 30
Figure 4.11: Typical Voice Spectrum
Voice coding techniques take advantage of features of the voice signal. In the time domain, we
can see that there is a lot of similarity between adjacent speech samples - this means that a
system that only sends differences between sample values will achieve some compression. We
can see tha there are a lot more values in samples with low intensity, than with high. This means
that we could use more bits to represent the low values than the high ones. This could be done in
a fixed way and A and mu law encodings do just this by choosing a logarithmic encoding. Or we
could adapt to the signal, and APCM does this. These techniques can be combined, and the
ADPCM (Adaptive Differential Pulse Code Modulation) achieves 50% savings over basic PCM
with no apparent loss of quality, and relatively cheap implementation.
More ingenious compression relies on two things: an appreciation of the actual model of speech
and a model of the listener. Such techniques usually involve recognizing the actual speech
production and synthesising a set of filters which are transmitted to the receiver and used to
reconstruct sound by applying them to raw ``sound'' from a single frequency source and a white
noise generator - examples of CODECs that are based on this idea are Linear Predictive Coding
(LPC) and CELP (Code Excited Linear Predictor. Including a model of how humans perceive
sound (so called ``psycho acoustics'') leads to more expensive, but highly effective compression
such as is used in MPEG audio CODECs.
Audio Input and Output
Audio signals to and from the real (analog) world have a less immediately obvious mapping to
the digital world. Audio signals vary depending on the application. Human speech has a well

Adri Jovin J.J.
Page | 31
understood spectrum, and set of characteristics, whereas musical input is much more varied, and
the human ear and perception and cognition systems behave rather differently in each case. For
example, when a speech signal degrades badly, humans make use of comprehension to
interpolate. This may be harder or easier to do with music depending on levels of expertise and
familiarity with the style/idiom.
Basically, for speech, the analog signal from a microphone is passed through several stages.
Firstly a band pass filter is applied eliminating frequencies in the signal that we are not interested
in (e.g. for telephone quality speech, above 3.6Khz). Then the signal is sampled, converting the
analog signal into a sequence of values, each of which represents the amplitude of the analogue
signal over a small discrete time interval. This is then quantised, or mapped into one of a set of
fixed values - e.g. for telephone quality speech, one of 2**8, or 256 possible values. These
values are then coded (represented in some standard form) for transmission or storage.
The process at the receiver is simply the reverse.
Audio Output
Audio output is generally made by some physical process interacting with the air (in space, of
course, there is no sound!). The air conveys the sound to your ear (or to a microphone or other
input device).
To produce output from a computer, we need to take a digital signal, convert it to analogue, and
use that to drive a loudspeaker. Many PCs now feature fairly standard hi-fi amplifiers and stereo
speaker systems.
Sounds can be created purely digitally (from synthesiszers) or partly digitally (from samples) or
naturally from the surrounding world (wind in the trees, rain on the roof), or from analog musical
instruments or from human speech.
Audio Output by People
People generate sounds by breathing air through the vocal chords, which resonate, and then
controlling the production of sound by varying the shape of their vocal tract, mouth, tongue and
so on. For the purposes of communication, speech is generally more useful than music, and
happens to use a constrained part of the frequency and power range that humans are capable of
generating - and a much more constrained part of the range than they are capable of hearing.
Typically, we can generate sounds over a dynamic range of 40 decibels, For recognisable speech,
the vast majority of important sounds are in the frequency range 60H to 8000Hz (compared with
music which is typically audible up to nearly 20KHz).
Speech is made up of sound units called phonemes (the smallest unit of distinguishable sound).
These are specific to a language, so for example ,we find that English and Japanese each have
phonemes that the other language does not (e.g. ``l'' and ``r'') (hence the difficulty in learning to
pronounce another distant language). We illustrate some of the international phonetic alphabet
for British English with example words in the table 4.1 below.

Adri Jovin J.J.
Page | 32
Vowels Diphthongs Semi-vowels Nasals Fricatives Affricatives Plosives
/i/ heed /gi/ buy /w/ was /m/ am /s/ sail /dg/ jaw /b/ bat
/I/ hid /ai/ by r/ ran /n/ an /S/ ship /tj/ chore /d/ disc
/e/ head /au/ bow /l/ lot /n/ sang
/ae/ had /ao/ bough /j/ yacht
Table 4.1: International Phonetic Alphabet for British English
Phonemes are vowels or consonants, where vowels are either pure or diphthongs (made of two
sounds), and consonants may be semi-vowel, fricative (use teeth to make), plosive (use lips ) or
nasal (use nose). Other factors influencing the sound we make are stress (change of strength),
rhythm and pace, and intonation (pitch).
Audio Input by People
We hear mainly through our ears which respond over a frequency range of around 20Khz. Stereo
is important to many human actions and even the phase difference between signals arriving from
the same source at each ear (as well as simple timing, since sound moves so slow compared with
light) gives us good directional hearing, although mainly at high frequencies. As people et older,
their high frequency accuracy decreases quite markedly, although this doesn't usually affect
speech recognition until old age.
Summary of Audio and Video Input and Output
Data (files etc.) typically compressed using simple schemes such as Run Length encoding, or
statistically based Huffman codes or dictionary based substitutional schemes such as the Lempel-
Ziv algorithms. Audio and Video are loss tolerant, so can use cleverer compression that discards
some information. Compression of 400 times is possible on video - useful given the base
uncompressed data rate of a 25 fps CCIR 601 image is 140Mbps.4.2 A lot of standards for this
now including schemes based on PCM, such as ADPCM, or on models such as LPC, and MPEG
Audio. Note that lossy compression of audio and video is not acceptable to some classes of user
(e.g. radiologist, or air traffic controller).
It is sometimes said that ``the eye integrates while the ear differentiates''. What is meant by this
is that the eye responds to stronger signals or higher frequencies with cumulative reaction, while
the ear responds less and less (i..e to double the pitch, you have to double the frequency - so we
hear a logarithmic scale as linear, and to double the loudness, you have to increase the power
exponentially too).
A video CODEC can be anything from the simplest A2D device, through to something that does
picture pre-processing, and even has network adapters build into it (i.e. a videophone!). A

Adri Jovin J.J.
Page | 33
CODEC usually does most of its work in hardware, but there is no reason not to implement
everything (except the a2d capture:-), in software on a reasonably fast processor.
The most expensive and complex component of a CODEC is the compression/decompression
part. There are a number of international standards, as well as any number of proprietary
compression techniques for video.
The ITU (was CCITT) Audio Family
The fundamental standard upon which all videoconferencing applications are based is G.711 ,
which defines Pulse Code Modulation(PCM). In PCM, a sample representing the instantaneous
amplitude of the input waveform is taken regularly, the recommended rate being 8000 samples/s
(50 ppm). At this sampling rate frequencies up to 3400-4000Hz are encodable. Empirically, this
has been demonstrated to be adequate for voice communication, and, indeed, even seems to
provide a music quality acceptable in the noisy environment around computers (or perhaps my
hearing is failing). The samples taken are assigned one of 212 values, the range being necessary
in order to minimize signal-to-noise ratio (SNR) at low volumes. These samples are then stored
in 8 bits using a logarithmic encoding according to either of two laws (A-law and =-law). In
telecommunications, A-law encoding tends to be more widely used in Europe, whilst =-law
predominates in the US However, since most workstations originate outside Europe, the sound
chips within them tend to obey =-law. In either case, the reason that a logarithmic compression
technique is preferred to a linear one is that it more readily represents the way humans perceive
audio. We are more sensitive to small changes at low volume than the same changes at high
volume; consequently, lower volumes are represented with greater accuracy than high volumes.
ADPCM
Adaptive Differential Pulse Code Modulation ADPCM (G.721) allows for the compression of
PCM encoded input whose power varies with time. Feedback of a reconstructed version of the
input signal is subtracted from the actual input signal, which is then quantised to give a 4 bit
output value. This compression gives a 32 kbit/s output rate. This standard was recently extended
in G.726 , which replaces both G.721 and G.723 , to allow conversion between 64 kbit/s PCM
and 40, 32, 24, or 16 kbit/s channels. G.727 is an extension of G.726 and issued for embedded
ADPCM on 40, 32, 24, or 16 kbit/s channels, with the specific intention of being used in
packetised speech systems utilizing the Packetized Voice Protocol (PVP), defined in G.764.
The encoding of higher quality speech (50Hz-7kHz) is covered in G.722 and G.725 , and is
achieved by utilizing sub-band ADPCM coding on two frequency sub-bands; the output rate is
64 kbit/s.
LPC
LPC (Linear Predictive Coding) is used to compress audio at 16 Kbit/s and below. In this
method the encoder fits speech to a simple, analytic model of the vocal tract. Only the

Adri Jovin J.J.
Page | 34
parameters describing the best-fit model is transmitted to the decoder. An LPC decoder uses
those parameters to generate synthetic speech that is usually very similar to the original. The
result is intelligible but machine-sound like talking.
CELP
CELP (Code Excited Linear Predictor) is quite similar to LPC. CELP encoder does the same
LPC modeling but then computes the errors between the original speech and the synthetic model
and transmits both model parameters and a very compressed representation of the errors. The
compressed representation is an index into an excitation vector (which can be considered like a
``code book'' shared between encoders and decoders. The result of CELP is a much higher
quality speech at low data rate.
MPEG AUDIO
High quality audio compression is supported by MPEG. MPEG I defines sample rates of 48
KHz, 44.1 KHz and 32 KHz. MPEG II adds three other frequencies , 16 KHz, 22,05 and 24
KHz. MPEG I allows for two audio channels where as MPEG II allows five audio channels plus
an additional low frequency enhancement channel.
MPEG defines three compression levels that is Audio Layer I, II and III. Layer I is the
simplest, a sub-band coder with a psycho-acoustic model. Layer II adds more advanced bit
allocation techniques and greater accuracy. Layer III adds a hybrid filterbank and non-uniform
quantization. Layer I, II and III gives increasing quality/compression ratios with increasing
complexity and demands on processing power.
Still Image
``A picture is worth a thousand words''. But an image, uncomrpessed is worth many megabytes.
How Big Is a Single Frame of Video?
First we consider the spatial size of analogue video when compared to the common formats for
digital video standards. A PAL television displays video as 625 lines and an NTSC television
displays 525 lines. Current televisions have an aspect ratio of 4:3, giving PAL a spatial
resolution of 833 x 625, and NTSC a resolution of 700 x 525, not all of which is visible. Most
common formats for digital video are related to the visible area for each of the television
standards. The size of video when using the international standard H.261, found in [#!h261!#] is
352 x 288 for the Common Image Format (CIF) format and 176 x 144 for the (Quarter CIF)
QCIF format, and 704 x 576 for the (Super CIF) SCIF format, where a CIF image is a quarter the
size of the visible area of a PAL image. For NTSC derived formats 640 x 480, 320 x 240, and
160 x 120 are common. Figure 4.12 shows the spatial size of these common resolutions with
respect to a PAL TV image.

Adri Jovin J.J.
Page | 35
Figure 4.12: The spatial size of digital video compared with a PAL TV image
It can be seen that digital images are all smaller than current television sizes. Moreover,
television images are significantly smaller than current workstation screen sizes which are
commonly of the order 1200 x 1000 pixels. Digital video utilizes even less of a workstation
screen.
Due to this significant size difference, some observers have commented that digital video often
looks like "moving postage stamps", on modern workstations.
For digital video, as with analogue video, a new frame is required every 1/25th second for PAL
and every 1/30th second for NTSC. If we assume that there are 24 bits per pixel in the digital
video and 30 frames per second, the amount of disc space required for such a stream of full-
motion video is shown in table 4.2. The table is presented for the amount of time the digital
video is shown and for a given spatial size in pixels.
Time:Size 640x480 320x240 160x120
1sec 27Mb 6.75Mb 1.68Mb
1min 1.6Gb 400Mb 100Mb
1hour 97Gb 24Gb 6Gb
1000hours 97Tb 24Tb 6Tb
Table 4.2: The amount of data for full-motion digital video
We can see that 1 hour of video with a resolution of 640 x 480 would consume 97 Gb of disc
space, which is significantly larger than most storage devices. An equivalent amount of analogue
video (i.e. a 1 hour video) , which has a higher resolution and also contains audio, would only
take between a half and a quarter of a video cassette, for a 120 minute or a 240 minute cassette,
respectively. However, although there are devices that can store this amount of data, there are
currently no digital storage devices which could store 97 Gb on half a device which is the size of
a video cassette. The data shown in the tables was collated by Larry Rowe of the Computer
Science Division - EECS, University of California at Berkeley, for his work on The Continuous
Media Player [#!rowe!#].
In order to reduce the amount of data used for digital video, it is common to use compression
techniques, such as the international standards H.261, MPEG [#!mpegrtp!#], or to use
proprietary techniques such as nv encoding [#!frederick!#] or CellB [#!cellb!#]. Rowe has also
estimated the amount of space used when compression techniques are used. Table 4.3 shows the

Adri Jovin J.J.
Page | 36
space needed when compressing video of size 640 x 480 pixels, and table 4.4 shows the space
used when compressing video of size 320 x 240 pixels. Both tables present data for a given scale
factor of compression and for the time the video is shown. The 97 Gb used for the 1 hour of 640
x 480 video can be reduced to approximately 1 Gb when compression is done at a scale factor of
100:1.
Time v. Scale None 3:1 25:1 (JPEG) 100:1 (MPEG)
1 sec 27 Mb 9 Mb 1.1 Mb 270 Kb
1 min 1.6 Gb 540 Mb 65 Mb 16 Mb
1 hour 97 Gb 32 Gb 3.9 Gb 970 Mb
Table 4.3: The amount of data for compressed video of size 640x480
Time v. Scale None 3:1 25:1 (JPEG) 100:1 (MPEG)
1 sec 6.75 Mb 2.25 Mb 270 Kb 68 Kb
1 min 400 Mb 133 Mb 16 Mb 4 Mb
1 hour 24 Gb 8 Gb 1 Gb 240 Mb
Table 4.4: The amount of data for compressed video of size 320x240
Although the table shows compression factors for MPEG, the H.261 standard uses a Discrete
Cosine Transform encoding function which is similar to that used in MPEG, therefore we can
expect the compression ratios to be of a similar order of magnitude. In reality, when encoding
real video the compression factor is not constant but variable because the amount of data
produced by the encoder is a function of motion. However, these figures do give a reasonable
estimation of what can be achieved.
It is significant that with digital video it is possible to dramatically reduce the amount of data
generated even further by reducing the perceived frame rate of the video from 30 frames a
second down to 15 or even 2 frames a second. This can be achieved by explicitly limiting the
number of frames or through a bandwidth limitation mechanism. In many multicast conferences
the bandwidth used is between 15 and 64 Kbps. Although the reduced frame rate video loses the
quality of full-motion video, it is perfectly adequate for many situations, particularly in
multimedia conferencing.
There are a large number of still image formats and compression schemes in use in the network
today. Common schemes include:
TIFF and GIF
These both use compression schemes based o nthe Lempel-Ziv type of algorithms described
earlier.
JPEG
This is from the Joint Photographic Experts Group in the International Organisation for
Standardization (ISO).

Adri Jovin J.J.
Page | 37
The first two of these still image schemes are discussed elsewhere in great detail. JPEG is
interesting as it is also the same baseline technology as is used partly in several populat moving
image compression schemes. The JPEG standard`s goal has been to develop a method for
continuous-tone image compression for both color and greyscale images. The standard define
four modes:
• Sequential In this mode each image is encoded in a single left-to-right, top-to-bottom
scan. This mode is the simplest and most implemented one in both hardware and software
implementation.
• Progressive In this mode the image is encoded in multiple scans. This is helpful for
applications in which transmission time is too long and the viewer prefers to watch the
image building in multiple coarse-to-clear passes.
• Lossless The image here is encoded to guarantee exact recovery of every source image
sample value. This is important to applications where any small loss of image data is
significant. Some medical applications do need that mode.
• Hierarchical Here the image is encoded at multiple resolutions, so that low-resolution
versions may be decoded without having to decode the higher resolution versions. This
mode is beneficial when transmission over packet switched networks. Only the data
significant for a certain resolution determined by the application can be transmitted, thus
allowing more applications to share the same network resources. In real time
transmission cases (e.g. an image pulled out of an information server and synchronized
with a real-time video clip), a congested network can start dropping packets containing
the highest resolution data resulting in a degraded quality of the image instead of delay.
JPEG uses the Discrete Cosine Transform to compress spatial redundancy within an image in all
of its modes apart from the lossless one where a predictive method issued instead.
As JPEG was essentially designed for the compression of still images, it makes no use of
temporal redundancy which is a very important element in most video compression schemes.
Thus, despite the availability of real-time JPEG video compression hardware, its use will be
quite limit due to its poor video quality.
Moving Image
Video Input and Output
Before you can digitize a moving image, you need to know what the analog form is, in terms of
resolution and frame rate. Unfortunately, there are three main standards in use. PAL is used in
the UK, while NTSC is used in the US and in JAPAN, and SECAM is used mainly in France and
Russia. The differences are in number of lines, frame rate, scan order and so forth. PAL is used
most widely - it uses 625 lines, and has a line rate of 50Hz. NTSC is used in the USA and Japan -
it uses 525 lines, and a line rate of 59.94Hz. SECAM is used in France and Russia These all
differ in number of lines, frame rate, interlace order and so on. There are also a number of
variants of PAL and NTSC (e.g. I/PAL is used in the UK and Ireland, while M/NTSC is the ITU
designation for the scheme used in the US and Japan).

Adri Jovin J.J.
Page | 38
Colour
Colour is very complex. Basically, light is from a spectrum (continuum), but we typically
manipulate colors by manipulating discrete things like pens, or the coloured dots of phosphor on
a CRT which emit light of a given intensity but at a single colour, when hit by an electron of a
given energy. There are several ways of mixing discrete colors to get a new colour that has the
right appearance to the human eye. The human eye does not perceive a spectrum, but rather all
colors as combinations of 3 so called primary colors, Red (435nm), Green (546nm) and Blue
(700nm).
These primaries can be added to produce secondaries, magenta, cyan and yellow. [The roles of
primary and secondary are reversed in pigments, over those in light, since the concern of a dye
maker is concerned with which colour is absorbed, rather than which is transmitted]. Most MM
users use colour far too much, however, in natural situations, it is very rich - this is a strange
paradox.
Colour Input by Humans
The human eye can perceive a very wide range of colors compared with gray scales. It actually
has different sensors for colour than for monochrome. Colour is detected by ``cones'' cells in the
retina that distinguish a range of different signals, while black and white (mono-chrome) is dealt
with by rods. Rods are actually sensitive to much lower light levels (intensity/power), and are
particularly good at handling motion. Cones are specialized to higher light levels (hence why
colour vision doesn't work in dim light levels such as during dawn/dusk/twilight).
Colour Input by Computers
A colour input device such as a video camera has a similar set of sensors to cones. These respond
to different wavelengths with different strengths. Essentially, a video camera is a digital device,
based around an array of such sensors, and a clock that sweeps across them the same way that
the electron gun in the back of a TV or computer display is scanned back and forth, and up and
down, to refresh the light emission from the dots on the screen. So, for a single, still frame, a
scan produces an array of reports of intensity, one element for each point in the back of the
camera. For a system with 3 colour sensor types, you get an array of triples, values of intensity of
light of each of the sensors at being a real. This is then converted into an analog signal for
normal analog recording. Some devices are emerging where the values can be directly input to a
computer rather than converted to analog, and then have to be converted to digital again by an
expensive frame grabber or video card. Given the range of intensities the human eye can
perceive isn't huge, they are usually stored digitally in a small number of bits - most usually 8 per
colour - hence a ``true'' colour display has 24 bits, 8 bits each for R, G and B. RGB is the most
commonly used computing colour model. CMY is just [1] - [RGB], and vice, versa. [0,0,0] is
black, and [255,255,255] is white.

Adri Jovin J.J.
Page | 39
Color Output by Computers and other Devices
Image or video output is just the reverse of input. Thus an area of memory is set aside for the
``frame buffer''. Data written here will be read by the video controller, and used to control the
signal to the display's electron gun for intensity of each of the colors for the corresponding pixel.
By changing what is in the frame buffer once per scan time, you get motion/animation etc. So to
play back digital video from disk, typically, you read it from disk to the frame buffer at the
appropriate rate, and you have a digital VCR (presumably with digital video tapes)!
``Video RAM'' is not usually quite the same as other memory since it is targeted at good row
then column scans rather than true RAM access.
VRAM stores n bits of each of RGB If n=8 , it is known ``True Color'' n < 8, can have colour
maps then typically values are indexes to tables. These Color maps lead to flicker or false colour
n=1 , is just plain monochrome.
Towards Compressed Video
Video compression can take away the requirement for the very high data rates and move video
transmission and storage into a very similar regime to that for audio. In fact, in terms of tolerance
for poor quality, it seems humans are better at adapting to poor visual information than poor
audio information.
A simple minded calculation shows the amount of data you might expect, and is shown in table
4.5.
1024 x 1024 pixels,
3 bytes per pixel (24 bit RGB)
25 Frames per second
Table 4.5: Liberal Estimate for Uncompressed Video Data Rate
yields 75Mbytes/second, or 600Mbps - this is right on the limit of modern transmission capacity.
Even in this age of deregulation and cheaper telecoms, and larger, faster disks, this is profligate.
On the other hand, for a scene with a human face in, as few as 64 pixels square, and 10 frames
per second might suffice for a meaningful image.
Table 4.6: Cautious Estimate for Uncompressed Video Data Rate
yields 122KBytes/Second, or just under 1 Mbps - this is achievable on modern LANs and high
speed WANs but still not friendly! Notice that in the last simple example, we did two things to
the picture.
1. We used less "space" for each frame by sending less "detail".
2. We sent frames less frequently since little is moving.

Adri Jovin J.J.
Page | 40
This is a clue as to how to go about improving things. Basically, if there isn't much information
to send, we avoid sending it. Spatial and temporal domain compression are both used in many of
the standards.
If a frame contains a lot of image that is the same, maybe we can encode this with less bits
without losing any information (run length encode, use logically larger pixels etc. etc.). On the
other hand, we can take advantage of other features of natural scenes to reduce the amount of bits
- for example, nature is very fractal, or self-similar:- there are lots of features, sky, grass, lines on
face etc., that are repetitive at any level of detail. If we leave out some levels of detail, the eye
(and human visual cortex processing) end up being fooled a lot of the time. The way that the eye
and the ear work (integration versus differentiation) means that video and audio compression are
very different things.
Hierarchical Coding
Hierarchical coding is based on the idea that coding will be in the form of quality hierarchy
where the lowest layer of hierarchy contains the minimum information for intelligibility.
Succeeding layers of the hierarchy adds increasing quality to the scheme.
This compression mechanism is ideal for transmission over packet switched networks where the
network resources are shared between many traffic streams and delays, losses and errors are
expected.
Packets will carry data from only one layer, accordingly packets can be marked according to
their importance for intelligibility for the end-user. The network would use these information as
a measure of what sort of packets to be dropped, delayed and what should take priority. It should
be noted that priority bits already exist in some protocols such as the IP protocol.
Hierarchical coding will also be ideal to deal with multicasting transmission over links with
different bandwidths. To deal with such problem in a non-hierarchical encoding scheme, either
the whole multicasting traffic adapts to the lowest bandwidth link capabilities thus degrading the
video/audio quality where it could have been better or causing the low link to suffer from
congestion and thus sites affected will lose some of the intelligibility in their received
video/audio. With hierarchical coding, low level packets can be filtered out whenever a low
bandwidth link is encountered thus preserving the intelligibility of the video/audio for the sites
affected by these links and still delivering a better quality to sites with higher bandwidth.
Schemes that are now in relatively commonplace use include H.261 for videotelephony, MPEG
for digital TV and VCRs and JPEG for still images. Most current standards are based on one
simple technique, so first lets look at that.
• That last idea was that levels of detail can be sent at different rates or priorities
• This can be useful if there are different users (e.g. in a TV broadcast, or Internet
multicast)

Adri Jovin J.J.
Page | 41
• It can also be useful for deciding what to lose in the face of overload or lack of disk
storage etc.
• Many of the video encodings (and still picture standards) are well suited to this.
Subband Coding
Subband coding is given as an example of an encoding algorithm that can neatly map onto
hierarchical coding. There are other examples of hierarchical encoding none of them is a
standard or widely used as the international standards such as H261 and MPEG.
Subband coding is based on the fact that the low spatial frequencies components of a picture do
carry most of the information within the picture. The picture can thus be divided into its spatial
frequencies components and then the coefficients are quantized describing the image band
according to their importance; lower frequencies being more important. The most obvious
mapping is to allocate each subband (frequency) to one of the hierarchy layers. If inter-frame
coding is used, it has to be adjusted as not to create any upward dependencies.
In the discrete cosine world, we can replace with just the cos(t), and the
integrals become sums. t just references a token or symbol in the original digital spatial sample
(for an image). Of course, an image is two-dimensional, which means we have to do the
transform for two spatial coordinates, rather than one temporal coordinate.
So to transform a block of image data, 8 by 8 pixels square into the frequency domain, we apply:
Let us have a look at how this works in practice. In the figure 4.13, we can see some artificial
images, and their transforms.
Figure 4.13: Some Examples of DCT'd data

Adri Jovin J.J.
Page | 42
In the transform, we can see how a given artifact in the input appears. For example, the lower left
corner of the DCT represents the DC component (average), whilst the corners represent
horizontal, vertical and other frequency information.
Finally, we should observe that there are a set of well known optimisations of the discrete
Fourier and Cosine transforms, based on decimation in time. These essentially all stem from the
fact that there is a lot of symmetry in the transform, and the relationship:
x=2, i=1, gives same cos() value as x=1, i=2
H.261
H261 is the most widely used international video compression standard for video conferencing.
This ITU (was CCITT) standard describes the video coding and decoding methods for the
moving picture component of a audiovisual service at the rates of p * 64 Kbps where p is in the
range of 1 to 30. The standard targets and is really suitable for applications using circuit switched
networks as their transmission channels. This is understandable as ISDN with both basic and
primary rate access was the communication channel considered within the framework of the
standard.
H.261 is usually used in conjunction with other control and framing standards such as H221,
H230 H242 and H320, of which more later.
H.261 Source Images format
The source coder operates on only non-interlaced pictures. Pictures are coded as luminance and
two color difference components(Y, Cb, Cr). The Cb and Cr matrices are half the size of the Y
matrix.
H261 supports two image resolutions, QCIF which is (144x176 pixels)and , optionally, CIF
which is(288x352).
The three main elements in an H.261 encoder as illustrated in 4.14 are:
Prediction
H261 defines two types of coding. INTRA coding where blocks of 8x8 pixels each are
encoded only with reference to themselves and are sent directly to the block transformation
process. On the other hand INTER coding frames are encoded with respect to another reference
frame.
A prediction error is calculated between a 16x16 pixel region (macroblock) and the
(recovered) correspondent macroblock in the previous frame. Prediction error of transmitted

Adri Jovin J.J.
Page | 43
blocks (criteria of transmission is not standardized) are then sent to the block transformation
process.
• Blocks are inter or intra coded
• Intra-coded blocks stand alone
• Inter-coded blocks are based on predicted error between the previous frame and this one
• Intra-coded frames must be sent with a minimum frequency to avoid loss of
synchronisation of sender and receiver.
Block Transformation
H261 supports motion compensation in the encoder as an option. In motion compensation a
search area is constructed in the previous (recovered) frame to determine the best reference
macro block. Both the prediction error as well as the motion vectors specifying the value and
direction of displacement between the encoded macroblock and the chosen reference is sent. The
search area as well as how to compute the motion vectors are not subject to standardization. Both
horizontal and vertical components of the vectors must have integer values in the range + 15 and
15 though
In block transformation, INTRA coded frames as well as prediction errors will be composed
into 8x8 blocks. Each block will be processed by a two-dimensional FDCT function. If this
sounds expensive, there are fast table driven algorithms and can be done in s/w quite easily, as
well as very easily in hardware.
Quantization & Entropy Coding
The purpose of this step is to achieve further compression by representing the DCT
coefficients with no greater precision than is necessary to achieve the required quality. The
number of quantizers are 1 for the INTRA dc coefficients and 31 for all others.
Entropy coding involves extra compression (non-lossy) is done by assigning shorter code-
words to frequent events and longer code-words to less frequent events. Huffman coding is
usually used to implement this step.
In other words, for a given quality, we can lose coefficients of the transform by using less bits
than would be needed for all the values This leads to a "coarser" picture. We can then entropy
code the final set of values by using shorter words for the most common values and longer ones
for rarer ones (like using 8 bits for three letter words in English:-)

Adri Jovin J.J.
Page | 44
Figure 4.14: H261 Encoder
H.261 Multiplexing
The video multiplexer structures the compressed data into a hierarchical bitstream that can be
universally interpreted.
The hierarchy has four layers:
1. Picture layer : corresponds to one video picture (frame)
2. Group of blocks: corresponds to 1/12 of CIF pictures or 1/3 of QCIF
3. Macroblocks : corresponds to 16x16 pixels of luminance and the two spatially
corresponding 8x8 chrominance components.
4. Blocks: corresponds to 8x8 pixels

Adri Jovin J.J.
Page | 45
H.261 Error Correction Framing
An error correction framing structure is described in the H261 standard. The frame structure is
shown in the figure. The BCH(511,493) parity is used to protect the bit stream transmitted over
ISDN and is optional to the decoder. The fill bit indicator allows data padding thus ensuring the
transmission on every valid clock cycle
H.263
H263+ is a new addition to the ITU H series and is aimed at extending the repertoire to Video
Coding for Low Bit Rate Communication. This makes it eminently suitable to a wide variety of
Internet access line speeds, and therefore also probably reasonably friendly to many Internet
Service Providers backbone speeds.
Existing A/V Standards and Video and the basic technology of CCD camera and of Television
and general CRT dictates frame grabbing at some particular resolution and rate. The choice of
resolution is complex. One could have fixed number of pixels, and aspect ratio, or allow a range
of choice of line rate and samples rates. H.261 and MPEG choose latter.
The line rate (a.k.a. Picture Clock Frequency - PCF) is 30,000/1001 or about 29.97Hz but one
can also use multiples of this. The chosen resolution for H.263 is dx*dy luminance and
chrominanace is just one half this in both dimensions. H.263+ then allows for sub-QCIF which is
128*96 pixels, QCIF - 176*144 pixels, CIF - 352*288 pixels, 4CIF (SCIF in the INRIA Ivs tool)
704*576 pixels and 16CIF 1408*1152 pixels. The designer can also choose a pixel aspect ratio;
the default is 288/3: 352/4 which is 12:11 (as per H.261). The picture area covered by standard
formats has aspect ratio of 4:3.
Luminance and chromnance sample positions as per H.261, discussed earlier in this chapter. The
structure of the coder is just the same too, although there are now two additional modes called
the ``slice'' and ``picture block'' modes.
A block is 16*16 Y and 8*8 Cb and Cr each; The Group of Block, or GOB refers to k*16 lines;
GOBS are numbered using a vertical scan starting with 0 to k, depending on the number of lines
in Picture. e.g. normally, when
lines<400
, k is 1. The number of GOBS per picture then is 6 for subQCIF, 9 for QCIF, 18 for CIF (and for
4CIF and 16CIF because of special rules).
Prediction works on Intra, inter, B, PB, EI or EP (the reference picture is smaller).
The Macroblock is 16 lines of Y, and the corresponding 8 each of Cb and Cr Motion vectors of
which we can receive 1 per macroblock.
There is some provision for other technology - we could envisage "intelligent" device in camera,
and only detect "objects" and motion - this is some ways off in the future and anyhow can be

Adri Jovin J.J.
Page | 46
done after the event with general s/w intelligence after dumb capture and compression (using
compression for hints.
H.263 then, extends H.261 over lower bitrate (not just p*64kbps design goal) and more features
for better quality and services, but the basic ideas same.
• Intra and Inter frame compression
• DCT block transform plus quantization
There are then a number of basic enhancements in H263 including
1. Continuous Presence Multi-point and Video Multiplex mode - basically 4 in 1 sub-bit-
stream transmission. This may be useful for conferences, tele-presence, surveillance and
so on
2. Motion Vectors can point outside picture
3. Arithmetic as well as variable length coding (VLC)
4. Advanced Prediction Mode which is also known as ``Overlapped Block Motion
Compensation'' uses 4 8*8 blocks instead of 1 16*16, This gives better detail.
5. PB Frames known as combined Predictive and Bi-Directional frames (like MPEG II).
6. FEC to help with transmission loss; Advanced Intra coding to help with interpolation;
Deblocking Filter mode, to remove blocking artifacts
7. Slice Structured Mode (re-order blocks so Slice layer instead of GOB layer is more delay
and loss tolerant for packet transport
8. Supplemental Enhancement Information, Freeze/Freeze Release and Enhancement and
Chroma Key (use external picture as merge/background etc...for mixing).
9. Improved PB mode, including 2 way motion vectors in PB mode
10. Reference Picture Selection
11. Temporal, SNR and Spatial Scalability mode; this allows receivers to drop B frames for
example - gives potential heterogeneity amongst receivers of multicast.
12. Reduced Resolution Update; Independent Segment decoding; Alternate INTER VLC
mode
13. Modified Quantization mode (can adjust up or down the amount of quantization to give
fine quality/bit-rate control.
Chroma Keying is a commonly used technology in TV, e.g. for picture in picture/superimpose
etc, for weather people and so on. The idea is to define some pixels in an image as ``transparent''
or ``semi-transparent'' and instead of showing these, a reference, background image is used (c.f.
transparent GIFs in WWW). We need an octet per pixel to define the keying color for Y, Cb and
Cr, each. The actual choice when there isn't an exact match is implementor defined.
MPEG
The aim of the MPEG-II video compression standard is to cater for the growing need of generic
coding methods for moving images for various applications such as digital storage and
communication. So unlike the H261 standard who was specifically designed for the compression

Adri Jovin J.J.
Page | 47
of moving images for video conferencing systems at p * 64Kbps , MPEG is considering a wider
scope of applications.
• Aimed at storage as well as transmission
• Higher cost and quality than H.261
• Higher minimum bandwidth
• Decoder is just about implementable in software
• Target 2Mbps to 8Mbps really.
• The "CD" of Video?
MPEG Source Images format
The source pictures consist of three rectangular matrices of integers: a luminance matrix (Y) and
two chrominance matrices (Cb and Cr).
The MPEG supports three formats:
4:2:0 format
In this format the Cb and Cr matrices shall be one half the size of the Y matrix in both
horizontal and vertical dimensions.
4:2:2 format
In this format the Cb and Cr matrices shall be one half the size of the Y matrix in horizontal
dimension and the same size in the vertical dimension.
4:4:4 format
In this format the Cb and Cr matrices will be of the same size as the Y matrix in both vertical
and horizontal dimensions.
Looking at some video capture cards (e.g. Intel's PC one) it may be hard to convert to this, but
then this is targeted at digital video tape and video on demand really.
MPEG frames
The output of the decoding process, for interlaced sequences, consists of a series of fields that are
separated in time by a field period. The two fields of a frame may be coded independently (field-
pictures) or can be coded together as a frame (frame pictures).
The diagram 4.15 shows the intra, predictive and bi-directional frames that MPEG supports:
Figure 4.15: MPEG Frames

Adri Jovin J.J.
Page | 48
An MPEG source encoder will consist of the following elements:
• Prediction (3 frame times)
• Block Transformation
• Quantization and Variable Length Encoding
MPEG Prediction
MPEG defines three types of pictures:
Intrapictures (I-pictures)
These pictures are encoded only with respect to themselves. Here each picture is composed
onto blocks of 8x8 pixels each that are encoded only with respect to themselves and are sent
directly to the block transformation process.
Predictive pictures (P-pictures)
These are pictures encoded using motion compensated prediction from a past I-picture or P-
picture. A prediction error is calculated between a 16x16 pixels region (macroblock) in the
current picture and the past reference I or P picture. A motion vector is also calculated to
determine the value and direction of the prediction. For progressive sequences and interlaced
sequences with frame-coding only one motion vector will be calculated for the P-pictures. For
interlace sequences with field-coding two motion vectors will be calculated. The prediction error
is then composed to 8x8 pixels blocks and sent to the block transformation
Bi-directional pictures (B-pictures)
These are pictures encoded using motion compensates predictions from a past and/or future I-
picture or P-picture. A prediction error is calculated between a 16x16 pixels region in the current
picture and the past as well as future reference I-picture or P-picture. Two motion vectors are
calculated. One to determine the value and direction of the forward prediction the other to
determine the value and direction of the backward prediction. For field-coding pictures in
interlaced sequences four motion vectors will thus be calculated.
It should be noted that a B-picture can never be used as a prediction picture (obviously!).
The method of calculating the motion vectors as well as the search area for the best predictor is
left to be determined by the encoder.
MPEG Block Transformation
In block transformation, INTRA coded blocks as well as prediction errors are processed by a
two-dimensional DCT function.
•Quantization

Adri Jovin J.J.
Page | 49
The purpose of this step is to achieve further compression by representing the DCT
coefficients with no greater precision than is necessary to achieve the required quality.
•Variable length encoding
Here extra compression (non-lossy) is done by assigning shorter code-words to
frequent events and longer code-words to less frequent events. Huffman coding is
usually used to implement this step.
MPEG Multiplexing
The video multiplexer structures the compressed data into a hierarchical bitstream that can be
universally interpreted.
The hierarchy has four layers :
Videosequence
This is the highest syntactic structure of the coded bitstream. It can be looked at as a random
access unit.
Group of pictures
This is optional in MPEG II. This corresponds to a series of pictures. The first picture in the
coded bitstream has to be an I picture. Group of pictures does assist random access. They can
also be used at scenes cuts or other cases where motion compensation is ineffective. Applications
requiring random access, fast-forwarder fast-reverse playback may use relatively short group of
pictures.
Picture
This would correspond to one picture in the video sequence. For field pictures in interlaced
sequences, the interlaced picture will be represented by two separate pictures in the coded
stream. They will be encoded in the same order that shall occur at the output of the decoder.
Slice
This corresponds to a group of Macroblocks. The actual number of Macroblocks within a slice
is not subject to standardization. Slices do not have to cover the whole picture. Its a requirement
that if the picture was used subsequently for predictions, then predictions shall only be made
from those regions of the picture that were enclosed in slices.
Macroblock
1. A macro block contains a section of the luminance component and the spatially
corresponding chrominance components. A 4:2:0 macroblock consists of 6 blocks (4Y, 1
Cb, 1Cr) A 4:2:2 Macroblock consists of 8 blocks (4Y, 2 Cb, 2 Cr) A4:4:4 Macroblock
consists of 12 blocks (4Y,4Cb, 4Cr)
2. Block which, as with H.261, corresponds to 8x8 pixels.
MPEG Picture Order

Adri Jovin J.J.
Page | 50
It must be noted that in MPEG the order of the picture in the coded stream is the order in which
the decoder process them. The reconstructed frames are not necessarily in the correct form of
display. The following example in table 4.7 shows such a case at the encoder output, in the coded
bitstream and at the decoder input, and at the decoder output.
At the encoder input
12 3 4 5 6 78 9 10 11 12 13
IB B P B B PB B I B B P
At the encoder output
14 2 3 7 5 610 8 9 13 11 12
IP B B P B BI B B P B B
At the decoder output
12 3 4 5 6 78 9 10 11 12 13
Table 4.7: MPEG Source, Encoder and Decoder Frame Sequence
Since the order of pictures at the decoder is not always in the display order, this leads to potential
for delays in the encoder/decoder loop This is also true of H.261 - at its highest compression
ratio, it may incur as much as 0.5 seconds delay - not very pleasant for interactive use!
Scaleable Extensions
The scalability tools specified by MPEG II are designed to support applications beyond that
supported by single layer video. In a scaleable video coding, it is assumed that given an encoded
bitstream, decoders of various complexities can decode and display appropriate reproductions of
coded video. The basic scalability tools offered are: data partitioning, SNR scalability, spatial
scalability and temporal scalability. Combinations of these basic scalability tools are also
supported and are referred to as hybrid scalability. In the case of basic scalability, two layers of
video referred to as the lower layer and the enhancement layer are allowed. Whereas in hybrid
scalability up to three layers are supported.
MPEG Extensions include:
Spatial scalable extension
This involves generating two spatial resolution video layers from a single video source such
that the lower layer is coded by itself to provide the basic spatial resolution and the enhancement
layer employs the spatially interpolated lower layer and carries the full spatial resolution of the
input video source.
SNR scalable extension
This involves generating two video layers of same spatial resolution but different video
qualities from a single video source. The lower layer is coded by itself to provide the basic video

Adri Jovin J.J.
Page | 51
quality and the enhancement layer is coded to enhance the lower layer. The enhancement layer
when added back to the lower layer regenerates a higher quality reproduction of the input video.
Temporal scalable extension
This involves generating two video layers whereas the lower one is encoded by itself to
provide the basic temporal rate and the enhancement layer is coded with temporal prediction
with respect to the lower layer. These layers when decoded and temporally multiplexed yield full
temporal resolution of the video source.
Data partitioning extension
This involves the partitioning of the video coded bitstream into two parts. One part will carry
the more critical parts of the bitstream such as headers, motion vectors and DC coefficients; the
other part will carry less critical data such as the higher DCT coefficients.
Profiles and levels
Profiles and levels provide a means of defining subsets of the syntax and semantics and
thereby the decoder capabilities to decode a certain stream. A profile is a defined sub-set of the
entire bitstream syntax that is defined by MPEG II. A level is a defined set of constraints
imposed on parameters in the bit stream.
To constrain the choice pragmatically, five MPEG II profiles are defined and four levels of
quality. However, It is important to realize specification is of an encoded stream rather than of an
actual compression and decompression mechanism. This leaves lots of options open to the
implementor Profiles allow us to scope these cjhoices (as in other standards, e.g. in
telecommunications) This is important, as the hard work (expensive end) is the encoder, while
the stream as specified, is generally easy however it Is implemented, to decode. The diagram
below (4.16) shows a comparison of the data rate out of an H.261 and an MPEG coder.
Figure 4.16: H261 versus MPEG data rates
MPEG II is now an ISO standard. Due to the forward and backward temporal compression used
by MPEG, a better compression and better quality can be produced. As MPEG does not limit the
picture resolution, high resolution data can still be compressed using MPEG. The scaleable
extensions defined by MPEG can map neatly on the hierarchical scheme explained in 2.1. The
out-of- order processing which occurs in both encoding and decoding side can introduce
considerable latencies. This is undesirable in video telephony and video conferencing.

Adri Jovin J.J.
Page | 52
Prices for hardware MPEG encoders are quite expensive at the moment though this should
change over the near future. The new SunVideo board (see below) does support MPEG I
encoding. Software implementation of MPEG I DECoders is already available.
MPEG III and IV
MPEG III was going to be a higher quality encoding for HDTV. It transpired after some studies
that MPEG II at higher rates is pretty good, and so MPEG III has been dropped.
MPEG IV is aimed at the opposite extreme - that of low bandwidth or low storage capacity
environments (e.g. PDAs). It is based around model-based image coding schemes (i.e. knowing
what is in the picture!). It is aimed at UP TO 64Kbps.
Region Coding
Intel's Digital Video Interactive compression scheme is based on the region encoding technique.
Each picture is divided into regions which in turn is split into subregions and so on, until the
regions can be mapped onto basic shapes to fit the required bandwidth and quality. The chosen
shapes can be reproduced well at the decoder. The data sent is a description of the region tree
and of the shapes at the leaves. This is an asymmetric coding, which requires large amount of
processing for the encoding and less for the decoding.
DVI ,though not a standard, started to play an important role in the market. SUN prototype
DIME board used DVI compression and it was planned to be incorporated in the new generation
of SUN videopix cards.
This turned out to be untrue. Intel canceled the development of the V3 DVI chips. SUN next
generation of VideoPix, the SunVideo card does not support DVI. The future of DVI is all in
doubt.
Wavelet, Vector Quantisation, and Fractal Compression
These are three techniques which are emerging from the research community for improved
quality or compression or even both.
Briefly, though Wavelets are a generalization of the Fourier Transforms. Vector quantisation is a
generalization of the quantisation mechanism, and fractal is a generalization of an information
free object based compression scheme.
All three are compute intensive, although there are many people working on codecs (hardware
and software) and we will probably see some low cost high quality results soon.

UNIT II INTERNETWORKING MULTIMEDIA
Adri Jovin J.J.
Page | 53
UNIT II
SUBNETWORK TECHNOLOGY
Broadband Services
These days telecommunications networks are global, so when information providers and content
owners determine the worth of their products and how to charge money for them, information
elements will ultimately link up online as distributed resources on a data highway (actually more
like a toll road), where you will pay to acquire and use multimedia-based information. Curiously,
the actual glass fiber cables that make up much of the physical backbone of the data highway are,
in many cases, owned by railroads and pipeline companies who simply buried the cable on
existing rights of way, where no special permits and environmental studies are necessary.
One railroad in the United States invested more than a million dollars in a special cable-laying
trenching car; in the United Kingdom, fiber-optic cable runs in the towpaths of the decaying
19th-century canal and barge system. Bandwidth on these fiber-optic lines is leased to others, so
competing retailers such as AT&T, Verizon, MCI, and Sprint may even share the same cable.
Full-text content from books and magazines is downloadable; feature movies are played at home;
real-time news feeds from anywhere on earth are available; lectures from participating
universities are monitored for education credits; street maps of cities are viewable—with
recommendations for restaurants, in any language—and online travelogues include testimonials
and video tracks. Just think—each of these interfaces or gateways to information is a multimedia
project waiting to be developed!
Interactive multimedia is delivered to many homes throughout the world. Interest from a
confluence of entertainment mega-corps, information publishers and providers, cable and
telephone companies, and hardware and software manufacturers is driving this inevitable
evolution, and profound changes in global communications strategy are on the drawing boards.
What will be piped through this new system for entertainment, reference, and lifelong learning
experiences are the very multimedia elements discussed in the chapters of this book, including
text, graphics, animation, sound, and video.
The actual content provided, let us hope, will be excellent fare, generated by thinking and caring
creative people using ideas that will propel all of us into a better world. Entertainment companies
that own content easily converted to multimedia projects are teaming up with cable TV
companies. Film studios are creating new divisions to produce interactive multimedia and
wealthy talents have formed new companies to join in on the action. Google is scanning millions
of books and periodicals. Even without a clear business model with known profits, large media
corporations are uniting to create huge conglomerates to control the content and delivery of
tomorrow’s information. Some companies will own the routes for carrying data, other companies
will own the hardware and software interfaces at the end of the line, at offices and homes. Some
will knit it all together and provide supply-on demand and billing services. Regardless of who
owns the roadways and the hardware boxes, multimedia producers will create the new literature

Adri Jovin J.J.
Page | 54
and the rich content sent along them. This is a fresh and exciting industry that is coming of age,
but one that is still faced with many growing pains.
We have been indoors too long. Let us now go outside and see if any interesting networking is
going on there. It turns out that quite a bit is going on there, and some of it has to do with the so-
called last mile. With the deregulation of the telephone system in many countries, competitors to
the entrenched telephone company are now often allowed to offer local voice and high-speed
Internet service. There is certainly plenty of demand. The problem is that running fiber, coax, or
even category 5 twisted pair to millions of homes and businesses are prohibitively expensive.
What is a competitor to do?
The answer is broadband wireless. Erecting a big antenna on a hill just outside of town and
installing antennas directed at it on customers' roofs is much easier and cheaper than digging
trenches and stringing cables. Thus, competing telecommunication companies have a great
interest in providing a multimegabit wireless communication service for voice, Internet, movies
on demand, etc. LMDS was invented for this purpose. However, until recently, every carrier
devised its own system. This lack of standards meant that hardware and software could not be
mass produced, which kept prices high and acceptance low.
Many people in the industry realized that having a broadband wireless standard was the key
element missing, so IEEE was asked to form a committee composed of people from key
companies and academia to draw up the standard. The next number available in the 802
numbering space was 802.16, so the standard got this number. Work was started in July 1999,
and the final standard was approved in April 2002. Officially the standard is called ''Air Interface
for Fixed Broadband Wireless Access Systems.'' However, some people prefer to call it a
wireless MAN (Metropolitan Area Network) or a wireless local loop. We regard all these terms
as interchangeable.
Like some of the other 802 standards, 802.16 was heavily influenced by the OSI model,
including the (sub)layers, terminology, service primitives, and more. Unfortunately, also like
OSI, it is fairly complicated. In the following sections we will give a brief description of some of
the highlights of 802.16, but this treatment is far from complete and leaves out many details. For
additional information about broadband wireless in general, see (Bolcskei et al., 2001; and
Webb, 2001). For information about 802.16 in particular, see (Eklund et al., 2002).
Comparison of 802.11 with 802.16
At this point you may be thinking: Why devise a new standard? Why not just use 802.11? There
are some very good reasons for not using 802.11, primarily because 802.11 and 802.16 solve
different problems. Before getting into the technology of 802.16, it is probably worthwhile
saying a few words about why a new standard is needed at all.
The environments in which 802.11 and 802.16 operate are similar in some ways, primarily in
that they were designed to provide high-bandwidth wireless communications. But they also
differ in some major ways. To start with, 802.16 provide service to buildings, and buildings are

Adri Jovin J.J.
Page | 55
not mobile. They do not migrate from cell to cell often. Much of 802.11 deals with mobility, and
none of that is relevant here. Next, buildings can have more than one computer in them, a
complication that does not occur when the end station is a single notebook computer. Because
building owners are generally willing to spend much more money for communication gear than
are notebook owners, better radios are available. This difference means that 802.16 can use full-
duplex communication; something 802.11 avoids keeping the cost of the radios low.
Because 802.16 runs over part of a city, the distances involved can be several kilometers, which
means that the perceived power at the base station can vary widely from station to station. This
variation affects the signal-to-noise ratio, which, in, turns, dictates multiple modulation schemes.
Also, open communication over a city means that security and privacy are essential and
mandatory.
Furthermore, each cell is likely to have many more users than will a typical 802.11 cell, and
these users are expected to use more bandwidth than will a typical 802.11 user. After all it is rare
for a company to invite 50 employees to show up in a room with their laptops to see if they can
saturate the 802.11 wireless networks by watching 50 separate movies at once. For this reason,
more spectrum is needed than the ISM bands can provide, forcing 802.16 to operate in the much
higher 10-to-66 GHz frequency range, the only place unused spectrum is still available.
But these millimeter waves have different physical properties than the longer waves in the ISM
bands, which in turn require a completely different physical layer. One property that millimeter
waves have is that they are strongly absorbed by water (especially rain, but to some extent also
by snow, hail, and with a bit of bad luck, heavy fog). Consequently, error handling is more
important than in an indoor environment. Millimeter waves can be focused into directional
beams (802.11 is omnidirectional), so choices made in 802.11 relating to multipath propagation
are moot here.
Another issue is quality of service. While 802.11 provides some support for real-time traffic
(using PCF mode), it was not really designed for telephony and heavy-duty multimedia usage. In
contrast, 802.16 are expected to support these applications completely because it is intended for
residential as well as business use.
In short, 802.11 was designed to be mobile Ethernet, whereas 802.16 was designed to be
wireless, but stationary, cable television. These differences are so big that the resulting standards
are very different as they try to optimize different things.
A very brief comparison with the cellular phone system is also worthwhile. With mobile phones,
we are talking about narrow-band, voice-oriented, low-powered, mobile stations that
communicate using medium-length microwaves. Nobody watches high-resolution, two-hour
movies on GSM mobile phones (yet). Even UMTS has little hope of changing this situation. In
short, the wireless MAN world is far more demanding than is the mobile phone world, so a
completely different system is needed. Whether 802.16 could be used for mobile devices in the
future is an interesting question. It was not optimized for them, but the possibility is there. For
the moment it is focused on fixed wireless.

Adri Jovin J.J.
Page | 56
The 802.16 Protocol Stack
The 802.16 protocol stack is illustrated in Fig. 4-31. The general structure is similar to that of the
other 802 networks, but with more sublayers. The bottom sublayer deals with transmission.
Traditional narrow-band radio is used with conventional modulation schemes. Above the
physical transmission layer comes a convergence sublayer to hide the different technologies from
the data link layer. Actually, 802.11 has something like this too, only the committee chose not to
formalize it with an OSI-type name.
Figure 4-31. The 802.16 protocol stack.
Although we have not shown them in the figure, work is already underway to add two new
physical layer protocols. The 802.16a standard will support OFDM in the 2-to-11 GHz frequency
range. The 802.16b standard will operate in the 5-GHz ISM band. Both of these are attempts to
move closer to 802.11.
The data link layer consists of three sublayers. The bottom one deals with privacy and security,
which is far more crucial for public outdoor networks than for private indoor networks. It
manages encryption, decryption, and key management.
Next comes the MAC sublayer common part. This is where the main protocols, such as channel
management, are located. The model is that the base station controls the system. It can schedule
the downstream (i.e., base to subscriber) channels very efficiently and plays a major role in
managing the upstream (i.e., subscriber to base) channels as well. An unusual feature of the
MAC sublayer is that, unlike those of the other 802 networks, it is completely connection
oriented, in order to provide quality-of-service guarantees for telephony and multimedia
communication.
The service-specific convergence sublayer takes the place of the logical link sublayer in the other
802 protocols. Its function is to interface to the network layer. A complication here is that 802.16
was designed to integrate seamlessly with both datagram protocols (e.g., PPP, IP, and Ethernet)
and ATM. The problem is that packet protocols are connectionless and ATM is connection
oriented. This means that every ATM connection has to map onto an 802.16 connection, in
principle a straightforward matter. But onto which 802.16 connection should an incoming IP
packet be mapped? That problem is dealt with in this sublayer.

Adri Jovin J.J.
Page | 57
The 802.16 Physical Layer
As mentioned above, broadband wireless needs a lot of spectrum, and the only place to find it is
in the 10-to-66 GHz range. These millimeter waves have an interesting property that longer
microwaves do not: they travel in straight lines, unlike sound but similar to light. As a
consequence, the base station can have multiple antennas, each pointing at a different sector of
the surrounding terrain, as shown in Fig. 4-32. Each sector has its own users and is fairly
independent of the adjoining ones, something not true of cellular radio, which is omnidirectional.
Figure 4-32. The 802.16 transmission environment.
Because signal strength in the millimeter band falls off sharply with distance from the base
station, the signal-to-noise ratio also drops with distance from the base station. For this reason,
802.16 employs three different modulation schemes, depending on how far the subscriber station
is from the base station. For close-in subscribers, QAM-64 is used, with 6 bits/baud. For
medium-distance subscribers, QAM-16 is used, with 4 bits/baud. For distant subscribers, QPSK
is used, with 2 bits/baud. For example, for a typical value of 25 MHz worth of spectrum, QAM-
64 gives 150 Mbps, QAM-16 gives 100 Mbps, and QPSK gives 50 Mbps. In other words, the
farther the subscriber is from the base station, the lower the data rate. The constellation diagrams
for these three modulation techniques were shown in Fig. 2-25.
Given the goal of producing a broadband system, and subject to the above physical constraints,
the 802.16 designers worked hard to use the available spectrum efficiently. One thing they did
not like was the way GSM and DAMPS work. Both of those use different but equal frequency
bands for upstream and downstream traffic. For voice, traffic is probably symmetric for the most
part, but for Internet access, there is often more downstream traffic than upstream traffic.
Consequently, 802.16 provides a more flexible way to allocate the bandwidth. Two schemes are
used, FDD (Frequency Division Duplexing) and TDD (Time Division Duplexing). The latter is
illustrated in Fig. 4-33. Here the base station periodically sends out frames. Each frame contains
time slots. The first ones are for downstream traffic. Then comes a guard time used by the
stations to switch direction. Finally, we have slots for upstream traffic. The number of time slots
devoted to each direction can be changed dynamically to match the bandwidth in each direction
to the traffic.

Adri Jovin J.J.
Page | 58Figure 4-33. Frames and time slots for time division duplexing.
Downstream traffic is mapped onto time slots by the base station. The base station is completely
in control for this direction. Upstream traffic is more complex and depends on the quality of
service required. We will come to slot allocation when we discuss the MAC sublayer below.
Another interesting feature of the physical layer is its ability to pack multiple MAC frames back-
to back in a single physical transmission. The feature enhances spectral efficiency by reducing
the number of preambles and physical layer headers needed.
Also noteworthy is the use of Hamming codes to do forward error correction in the physical
layer. Nearly all other networks simply rely on checksums to detect errors and request
retransmission when frames are received in error. But in the wide area broadband environment,
so many transmission errors are expected that error correction is employed in the physical layer,
in addition to checksums in the higher layers. The net effect of the error correction is to make the
channel look better than it really is (in the same way that CD-ROMs appear to be very reliable,
but only because more than half the total bits are devoted to error correction in the physical
layer).
The 802.16 MAC Sublayer Protocol
The data link layer is divided into three sublayers, as we saw in Fig. 4-31. Suffice it to say that
encryption is used to keep secret all data transmitted. Only the frame payloads are encrypted; the
headers are not. This property means that a snooper can see who is talking to whom but cannot
tell what they are saying to each other.
At the time a subscriber connects to a base station, they perform mutual authentication with RSA
public-key cryptography using X.509 certificates. The payloads themselves are encrypted using a
symmetric-key system, either DES with cipher block chaining or triple DES with two keys. AES
(Rijndael) is likely to be added soon. Integrity checking uses SHA-1. Now that was not so bad,
was it?
Let us now look at the MAC sublayer common part. MAC frames occupy an integral number of
physical layer time slots. Each frame is composed of sub-frames, the first two of which are the
downstream and upstream maps. These maps tell what is in which time slot and which time slots
are free. The downstream map also contains various system parameters to inform new stations as
they come on-line.
The downstream channel is fairly straightforward. The base station simply decides what to put in
which subframe. The upstream channel is more complicated since there are competing

Adri Jovin J.J.
Page | 59
uncoordinated subscribers that need access to it. Its allocation is tied closely to the quality-of-
service issue. Four classes of service are defined as follows:
1. Constant bit rate service.
2. Real-time variable bit rate service.
3. Non-real-time variable bit rate service.
4. Best-efforts service.
All service in 802.16 is connection-oriented, and each connection gets one of the above classes
of service, determined when the connection is set up. This design is very different from that of
802.11 or Ethernet, which have no connections in the MAC sublayer.
Constant bit rate service is intended for transmitting uncompressed voice such as on a T1
channel. This service needs to send a predetermined amount of data at predetermined time
intervals. It is accommodated by dedicating certain time slots to each connection of this type.
Once the bandwidth has been allocated, the time slots are available automatically, without the
need to ask for each one.
Real-time variable bit rate service is for compressed multimedia and other soft real-time
applications in which the amount of bandwidth needed each instant may vary. It is
accommodated by the base station polling the subscriber at a fixed interval to ask how much
bandwidth is needed this time.
Non-real-time variable bit rate service is for heavy transmissions that are not real time, such as
large file transfers. For this service the base station polls the subscriber often, but not at rigidly-
prescribed time intervals. A constant bit rate customer can set a bit in one of its frames
requesting a poll in order to send additional (variable bit rate) traffic.
If a station does not respond to a poll k times in a row, the base station puts it into a multicast
group and takes away its personal poll. Instead, when the multicast group is polled, any of the
stations in it can respond, contending for service. In this way, stations with little traffic do not
waste valuable polls.
Finally, best-efforts service is for everything else. No polling is done and the subscriber must
contend for bandwidth with other best-efforts subscribers. Requests for bandwidth are done in
time slots marked in the upstream map as available for contention. If a request is successful, its
success will be noted in the next downstream map. If it is not successful, unsuccessful
subscribers have to try again later. To minimize collisions, the Ethernet binary exponential
backoff algorithm is used.
The standard defines two forms of bandwidth allocation: per station and per connection. In the
former case, the subscriber station aggregates the needs of all the users in the building and makes
collective requests for them. When it is granted bandwidth, it doles out that bandwidth to its
users as it sees fit. In the latter case, the base station manages each connection directly.

Adri Jovin J.J.
Page | 60
The 802.16 Frame Structure
All MAC frames begin with a generic header. The header is followed by an optional payload and
an optional checksum (CRC), as illustrated in Fig. 4-34. The payload is not needed in control
frames, for example, those requesting channel slots. The checksum is (surprisingly) also optional
due to the error correction in the physical layer and the fact that no attempt is ever made to
retransmit real-time frames. If no retransmissions will be attempted, why even bother with a
checksum?
Figure 4-34. (a) A generic frame. (b) A bandwidth request frame.
A quick rundown of the header fields of Fig. 4-34(a) is as follows. The EC bit tells whether the
payload is encrypted. The Type field identifies the frame type, mostly telling whether packing
and fragmentation are present. The CI field indicates the presence or absence of the final
checksum. The EK field tells which of the encryption keys is being used (if any). The Length
field gives the complete length of the frame, including the header. The Connection identifier tells
which connection this frame belongs to. Finally, the HeaderCRC field is a checksum over the
header only, using the polynomial x8
+ x2
+ x + 1.
A second header type, for frames that request bandwidth, is shown in Fig. 4-34(b). It starts with a
1 bit instead of a 0 bit and is similar to the generic header except that the second and third bytes
form a 16-bit number telling how much bandwidth is needed to carry the specified number of
bytes. Bandwidth request frames do not carry a payload or full-frame CRC.
A great deal more could be said about 802.16, but this is not the place to say it. For more
information, please consult the standard itself.
ATM and IP
The two basic tasks of an intermediate node in a packet switched network are[#!Illus!#]: to
forward packets, maintaining as far as economically possible, appropriate timing relationship
between packets provided that they meet the service contract; and to deliver them along the
appropriate route to the destination.
The Internet[#!IP!#][#!TCP!#] has hitherto defined a simple service model that does not offer
any definition of the timing model, and it has sufficed to provide a single FIFO queue in a router.
In contrast, the path selection mechanisms in the Internet have been rich, typically permitting
rapid deployment of new routers, and rapid response to changes in traffic patterns.

Adri Jovin J.J.
Page | 61
This has been achieved by making a hop-by-hop, packet-by-packet, routing lookup. Some people
confuse this with a routing decision - in fact, the routing decision is made when a path
computation is done, which is typically whenever new routing information is available to each
node, and is a management task, that often operates somewhat slower than the route
lookup/forwarding decision.
However, to give flexibility, IP packets all contain source and destination information, which
permits a choice of route.
Recently, to add further services, the Internet Standards have been enhanced[#!white!#] to
provide a signalling protocol, and a family of service models based on the theory in Parekh's
work[#!gps1!#][#!gps2!#]. This shows how a Weighted Fair Queueing system can provide
bounded delays for variable length packet flows, provided that the traffic is constrained by a
leaky bucket, and that an admission test (for the appropriate leaky bucket specification) is carried
out at the edge (or at each hop) of the path - this is known as a ``flow specification'', and
subsequent packets are matched to the admitted flow by classifying them based on the source,
destination and transport level ports (application level multiplexing I-D's). [#!Giga!#] This is the
Integrated Services Internet.
There has been some doubt about whether it is possible to build routers fast enough to classify
packets according to their flow-spec, and to carry out a WFQ insertion.
In contrast to this, two other hybrid approaches to building a fast Internet have been proposed:
1. Frame Relay or ATM switch fabric.
2. Hybrid switch/router nodes.
The first approach envisages a network provided (presumably) by telecommunications carriers
made up of traditional virtual circuit based packet switches based around either Frame Relay
(today) or ATM (tomorrow). Routers would operate between site networks, and act as access
devices. Here there is a clear division of slow changing, backbone provisioning, and more rapid
changing edge networks - this is not terribly different from todays systems, although there could
be more flexible access to differential services from the routers (e.g. interaction between RSVP
requests to routers, and router access via say Q.2931 to backbone QoS support).
The second approach is a more integrated approach, attempting to capitalise on the benefits of
virtual circuits at one level, and on the flexibility advantages of dynamic IP routing at another.
All nodes provide both functions in some manner. There are four main proposals for how to
achieve this latter architecture.
The principal performance gain that is expected from all the proponents of these schemes is that
the switch designers for ATM capitalises on the fixed size header, with the short VCI/VPI field,
that permits simple tables for the lookup for forwarding, and the small fixed size cell, that
permits low latency, and jitter, in combining bursty flows for forwarding. Sometimes, there is a
mistaken claim that the queue insertion procedure could be faster than for Integrated Services IP

Adri Jovin J.J.
Page | 62
- in fact, recent work by Hui Zhang[#!wf2q!#][#!scaleswitch!#] shows that this is not at all the
case.
1. CSR - Cell Switched Router - from Toshiba (and others)
This approach simply uses normal IP routing for forwarding packets, but shreds them, so
that the store-and-forwarding latency is much lower.
2. Flow Label - Implemented in Ipsilon devices
This is a smart technique for matching a flow of IP packets onto a virtual circuit,
dynamically, Essentially, flow characteristics are pre-programmed into a recogniser,
which only opens VCs for individual flows that are expected to persist long enough.
Short flows are bundled onto a default VC.
3. Tag Switching - Proposed by Cisco
All routes from the routing table are used to generate tags, which are then used in the
backbone to indicate a VC to follow.
4. ARIS - Aggregated Route-Based IP Switching
All backbone egress points are used to generate tags which are used to bundle together
like minded flows onto a VC.
There are a number of problems with all these hybrid approaches, including: VC setup times;
Exhausting VCI/VPI tables; management complexity; difficulty supporting mobility; difficulty
supporting multicast (many-to-many); possible lack of fault tolerance.
Mapping classes and QoS
The Integrated Services model has an initial deployment scenario of routers connected together
by point to point links. In that situation, packet scheduling for service classes needs only to be
deployed within the routers, to provide the service overall.
However, many parts of the Internet involve other interconnection technologies between routers.
Two common, but extremely different situations are:
• Routers interconnected via Local Area Networks, such as Ethernet, FDDI, 100Mbps
Ethernet, 100VG, and so forth.
• Routers interconnected by so-called ``Non Broadcast Multiple Access'' (NBMA)
networks such as Frame Relay, SMDS and ATM - these are often referred to as switched
clouds, typically because of the way that they are drawn on inter-network maps.
The Integrated Services over Specific Lower Layers, Working Group of the IETF has defined the
mapping of some of the IP level services onto services provided at the lower layer.
In some cases, the Link Layer as currently deployed cannot support the upper layer services with
any reasonable guarantee, and so some enhancement is typically called for - one such case is
Ethernet where the shared Media Access is so non-deterministic, that an enhancement based on

Adri Jovin J.J.
Page | 63
some form of distributed bandwidth manager is required to provide anything beyond best effort
IP service over an Ethernet hop.
In the case of NBMA networks, particularly ATM, a much richer variety of services is available
at the lower layer for IP to utilise. In fact, there are multiple possible mappings from the high
level IP service, to the proposed ``Bearer Service Classes'' of ATM - for example:
• Guaranteed Service could be provided over CBR or rtVBR.
• Controlled Load could be provided over CBR, rtVBR, nrtVBR or ABR
• Best Effort could be provided over CBR, rtVBR, nrtVBR, ABR or UBR
Again, this area is very active, and simplification will no doubt be sought and found by the
market. In the next 3 subsections, we briefly describe the ``pure'' IP approach.
Topology Control
One of the principle reasons for the success of IP is the flexibility for addressing and routing. At
the same time, this has led to problems . Stability of routing is reported to be getting worse, and
there is some concern about the exhaustion of the global IPv4 address space. This is being solved
by the introduction of IPv6, but the performance problems for route lookup might seem to be
exacerbated by this. In fact, the work of Degermark and others at SICS[#!Rout!#], shows that it
is feasible to construct a novel data structure that not only permits fast routing lookup, it actually
reduces the size of the routing table as well. In fact, the entire routing table of today’s Internet
can be fitted into the on-chip data cache of a common desktop processor (Pentium) and a lookup
(``longest match prefix'') computed in 8 memory accesses in the worst case. The scaling
complexity of the data structure is with the route depth, not with the size of the network, and so
should not slow down much as we move to a billion end systems.
This is useful since it permits us to consider using IP addresses (and possibly fully generalised
port specifications) as the key, the basis for deciding what to do with a packet as well as where to
send it. This gives high degrees of flexibility (one can alter QoS mid flow, as well as route).
QoS Control
QoS control requires some number of alternate queues, as well as some form of admission,
policing and possibly shaping.
Assuming that admission and policing can be done on small numbers of flows at the ``edge''
(ingress) of the network, then we can police (and shape) aggregate flows as they approach the
core of the network (recent results from Roch Guerin's group at TJ Watson show that this in fact
gets higher utilisation). Then the problem is that of the performance of queue insertion.

Adri Jovin J.J.
Page | 64
Queue Insertion/Lookup Performance
Queue insertion for WFQ is typically a sort algorithm (actually, it depends on the interleave of
the arrival processes - if they are loosely synchronised, then it is possible that one may only
rarely have to do more than a few swaps).
Hui Zhang's work has shows that for CBR (Guaranteed Service in Integrated Service Internet
Terminology), a different queueing algorithm called Worst Case Fair Weighted Fair Queueing
achieves better delay jitter bounds, and yet can have O(1) insertion time.
Conclusions on IP/ATM
It would appear that a purist IP architecture for all switching nodes in the Internet is both
feasible, and for management reasons (and therefore cost), attractive. Because (and not despite)
of the work done on QoS and scalable ATM switch design, it would seem that the exposure of
ATM at any interface is completely unnecessary, except perhaps, at modest speed links (ADSL,
DVB, or 25Mbps Domestic networks?), where the reduced latency for voice/video on
multiplexed at finer grain may need the cut through cell size.
IPv6
IPv6 is the next generation of the Internet Protocol (IP). It has primarily been developed to solve
the problem of the eventual exhaustion of the IPv4 address space, but also offers other
enhancements. IPv6 addresses are 16 bytes long, in contrast to IPv4’s 4 byte addresses. Other
features of IPv6 include:
• Address structure improvements:
• globally unique addresses with more levels of addressing hierarchy, toreduce the
size of routing tables
• autoconfiguration of addresses by hosts
• improved scalability of multicast routing, by adding a “scope” field to multicast
addresses
• a new type of address, the “anycast address”, which is used to send a packet to
any one of a group of devices.
• Removal of the need for packet fragmentation en-route, by dynamic determination of the
largest packet size that is supported by every link in the path. A link’s MTU (Maximum
Transmission Unit) must be at least 1280 bytes, compared with 576 bytes for IPv4.
• Traffic Class, which allows a packet to be labelled with an appropriate priority. If the
network becomes congested, the lowest priority packets are dropped.
• Flow labels, which indicate to intermediate routers that packets are part of a flow, and
that this flow requires a particular type of service. This feature enables, for example, real-
time processing of data streams. It also increases routing speed because the forwarding
router need only check the flow label, not the rest of the header. The handling indicated
by the flow label can be done by the IPv6 Hop-by-Hop header, or by a separate protocol
such as RSVP.

Adri Jovin J.J.
Page | 65
• Mandatory authentication and data integrity protocols, through IPsec. IPsec is optional in
IPv4.
An IPv6 network can contain three types of nodes, where node is a general term that refers to
any IPv6-aware device. A host is a device on the network that is not a router. For example, a host
may be a printer or a computer. A router may also act as a host. A router is a device on the
network that directs the flow of IPv6 packets. For example, a router may be a router or a Layer 3
switch. A destination is a host to which packets are specifically sent.
The 6bone
The 6bone is an experimental virtual network of nodes that support IPv6 packets, tunnelled
together through the existing IPv4 Internet. Most of the nodes are workstations or similar
machines, with IPv6-capable operating systems.
The 6bone is part of the transition to IPv6. Its purpose is to provide an environment in which
IPv6 can be tested and procedures for IPv6 can be developed. When IPv6 is sufficiently
developed and being used widely, the 6bone will probably disappear.
IPv6 Addresses and Prefixes
IPv6 addresses are hexadecimal, and are made up of eight pairs of octets separated by colons (:).
An example of a valid address is fe80:0000:0000:0000:0260:0000:97ff:64aa. In the interests of
brevity, addresses can be abbreviated in two ways:
• Leading zeros can be omitted, so this address can be written as
fe80:0:0:0:260:0:97ff:64aa.
• Consecutive zeros can be replaced with a double colon, so this address can be written as
fe80::260:0:97ff:64aa. Note that a double colon can replace any number of consecutive
zeros, but an address can only contain one double colon.
Like IPv4 addresses, a proportion of the leftmost bits of the IPv6 address can be used to indicate
the subnet, rather than a single node. This part of the address is called the prefix. Prefixes
provide the equivalent functionality to a subnet mask in IPv4, allowing a subnet to be addressed,
rather than a single node. If a prefix is specified, the IPv6 address is followed by a slash and the
number of bits that represent the prefix. For example, 3ffe::/16 indicates that the first 16bits
(3ffe) of the address 3ffe:0:0:0:0:0:0:0 represent the prefix. Like IPv4 addresses, IPv6 addresses
are attached to interfaces.
Like IPv4 addresses, IPv6 addresses are attached to interfaces.
Unicast Addresses
A unicast address is attached to a single interface, and is used to deliver packets only to that
interface.

Adri Jovin J.J.
Page | 66
A number of special addresses have been defined:
• IPv4-compatible and IPv4-mapped addresses. IPv4-compatible addresses are used to
tunnel IPv6 packets across an IPv4 network. IPv4-mapped addresses are used by an IPv6
host to communicate with an IPv4 host. The IPv6 host addresses the packet to the
mapped address.
• Link-local addresses, which can only be used on the local network that the interface is
attached to. The link-local prefix is fe80::/10. Different interfaces on a device may have
the same link-local address.
• Site-local addresses, which are used in internal or private networks. These addresses are
analogous to the IPv4 private addresses 10.x.x.x and 192.168.x.x.
• The Loopback address, consisting of ::1, which is the equivalent of the IPv4 loopback
address, and allows a host to send packets to itself.
• The Unspecified address, consisting of ::, which is the equivalent of the IPv4 unspecified
address, and is used as a source address by hosts during the auto-configuration process.
Multicast Addresses
IPv6 multicast addresses are also used to provide equivalent functionality to broadcast addresses
in IPv4. Broadcast addresses are not supported in IPv6. A multicast address identifies a group of
interfaces, and packets are sent to all interfaces in that group.
Among the special addresses which have been defined are addresses which allow multicasting
to:
• All interfaces on a particular host (FF01::1)
• All nodes on a local network (FF01::2)
• All routers on the local link (FF02::2)
• All routers on the local site (FF05::2).
Anycast Addresses
An anycast address is a unicast address that is attached to more than one interface. If a packet is
sent to an anycast address it will be delivered to the nearest interface with that address, with the
definition of “nearest” depending on the protocol used for routing. If the protocol is RIPv6, the
nearest interface will be the one which is the shortest number of hops away.
Anycast addresses can only be assigned to routers, and packets cannot originate from an anycast
address. A router must be configured to know if it is using an anycast address, because the
address format cannot be distinguished from that of a unicast address.
Only one anycast address has been predefined, the subnet-router addresses. The subnet-router
address is used to send messages to the nearest router on a subnet, and consists of the subnet’s
prefix followed by zeros.

Adri Jovin J.J.
Page | 67
IPv6 Headers
The basic unit of data sent through an internet is called a packet in IPv6. A packet consists of a
header followed by the data (Figure 14-1 on page 14-6, Table 14-1 on page 14-6). The header
contains the information necessary to move the packet across the internet. It must be able to cope
with missing and duplicated packets as well as possible fragmentation (and reassembly) of the
original packet.
The IPv6 header is similar to the shorter IPv4 header, which is described in Figure 13-1 and
Table 13-1 on page 13-6 of Chapter 13, Internet Protocol (IP). Although the IPv6 header is only
twice as long as the IPv4 header (40 bytes instead of 20 bytes), it contains four times the address
space size (128 bits instead of 32 bits).
The Basic IPv6 Header
The IPv6 header no longer contains the header length, identification, flags, fragment offset and
header checksum fields. Some of these options have been placed in extension headers. The Time
To Live field has been replaced with a hop limit, and the IPv4 Type of Service field is now
replaced with a Traffic Class field.
Figure 14-1: The IPv6 packet.

Adri Jovin J.J.
Page | 68
Table 14-1: General description of fields in an IPv6 packet header .
The IPv4 header fields that have been changed in IPv6 are:
• Type of Service: The type of service that a connection should receive is indicated in
IPv6 by the Flow Label field in the IPv6 header.
• Fragmentation information (the Identification field, the Flags field and the
Fragment Offset field): In most cases fragmentation will not occur in IPv6, and if it
does, packets will only be fragmented at their source, not en route. Therefore, the
fragmentation information is now contained in an extension header, to reduce the size of
the basic IPv6 header.
• Header Checksum: This option has not been provided in IPv6. This is partly because
transport protocols implement checksums and partly because of the availability of the
IPsec authentication header (AH) in IPv6.
• Options: Extension headers now handle all the optional values associated with IPv6
packets. The biggest advantage of this scheme is that the size of the basic IP header is a
constant.

Adri Jovin J.J.
Page | 69
Extension Headers
In IPv6, many of the less commonly used fields in the IPv4 header (or their equivalents) have
become extension headers, which are placed after the basic IPv6 header. The length of each
header must be a multiple of 8 bytes.
In IPv6, many of the less commonly used fields in the IPv4 header (or their equivalents) have
become extension headers, which are placed after the basic IPv6 header. The length of each
header must be a multiple of 8 bytes.
IPv6 nodes that originate packets are required to place extension headers in a specific order:
1. The basic IPv6 header, which must come immediately before the extension headers.
2. The Hop-by-Hop header, which specifies options that must be examined by every node in
the routing path.
3. A Destination Options header, to specify options to be processed by the first destination
or final destination. The destination options header is the only extension header that may
be present more than once in the IPv6 packet.
4. The Routing header, which allows a static path to be specified for the packet, if the
dynamically-determined path is undesirable.
5. The Fragment header, which indicates that the source node has fragmented the packet,
and contains information about the fragmentation.
6. The Authentication header (AH), which verifies the integrity of the packet and its
headers.
7. The Encapsulating Security Payload (ESP) header, which encrypts a packet and verifies
the integrity of its contents.
8. The Upper Layer Protocol header, which indicates which protocol a higher layer (such as
the transport layer) is to process the packet with (for example, TCP).
IPv6 Routing
Routing in IPv6 is almost identical to IPv4 routing under CIDR, except that the addresses are
128-bit IPv6 addresses instead of 32-bit IPv4 addresses.
Routing Information Protocol, RIPv6
RIP is a simple distance vector protocol which defines networks based on how many hops they
are from the router. Once a network is more than 15 hops away (one hop is one link) it is not
included in the routing table.
Extensions to RIPv2 to support IPv6 are:

Adri Jovin J.J.
Page | 70
• The address field of a routing entry is expanded to 128 bits to allow IPv6 prefixes
• The 32-bit RIPv2 subnet mask field is replaced by an 8-bit prefix length field
• Authentication is removed in RIPv6
• The size of a routing packet is no longer arbitrarily limited
• RIPv6 is able to specify the next hop, instead of simply allowing the recipient of the
update to set the next hop to the sender of the update.
In RIPv6, each router uses a routing table to keep track of every destination that is reachable
throughout the system. Each entry in the routing table contains:
• The IPv6 prefix of the destination
• A metric, which represents the total cost of getting a packet from the router to that
destination
• The IPv6 address of the next router along the path to the destination
• A flag to indicate that information about the route has changed recently
• Various timers associated with the route.
IPv6 Filtering
With the increase in connections to the Internet, and the interconnection of networks from
different organisations, filtering of data packets is an important mechanism in ensuring that only
legitimate connections are allowed. Security can never be perfect while connections to other
networks exist, but filters allow network managers to manage the permissible free access, while
restricting users who do not have permission.
Resource Reservation
RSVP
The protocol that has been devised to establish a reservation in the network for particular flow
classes is called the Resource ReserVation Protocol, or RSVP [Zhang, 94]. It might be more
accurate to describe it as a dual function protocol, that serves to install knowledge about classes
of traffic in the network (a filter specification) as well as what particular type or quality of
service that these flows will need (a flow specification). This separation is important, as the filter
specification can be re-used in a number of ways. The simplest way that the filter specification is
reusable is the fact that it can refer to a flow received by an application, rather than to one that is
sent. This means that a sender can send video at a rat convenient to them, but that receivers can
select the sub-band rates that are most convenient to each of them. This receiver based
reservation is quite different from the telephony model of how to do things, and fits well with the
multicast model described in some detail in the next chapter - it is a bit like the way people can

Adri Jovin J.J.
Page | 71
choose what size TV screen they have independent of the TV transmitted signal (or choose
whether the TV is color, or black & white, or has mono or stereo audio), The second way that the
filter generalizes the idea of a reservation is that it can include a wild card, so that it can refer to
groups of sources. For example, in an audio conference, there is no necessity to set aside
resources for all the audio feeds at once, since humans are typically capable of organizing
themselves in a conversation so that there is only one (or one and a bit!) person speaking at any
one time. Hence, sources will ask for reservations that are only marginally more than a unicast
audio reservation for a multi-way audio conference. Flow Specifications are cast in terms of the
class of service, combined with the quantitative parameters as needed. For example, a mean rate
combined with a burstiness (also known as Leaky Bucket parameters by analogy with a water
bucket with a given volume and size of leak) suffice to characterize many applications. Future
versions of the Internet Protocol will incorporate tags (known as Flow Identifiers) which may
make the classification of packets by newer routers a more efficient task. Again, the Flow
identifier has been designed with possible future bearer networks such as ATM in mind. Thus it
is 24 bits in size, just as the VCI/VPI field is in ATM used for roughly the same function.
RSVP Status: IETF Proposed Standard RFC 2205
ISSLL Status: IETF Work in Progress.
IPv6 Status: IETF Last Call
Table 2.4: Integrated Services Internet function Component Status
It is worth noting that RSVP is not the only IP reservation protocol that has been designed for
this purpose. Others include ST-II[#!delgrossi!#] and ST-II+[#!rsvpvstii!#] which incidentally
contain some interesting architectural differences to RSVP such as the use of hard-state and
sender-initiated reservations rather than soft-state and receiver-initiated reservations as in RSVP.
With hard-state the network is responsible for reliably maintaining router state whereas with
soft- state the responsibility is passed to the end-systems which must generate periodic refreshes
to prevent state timeout.
An earlier Internet signaling protocol, ST-II+ permits both sender and receiver-initiated
reservations, ST-II permits sender-initiated reservations only. For further discussion of
alternatives we refer the interested reader to Mitzel[#!rsvpvstii!#] However in this chapter and
book the only reservation protocol we consider in detail is RSVP since currently this has the
most industry support.
RSVP is a novel signaling protocol in at least three ways:

Adri Jovin J.J.
Page | 72
1. It accommodates multicast, not just point-to-multipoint (one-to-many) reservations. To
this end, the receiver driven request model permits heterogeneity, in principle, and the
filter mechanism allows for calls that reserve resources efficiently for the aggregate
traffic flow (e.g. for audio conferencing).
2. It uses soft state, which means that it is tolerant of temporary loss of function without
entailing fate-sharing between the end systems and the network nodes. This means that
QoS routing can be deployed separately (in more than one way!).
3. RSVP is quite straightforward in packet format and operations, and so is relatively low
cost in terms of implementation in end systems and routers. One thing that RSVP is not is
a routing protocol. RSVP does not support QoS-dependent routing itself (in other words,
such routing is independent of RSVP, and could precede or follow reservations).
RSVP identifies a communication session by the combination of destination address, transport
layer protocol type and destination port number. It is important to note that each RSVP operation
only applies to packets of a particular session and as such every RSVP message must include
details of the session to which it applies.
Although RSVP is applicable to both unicast and multicast sessions we concentrate here on the
more complicated multicast case.
Figure 2.3: Direction of RSVP messages
RSVP is not a routing protocol, it is a signaling protocol; it is merely used to reserve resources
along the existing route set up by whichever underlying routing protocol is in place. Figure 2.3
shows an example of RSVP for a multicast Session involving one sender, S1 and three receivers,
RCV1 - RCV3.
The primary messages used by RSVP are the Path message which originates from the traffic
sender and the Resv message which originates from the traffic receivers. The primary roles of the

Adri Jovin J.J.
Page | 73
Path message are firstly, to install reverse routing state in each router along the path and
secondly, to provide receivers with information about the characteristics of the sender traffic and
end-to-end path so that they can make appropriate reservation requests. The primary role of the
Resv message is to carry reservation requests to the routers along the distribution tree between
receivers and senders. Returning now to Figure 2.3, as soon as S1 has data to send it begins
periodically forwarding RSVP Path messages to the next hop, R1 down the distribution tree.
RSVP messages can be transported raw within IP datagrams using protocol number 46 although
hosts without this raw I/O capability may first encapsulate the RSVP messages within a UDP
header.
Reservation Styles and Merging
Associated with each reservation made at a router's interface is a Filterspec describing the
packets to which the reservation applies along with an effective Flowspec. Both the Filterspec
and effective Flowspec are obtained from a merging process applied to selected Resv messages
arriving on the routers interface. The rules for merging are dependent upon the reservation
``Style'' of each Resv message as described below. In addition the router calculates the Filterspec
and Flowspec of Resv messages to be sent to the previous hop(s) upstream by applying Style-
dependent merging of stored reservation state. Any changes to stored reservation state that result
in changes to the Resv messages to be sent upstream will cause an updated Resv message to be
sent upstream immediately. Otherwise Resv messages are created based on stored reservation
state and sent upstream periodically. As for path state all reservation state is stored in routers
using soft-state and consequently relies on periodic refreshes via Resv messages to prevent state
timeout. The periodic Resv message is necessary and sufficient to prevent reservation state
timeout. Of course, if a router crashes, a Path message is necessary after reboot so that the Resv
can rendezvous with it.
In addition just as a PathTear message exists to explicitly tear down path state, a ResvTear
message exists to explicitly tear down reservation state.
Currently three reservation Styles are permissible as described below and illustrated in Figures
2.4 to 2.8 where the convention Style(FilterspecFlowspec) is used to summarise the requests
made by the Resv messages. It should be noted that the merging processes described below apply
only to packets of the same Session(This is true of any RSVP process). Also merging can only
occur between messages with the same reservation style. Details of the reservation styles are as
follows where it is assumed that each interface I in Figures 2.4 to 2.6 is routable to each of the
router's other interfaces.

Adri Jovin J.J.
Page | 74
Fixed Filter (FF)
(distinct reservation and explicit sender selection)
The Filterspec of each FF reservation installed at an interface consists of a single sender only.
The effective Flowspec of the reservation installed is the maximum of all FF reservation requests
received on that interface for that particular sender. In cases where the interface connects to a
shared medium LAN Resv messages from multiple next hops may be received.
The Flowspec of the FF Resv message unicast to the previous hop of a particular sender is
given by the maximum Flowspec of all reservations installed in the router for that particular
sender.
Figure 2.4: Fixed Filter Reservation Example.
Wildcard Filter (WF)
(shared reservation and wildcard sender selection)
The Filterspec of each WF reservation installed at an interface is wildcard and matches on any
sender from upstream. The effective Flowspec installed is the maximum from all WF reservation
requests received on that particular interface. The Flowspec of each WF Resv message unicast to
a previous hop upstream is given by the maximum Flowspec of all WF reservations installed in
the router. More strictly speaking, only WF reservations whose ``Scope'' applies to the interface
out of which the Resv message is sent are considered for this second merging process. Scope
details are required for WF reservations on non-shared trees to prevent looping. Further details
can be found in [#!rsvpv1!#].
Figure 2.5: Wildcard Filter Reservation Example.

Adri Jovin J.J.
Page | 75
Shared Explicit (SE)
(shared reservation and explicit sender selection)
The Filterspec of each SE reservation installed at an interface contains a specific set of senders
from upstream and is obtained by taking the union of the individual Filterspecs from each SE
reservation request received on that interface. The effective Flowspec installed is the maximum
from all SE reservation requests received on that particular interface. The Filterspec of a SE Resv
message unicast out of an interface to a previous hop upstream is the union of all senders whose
previous hop is via that interface and who are contained in the Filterspec of at least one SE
reservation in the router. Likewise the Flowspec of this SE Resv message is given by the
maximum Flowspec of all SE reservations whose Filterspecs contain at least one sender whose
previous hop is via that interface.
Figure 2.6: Shared Explicit Reservation Example.
SE and WF styles are useful for conferencing applications where only one sender is likely to be
active at once in which case reservation requests for say twice the sender bandwidth could be
reserved in order to allow an amount of over-speaking. Although RSVP is unaware of which
service(Controlled-Load or Guaranteed) reservations refer to, RSVP is able to identify those
points in the distribution tree that require reshaping in the event that the reservations are for
Guaranteed Service as described in section 2.8.2. Consequently at all such points RSVP informs
the traffic control mechanisms within the appropriate router accordingly although such action
will only result in reshaping if the reservation is actually for Guaranteed Service.
Path Messages
Each Path message includes the following information:
• Phop, the address of the last RSVP-capable node to forward this Path message. This
address is updated at every RSVP-capable router along the path.

Adri Jovin J.J.
Page | 76
• The Sender Template, a filter specification identifying the sender. It contains the IP
address of the sender and optionally the sender port(in the case of IPv6 a flow label may
be used in place of the sender port).
• The Sender Tspec defining the sender traffic characteristics.
• An optional Adspec containing OPWA information(see sections 2.9.4 and 2.9.5) which is
updated at every RSVP-capable router along the path to attain end-to-end significance
before being presented to receivers to enable them to calculate the level of resources that
must be reserved to obtain a given end-to-end QOS.
Processing and Propagation of Path Messages
Each intermediate RSVP-capable router along the distribution tree intercepts Path messages and
checks them for validity. If an error is detected then the router will drop the Path message and
send a PathErr message upstream to inform the sender who can then take appropriate action.
Assuming the Path message is valid the router does the following: Update the path state entry for
the sender identified by the Sender Template. If no path state exists then create it. Path state
includes the Sender Tspec, the address, Phop of the previous hop upstream router and optionally
an Adspec. The Phop address needs to be stored in order to route Resv messages in the reverse
direction up the tree. The Sender Tspec provides a ceiling to clip any inadvertently over-
specified Tspecs subsequently received in Resv messages Set cleanup timer equal to cleanup
timeout interval and restart timer. Associated with each path state entry is a cleanup timer, the
expiration of which triggers deletion of the path state. Expiration of the timer will be prevented if
a Path message for the entry is received at least once every cleanup timeout interval. This is the
so-called RSVP soft state mechanism and ensures that state automatically times out if routing
changes while subsequent Path messages install state along the new routing path. In this way the
use of soft-state rather than hard- state helps to maintain much of the robustness of the initial
Internet design concepts whereby all flow-related state was restricted to the end
systems[#!clark88!#]
The router is also responsible for generating Path messages based on the stored path state and
forwarding them down the routing tree making sure that for each outgoing interface the
Adspec(see section 2.9.4) and Phop objects are updated accordingly. Path messages will be
generated and forwarded whenever RSVP detects any changes to stored path state or is informed
by the underlying routing protocol of a change in the set of outgoing interfaces in the data
forwarding path. Otherwise, a Path message for each specific path state entry is created and
forwarded every refresh period timeout interval in order to refresh downstream path state. The

Adri Jovin J.J.
Page | 77
refresh period timeout interval is several times smaller than the cleanup timeout interval so that
occasional lost Path messages can be tolerated without triggering unnecessary deletion of path
state.
However it is still a good idea that a minimum network bandwidth and router processing
resources be configured for RSVP messages to protect them from congestion losses.
Although all path state would eventually timeout in the absence of any refreshes via Path
messages, RSVP includes an additional message, PathTear to expedite the process. PathTear
messages travel across the same path as Path messages and are used to explicitly tear down path
state. PathTear messages are generated whenever a path state entry is deleted and so a PathTear
message generated by a sender will result in deletion of all downstream path state for that sender.
Typically, senders do this as soon as they leave the communications session. Also, deletion of
any path state entry triggers deletion of any dependent reservation state.
Adspec
The Adspec is an optional object descriptor that the sender may include in its generated Path
messages in order to advertise to receivers the characteristics of the end to end communications
path. This information can be used by receivers to determine the level of reservation required in
order to achieve their desired end to end QOS. The
Adspec consists of a message header, a Default General Parameters part and at least one of the
following; Guaranteed Service part, Controlled-Load Service part. Omission of either the
Guaranteed or Controlled-Load Service part is an indication to receivers that the omitted service
is not available. This feature can be used in a multicast Session to force all receivers to select the
same service. At present RSVP does not accommodate heterogeneity of services between
receivers within a given multicast Session, although, within the same service model, the
parameters may differ for receivers in the same session, so the core objective of supporting
heterogeneity is mainly met.
Default General Part
The Default General Parameters part includes the following fields which are updated at each
RSVP-capable router along the path in order to present end-to-end values to the receivers.
• Minimum Path Latency (summation of individual link latencies). This parameter
represents the end-to-end latency in the absence of any queuing delay. In the case of

Adri Jovin J.J.
Page | 78
Guaranteed Service, receivers can add this value to the bounded end-to-end queuing
delay to obtain the overall bounded end-to-end delay.
• Path Bandwidth (minimum of individual link bandwidths along the path)
• Global Break bit - This bit is cleared when the Adspec is created by the sender.
Encountering any routers that do not support RSVP will result in this bit being set to one
in order to inform the receiver that the Adspec may be invalid.
• Integrated Services(IS) Hop count - incremented by one at every RSVP/IS-capable router
along the path.
• PathMTU - Path Maximum Transmission Unit (minimum of MTUs of individual links
along the path).
Correct functioning of IETF Integrated Services requires that packets of a data flow are never
fragmented. It might be possible to devise a scheme to support QoS for fragmented traffic, but
the key problem of how loss of fragment results in loss of overall datagram is hard to work
around!
This also means that the value of M in the Tspec of a reservation request must never exceed the
MTU of any link to which the reservation request applies to. MTU Discovery can be employed
to avoid this.
A receiver can ensure that this requirement is met by setting the value of M in the Tspec of it's
reservation request to the minimum of the PathMTU values received in ``relevant'' Path
messages. The value of M in each generated reservation request may be further reduced on the
way to each sender if merging of Resv messages occurs(see section 2.9.2). The minimum value
of M from the Tspec of each Resv message received by the sender should then be used by the
sending application as the upper limit on the size of packets to receive special QOS. In this way
fragmentation of these packets will never occur. In cases where the last hop to a sender is a
shared medium LAN the sender may receive Resv messages across the same interface from
multiple next hop routers.
The specification [#!rsvpandintserv!#] recommends that the value of M in the Sender Tspec,
which has played no part in the above MTU negotiation process, should be set equal to the
maximum packet size that the sender is capable of generating rather than what it is currently
sending.
Guaranteed Service Part
The Guaranteed Service part of the Adspec includes the following fields which are updated at
each RSVP-capable router along the path in order to present end-to-end values to the receivers.

Adri Jovin J.J.
Page | 79
• Ctot - end to end composed (as explained above, from the sum of each router/hop's
estimate of the error) value for C.
• Dtot - end to end composed value for D.
• CSum - composed value for C since last reshaping point.
• DSum - composed value for D since last reshaping point (CSum and Dsum values are
used by reshaping processes at certain points along the distribution tree).
• Guaranteed Service Break Bit - This bit is cleared when the Adspec is created by the
sender. Encountering any routers that do support RSVP/IS but do not support Guaranteed
Service will result in this bit being set to one in order to inform the receiver that the
Adspec may be invalid and the service cannot be guaranteed.
• Guaranteed Service General Parameters Headers/Values - These are optional but if any
are included then each one overrides the corresponding value given in the Default
General Parameters fragment as far as a receiver wishing to make a Guaranteed Service
reservation is concerned. These override parameters could for example be added by
routers along the path that have certain service-specific requirements. For example a
router may have been configured by network management so that Guaranteed Service
reservations can only take up a certain amount, Bgs of the outgoing link bandwidth.
Consequently if the Default Path bandwidth value in the Adspec to be sent out of this
interface is greater than Bgs then a Guaranteed Service Specific Path bandwidth header
and value equal to Bgs may be included in the Adspec. As for Default General
Parameters, any Service- Specific General Parameters must be updated at each RSVP
hop.
Controlled Load part
The Controlled-Load Service part of the Adspec includes the following fields which are updated
at each RSVP-capable router along the path in order to present end-to-end values to the
receivers.
• Controlled-Load Service Break Bit - This bit is cleared when the Adspec is created by the
sender. Encountering any routers that do support RSVP/IS but do em not support
Controlled-Load will result in this bit being set to one in order to inform the receiver that
the Adspec may be invalid and the service cannot be guaranteed.
• Controlled-Load Service General Parameters Headers/Values - As for the Guaranteed
Service fragment, override Service-Specific General Parameters may be added to the
Controlled-Load Service fragment.

Adri Jovin J.J.
Page | 80
Making a Reservation using One Pass with Advertising (OPWA)
One Path With Advertising(OPWA), refers to the reservation model for the case where the
sender includes an Adspec in its Path messages to enable the receiver to determine the end-to-
end service that will result from a given reservation request. If the sender omits the Adspec from
its Path messages then the reservation model is referred to simply as One Pass in which case
there is no easy way for the receiver to determine the resulting end-to-end service. The objective
of this aspect of the RSVP reservation model, both for One Pass and One Pass with Advertising
is to minimise the latency (in terms of number of handshakes between senders and recipients)
before a reservation is in place. Here we consider the OPWA case. Let us assume that the sender
omits the Controlled-Load Service data fragment from the Adspec thereby restricting each
receiver to reservation of Guaranteed Service only. Upon receiving Path messages the receiver
extracts the following parameters from the Sender Tspec: r, b, p, m. In addition the following are
extracted from the Adspec: Minimum Path Latency, Ctot, Dtot, PathMTU, Path Bandwidth. In a
way similar to incremental route calculation, OPWA permits incremental accumulation of the
delay for a reservation. The required bound on end-to-end queuing delay, Qdelreq is now
calculated by subtracting the Minimum Path Latency from the value of end-to-end delay required
by the receiver’s application. Typically, the receiver would then perform an initial check by
evaluating equation (2) for R equal to the peak rate, p. If the resultant delay was greater than or
equal to Qdelreq then equation (2) would be used for calculation of the minimum value of R
necessary to satisfy Qdelreq. Otherwise equation (1) would be used for this purpose. This
minimum value of R is then obtained by inserting Qdelreq into either equation (1) or (2) along
with M(given by PathMTU), Ctot, Dtot, r, b, p, as appropriate. If the obtained value of R exceeds
the Path Bandwidth value as obtained from the Adspec of the received Path message then it must
be reduced accordingly. The receiver can now create a reservation specification, Rspec
comprising firstly the calculated value, R of bandwidth to be reserved in each router, and
secondly a Slack Term that is initialised to zero In some cases even with R set to the minimum
permissible value of r the resultant end-to-end queuing delay as given by eqs (1) and (2) will still
be less than Qdelreq in which case the difference can be represented in a non-zero slack term. In
addition there are other scenarios explained in section 2.9.6 in which the slack term may not be
initialised to zero. The Rspec can now be used in the creation of a Resv message which also
includes the following:
• An indication of the reservation style which can be FF, SE or WF.
• A filter specification, Filterspec (omitted for the case of WF reservation style). This is
used to identify the sender(s) and the format is identical to that of the Sender Template in
a Path message.

Adri Jovin J.J.
Page | 81
• A flow specification, Flowspec comprising the Rspec and a traffic specification, Tspec.
Tspec is usually set equal to the Sender Tspec except M will be given by PathMTU
obtained from the received Adspec.
• Optionally a reservation confirm object, ResvConf containing the IP address of the
receiver. If present this object indicates that the node accepting this reservation request at
which propagation of the message up the distribution tree finishes should return a
ResvConf message to the receiver to indicate that there is a high probability that the end-
to-end reservation has been successfully installed.
In practice there are certain scenarios in which a ResvConf message might be received by a
receiver only for the request to be rejected shortly afterwards.
The Resv message is now sent to the previous hop upstream as obtained from the stored path
state. Upon reaching the next upstream router the Resv messages can be merged with other Resv
messages arriving on the same interface according to certain rules as described later to obtain an
effective Flowspec and Filterspec. The following action is then taken.
The effective Flowspec is passed to the traffic control module within the router which applies
both admission control and policy control to determine whether the reservation can be accepted.
Admission control is concerned solely about whether enough capacity exists to satisfy the
request while policy control also takes into account any additional factors that need to be
considered (e.g. certain policies may limit a users reserved bandwidth even if spare bandwidth
exists). If the reservation attempt is denied then any existing reservations are left unaltered and
the router must send a ResvErr message downstream. If the reservation request is accepted then
reservation state is set up in accordance with the effective Flowspec and Filterspec. In accepting
the request it may be permissible to alter the Rspec associated with the reservation from (Rin,
Sin) to (Rout, Sout) in accordance with the rules described earlier
The resultant reservation may then be merged with other reservations in accordance with the
rules in section 2.9 to obtain a new Resv message that is sent to the next router upstream, the
address of which is obtained from the stored path state.
Slack term
When a receiver generates an Rspec for a Resv message to be sent for a Guaranteed Service
reservation request it must include a Slack Term, S(ms) as well as the amount of bandwidth, R to
be installed in each router along the path. S represents the amount by which the end-to-end delay
bound will be below the end-to-end delay required by the application assuming each router along

Adri Jovin J.J.
Page | 82
the path reserves R bandwidth according to the Guaranteed Service fluid approximation.
Inclusion of a non-zero Slack Term offers the individual routers greater flexibility in making
their local reservations. In certain circumstances this greater flexibility could increase the chance
of an end-to- end reservation being successful. Some routers have deadline based schedulers that
decouple rate and delay guarantees. Such a scheduler may sometimes be unable to meet it's
deadline requirement for Guaranteed Service in which case it might still be able to accept the
reservation providing the Slack Term is at least as large as the excess delay. The excess delay
would then be subtracted from the Slack Term before unicasting the Resv message to the
previous hop upstream. Similarly a rate based scheduler might be able to admit a reservation
request by reserving less than the requested bandwidth and unicasting the reduced reservation
request to a previous hop upstream provided it could extract enough slack. Any router using
available slack to reduce it's reservation must conform to the rules in equation (3) to ensure that
the end-to-end delay bound remains satisfied. where:
• Ctoti is the cumulative sum of the error terms, C for all the routers that are upstream of,
and including the current element, i.
• (Rin, Sin) is the reservation request received by router, i.
• (Rout, Sout) is the modified reservation request unicast to the previous hop router
upstream.
An example of how intelligent use of the Slack Term can increase the probability of an end-to-
end reservation request being accepted is illustrated in Figures 2.7 and 2.8.
Suppose the token bucket rate (as defined earlier) of the data to be sent is 1.5Mb/s and the
receiver has calculated from the Tspec and Adspec parameters in received Path messages that the
desired end-to-end delay can be achieved by a reservation of (R=2.5Mb/s, S=0) which is then
requested in Figure 2.7. However because R3 only has 2Mb/s of unused bandwidth and there is
no slack available the reservation is denied. In Figure 2.8 the reservation is increased to
R=3Mb/s and the amount by which such a reservation would be within the required delay bound
is put in the Slack Term(S>0). R5 and R6 reserve the requested 3Mb/s. R3 can only reserve a
value of 2Mb/s which if used as the new reservation value in the propagated Resv message will
cause an increase in the end to end delay bound. R3 can calculate this increase, di and if it is less
than the value of the Slack Term, S1 in the received Resv message then the request can be
accepted and a reservation of 2Mb/s installed in R3. R3 will then set the Rspec in the Resv
message to (R=2Mb/s, S2=S1-di) before unicasting it to the next hop upstream which results in
R2 and R1 also reserving 2Mb/s.The end-to-end delay bound of the reserved path is now no
greater than for a reservation of 2.5Mb/s in every router if that were possible.

Adri Jovin J.J.
Page | 83
Figure 2.7: R1=2.5Mb/s, S1=0. Reservation request denied.
Figure 2.8: R1=3Mb/s, S1>0, R2=2Mb/s, S2<S1. Reservation accepted.
Buffer (Queue) Management
Scheduling is often linked to queue (buffer) management schemes. They are used, for example,
to establish the dropping strategy when the buffer is full. There are three possibilities:
1. Tail drop, which discharges the last arrived packets;
2. Front drop, which eliminates the first packet in the queue;
3. Random drop, which discharges a randomly selected packet within the queue.
There are also dynamic schemes to drop packets before saturation. Random Early Detection –
RED – is an algorithm really implemented in commercial routers: it detects incipient congestion
by computing the average queue size. If the average queue size exceeds a fixed threshold, RED
drops or marks each arriving packet with a certain probability, where the exact probability is a
function of the average queue size. The scheme is thought to help transport layer protocols such
as TCP, to avoid entering severe congestion. There are some modifications of the original
scheme. Weighted Random Early Detection (WRED) drops packets selectively based on their
priority (typically IP Precedence) by following the rule that packets with a higher IP precedence
are less likely to be dropped than packets with a lower precedence. Flow-based WRED is a

Adri Jovin J.J.
Page | 84
particular feature of WRED, which forces it to afford greater fairness to all flows depending on
how packets are dropped.
Traffic Shaping
Before the network can make QoS guarantees, it must know what traffic is being guaranteed. In
the telephone network, this characterization is simple. For example, a voice call (in
uncompressed format) needs 64 kbps and consists of one 8-bit sample every 125 μsec. However,
traffic in data networks is bursty. It typically arrives at nonuniform rates as the traffic rate varies
(e.g., videoconferencing with compression), users interact with applications (e.g., browsing a
new Web page), and computers switch between tasks. Bursts of traffic are more difficult to
handle than constant-rate traffic because they can fill buffers and cause packets to be lost.
Traffic shaping is a technique for regulating the average rate and burstiness of a flow of data that
enters the network. The goal is to allow applications to transmit a wide variety of traffic that suits
their needs, including some bursts, yet have a simple and useful way to describe the possible
traffic patterns to the network. When a flow is set up, the user and the network (i.e., the customer
and the provider) agree on a certain traffic pattern (i.e., shape) for that flow. In effect, the
customer says to the provider ‘‘My transmission pattern will look like this; can you handle it?’’
Sometimes this agreement is called an SLA (Service Level Agreement), especially when it is
made over aggregate flows and long periods of time, such as all of the traffic for a given
customer. As long as the customer fulfills her part of the bargain and only sends packets
according to the agreed-on contract, the provider promises to deliver them all in a timely fashion.
Traffic shaping reduces congestion and thus helps the network live up to its promise. However,
to make it work, there is also the issue of how the provider can tell if the customer is following
the agreement and what to do if the customer is not. Packets in excess of the agreed pattern
might be dropped by the network, or they might be marked as having lower priority. Monitoring
a traffic flow is called traffic policing.
Shaping and policing are not so important for peer-to-peer and other transfers that will consume
any and all available bandwidth, but they are of great importance for real-time data, such as
audio and video connections, which have stringent quality-of-service requirements.
Leaky and Token Buckets
We have already seen one way to limit the amount of data an application sends: the sliding
window, which uses one parameter to limit how much data is in transit at any given time, which
indirectly limits the rate. Now we will look at a more general way to characterize traffic, with the
leaky bucket and token bucket algorithms. The formulations are slightly different but give an
equivalent result.

Adri Jovin J.J.
Page | 85
Try to imagine a bucket with a small hole in the bottom, as illustrated in Fig. 5-28(b). No matter
the rate at which water enters the bucket, the outflow is at a constant rate, R, when there is any
water in the bucket and zero when the bucket is empty. Also, once the bucket is full to capacity
B, any additional water entering it spills over the sides and is lost.
Figure 5-28. (a) Shaping packets. (b) A leaky bucket. (c) A token bucket.
This bucket can be used to shape or police packets entering the network, as shown in Fig. 5-
28(a). Conceptually, each host is connected to the network by an interface containing a leaky
bucket. To send a packet into the network, it must be possible to put more water into the bucket.
If a packet arrives when the bucket is full, the packet must either be queued until enough water
leaks out to hold it or be discarded. The former might happen at a host shaping its traffic for the
network as part of the operating system. The latter might happen in hardware at a provider
network interface that is policing traffic entering the network. This technique was proposed by
Turner (1986) and is called the leaky bucket algorithm.
A different but equivalent formulation is to imagine the network interface as a bucket that is
being filled, as shown in Fig. 5-28(c). The tap is running at rate R and the bucket has a capacity
of B, as before. Now, to send a packet we must be able to take water, or tokens, as the contents
are commonly called, out of the bucket (rather than putting water into the bucket). No more than
a fixed number of tokens, B, can accumulate in the bucket, and if the bucket is empty, we must
wait until more tokens arrive before we can send another packet. This algorithm is called the
token bucket algorithm.
Leaky and token buckets limit the long-term rate of a flow but allow shortterm bursts up to a
maximum regulated length to pass through unaltered and without suffering any artificial delays.
Large bursts will be smoothed by a leaky bucket traffic shaper to reduce congestion in the
network. As an example, imagine that a computer can produce data at up to 1000 Mbps (125
million bytes/sec) and that the first link of the network also runs at this speed. The pattern of

Adri Jovin J.J.
Page | 86
traffic the host generates is shown in Fig. 5-29(a). This pattern is bursty. The average rate over
one second is 200 Mbps, even though the host sends a burst of 16,000 KB at the top speed of
1000 Mbps (for 1/8 of the second).
Figure 5-29. (a) Traffic from a host. Output shaped by a token bucket of rate 200 Mbps and
capacity (b) 9600 KB and (c) 0 KB. Token bucket level for shaping with rate 200 Mbps and
capacity (d) 16,000 KB, (e) 9600 KB, and (f) 0 KB.
Now suppose that the routers can accept data at the top speed only for short intervals, until their
buffers fill up. The buffer size is 9600 KB, smaller than the traffic burst. For long intervals, the
routers work best at rates not exceeding 200 Mbps (say, because this is all the bandwidth given
to the customer). The implication is that if traffic is sent in this pattern, some of it will be
dropped in the network because it does not fit into the buffers at routers. To avoid this packet
loss, we can shape the traffic at the host with a token bucket. If we use a rate, R, of 200 Mbps
and a capacity, B, of 9600 KB, the traffic will fall within what the network can handle. The
output of this token bucket is shown in Fig. 5-29(b). The host can send full throttle at 1000 Mbps
for a short while until it has drained the bucket. Then it has to cut back to 200 Mbps until the
burst has been sent. The effect is to spread out the burst over time because it was too large to
handle all at once. The level of the token bucket is shown in Fig. 5- 29(e). It starts off full and is
depleted by the initial burst. When it reaches zero, new packets can be sent only at the rate at
which the buffer is filling; there can be no more bursts until the bucket has recovered. The bucket
fills when no traffic is being sent and stays flat when traffic is being sent at the fill rate.
We can also shape the traffic to be less bursty. Fig. 5-29(c) shows the output of a token bucket
with R = 200 Mbps and a capacity of 0. This is the extreme case in which the traffic has been
completely smoothed. No bursts are allowed, and the traffic enters the network at a steady rate.
The corresponding bucket level, shown in Fig. 5-29(f), is always empty. Traffic is being queued

Adri Jovin J.J.
Page | 87
on the host for release into the network and there is always a packet waiting to be sent when it is
allowed. Finally, Fig. 5-29(d) shows the bucket level for a token bucket with R = 200 Mbps and
a capacity of B = 16,000 KB. This is the smallest token bucket through which the traffic passes
unaltered. It might be used at a router in the network to police the traffic that the host sends. If
the host is sending traffic that conforms to the token bucket on which it has agreed with the
network, the traffic will fit through that same token bucket run at the router at the edge of the
network. If the host sends at a faster or burstier rate, the token bucket will run out of water. If
this happens, a traffic policer will know that the traffic is not as described. It will then either drop
the excess packets or lower their priority, depending on the design of the network. In our
example, the bucket empties only momentarily, at the end of the initial burst, and then recovers
enough for the next burst. Leaky and token buckets are easy to implement. We will now describe
the operation of a token bucket. Even though we have described water flowing continuously into
and out of the bucket, real implementations must work with discrete quantities. A token bucket is
implemented with a counter for the level of the bucket. The counter is advanced by R /ΔT units
at every clock tick of ΔT seconds. This would be 200 Kbit every 1 msec in our example above.
Every time a unit of traffic is sent into the network, the counter is decremented, and traffic may
be sent until the counter reaches zero.
When the packets are all the same size, the bucket level can just be counted in packets (e.g., 200
Mbit is 20 packets of 1250 bytes). However, often variable-sized packets are being used. In this
case, the bucket level is counted in bytes. If the residual byte count is too low to send a large
packet, the packet must wait until the next tick (or even longer, if the fill rate is small).
Calculating the length of the maximum burst (until the bucket empties) is slightly tricky. It is
longer than just 9600 KB divided by 125 MB/sec because while the burst is being output, more
tokens arrive. If we call the burst length S sec., the maximum output rate M bytes/sec, the token
bucket capacity B bytes, and the token arrival rate R bytes/sec, we can see that an output burst
contains a maximum of B + RS bytes. We also know that the number of bytes in a maximum
speed burst of length S seconds is MS. Hence, we have
B + RS = MS
We can solve this equation to get S = B /(M − R). For our parameters of B = 9600 KB, M = 125
MB/sec, and R = 25 MB/sec, we get a burst time of about 94 msec. A potential problem with the
token bucket algorithm is that it reduces large bursts down to the long-term rate R. It is
frequently desirable to reduce the peak rate, but without going down to the long-term rate (and
also without raising the long-term rate to allow more traffic into the network). One way to get
smoother traffic is to insert a second token bucket after the first one. The rate of the second
bucket should be much higher than the first one. Basically, the first bucket characterizes the
traffic, fixing its average rate but allowing some bursts. The second bucket reduces the peak rate

Adri Jovin J.J.
Page | 88
at which the bursts are sent into the network. For example, if the rate of the second token bucket
is set to be 500 Mbps and the capacity is set to 0, the initial burst will enter the network at a peak
rate of 500 Mbps, which is lower than the 1000 Mbps rate we had previously. Using all of these
buckets can be a bit tricky. When token buckets are used for traffic shaping at hosts, packets are
queued and delayed until the buckets permit them to be sent. When token buckets are used for
traffic policing at routers in the network, the algorithm is simulated to make sure that no more
packets are sent than permitted. Nevertheless, these tools provide ways to shape the network
traffic into more manageable forms to assist in meeting quality-of-service requirements.
Packet Scheduling
Being able to regulate the shape of the offered traffic is a good start. However, to provide a
performance guarantee, we must reserve sufficient resources along the route that the packets take
through the network. To do this, we are assuming that the packets of a flow follow the same
route. Spraying them over routers at random makes it hard to guarantee anything. As a
consequence, something similar to a virtual circuit has to be set up from the source to the
destination, and all the packets that belong to the flow must follow this route. Algorithms that
allocate router resources among the packets of a flow and between competing flows are called
packet scheduling algorithms. Three different kinds of resources can potentially be reserved for
different flows:
1. Bandwidth.
2. Buffer space.
3. CPU cycles.
The first one, bandwidth, is the most obvious. If a flow requires 1 Mbps and the outgoing line
has a capacity of 2 Mbps, trying to direct three flows through that line is not going to work.
Thus, reserving bandwidth means not oversubscribing any output line.
A second resource that is often in short supply is buffer space. When a packet arrives, it is
buffered inside the router until it can be transmitted on the chosen outgoing line. The purpose of
the buffer is to absorb small bursts of traffic as the flows contend with each other. If no buffer is
available, the packet has to be discarded since there is no place to put it. For good quality of
service, some buffers might be reserved for a specific flow so that flow does not have to compete
for buffers with other flows. Up to some maximum value, there will always be a buffer available
when the flow needs one.
Finally, CPU cycles may also be a scarce resource. It takes router CPU time to process a packet,
so a router can process only a certain number of packets per second. While modern routers are
able to process most packets quickly, some kinds of packets require greater CPU processing,

Adri Jovin J.J.
Page | 89
such as the ICMP packets we will describe in Sec. 5.6. Making sure that the CPU is not
overloaded is needed to ensure timely processing of these packets.
Packet scheduling algorithms allocate bandwidth and other router resources by determining
which of the buffered packets to send on the output line next. We already described the most
straightforward scheduler when explaining how routers work. Each router buffers packets in a
queue for each output line until they can be sent, and they are sent in the same order that they
arrived. This algorithm is known as FIFO (First-In First-Out), or equivalently FCFS (First-Come
First-Serve). FIFO routers usually drop newly arriving packets when the queue is full. Since the
newly arrived packet would have been placed at the end of the queue, this behavior is called tail
drop. It is intuitive, and you may be wondering what alternatives exist. In fact, the RED
algorithm we described in Sec. 5.3.5 chose a newly arriving packet to drop at random when the
average queue length grew large. The other scheduling algorithms that we will describe also
create other opportunities for deciding which packet to drop when the buffers are full. FIFO
scheduling is simple to implement, but it is not suited to providing good quality of service
because when there are multiple flows, one flow can easily affect the performance of the other
flows. If the first flow is aggressive and sends large bursts of packets, they will lodge in the
queue. Processing packets in the order of their arrival means that the aggressive sender can hog
most of the capacity of the routers its packets traverse, starving the other flows and reducing
their quality of service. To add insult to injury, the packets of the other flows that do get through
are likely to be delayed because they had to sit in the queue behind many packets from the
aggressive sender. Many packet scheduling algorithms have been devised that provide stronger
isolation between flows and thwart attempts at interference (Bhatti and Crowcroft, 2000). One of
the first ones was the fair queueing algorithm devised by Nagle (1987). The essence of this
algorithm is that routers have separate queues, one for each flow for a given output line. When
the line becomes idle, the router scans the queues round-robin, as shown in Fig. 5-30. It then
takes the first packet on the next queue. In this way, with n hosts competing for the output line,
each host gets to send one out of every n packets. It is fair in the sense that all flows get to send
packets at the same rate. Sending more packets will not improve this rate. Although a start, the
algorithm has a flaw: it gives more bandwidth to hosts that use large packets than to hosts that
use small packets. Demers et al. (1990) suggested an improvement in which the round-robin is
done in such a way as to simulate a byte-by-byte round-robin, instead of a packet-by-packet
round-robin. The trick is to compute a virtual time that is the number of the round at which each
packet would finish being sent. Each round drains a byte from all of the queues that have data to
send. The packets are then sorted in order of their finishing times and sent in that order.
This algorithm and an example of finish times for packets arriving in three flows are illustrated
in Fig. 5-31. If a packet has length L, the round at which it will finish is simply L rounds after the

Adri Jovin J.J.
Page | 90
start time. The start time is either the finish time of the previous packet, or the arrival time of the
packet, if the queue is empty when it arrives.
Figure 5-30. Round-robin fair queueing.
Figure 5-31. (a) Weighted Fair Queueing. (b) Finishing times for the packets.
From the table in Fig. 5-32(b), and looking only at the first two packets in the top two queues,
packets arrive in the order A, B, D, and F. Packet A arrives at round 0 and is 8 bytes long, so its
finish time is round 8. Similarly the finish time for packet B is 11. Packet D arrives while B is
being sent. Its finish time is 9 byte-rounds after it starts when B finishes, or 20. Similarly, the
finish time for F is 16. In the absence of new arrivals, the relative sending order is A, B, F, D,
even though F arrived after D. It is possible that another small packet will arrive on the top flow
and obtain a finish time before D. It will only jump ahead of D if the transmission of that packet
has not started. Fair queueing does not preempt packets that are currently being transmitted.
Because packets are sent in their entirety, fair queueing is only an approximation of the ideal
byte-by-byte scheme. But it is a very good approximation, staying within one packet
transmission of the ideal scheme at all times.
One shortcoming of this algorithm in practice is that it gives all hosts the same priority. In many
situations, it is desirable to give, for example, video servers more bandwidth than, say, file
servers. This is easily possible by giving the video server two or more bytes per round. This
modified algorithm is called WFQ (Weighted Fair Queueing). Letting the number of bytes per
round be the weight of a flow, W, we can now give the formula for computing the finish time:
Fi = max(Ai ,Fi −1)+Li /W

Adri Jovin J.J.
Page | 91
where Ai is the arrival time, Fi is the finish time, and Li is the length of packet i. The bottom
queue of Fig. 5-31(a) has a weight of 2, so its packets are sent more quickly as you can see in the
finish times given in Fig. 5-31(b). Another practical consideration is implementation complexity.
WFQ requires that packets be inserted by their finish time into a sorted queue. With N flows, this
is at best an O(logN) operation per packet, which is difficult to achieve for many flows in high-
speed routers. Shreedhar and Varghese (1995) describe an approximation called deficit round
robin that can be implemented very efficiently, with only O(1) operations per packet. WFQ is
widely used given this approximation.
Other kinds of scheduling algorithms exist, too. A simple example is priority scheduling, in
which each packet is marked with a priority. High-priority packets are always sent before any
low-priority packets that are buffered. Within a priority, packets are sent in FIFO order.
However, priority scheduling has the disadvantage that a burst of high-priority packets can starve
low-priority packets, which may have to wait indefinitely. WFQ often provides a better
alternative. By giving the high-priority queue a large weight, say 3, high-priority packets will
often go through a short line (as relatively few packets should be high priority) yet some fraction
of low priority packets will continue to be sent even when there is high priority traffic. A high
and low priority system is essentially a two-queue WFQ system in which the high priority has
infinite weight.
As a final example of a scheduler, packets might carry timestamps and be sent in timestamp
order. Clark et al. (1992) describe a design in which the timestamp records how far the packet is
behind or ahead of schedule as it is sent through a sequence of routers on the path. Packets that
have been queued behind other packets at a router will tend to be behind schedule, and the
packets that have been serviced first will tend to be ahead of schedule. Sending packets in order
of their timestamps has the beneficial effect of speeding up slow packets while at the same time
slowing down fast packets. The result is that all packets are delivered by the network with a more
consistent delay.
Throughput
In communication networks, such as Ethernet or packet radio, throughput or network throughput
is the average rate of successful message delivery over a communication channel. This data may
be delivered over a physical or logical link, or pass through a certain network node. The
throughput is usually measured in bits per second (bit/s or bps), and sometimes in data packets
per second or data packets per time slot.

Adri Jovin J.J.
Page | 92
The system throughput or aggregate throughput is the sum of the data rates that are delivered to
all terminals in a network.
The throughput can be analyzed mathematically by means of queueing theory, where the load in
packets per time unit is denoted arrival rate λ, and the throughput in packets per time unit is
denoted departure rate μ.
Throughput is essentially synonymous to digital bandwidth consumption.
Jitter Performance
Jitter is the undesired deviation from true periodicity of an assumed periodic signal in electronics
and telecommunications, often in relation to a reference clock source. Jitter may be observed in
characteristics such as the frequency of successive pulses, the signal amplitude, or phase of
periodic signals. Jitter is a significant, and usually undesired, factor in the design of almost all
communications links (e.g., USB, PCI-e, SATA, OC-48). In clock recovery applications it is
called timing jitter.[1]
Jitter can be quantified in the same terms as all time-varying signals, e.g., RMS, or peak-to-peak
displacement. Also like other time-varying signals, jitter can be expressed in terms of spectral
density (frequency content).
Jitter period is the interval between two times of maximum effect (or minimum effect) of a signal
characteristic that varies regularly with time. Jitter frequency, the more commonly quoted figure,
is its inverse. ITU-T G.810 classifies jitter frequencies below 10 Hz as wander and frequencies at
or above 10 Hz as jitter.[2]
Jitter may be caused by electromagnetic interference (EMI) and crosstalk with carriers of other
signals. Jitter can cause a display monitor to flicker, affect the performance of processors in
personal computers, introduce clicks or other undesired effects in audio signals, and loss of
transmitted data between network devices. The amount of tolerable jitter depends on the affected
application.
Sampling jitter
In analog to digital and digital to analog conversion of signals, the sampling is normally assumed
to be periodic with a fixed period - the time between every two samples is the same. If there is
jitter present on the clock signal to the analog-to-digital converter or a digital-to-analog
converter, the time between samples varies and instantaneous signal error arises. The error is

Adri Jovin J.J.
Page | 93
proportional to the slew rate of the desired signal and the absolute value of the clock error.
Various effects such as noise (random jitter), or spectral components (periodic jitter) can come
about depending on the pattern of the jitter in relation to the signal. In some conditions, less than
a nanosecond of jitter can reduce the effective bit resolution of a converter with a Nyquist
frequency of 22 kHz to 14 bits.
This is a consideration in high-frequency signal conversion, or where the clock signal is
especially prone to interference.
Packet jitter in computer networks
In the context of computer networks, the term jitter is often used as a measure of the variability
over time of the packet latency across a network. A network with constant latency has no
variation (or jitter). Packet jitter is expressed as an average of the deviation from the network
mean latency. However, for this use, the term is imprecise. The standards-based term is packet
delay variation (PDV). PDV is an important quality of service factor in assessment of network
performance.
Jitter metrics
For clock jitter, there are three commonly used metrics: absolute jitter, period jitter, and cycle to
cycle jitter.
Absolute jitter is the absolute difference in the position of a clock's edge from where it would
ideally be.
Period jitter (aka cycle jitter) is the difference between any one clock period and the
ideal/average clock period. Accordingly, it can be thought of as the discrete-time derivative of
absolute jitter. Period jitter tends to be important in synchronous circuitry like digital state
machines where the error-free operation of the circuitry is limited by the shortest possible clock
period, and the performance of the circuitry is limited by the average clock period. Hence,
synchronous circuitry benefits from minimizing period jitter, so that the shortest clock period
approaches the average clock period.
Cycle-to-cycle jitter is the difference in length/duration of any two adjacent clock periods.
Accordingly, it can be thought of as the discrete-time derivative of period jitter. It can be
important for some types of clock generation circuitry used in microprocessors and RAM
interfaces.

Adri Jovin J.J.
Page | 94
Since they have different generation mechanisms, different circuit effects, and different
measurement methodology, it is useful to quantify them separately.
In telecommunications, the unit used for the above types of jitter is usually the Unit Interval
(abbreviated UI) which quantifies the jitter in terms of a fraction of the ideal period of a bit. This
unit is useful because it scales with clock frequency and thus allows relatively slow interconnects
such as T1 to be compared to higher-speed internet backbone links such as OC-192. Absolute
units such as picoseconds are more common in microprocessor applications. Units of degrees
and radians are also used.
If jitter has a Gaussian distribution, it is usually quantified using the standard deviation of this
distribution (aka. RMS). Often, jitter distribution is significantly non-Gaussian. This can occur if
the jitter is caused by external sources such as power supply noise. In these cases, peak-to-peak
measurements are more useful. Many efforts have been made to meaningfully quantify
distributions that are neither Gaussian nor have meaningful peaks (which is the case in all real
jitter). All have shortcomings but most tend to be good enough for the purposes of engineering
work. Note that typically, the reference point for jitter is defined such that the mean jitter is 0.
In networking, in particular IP networks such as the Internet, jitter can refer to the variation
(statistical dispersion) in the delay of the packets.
Types
Random jitter
Random Jitter, also called Gaussian jitter, is unpredictable electronic timing noise. Random jitter
typically follows a Gaussian distribution or Normal distribution. It is believed to follow this
pattern because most noise or jitter in an electrical circuit is caused by thermal noise, which has a
Gaussian distribution. Another reason for random jitter to have a distribution like this is due to
the central limit theorem. The central limit theorem states that composite effect of many
uncorrelated noise sources, regardless of the distributions, approaches a Gaussian distribution.
One of the main differences between random and deterministic jitter is that deterministic jitter is
bounded and random jitter is unbounded.
Deterministic jitter
Deterministic jitter is a type of clock timing jitter or data signal jitter that is predictable and
reproducible. The peak-to-peak value of this jitter is bounded, and the bounds can easily be
observed and predicted. Deterministic jitter can either be correlated to the data stream (data-

Adri Jovin J.J.
Page | 95
dependent jitter) or uncorrelated to the data stream (bounded uncorrelated jitter). Examples of
data-dependent jitter are duty-cycle dependent jitter (also known as duty-cycle distortion) and
inter-symbol interference. Deterministic jitter (or DJ) has a known non-Gaussian probability
distribution.
Total jitter
Total jitter (T) is the combination of random jitter (R) and deterministic jitter (D):
T = Dpeak-to-peak + 2× n×Rrms,
in which the value of n is based on the bit error rate (BER) required of the link.
A common bit error rate used in communication standards such as Ethernet is 10−12
.
Mitigation
Anti-jitter circuits
Anti-jitter circuits (AJCs) are a class of electronic circuits designed to reduce the level of jitter in
a regular pulse signal. AJCs operate by re-timing the output pulses so they align more closely to
an idealised pulse signal. They are widely used in clock and data recovery circuits in digital
communications, as well as for data sampling systems such as the analog-to-digital converter and
digital-to-analog converter. Examples of anti-jitter circuits include phase-locked loop and delay-
locked loop. Inside digital to analog converters jitter causes unwanted high-frequency
distortions. In this case it can be suppressed with high fidelity clock signal usage.
Jitter buffers
Jitter buffers or de-jitter buffers are used to counter jitter introduced by queuing in packet
switched networks so that a continuous playout of audio (or video) transmitted over the network
can be ensured. The maximum jitter that can be countered by a de-jitter buffer is equal to the
buffering delay introduced before starting the play-out of the mediastream. In the context of
packet-switched networks, the term packet delay variation is often preferred over jitter.
Some systems use sophisticated delay-optimal de-jitter buffers that are capable of adapting the
buffering delay to changing network jitter characteristics. These are known as adaptive de-jitter
buffers and the adaptation logic is based on the jitter estimates computed from the arrival
characteristics of the media packets. Adaptive de-jittering involves introducing discontinuities in
the media play-out, which may appear offensive to the listener or viewer. Adaptive de-jittering is
usually carried out for audio play-outs that feature a VAD/DTX encoded audio, that allows the
lengths of the silence periods to be adjusted, thus minimizing the perceptual impact of the
adaptation.

Adri Jovin J.J.
Page | 96
Dejitterizer
A dejitterizer is a device that reduces jitter in a digital signal. A dejitterizer usually consists of an
elastic buffer in which the signal is temporarily stored and then retransmitted at a rate based on
the average rate of the incoming signal. A dejitterizer is usually ineffective in dealing with low-
frequency jitter, such as waiting-time jitter.
Filtering
A filter can be designed to minimize the effect of sampling jitter.
----X-----------X----

UNIT III INTERNETWORKING MULTIMEDIA
Adri Jovin J.J.
Page | 97
UNIT III
MULTICAST AND TRANSPORT PROTOCOL
Multicast over shared media network
When there is a requirement to send data to many receivers simultaneously, there are two
traditional methods that can be used: repeated transmission and broadcast. Repeated transmission
may be acceptable if the cost is low enough and delivery can be spread out over time, as with
junk mail or electronic mailing lists. Otherwise a broadcast solution is required. With real-time
multimedia, repeated delivery is feasible, but only at great expense to the sender, who must
invest in large amounts of bandwidth. Similarly, traditional broadcast channels have been very
expensive if they cover significant numbers of recipients or large geographic areas. However, on
the internet, there is an alternative solution. IP Multicast effectively turns the Internet into a
broadcast channel, but one that anyone can send to without having to spend huge amounts of
money on transmitters and government licenses. It provides efficient, timely and global many-to-
many distribution of data, and as such may become the broadcast medium of choice in the future.
The Internet is a datagram network. This means that anyone can send a packet to a destination
without having to pre-establish a path. Of course, the boxes along the way must have either pre-
computed a set of paths, or be pretty fast at calculating one as needed, and typically, the former
approach is used. However, the sending host need not be aware of or participate in the complex
route calculation; nor need it take part in a complex signaling or call setup protocol. It simply
addresses the packet to the right place, and sends it. As discussed briefly in chapters one and two,
this may be a more complex procedure if the sending or receiving systems need more than the
default performance that a path or network might offer, but it is the default model. Adding
multicast to the Internet does not alter the basic model. A sending host can still simply send, but
now there is a new form of address, the multicast or host group address. Unlike unicast
addresses, hosts can dynamically subscribe to multicast addresses and by so doing cause
multicast traffic to be delivered to them. Thus the IP Multicast service model can be summarised:
• Senders send to a multicast address.
• Receivers express an interest in a multicast address.
• Routers conspire to deliver traffic from the senders to the receivers.
Sending multicast traffic is no different from sending unicast traffic except that the destination
address is slightly special. However, to receive multicast traffic, an interested host must tell its
local router that it is interested in a particular multicast group address, which it does using the
Internet Group Management Protocol (IGMP).
Point to multipoint communication is nothing new. We are all used to the idea of broadcast TV
and Radio, where a shared medium (the RF spectrum) is partitioned up amongst users (basically,
transmitter or TV and radio station owners). It is a matter of regulation that there is typically only
one particular unique sender of particular content on a particular frequency, although there are
other parts of the RF spectrum given over to free use for multi-party communication (e.g. police
radio, citizen band radio, etc etc). The Internet multicast model[#!igmp!#] is very similar. The

Adri Jovin J.J.
Page | 98
idea is to convert the mesh wide area network that is the Internet (whether the public Internet, or
a private Enterprise net, or Intranet makes no difference to the model), into a shared resource for
senders to send to multiple participants, or groups. To make this group communication work for
large scale systems (in the sense of a large number of recipients for a particular group, or in the
sense of a large number of senders to a large number of recipients, or in the sense of a large
number of different groups), it is necessary, both for senders, and for the routing functions to
support delivery, to have a system that can be largely independent of the particular recipients at
any one time. In other words, just as a TV or Radio station do not know who is listening when,
an Internet multicast sender does not know who might receive packets it sends. If this sends
alarm bells ringing about security, it shouldn't. A unicast sender has no assurance about who
receives their packets either. Assurances about disclosure (privacy) and authenticity of
sender/recipient are largely separate matters from simple packet delivery models. We discuss
these security matters in some detail in chapter 10.
Basically, the model is an extension of the datagram model, and uses the fact that the datagram is
a self contained communications unit, that not only conveys data from source to destination, but
also conveys the source and destination, in other words, in some senses, datagrams signal their
own path, both with a source and a destination address in every packet. By adding a range of
addresses for sending to a group, and providing independence between the address allocation and
the rights to send to a group, the analogy between RF spectrum and the Internet multicast space
is maintained. Some mechanism, as yet unspecified, is used to choose dynamically which
address to send to. In chapter 7, we will take a look at some protocols for carrying out this
function, but suffice it to say that for now, the idea is that somehow, elsewhere, the address used
for a multicast session or group communication activity is chosen so that it does not clash with
other uses or users, and is advertised. Imagine the equivalent in the RF spectrum. A New TV or
radio station's manager (or multicast session creator) looks in a TV listings magazine, or in a
government or regulatory bodies publications for a list of frequencies (say a bulletin board) in
use, and finds one that is free. We write to the TV listings magazine (the bulletin board) and have
them advertise this new address. We can now safely send, and receivers find out where to
``receive on''. It is very important to note that unlike the RF spectrum, an IP packet to be
multicast carries a unique source identifier, in that such packets are sent with the normal unicast
IP address of the interface of the sending host they were sent from. It is also worth noting that an
address that is being used to signify a group of entities must surely be a logical address (or in
some senses a name) rather than a topological or topographical identifier. We shall see that this
means there must be some service that maps such a logical identifier to a specific set of locations
in the same way that a local unicast address must be mapped (or bound) to a specific location. In
the multicast case, this mapping is distributed. Note also that multicast internet addresses are in
some sense ``host group'' addresses, in that they indicate a set of hosts to deliver to. In the
Internet model , there is a further level of multiplexing, that of transport level ports, and there is
room for some overlap of functionality, since a host may receive packets sent to multiple
multicast addresses on the same port, or multiple ports on the same multicast address. We will
look at this further in chapter five. There are a number of questions this model raises about
address and group management, such as how these addresses are allocated. The area requiring
most change, though, is in the domain of the routing. Somehow the routers must be able to build
a distribution tree from the senders to all the receivers for each multicast group. The senders
don't know who the receivers are (they just send their data), and the receivers don't know who

Adri Jovin J.J.
Page | 99
the senders are (they just ask for traffic destined for the group address), so the routers have to do
something without help from the hosts.
Host Functions
IGMP v1 Status: IETF Standard (RFC 1112)
IGMP v2 Status: IETF Proposed Standard (RFC 2236)
IGMP v3 Status: IETF Work in Progress
As we stated above, host functionality is extended through the use of the IGMP protocol. Hosts,
and routers which we will look at later, must be able to deal with new forms of addresses. When
IP version 4 addressing was first designed, it was divided into classes as shown in table 3.1.
Class A 1.0.0.0 to 126.255.255.255
Class B 128.0.0.0 to 191.255.255.255
Class C 192.0.0.0 to 223.255.255.255
Class D 224.0.0.0 to 239.255.255.255
Table 3.1: Class D addresses are Multicast
Originally class A was intended for large networks, B for middle size networks and C for small
networks. Class D was later allocated for multicast addresses. Since then, classless addressing
has been introduced to solve internet scaling problems, and the rules for class A, B and C no
longer hold, but class D is still reserved for multicast, and so all IPv4 multicast addresses start
with the high order 4 bit nibble: 1110 In other words, from the 232
possible addresses, 228
are
multicast. This means that there can be up to around 270 Million different groups, each with as
many senders as can get unicast addresses! This is many orders of magnitude more than the RF
spectrum allows for typical analog frequency allocations.
For a host to support multicast, the host service interface to IP must be extended in 3 ways:
1. A host must be able to join a group. This means it must be able to re-program its network
level, and possibly, consequentially, the lower levels, to be able to receive packets
addressed to multicast group addresses.
2. An application that has joined a multicast group and then sends to that group must be able
to select whether it wants the host to loop-back the packets it sent so that it receives its
own packets.
3. A host should be able to limit the scope with which multicast messages are sent. The
Internet Protocol contains a time to live field, used originally to limit the lifetime of
packets on the network, both for safety of upper layers, and for prevention of traffic
overload during temporary routing loops. It is used in multicast to limit how ``far'' a
packet can go from the source. We will see below how scoping can interact with routing.
When an application tells the host networking software to join a group, the host software checks
to see if the host is a member of the group. If not, it makes a note of the fact, and sends out an

Adri Jovin J.J.
Page | 100
IGMP join message. It also maps the IP address to a lower level address and reprograms its
network interface to accept packets sent to that address. There is a refinement here: a host can
join ``on an interface''; that is, hosts that have more than one network card can decide which one
(or more than one) they wish to receive multicast packets via. The implication of the multicast
model is that it is ``pervasive'' so that it is usually only necessary to join on one interface3.2.
Taking a particular example to illustrate the IP level to link level mapping process, if a host joins
an IP multicast group using an Ethernet interface, there is a mapping from the low 24 bits of the
multicast address into the low 24 (out of 48) bits of the ethernet address. Since this is a many-to-
one mapping, there may be multiple IP multicast groups occupying the same ethernet address on
a given wire, though it may be made unlikely by the address allocation scheme (as discussed in
chapter 7). An ethernet LAN is a shared medium network, thus local addressing of packets to an
ethernet group means that they are received by ethernet hardware and delivered to the host
software, only of those hosts with members of the relevant IP group. This means that host
software is generally saved the burden of filtering out irrelevant packets. Where there is an
ethernet address clash, software can filter the packets efficiently. How IGMP works can be
summarised as follows:
• When a host first joins a group, it programs its ethernet interface to accept the relevant
traffic, and it sends an IGMP join message on its local network. This informs any local
routers that there is a receiver for this group now on this subnet.
• The local routers remember this information, and arrange for traffic destined for this
address to be delivered to the subnet.
• After a while, the routers wonder if there is still any member on the subnet, and send an
IGMP query message to the multicast group. If the host is still a member, it replies with a
new Join message unless it hears someone else do so first. Multicast traffic continues to
be delivered.
• Eventually the application finishes, and the host no longer wants the traffic. It reprograms
its ethernet interface to reject the traffic, but the packets are still sent until the router
times the group out and sends a query to which no-one responds. The router then stops
delivering the traffic.
Thus joining a multicast group is quick, but leaving can be slow with IGMP version 1. IGMP
version 2 reduces the leave latency by introducing a "leave" message, and an assorted set of rules
to prevent one receiver leaving from disconnecting others. IGMP version 3 introduces the idea of
source-specific joining and leaving whereby a host can subscribe (or reject) traffic from
individual senders rather than the group as a whole, at the expense of more complexity and extra
state in routers.
Multicast Routing and Addressing
Routers conspire to deliver packets in the Internet. Everything in any part of the Internet that
wants to be reached must have an address. The address tells the computers in the Internet (hosts
and routers) where something is topologically. Thus the address is also hierarchical. My
computer's address is 128.16.8.88. We asked our Internet provider for a network number. We
were given the number 128.16.x.y. We could fill in the x and y how we liked, to number the
computers on our network. We divided our computers into groups on different LAN segments,

Adri Jovin J.J.
Page | 101
and numbered the segments 1-256 (x), and then the hosts 1-256 (y) on each segment. When your
organisation asks for a number for its net, it will be asked how many computers it has, and
assigned a network number big enough to accommodate that number of computers. Nowadays, if
you have a large network, you will be given a number of numbers! Everything in the Internet
must be reachable. The route to a host will traverse one or more networks. The easiest way to
picture a route is by thinking of how a letter to a friend in a foreign country gets there. You post
the letter in a postbox. It is picked up by a postman (LAN), and taken to a sorting office (router).
There, the sorter looks at the address, and sees that the letter is for another country, and sends it
to the sorting office for international mail. This then carries out a similar procedure. And so on,
until the letter gets to its destination. If the letter was for the same 'network' then it would get
immediately locally delivered. Notice the fact that all the routers (sorting offices) don't have to
know all the details about everywhere, just about the next hop to go to. Notice the fact that the
routers (sorting offices) have to consult tables of where to go next (e.g. international sorting
office). Routers chatter to each other all the time figuring out the best (or even just usable) routes
to places. Note sure this is the right place for this - mjh
The way to picture this is to imagine a road system with a person standing at every intersection
who is working for the Road Observance Brigade. This person (Rob) reads the road names of the
roads meeting at the intersection, and writes them down on a card, with the number 0 after each
name. Every few minutes, Rob holds up the card to any neighbour standing down the road at the
next intersection. If they are doing the same, Rob writes down their list of names, but adds 1 to
the numbers read off the other card. After a while, Rob is now telling people about the
neighbours roads several roads away! Of course, Rob might get two ways to get somewhere!
Then, he crosses out the one with the larger number. To support multicast, routers need to know
where the recipients are (or possibly, where they are not!). The first step in this is for routers to
take part in the IGMP.
Multicast Routing
Given the multicast service model described above, and the restrictions that senders and
receivers don't know each others' location or anything about the topology, how do routers
conspire to deliver traffic from the senders to the receivers?
We shall assume that if a sender and a receiver did know about each other, that they could each
send unicast packets to the other. In other words, there is a network with bidirectional paths and
an underlying unicast routing mechanism already running. Given this, there is a spectrum of
possible solutions. At one extreme, we can flood data from the sender to all possible receivers
and have the routers for networks where there are no receivers prune off their branches of the
distribution tree. At the other extreme, we can communicate information in a multicast routing
protocol conveying the location of all the receivers to the routers on the paths to all possible
senders. Neither of methods is terribly desirable on a global scale, and so the most interesting
solutions tend to be hybrid solutions that lie between these extremes.
In the real world there are many different multicast routing protocols, each with its own
advantages and disadvantages. We shall explain each of the common ones briefly, as a working
knowledge of their pros and cons helps us understand the practical limits to the uses of multicast.

Adri Jovin J.J.
Page | 102
Flood and Prune Protocols
DVMRP Status: IETF Experimental Standard (RFC 1075)
DM-PIM Status: IETF Work in progress
These protocols are more correctly known as reverse-path multicast algorithms. When a sender
first starts sending, traffic is flooded out through the network. A router may receive the traffic
along multiple paths on different interfaces, in which case it rejects any packet that arrives on
any other interface other than the one it would use to send a unicast packet back to the source. It
then sends a copy of each packet out of each interface other than the one back to the source. In
this way each link in the whole network is traversed at most once in each direction, and the data
is received by all routers in the network. So far, this describes reverse-path broadcast. Many parts
of the network will be receiving traffic although there are no receivers there. These routers know
they have no receivers (otherwise IGMP would have told them) and they can then send prune
messages back towards the source to stop unnecessary traffic flowing. Thus the delivery tree is
pruned back to the minimal tree that reaches all the receivers. The final distribution tree is what
would be formed by the union of shortest paths from each receiver to the sender, and so this type
of distribution tree is known as a shortest-path tree.
Two commonly used multicast routing protocols fall in the class - DVMRP (the Distance Vector
Multicast Routing Protocol) and Dense-mode PIM (Protocol Independent Multicast). The
primary difference between these is that DVMRP computes its own routing table to determine
the best path back to the source, whereas DM-PIM uses that of the underlying unicast routing
hence the term "Protocol Independent".
It should be fairly obvious that sending traffic everywhere and getting people to tell you what
they don't want is not a terribly scalable mechanism. Sites get traffic they don't want (albeit very
briefly), and routers not on the delivery tree need to store prune state. For example, if a group has
one member in the UK and two in France, routers in Australia still get some of the packets and
need to hold prune state to prevent more packets arriving! However, for groups where most
places actually do have receivers (receivers are ``densely'' distributed), this sort of protocol
works well, and so although these protocols are poor choices for a global scheme, they might be
appropriate within some organisations.
MOSPF
This isn't really a category, but a specific instance of a protocol. MOSPF is the multicast
extension to OSPF (Open Shortest Path First) which is a unicast link-state routing protocol.
Link-state routing protocols work by having each router send a routing message periodically
listing its neighbours and how far away they are. These routing messages are flooded throughout
the entire network, and so every router can build up a map of the network which it can then use
to build forwarding tables (using a Dijkstra algorithm) to decide quickly which is the correct next
hop for send a particular packet.

Adri Jovin J.J.
Page | 103
Extending this to multicast is achieved simply by having each router also list in a routing
message the groups for which it has local receivers. Thus given the map and the locations of the
receivers, a router can also build a multicast forwarding table for each group.
MOSPF also suffers from poor scaling. With flood-and-prune protocols, data traffic is an
implicit message about where there are senders, and so routers need to store unwanted state
where there are no receivers. With MOSPF there are explicit messages about where all the
receivers are, and so routers need to store unwanted state where there are no senders. However,
both types of protocol build very efficient distribution trees.
Center-based Trees
Rather than flooding the data everywhere, or flooding the membership information everywhere,
algorithms in the center-based trees category map the multicast group address to a particular
unicast address of a router, and they build explicit distribution trees centered around this
particular router. There are three main problems that need to be solved to get this approach to
work:
1. How to perform the mapping from group address to center-address?
2. How to choose the location of the center so that the distribution trees are efficient?
3. How to actually construct the tree given the center address.
Different protocols have come up with different solutions to these problems. There are three
center-based tree protocols worth us exploring in a little detail because they illustrate different
approaches: Core-Based Trees (CBT), Sparse-mode PIM (SM-PIM) and the Border Gateway
Multicast Protocol (BGMP).
Core-Based Trees
CBT was the earliest center-based tree protocol, and is the simplest.
When a receiver joins a multicast group, its local CBT router looks up the multicast address and
obtains the address of the Core router for the group. It then sends a Join message for the group
towards the Core. At each router on the way to the core, forwarding state is instantiated for the
group, and an acknowledgment is sent back to the previous router. In this way, a multicast tree is
built, as shown in figure 3.1.

Adri Jovin J.J.
Page | 104
Figure 3.1: Formation of a CBT bidirectional shared tree
If a sender (that is a group member) sends data to the group, the packets reach its local router,
which forwards them to any of its neighbours that are on the multicast tree. Each router that
receives a packet forwards it out of all it its interfaces that are on the tree except the one the
packet came from. The style of tree CBT builds is called a "bidirectional shared tree", because
the routing state is "bidirectional" - packets can flow both up the tree towards the core and down
the tree away from the core depending on the location of the source, and "shared" by all sources
to the group. This is in contrast to "unidirectional shared trees" built be SM-PIM as we shall see
later.
IP Multicast does not require senders to a group to be members of the group, so it is possible that
a sender's local router is not on the tree. In this case, the packet is forwarded to the next hop
towards the Core. Eventually the packet will either reach a router that is on the tree, or it will
reach the core, and it is then distributed along the multicast tree.
CBT also allows multiple Core routers to be specified which adds a little redundancy in case the
core becomes unreachable. CBT never properly solved the problem of how to map a group
address to the address of a core. In addition, good core placement is a hard problem. Without
good core placement, CBT trees can be quite inefficient, and so CBT is unlikely to be used as a
global multicast routing protocol.
However, within a limited domain, CBT is very efficient in terms of the amount of state that
routers need to keep. Only routers on the distribution tree for a group keep forwarding state for
that group, and no router needs to keep information about any source, and thus CBT scales much

Adri Jovin J.J.
Page | 105
better than flood-and-prune protocols, especially for sparse groups where only a small proportion
of subnetworks have members.
Sparse-Mode PIM
The work on CBT encouraged others to try to improve on its limitations while keeping the good
properties of shared trees, and Sparse-Mode PIM was one result. The equivalent of a CBT Core
is called a Rendezvous Point (RP) in PIM, but it largely serves the same purpose.
When a sender starts sending, irrespective of whether it is a member or not, its local router
receives the packets and maps the group address to the address of the RP. It then encapsulates
each packet in another IP packet (imagine putting one letter inside another differently addressed
envelope) and sends it unicast directly to the RP.
When a receiver joins the group, its local router initiates a Join message that travels hop-by-hop
to the RP instantiating forwarding state for the group. However, this state is unidirectional state -
it can only be used by packets flowing from the RP towards the receiver, and not for packets to
flow back up the tree towards the RP. Data from senders is de-encapsulated at the RP and flows
down the shared-tree to all the receivers.
The important advance that SM-PIM made over CBT was to realize that discovering who the
senders are could be separated from building efficient trees from those senders to receivers.
Figure 3.2: Formation of a Sparse Mode PIM tree

Adri Jovin J.J.
Page | 106
Thus SM-PIM unidirectional trees are not terribly good distribution trees, but do start data
flowing to the receivers. Once this data is flowing, a receiver's local router can then initiate a
transfer from the shared tree to a shortest-path tree by sending a source-specific join message
towards the source, as shown in figure 3.2. When data starts to arrive along the shortest-path tree,
a prune message can be sent back up the shared tree towards the source to avoid getting the
traffic twice.
Unlike other shortest-path tree protocols such as DVMRP and DM-PIM where prune state exists
everywhere there are no receivers, with SM-PIM, source-specific state only exists on the
shortest-path tree. Also low bandwidth sources such as those sending RTCP receiver reports do
not trigger the transfer to a shortest path tree, which further helps scaling by eliminating
unnecessary source-specific state.
Because SM-PIM can optimise its distribution trees after formation, it is less critically dependent
on the RP location than CBT is on the Core location. Hence the primary requirement for
choosing an RP is load-balancing. To perform multicast-group to RP mapping, SM-PIM pre-
distributes a list of candidates to be RPs to all routers. When a router needs to perform this
mapping, it uses a special hash function to hash the group address into the list of candidate RPs
to decide the actual RP to join. Except in exceptional failure circumstances, all the routers within
the domain will perform the same hash, and come up with the same choice of RP. The RP may
or may not be in an optimal location, but this is offset by the ability to switch to a shortest path
tree.
The dependence on this hash function and the requirement to achieve convergence on a list of
candidate RPs does however limit the scaling of SM-PIM. As a result, it is also best deployed
within a domain, although the size of such a domain may be quite large.
Border Gateway Multicast Protocol
The Border Gateway Multicast Protocol (BGMP) is an attempt to design a true inter-domain
multicast routing protocol; one that can scale to operate in the global Internet. DVMRP and DM-
PIM will not do this because their flood-and-prune nature requires off-tree routers to keep per-
source state. MOSPF will not do this because OSPF does not scale well enough, and MOSPF
(which also distributes receivership information) scales worse. CBT and SM-PIM will not do this
because the scalability of the mechanisms they use to perform group-to-RP mapping limits them.
BGMP was based on ideas from the intra-domain protocols above, but has a slightly different
goal - it does not build trees of routers, but rather it builds bidirectional shared-trees of domains.
Within a domain, any of the intra-domain multicast routing protocols can be used, and BGMP
then provides multicast routing between the domains.
BGMP builds trees of domains that are similar to CBT trees of routers - they are bidirectional
shared-trees built by sending explicit join messages towards a root domain. However BGMP can
also build source-specific branches, which are similar in concept to source-specific trees in SM-

Adri Jovin J.J.
Page | 107
PIM, but do not always reach as far as the source. The principle problem that prevents CBT and
SM-PIM from scaling is that of mapping a multicast address to the unicast address of a
Rendezvous Point or Core. BGMP solves this problem through a close tie-in with a hierarchical
multicast address allocation scheme called Multicast Address-Set Claim (MASC). We will
describe MASC in more detail in chapter ???. MASC allocates ranges of multicast addresses to
domains. These address-ranges are distributed to Border-Routers world-wide as group-routes
using BGP routing (which is used for unicast inter-domain routing). Such a group-route indicates
the path to the root-domain for that range of multicast addresses. To make this scale, MASC
allocates the address ranges dynamically in a manner that is aggregatable, so the number of
group routes that need to be stored at each border-router is relatively small.
Figure 3.3: Formation of a BGMP Shared Tree
Figure 3.3 illustrates how BGMP builds a distribution tree given than group-routes have already
been distributed. In this case all the domains are running DM-PIM because this simplifies the
example, but in reality they would likely be running a mixture of different protocols.
Receiver R joins the multicast group 224.1.128.254. Its local router sends a Domain-Wide
Report (DWR) to all the border routers in domain A. Router A1 discovers that the best route to
the root for 224.1.128.254 is 224.1/16 received from its external peer router B1, so it sends a
BGMP Join message to B1. Router B1 in turn looks up 224.1.128.254 and discovers its best
route is 224.1.128/20 that it received from its internal peer router B3, so it sends a BGMP join
message to B3. B3 repeats the process and sends a BGMP join to C1. Router C1 is a border

Adri Jovin J.J.
Page | 108
router for the domain which has been allocated 224.1.128.254 (along with other groups), so the
join message has now reached the root domain and need travel no further.
Now source S in domain D starts sending to 224.1.128.254. Its data floods through domain D
and reaches all the border routers. D2 is not on the best path to the root domain, so it sends a
DM-PIM prune message back towards S, but router D1 is on the best path to the root domain. It
has no state for group 224.1.128.254, but forwards the data anyway to C2. C2 forwards the data
into the DM-PIM domain C, and it floods through the domain. Router C3 has no state for the
group, and sends a prune in response, but router C1 is on the shared tree for the group, and so
forwards the data to B3. B3 wishes to forward the data into B, but it is not on the best path back
to S. If it merely multicast the data into B, routers within B would drop the data as DM-PIM
requires the distribution tree to be source-based. Instead, it encapsulates the data and sends it to
the correct entry router (as determined by its unicast routing table), in this case B2. B2 then
decapsulates the data and multicasts it into domain B, where it floods through the domain. B1 in
turn forwards the data along the shared tree to domain A and hence to R.
Figure 3.4: Forming a BGMP Shortest-path branch
At this stage, as shown in figure 3.4A, data is flowing from S to R along the shared tree. It has
flooded through the domains along the path, but has been pruned back within the domains to just
the minimal tree required to deliver the data. However, B3 is having to encapsulate the data to
B2, which is undesirable as routers typically are not very good at encapsulation, and also as there
is a better path that the data could have taken. As B2 is not on the shared tree, it is permitted to
initiate a shortest-path branch by sending a source-specific join for source S to D2. When D2
receives this join, it grafts itself onto the DM-PIM tree within domain D, and traffic starts to flow
to B2. B2 then sends a BGMP prune to B3 and starts dropping the encapsulated packets to

Adri Jovin J.J.
Page | 109
prevent it receiving two copies of the data. The prune will propagate up to the root domain if it
encounters no other branches of the shared tree on the way.
The example above used Dense-Mode PIM within the domains, but any other inter-domain
multicast routing protocol could be used instead. Each has its own set of rules for how to
interoperate with BGMP, but at least each does not then need an additional set of rules for how
to interoperate with every other intra-domain multicast routing protocol, which greatly simplifies
things from an operational point of view.
Deciding where to put the root for any shared tree is a hard problem. BGMP places the root in
the domain which has been allocated the multicast address. Hence if the session initiator obtains
the address from its local multicast address allocation server, then the tree will be rooted in the
session initiators domain. For many uses of multicast such a TV-style broadcasts, this is optimal.
For other uses of multicast, with many senders, it may be less optimal, but it is still a reasonable
default. Without knowing the receivership in advance, it is difficult to do better than this.
Multicast Scoping
When applications operate in the global MBone, it is clear that not all groups should have global
scope. This is especially the case for performance reasons with flood and prune multicast routing
protocols, but it also the case with other routing protocols for application security reasons and
because multicast addresses are a scarce resource. Being able to constrain the scope of a session
allows the same multicast address to be in use at more than one place so long as the scope of the
sessions does not overlap.
Multicast scoping can currently be performed in two ways which are known as TTL scoping and
administrative scoping. Currently TTL scoping is most widely used, with only a very few sites
making use of administrative scoping.
TTL Scoping
When an IP packet is sent, an IP header field called Time To Live (TTL) is set to a value
between zero and 255. Every time a router forwards the packet, it decrements the TTL field in
the packet header, and if the value reaches zero, the packet is dropped. The IP specification also
states that TTL should be decremented if a packet is queued for more than a certain amount of
time, but this is rarely implemented these days. With unicast, TTL is normally set to a fixed
value by the sending host (64 and 255 are commonly used) and is intended to prevent packets
looping forever, and also forms a part of the formal proof that the TCP close semantics are safe.
With IP multicast, TTL can be used to constrain how far a multicast packet can travel across the
MBone by carefully choosing the value put into packets as they are sent. However, as the
relationship between hop-count and suitable scope regions is poor at best, the basic TTL
mechanism is supplemented by configured thresholds on multicast tunnels and multicast-capable
links. Where such a threshold is configured, the router will decrement the TTL, as with unicast
packets, but then will drop the packet if the TTL is less than the configured threshold. When
these thresholds are chosen consistently at all of the borders to a region, they allow a host within

Adri Jovin J.J.
Page | 110
that region to send traffic with a TTL less than the threshold, and to know that the traffic will not
escape that region.
An example of this is the multicast tunnels and links to and from Europe, which are all
configured with a TTL threshold of 64. Any site within Europe that wishes to send traffic that
does not escape Europe can send with a TTL of less than 64 and be sure that their traffic does not
escape.
However, there are also likely to be thresholds configured within a particular scope zone - for
example most European countries use a threshold of 48 on international links within Europe, and
as TTL is still decremented each time the packet is forwarded, it is good practice to send
European traffic with a TTL of 63, which allows the packet to travel 15 hops before it would fail
to cross a European international link.
Administrative Scoping
There are circumstances where it is difficult to consistently choose TTL thresholds to perform
the desired scoping. In particular it is impossible to configure overlapping scope regions as
shown in figure 3.5, and there are a number of other problems with TTL scoping, and so more
recently, administrative scoping has been added to the multicast forwarding code in
mrouted[#!mrouted!#] and in most router implementations. Administrative scoping allows the
configuration of a boundary by specifying a range of multicast addresses that will not be
forwarded across that boundary in either direction
Figure 3.5: Overlapping scope zones possible with administrative scoping
Scoping Deployment
Administrative scoping is much more flexible than TTL scoping, but suffers from a number of
disadvantages. In particular, it is not possible to tell from the address of a packet where it will go
unless all the scope zones that the sender is within are known. Also, as administrative boundaries
are bi-directional, one scope zone nested within or overlapping another must have totally
separate address ranges. This makes their allocation difficult from an administrative point of
view, as the ranges ought to be allocated on a top-down basis (largest zone first) in a network
where there is no appropriate top-level allocation authority. Finally, it is easy to misconfigure a
boundary by omitting or incorrectly configuring one of the routers - with TTL scoping it is likely
that in many cases a more distant threshold will perform a similar task lessening the
consequences, but with administrative scoping there is less likelihood that this is the case.
For these reasons administrative scoping has been viewed by many network administrators as a
specialty solution to difficult configuration problems, rather than as a replacement for TTL
scoping, and the MBone still very much relies on TTL scoping.

Adri Jovin J.J.
Page | 111
NBMA networks
A non-broadcast multiple access network (NBMA) is a computer network to which multiple
hosts are attached, but data is transmitted only directly from one computer to another single host
over a virtual circuit or across a switched fabric. NBMA networks do support multicast or
broadcast traffic manually (pseudo-broadcasts). Some common examples of nonbroadcast
network technologies include Asynchronous Transfer Mode (ATM), Frame Relay, X.25, and
home power line networking. NBMA (non-broadcast multiple access) is one of four network
types in the OSPF (Open Shortest Path First) communications protocol. NBMA is used to
accurately model X.25 and frame relay environments in multiple-access networks where there
are no intrinsic broadcast and multicast capabilities. The other OSPF network types are:
broadcast, point-to-point, and point-to-multipoint. In an NBMA configuration, OSPF sends
HELLO packets (packets sent periodically to establish and confirm neighbor relationships
between routers) to each router one at a time rather than multicasting them. The HELLO timer
(which tells the router how often to send HELLO packets) is extended from 10 to 30 seconds and
the dead router timer (which tells the router how long to wait before it decides that a neighboring
router is not functioning) is extended from 40 to 120 seconds.
Non Broadcast Multiple Access (NBMA) networks may be utilized in a variety of ways. At one
extreme, they can be used to simply provide administratively configurable point to point service,
sufficient to interconnect IPv6 routers (and even IPv6 hosts, in certain situations). At the other
extreme, NBMA networks that support dynamic establishment and teardown of Virtual Circuits
(or functional equivalents) may be used to emulate the service provided to the IPv6 layer by
conventional broadcast media such as Ethernet. Typically this emulation requires complex
convergence protocols, particularly to support IPv6 multicast.
Certain NBMA networks may provide a form of connectionless service (e.g. SMDS). In these
cases, a "call" or "VC" shall be considered to implicitly exist if the sender has an NBMA
destination address to which it can transmit packets whenever it desires.
Neighbor Discovery
A key difference between this architecture and previous IP over NBMA protocols is its
mechanism for supporting IPv6 Neighbor Discovery. The IPv4 world evolved an approach to
address resolution that depended on the operation of an auxiliary protocol operating at the 'link
layer' - starting with Ethernet ARP (RFC 826). In the world of NBMA (Non Broadcast, Multiple
Access) networks ARP has been applied to IPv4 over SMDS (RFC 1209) and IPv4 over ATM
(RFC 1577). More recently the ION working group has developed NHRP (Next Hop Resolution
Protocol), a general protocol for performing intra-subnet and inter-subnet address resolution
applicable to a range of NBMA network technologies. IPv6 developers opted to migrate away
from a link layer specific approach, choosing to combine a number of tasks into a protocol
known as Neighbor Discovery, intended to be non-specific across a number of link layer
technologies. A key assumption made by Neighbor Discovery's actual protocol is that the link
technology underlying a given IP interface is capable of native multicasting. This is not
particularly true of most NBMA network services, and usually requires convergence protocols to
emulate the desired service. (The MARS protocol, RFC 2022, is an example of such a
convergence protocol.) This document augments and optimizes the MARS protocol for use in

Adri Jovin J.J.
Page | 112
support of IPv6 Neighbor Discovery, generalizing the applicability of RFC 2022 beyond ATM
networks.
A shortcut is an NBMA level call (VC) directly connecting two IP endpoints that are logically
separated by one or more routers at the IP level. IPv6 packets traversing this VC are said to
'shortcut' the routers that are in the logical IPv6 path between the VC's endpoints. NBMA
shortcuts are a mechanism for minimizing the consumption of resources within an IP over
NBMA cloud (e.g. router hops and NBMA VCs). It is important that NBMA shortcuts are
supported whenever IP is deployed across NBMA networks capable of supporting dynamic
establishment of calls (SVCs or functional equivalent). For IPv6 over NBMA, shortcut
discovery and management is achieved through a mixture of Neighbor Discovery and NHRP.
Power line networks
The ITU-T G.hn standard provides a specification for creating a high-speed (up to 1 Gigabit/s)
local area network using existing home power lines, phone lines and coaxial cables.
Because of multipath propagation, power lines use frequency-selective channels. Channel
frequency response is different for each pair of transmitter and receiver, so modulation
parameters are unique for each transmitter and receiver pair. Since each pair of devices uses a
different modulation scheme for communication, other devices may not be able to demodulate
the information sent between them
Split horizon route advertisement
In NBMA networks a special technique called split horizon route advertisement must be
employed by distance-vector routing protocols in order to prevent routing loops. This family of
protocol relies on link layer broadcasting for route advertisement propagation, so when this
feature is absent, it has to be emulated with series of unicast transmissions, which may result in
receiver node sending a route advertisement back to the node it has just received it from.
Reliable transport protocols (Reliable Multicast Transport)
Reliable multicast is sometimes regarded as something of an oxymoron.
When people talk about "Reliable Multicast", they usually mean a single protocol at a single
'layer' of a protocol stack, typically the transport layer (although I've seen people propose it in the
network and even link (ATM!) layers too), that can act as any layered protocol can - to provide
common functionality for applications (higher layers) that need it.
So what's wrong with that?
Well, possibly 3 things (or more):
1. Fate sharing
2. Performance
3. Semantics

Adri Jovin J.J.
Page | 113
Fate Sharing
Fate sharing in unicast applications means that so long as there is a path that IP can find between
two applications, then TCP can hang on to the connection as long as they like. However, if either
party fails, the connection certainly fails.
Fate sharing between multicast end points is a more subtle idea. Should 'reliability' extend to
supporting the connection for k recipients failing? Clearly this will be application specific (just
as timing out on not getting liveliness out of a unicast connection is for TCP - we must permit
per recipient Timeouts/Failures).
Performance
When a talks to b, the performance is limited by 1 path. Whatever can be done to improve the
throughput (or delay bound) is done by IP (e.g. load sharing the traffic over multiple paths).
When a talks to b,c,d,e,f, should the throughput or delay be that sustainable by the slowest or
average?
Semantics
As well as performance and failure modes, N-way reliable protocols can have different service
models. We could support reliable 1-n, reliable n-1 and reliable n-m.
Applications such as software distribution are cited as classic 1-n requirements. Telemetry is
given as a n-1 reliable protocol. Shared whiteboards are cited as examples of n-m.
Now the interesting thing is to look at the reliability functions needed in these. 1-n and n-1 are
effectively simplex bulk transfer applications. In other words, the service is one where reliability
can be dealt with by "rounding up' the missing bits at the end of the transfer. Since this need not
be especially timely, there is no need for this to be other than end to end, and application based.
3.6
On the other hand n-m processes such as whiteboards need timely recovery from outages. The
implication is that the "service' is best done somewhat like the effect of having n*n-1/2 TCP
connections. If used in the WAN, the recovery may best be distributed, since requests for
recovery will implode down the very links that are congested or error prone and caused the need
for recovery.
Now there are different schemes for creating distributed recovery. If the application semantics
are that operations (ALF packetsworths...) are sequenced in a way that the application can index
them, then any member of a multicast session can efficiently help any other member to recover
(examples of this include mark Handley's Network Text tool...). On the other hand, packet-based
recovery can be done from data within the queues between network/transport and application, if
they are kept at all members in much the same way as a sender in a unicast connection keeps a
copy of all un-acknowledged data. The problem with this is that because its multicast, we don't
have a positive acknowledgment system. Because of that, there is no way to inform all end points

Adri Jovin J.J.
Page | 114
when they can safely discard the data in the 'retransmit' queue. Only the application really knows
this!
Well, this is not to say that there isn't an obvious toolkit for reliable multicast support - it would
certainly be good to have RTP style media timestamps (determined by the application, but filled
in by the system). It would be good to have easy access to a time-stamped based receive queue
so applications could use this to do all the above. It might be neat to have virtual token ring,
expanding ring search, token tree and other toolkits to support retransmit 'helper' selection....
So, drawing a table of where we might put functions to provide reliability (retransmit),
sequencing and performance (adaptive playout say versus end to end, versus hop by hop delay
constraint), we can see the following picture:
Recovery Sequency Dalliance
Network not in our internet ditto int-serv
Transport one-many y adaptive
Application many-many operation semantics adaptive
Table 3.2: Reliable Multicast Semantics
TCP Adaption Algorithms
Before we look at the mechanisms to multimedia adaption, it is worth explaining how the current
system survives at all for data flows! It is essentially achieved by smoothly degrading everyone's
performance smoothly as load increases (rather than blocking access) that the current Internet
continues to function. This is done through a variety of adaption algorithms, both for data and for
multimedia applications. Adaption in protocols was first introduced in TCP in around
1988[#!van:88!#].
Adaption in TCP is both to the round trip time, in order to dynamically tune retransmit timers for
reliable delivery, and of the send rate in order to adapt to the achievable transfer rate between the
sender and receiver (possible due to network bottlenecks, or receiver interface performance
problems). The same techniques can often be applied in other protocols, in particular for
multimedia services to operate over time varying network services.
Adaption to delay at the receiver can be used for two things:
1. Adaptive playout buffer to smooth playout so that a fixed rate media device (e.g. within a
single video frame scan, or a dumb CBR audio device) isn't starved of data, or overrun.
2. Syncronisation of streams from different source (timestamp) can be achieved at a
receiver.
The first of these is done by looking at the Inter-Arrival Time variation, and calculating a
"rolling average".
It is needed to deal with the fact that there are variations in network delay for two reasons:

Adri Jovin J.J.
Page | 115
• Other traffic causes long term average to vary.
• Bursts of one's own traffic cause ones own delay to vary quickly.
The usual algorithm for this is an ``Exponential Weighted Moving Average'':
Assume we measure the arrival time for each (i'th) packet as IATi then the simple average would
be: sumi=1n
IATi, divided by the number of packets.
But since the average is not fixed, we give the most recent measurements a lot more weight than
the older ones by using:
In other words, we give alpha's worth of credence to the latest measurement, and only 1 - alpha
to ALL the previous ones. It is only a coincidence that the equation for a rolling, moving average
for IAT estimation is the same as the one for TCP's RTT estimation. It is worth noting, however,
that in both cases, the requirement is only for local clocks not to drift too fast so that the
measurement of successive arrival times of packets is accurate w.r.t the previous ones; but no
clock synchronisation are needed.
This is sometimes not used, as it includes all the past, and if there is a fundamental change in the
network (e.g. a re-route) then a system that eliminates outlying points quickly may be better.
Henning Sculzrinne's paper suggests a "band pass filter" approach to the estimation of mean
IAT, based on taking only the sum of the smallest of a set of measurements recently, over the
number of them.
Once you have a mean IAT, then you can calculate the current required playout buffer, since it is
roughly twice the variation in inter-arrival.
Given two streams, to synchronise their playout at a receiver we need to know the delay from
each source to each destination, and the clock offsets in case the clocks in the two (or more)
senders are out of step with the receiver. This requires the exchange of packets including each
sender’s statement of the current clock from its view. Assume that the delay in each direction on
the net is the same (and if you don't it is impossible to solve this):
1. send a packet from s to d with source time in it (t1) and it arrives at d when d's clock
reads t2
2. send the packet back to s with t1, t2 and t3, the time on d's clock when it sends it
3. s gets the response at t4 by its clock.
4. if s and d have clocks that differ by offset, and the network delay is d then
t4 = t3+offset+delay

Adri Jovin J.J.
Page | 116
but
t2=t1-offset+delay
so
delay = (t4+t2-t3+t1)/2
and offset can be similarly calculated. we then do this for several goes, and keep
a mean and variance.
TCP Adaption Status: IETF Proposed Standard RFC 2001
RTP Status: IETF Proposed Standard RFC 1889.
Table 5.1: Status of Protocols with Adaption Algorithms
Playout Algorithms
For a variety of reasons, samples may arrive at different times than they were sent - this variation
can be coped with so long as the mean rate of delivery of samples is maintained, and the
variation is a second order (less important) effect compared with the delivery delay. A playout
buffer can accommodate a fixed variation in arrival. If the arrival rate itself varies, it is possible
to use an Adaptive Playout buffer, which is continually re-calculated.
As explained above, the Internet typically currently provides no guarantees. The throughput and
delay along a path can vary quite drastically as other traffic comes and goes. When the network
is overloaded, packets get lost leaving gaps in the information flow at a receiver. This is
illustrated below. Two basic techniques have emerged to deal with these two problems:
1. Audio, Video and other interactive application receivers generally use an adaptive
playout scheme.
2. Senders are generally adaptive to reported network conditions, falling back to lesser
quality as the network becomes more highly loaded, and only, gingerly, increasing the
quality and subsequent load as the network is perceived to have more spare capacity.
The way these two techniques work is quite ingenious, but once seen, relatively simple. All
sources that are generating information with time structure use a protocol called RTP, the Real
Time Protocol, which places a media timestamp in each packet that is sent. All receivers use this
time stamp for two purposes:
The inter-arrival time distribution is monitored. If the delay on the path varies, it will probably
vary fairly smoothly, with some sort of reasonable probability distribution. By monitoring the
mean difference between interarrival times, and adding this to a playout buffer that is used to
delay sending thing between the receiving application and the output device (video window,
audio, whiteboard , etc.), the receiver can be assured to a high degree of chance that it wont be
starved of data (run out of steam). See figures 5.1 and 5.2 and

Adri Jovin J.J.
Page | 117
Figure 5.1: Typical Media Tool Data Flows
Figure 5.1 shows the components of a playout buffer. These include the mixing of streams from
multiple sources, which can also be used to synchronise media. In figure 5.2, we can see the
reason for this requirement graphically displayed: interference from other traffic causes jitter
within the network.
Figure 5.2: End to end requirement for playout adaption
Receivers monitor gaps in the inter-arrival times (that correspond to missing data, as opposed
just to, say, silence in an audio stream). Periodically, Mbone applications report the statistics
about particular sources by multicasting a report to the same group. A sender/source can use this
report to calculate whether the network appears congested or not. The scheme used to adjust the
sending rate is basically that used in TCP, but must be implemented by degrading the quality of
the input media for audio and video - many video compression schemes are easily altered to
permit this. The total amount of traffic generated by these quality reports is constrained to be a
constant percentage of any conference session (typically around 5 conditions is that as a
conference grows, the number of samples of different parts of the net gets better and better,

Adri Jovin J.J.
Page | 118
hence the quality of information in fact improves, even though the quantity from any given
receiver decreases. The receivers use the reception of other receivers reports to give an estimate
of the number of receivers, and hence to reduce the frequency with which they send reports in a
fully distributed way.
RTP
The Real Time Protocol is the Internet Standard for conveying media streams between
interactive participants.
RTP Packet Format
Table 5.2: RTP Packet Format
The first twelve octets are present in every RTP packet, while the list of CSRC identifiers is
present only when inserted by a mixer.
version (V)
: 2 bits
This field identifies the version of RTP. The version defined by this specification is two (2).
padding (P)
: 1 bit
If the padding bit is set, the packet contains one or more additional padding octets at the end
which are not part of the payload.
extension (X)
: 1 bit
If the extension bit is set, the fixed header is followed by exactly one header extension, with a
format defined in Section 5.2.1.

Adri Jovin J.J.
Page | 119
CSRC count (CC)
: 4 bits
The CSRC count contains the number of CSRC identifiers that follow the fixed header.
marker (M)
: 1 bit
The interpretation of the marker is defined by a profile. It is intended to allow significant
events such as frame boundaries to be marked in the packet stream.
payload type (PT)
: 7 bits
This field identifies the format of the RTP payload and determines its interpretation by the
application.
sequence number
: 16 bits
The sequence number increments by one for each RTP data packet sent, and may be used by
the receiver to detect packet loss and to restore packet sequence.
timestamp
: 32 bits
The timestamp reflects the sampling instant of the first octet in the RTP data packet. The
sampling instant must be derived from a clock that increments monotonically and linearly in time
to allow synchronization and jitter calculations
SSRC
: 32 bits
The SSRC field identifies the synchronization source.
CSRC list
: 0 to 15 items, 32 bits each
The CSRC list identifies the contributing sources for the payload contained in this packet. The
number of identifiers is given by the CC field. If there are more than 15 contributing sources,
only 15 may be identified. CSRC identifiers are inserted by mixers, using the SSRC identifiers of
contributing sources.

Adri Jovin J.J.
Page | 120
RTP Header Compression
The combination of the IP, UDP and RTP control information adds up to a significant overhead
for small media samples, particularly over low speed links, commonly in use by the domestic
and small office user dialling up their Internet Service Provider at a few tens of kilobits per
second. An Internet Protocol Datagram has a 20 byte header, while the UDP header is 8 bytes
(source and destination ports, plus length and checksum field). The RTP header adds 12 bytes to
this, making a total of 40 bytes of control for a single sample, in some cases (especially where
sampling/serialising delay is a problem) as little as 20ms worth of 8KHz speech.
Luckily, by far the common case of such usage is as stated when dialling up an ISP, where the
access router is connected to a high speed backbone. There is already a technique for reducing
the overhead of packets in such circumstances, designed for compressing TCP/IP headers
specified in RFC 1144 by Jacobson[#!ctcp!#]. Casner and Jacobson adapted this technique to
RTP headers[#!crtp!#], noting certain particular differences.
The technique consists of two parts: noting fields in the packet headers that do not change over
the life of a flow; and noting that there are few flows at the ``edge of the network'' so that such
information can be conveyed over the first hop by a single packet, and subsequently referred to
by a short ``connection identifier'', which serves to index the full state so that the first hop router
can reconstruct the full Internetwork packet. In RTP, it turns out that there are also fields that
change only by the same amount from each packet to the next, except in exceptional
circumstances, so that this second order information can also be stored in the compression state
vector at the first hop routers. Note also that this compression state is ``soft state'' in that it can be
recovered simply by loss since the packet conveys enough implicit information that end-to-end
checksums are still computed, and hop-wise recomputed from the state vector and from the
remaining data in the compressed header. In other words, if the router resets, or the route
changes, or the end system radically alters state, the invalid checksum causes a reset of the
compressed state, and an exchange of a full packet re-creates the necessary state from full anew.
The work is still in progress, although there are several implementations in common use, and
shows a typical reduction to a header size of 3-4 bytes (better than ten-fold reduction in control
overhead).
RTP Multiplexing
There are a number of circumstances in which one might wish to carry multiple media flows
within a single RTP data payload between two points. The two most important cases are: IP
paths between Internet Telephony Gateways; special hardware devices such as CODECs with
non-negotiable multiplexed media.
There are at least two ways to multiplex data in RTP packets. One could (e,g. in the telephony
case) assume that all the samples have the same payload types, and are just offset in different
end-to-end flows. Here we need a mapping table in the gateways, that indicates the offset for
each payload type and a list of the flows in each packet. The second approach might be a more
generic one suggested by Mark Handley recently in work in progress, which is to adapt the ideas
from the previous section concerning RTP header compression and to allow for multiple
compressed headers within a single RTP packet, one for each of the samples. This latter

Adri Jovin J.J.
Page | 121
approach would not use precisely the same compression algorithm, since the fields differ for
different reasons, but would permit multiple different media to be efficiently encapsulated in a
single packet. This might address both types of application of RTP multiplexing more
effectively.
RTCP
RTCP is the Real-time Transport Control Protocol, which may be used as a lightweight
companion to RTP to convey a number of statistics and other information about an RTP flow
between recipients and senders.
The header is illustrated below.
Table 5.3: The Real Time Control Protocol

Adri Jovin J.J.
Page | 122
SR or RR
: The first RTCP packet in the compound packet must always be a report packet to facilitate
header validation as described in Appendix A.2. This is true even if no data has been sent nor
received, in which case an empty RR is sent, and even if the only other RTCP packet in the
compound packet is a BYE.
Additional RRs
: If the number of sources for which reception statistics are being reported exceeds 31, the
number that will fit into one SR or RR packet, then additional RR packets should follow the
initial report packet.
SDES
: An SDES packet containing a CNAME item must be included in each compound RTCP
packet. Other source description items may optionally be included if required by a particular
application, subject to bandwidth constraints (see Section 6.2.2).
BYE or APP
: Other RTCP packet types, including those yet to be defined, may follow in any order, except
that BYE should be the last packet sent with a given SSRC/CSRC. Packet types may appear
more than once.
Payloads
There are quite a few payloads defined for RTP now, ranging from oens designed to carry video
and audio formerly available on hardware CODECs only, such as MPEG, H.261, and the various
CCITT/ITU voice coding schemes, through to some more generic payloads designed to carry
redundantly coded audio such as that used by INRIA's Freephone, and UCL's RAT tools.
There has been some discussion in the IETF of a payload for multiplexed media within a single
stream (such as the way the MOPEG systems stream of H221 framed stream emerge in a single
bitstream from hardware CODEC). To date, the disadvantages of this have outweighed any
perceived advantage.
RTCP Scaling Properties and Timer Considerations
The intention and original design goal of RTCP messages was for them to act as a distributed
source of lightweight session data that would allow a range of highly fault tolerant, and
reasonable scale mechanisms to be built including:
membership
RTCP reports carry the identifier of participants. There are also ``BYE'` messages which
indicate the departure of a participant.

Adri Jovin J.J.
Page | 123
loss statistics
RTCP reports convey information about packet losses. These could be used not only as health
reports, but also to select a CODEC or compression level for sources capable of adaption.
RTCP reports are designed to be sent periodically, with a frequency inversely proportional to the
number of members. This can be set to constrain the bandwidth consumed by RTCP to be a
known fixed fraction of the total capacity required for a many-to-many session. For multimedia
conferencing this is an excellent solution for many applications requirements for such data.
However there are three circumstances where this approach creates a problem: firstly, when a
session starts, the members do not know the number of other members - initial membership for a
scheduled session could rise very sharply, leading to a flood of initial packets; secondly, this
effect can also happen for BYE messages; finally, for sessions where there is some form of floor
control, or few senders compared with recipients, the fact that the RTCP reports are multicast
means that they arrive at all participants- this may still represent a non-trivial fraction of the
capacity to a given participant - in some cases, the state for large membership may impact on
memory for small multicast devices (Internet Telephones).
----X-----------X----

UNIT IV INTERNETWORKING MULTIMEDIA
Adri Jovin J.J.
Page | 124
UNIT IV
MEDIA ON-DEMAND
STORAGE AND MEDIA SERVERS
As you add more memory and storage space to your computer, you can expect your computing
needs and habits to keep pace, filling the new capacity.
To estimate the memory requirements of a multimedia project—the space required on a hard
disk, thumb drive, CD-ROM, or DVD, not the random access memory (RAM) used while your
computer is running— you must have a sense of the project’s content and scope. Color images,
text, sound bites, video clips, and the programming code that glues it all together require
memory; if there are many of these elements, you will need even more. If you are making
multimedia, you will also need to allocate memory for storing and archiving working files used
during production, original audio and video clips, edited pieces, and final mixed pieces,
production paperwork and correspondence, and at least one backup of your project files, with a
second backup stored at another location. It is said that when John von Neumann, often called
“the father of the computer,” was designing the ENIAC computer in 1945, there was an
argument about how much memory this first computer should have. His colleagues appealed for
more than the 2K Dr. von Neumann felt was sufficient. In the end, he capitulated and agreed to
install 4K in the ENIAC, commenting “...but this is more memory than you will ever need.”
Random Access Memory (RAM)
If you are faced with budget constraints, you can certainly produce a multimedia project on a
slower or limited-memory computer. On the other hand, it is profoundly frustrating to face
memory (RAM) shortages time after time, when you’re attempting to keep multiple applications
and files open simultaneously. It is also frustrating to wait the extra seconds required of each
editing step when working with multimedia material on a slow processor.
In spite of all the marketing hype about processor speed, this speed is ineffective if not
accompanied by sufficient RAM. A fast processor without enough RAM may waste processor
cycles while it swaps needed portions of program code into and out of memory. In some cases,
increasing available RAM may show more performance improvement on your system than
upgrading the processor chip.
Read-Only Memory (ROM)
Unlike RAM, read-only memory (ROM) is not volatile. When you turn off the power to a ROM
chip, it will not forget, or lose its memory. ROM is typically used in computers to hold the small
BIOS program that initially boots up the computer, and it is used in printers to hold built-in
fonts. Programmable ROMs (called EPROMs) allow changes to be made that are not forgotten
when power is turned off.

Adri Jovin J.J.
Page | 125
Hard Disks
Adequate storage space for your production environment can be provided by large-capacity hard
disks, server-mounted on a network. As multimedia has reached consumer desktops, makers of
hard disks have built smaller-profile, larger-capacity, faster, and less-expensive hard disks.
As network and Internet servers drive the demand for centralized data storage requiring terabytes
(one trillion bytes), hard disks are often configured into fail-proof redundant arrays offering
built-in protection against crashes.
Flash Memory or Thumb Drives
These flash memory data storage devices are about the size of a thin cigarette lighter and can be
integrated with USB or FireWire interfaces to store from eight megabytes to several GB of data.
They are available in every color of the rainbow, are extremely portable, and, because they have
fewer moving parts, are more reliable than disk drives. Consisting of a small printed circuit
board encased in a sturdy metal or plastic casing with a USB connector covered with a cap, the
flash drive is trendy as a status symbol, and convenient to use. This same solid-state storage is
used in digital cameras, cell phones, and audio recording devices, and for solid state hard drives
(no spinning platters or moving parts) that are found in some net-books and other handheld
devices.
CD-ROM Discs
Compact disc read-only memory (CD-ROM) players have become an integral part of the
multimedia development workstation and are an important delivery vehicle for mass-produced
projects. A wide variety of developer utilities, graphic backgrounds, stock photography and
sounds, applications, games, reference texts, and educational software are available on this
medium. CD-ROM players have typically been very slow to access and transmit data (150 KBps,
which is the speed required of consumer Audio CDs), but developments have led to double-,
triple-, quadruplespeed, 24x, 48x, and 56x drives designed specifically for computer (not Red
Book Audio) use. These faster drives spool up like washing machines on the spin cycle and can
be somewhat noisy, especially if the inserted compact disc is not evenly balanced. With a
compact disc recorder, you can make your own CDs, using CD-recordable (CD-R) blank discs to
create a CD in most formats of CD-ROM and CD-Audio. Software, such as Roxio’s Toast and
Easy CD Creator, lets you organize files on your hard disk(s) into a “virtual” structure, and then
writes them to the CD in that order. CD-R discs are manufactured differently than normal CDs
but can play in any CD-Audio or CD-ROM player. These write-once, enhanced CDs make
excellent high-capacity file archives and are used extensively by multimedia developers for pre-
mastering and testing CD-ROM projects and titles. Because they have become very inexpensive,
they are also used for short-run distribution of finished multimedia projects and data backup. A
CD-RW (read and write) recorder can rewrite 700MB of data to a CD-RW disc about 1,000
times.

Adri Jovin J.J.
Page | 126
Digital Versatile Discs (DVD)
In December 1995, nine major electronics companies (Toshiba, Matsushita, Sony, Philips, Time
Warner, Pioneer, JVC, Hitachi, and Mitsubishi Electric) agreed to promote a new optical disc
technology for distribution of multimedia and feature-length movies called Digital Versatile Disc
(DVD) (see Table 7-3).With a DVD capable not only of gigabyte storage capacity but also full-
motion video (MPEG2) and high-quality audio in surround sound, this is an excellent medium
for delivery of multimedia projects. Commercial multimedia projects will become more
expensive to produce, however, as consumers’ performance expectations rise. There are three
types of DVD, including DVD-Read Write, DVD-Video, and DVD-ROM. These types reflect
marketing channels, not the technology.
Blu-ray Discs
Driven by the implementation of High Definition TV (HDTV) and by the motion picture
industry, a new technology was needed to increase storage capacity and throughput beyond
DVD. Two competing and incompatible solutions were promoted and a war was fought in the
marketplace between HD-DVD, backed by Toshiba, and Blu-ray, backed by Sony. By 2008,
Toshiba had sold about one million HD-DVD players, but Sony had sold close to ten million
Blu-ray players, which were also included in popular PlayStation game machines. Toshiba
announced it was quitting. Blu-ray is promoted not only for high definition television recording
and high definition video distribution, but also for high definition camcorder archiving, mass
data storage, and digital asset management and professional storage when used as a recording
medium in BD-R format.
Table 7-3 DVD and Blu-ray Specifications
Media Server
A media server refers either to a dedicated computer appliance or to a specialized application
software, ranging from an enterprise class machine providing video on demand, to, more

Adri Jovin J.J.
Page | 127
commonly, a small personal computer or NAS (Network Attached Storage) for the home,
dedicated for storing various digital media (meaning digital videos/movies, audio/music, and
picture files).
Purpose
By definition a media server is a device that simply stores and shares media. This definition is
vague, and can allow several different devices to be called Media Servers. It may be a simple
Network-attached storage, a Home theater PC (HTPC) running Windows XP Media Center
Edition, MediaPortal or MythTV, or a commercial web server that hosts media for a large web
site. In a home setting, a media server acts as an aggregator of information: video, audio, photos,
books, etc. These different types of media (whether they originated on DVD, CD, digital camera,
or in physical form) are stored on the media server's hard drive. Access to these is then available
from a central location. It may also be used to run special applications that allow the user(s) to
access the media from a remote location via the internet.
Hardware
The only requirement for a media server is a method of storing media and a network connection
with enough bandwidth to allow access to that media. Depending on the uses and applications
that it runs, a media server may require large amounts of RAM, or a powerful, multicore CPU. A
RAID may be used to create a large amount of storage, though it is generally not necessary in a
home media server to use a RAID that gives a performance increase because current home
network transfer speeds are slower than that of most current hard drives. However, a RAID
configuration may be used to prevent loss of the media files due to disk failure as well. Many
Media servers also have the ability to capture the media. This is done with specialized hardware
such as TV tuner cards. Analog TV tuner cards can capture video from analog broadcast TV and
output from cable/satellite set top boxes. This analog video then needs to be encoded in digital
format to be stored on the Media server. This encoding can be done with software running on the
Media server computer or by hardware on the TV tuner card. Digital TV tuner cards take input
from broadcast digital TV. In the USA, the ATSC standard is used. In most of the rest of the
world, DVB-T is the accepted standard. Since these transmissions are already digital, they do not
need to be encoded.
RAID
RAID - Redundant Array of Inexpensive Disks - is a method of combining several hard drives
into one logical unit. It can offer fault tolerance and higher throughput levels than a single hard
drive or group of independent hard drives.
RAID is a mature technology that speeds up data access while at the same time protecting your
data from hard disk failure. RAID is quickly becoming a necessary component in every network
since data loss and downtime can prove both fatal and financially destructive. Most networks are
designed to provide instant access to massive amounts of data. More and more employees have

Adri Jovin J.J.
Page | 128
to access customer and other databases. Intranets and corporate Web sites provide access to huge
databases online.
Raid Components and Concepts
First, let us define Logical Arrays as a split or combination of Physical Arrays, which in turn are
one or more Physical Drives that are simply the individual hard disks that comprise these arrays.
Logical Drives are then made of one or more Logical Arrays.
Mirroring refers to complete redundancy of data on identical disks. The data that is being written
on one Logical Array is completely duplicated on a similar array thereby providing 100% data
redundancy. The cost associated with mirroring is that the amount of available storage is reduced
by 50%; writes are slightly slower albeit reads are faster in some situations.
Striping refers to a technique that allows Physical Drives in a Logical Array to be used in parallel
in order to gain in performance. In this technique, data is broken down in Byte or Block levels or
stripes, where every Byte or Block is written to a separate disk in the array. Byte level can at
times be a 512-byte sector, while Block size can be selected from variety of choices. The gain in
performance is similar between Reads and Writes.
In some RAID levels, striping is combined with a technique called Parity to enhance fault
tolerance. Parity, similar to parity in memory, is simply adding a Block (Byte) of calculated
parity data to several Blocks (Bytes) in such a way that any one of the Blocks (Bytes) can be
reconstructed in case of loss, from the remainder of the Blocks (Bytes) and the parity Block
(Byte). While Parity gains from performance of striping, its disadvantages are more complexity
and loss of some disk space, which is taken up by parity information.
There are many ways to combine RAID techniques. Some standardized combinations are
referred to as RAID Levels, even though ‘Level’ in this context does not denote any hierarchy or
advantage. Levels are independent and different. Some RAID levels combine multiple other
levels to achieve certain aims.
Mirroring is used in levels 1, 0+1, 10 (1+0). Striping without parity is used in level 0, 0+1, and
10. Striping with Block level parity is used in level 5 and 6. While the minimum number of
drives required at each level are noted, there is no inherent maximum to number of drives in
arrays other than the one imposed by controllers.
RAID-0 – Striping
Simple striping is used in this level to gain in performance. This level does not offer any
redundancy. Data is broken into stripes of user-defined size and written to a different drive in the
array. Minimum of two disks are required. It uses 100% of the storage capacity since no
redundant information is written. Recommended use for this level is when your data changes
infrequently and is backed up regularly and you require high-speed access. Web servers,

Adri Jovin J.J.
Page | 129
graphics design, audio and video editing, and online gaming are some example applications that
might benefit from this level.
RAID-1 - Mirroring
This level uses mirroring and data is duplicated on two drives. If either fails, the other continues
to function until the failed drive is replaced. At the cost of 50% of available capacity, this level
provides very high availability. Rebuild of failed drives is relatively fast. Read performance is
good and write performance is fair compared to single drive read and write. A minimum of 2
drives is required. Whenever the need for high availability and vital data are involved, this level
is a good candidate for use.
RAID-2, RAID-3, and RAID-4
RAID-4 interleaves stripes like RAID-0, but it requires an additional drive just to store the
parity, which is used to provide redundancy. In a RAID-4 system, if any one of the disks fails,
the data on the remaining disks can be used to reconstruct the data that was on the failed disk.
Even if the parity disk fails, the other disks are still intact. Thus RAID-4 can survive the failure
of any of its disks. RAID-2 and RAID-3 are seldom used anymore, and have mostly been made
obsolete by modern disk technology. RAID-2 stores ECC information instead of parity, but since
all modern disk drives incorporate ECC, RAID-2 offers little additional protection. RAID-3 is
similar to RAID-4, except that it uses the smallest possible stripe size. As a result, any given read
will involve all disks, making overlapping I/O requests difficult/impossible. In order to avoid
delay due to rotational latency, RAID-3 requires that all disk drive spindles be synchronized.
Most modern disk drives lack spindle-synchronization ability, so RAID-3 is no longer used.
RAID-5 -- Striping with Parity
One of the most popular RAID techniques, it uses Block Striping of data along with parity and
writes them to all drives. In contrast to the RAID levels that write the parity information to a
single drive and use the rest of the drives for data blocks, RAID-5 distributes the parity blocks
amongst all drives, keeping parity separate from the data blocks generating it.
RAID-5 systems require a minimum of 3 disks. The impact on capacity is equivalent to
removing one drive from the array. If any one drive fails, the array is said to be degraded, and the
data blocks residing on that drive can be derived from parity and data on remainder of the drives.
RAID controllers usually allow a hot spare drive to be configured that is used when the array is
degraded and the array can be rebuilt in the background while normal operation continues.
RAID-5 combines good performance, good fault tolerance, with high efficiency. It is best suited
for transaction processing and is often used for “general purpose” service, as well as for
relational database applications, enterprise resource planning and other business systems.
RAID-6 -- Striping with dual Parity
This level is identical to level 5 except that it calculates and writes an extra parity block to all
drives. While this will have the effect of reducing the useable capacity by one more drive, it

Adri Jovin J.J.
Page | 130
reduces the window of vulnerability during the RAID-5 rebuilds considerably and can withstand
the failure of a second drive during rebuilds. The advantages of RAID-6 becomes even more
pronounced as the capacity of SATA drives go up and rebuilds take longer to finish. While
calculating a second parity has a negative impact on performance in software based RAID
systems, the effect is very minimal when hardware RAID engines that have built in circuitry to
do the parity calculations are used. RAID-6 requires a minimum of four drives to be
implemented and the usable capacity is always 2 less than the number of available disk drives in
the RAID set. Applications suited for this level are the same as those of level 5.
RAID-10 –- A Stripe of Mirrors
RAID-10 is an example of combining two RAID levels to achieve more targeted results. It is
often confused with its brethren level 0+1 that is referred to as “Mirrored Stripes.” While in each
case drives are mirrored and blocks are striped to these drives, in RAID-10, Blocks are striped to
N/2 sets of mirrored drives (N being number of drives inthe array) while in level 0+1, blocks are
striped to 2 mirrored sets each containing N/2 drives.
Because of RAID-10’s mirroring, the storage efficiency is at 50%. This level offers excellent
fault tolerance and availability. It is recommended for applications requiring high performance
and high reliability that are willing to sacrifice the efficiency (twice the number of drives to
achieve the capacity). These include enterprise servers and moderate size database systems.
RAID–50 -- Striping across multiple RAID--5’s
Also referred to as level 5+0. It combines Block Striping with distributed parity with straight
Block Striping of level 0. In other words it uses a Block Stripe of level 0 on Level 5 elements.
The minimum number of drives is 6 and the capacity can be derived from subtracting one drive
for each set of Level 5 elements. As an example, a 6-drive array would have the capacity of the
five drives. Level 50 is recommended when high fault tolerance, large capacity, and random
read/writes are required. It is sometimes used for large databases.
RAID-60 Striping across multiple RAID-6’s
Also referred to as level 6+0, combines multiple RAID-6 sets with RAID-0 (striping). Dual
parity allows the failure of two disks in each RAID-6 array. Striping helps to increase capacity
and performance without adding disks to each RAID-6 array (which would decrease data
availability and could impact performance in degraded mode).
Benefits of RAID
RAID provides increased storage capacities, and protects your important data from hard drive
failure.
There are multiple benefits of using RAID:
• Reliability and Scalability
• Real-time data recovery with uninterrupted access when a hard drive fails

Adri Jovin J.J.
Page | 131
• System uptime and network availability and protection from loss
• Protection against data loss
• Multiple drives working together increase system performance
A disk system with RAID capability can protect its data and provide on-line, immediate access to
its data, despite a single disk failure (some RAID storage systems can withstand two concurrent
disk failures). RAID capability also provides for the on-line reconstruction of the contents of a
failed disk to a replacement disk.
RAID offers faster hard drive performance and nearly complete data safety. Storage
requirements are expanding as file sizes get bigger and rendering needs get more complex. If you
handle very large images or work on audio and video files, faster data throughput means
enhanced productivity. RAID can be backed up to tape while the system is in use.
RAID Levels
The most common RAID levels are shown below in a tabular format, complete with pros and
cons and uses for the given type of RAID.

Adri Jovin J.J.
Page | 132

Adri Jovin J.J.
Page | 133

Adri Jovin J.J.
Page | 134

Adri Jovin J.J.
Page | 135

Adri Jovin J.J.
Page | 136

Adri Jovin J.J.
Page | 137

Adri Jovin J.J.
Page | 138
RAID 5

Adri Jovin J.J.
Page | 139RAID 6

Adri Jovin J.J.
Page | 140
Voice and Video over IP
Digitizing and Encoding
A coder/decoder (codec) converts an analog signal to digital form. The most common type is a
waveform coder which measures the amplitude of the input at regular intervals. The main tradeo
is between quality of reproduction and then size of the digital representation. The phone system
uses the Pulse Code Modulation (PCM) standard that takes an 8-bit sample every 125 µ (8000
times per second, 64 Kbps). Storing a 128 second audio clip requires one megabyte of memory.
There are three ways to reduce the amount of data.
• Take fewer samples per second.
• Use fewer bits to encode each sample.
• Use a digital compression scheme. This is most useful when delay is unimportant.
Audio and Video Transmission And Reproduction
Most audio and video applications are classified as real-time because they require timely
transmission and delivery. At issue is delivering a stream of data at exactly the same rate that the
sender used. The phone system introduced an isochronous architecture. The entire system is

Adri Jovin J.J.
Page | 141
engineered to deliver output with exactly the same timing as was used to generate input. If it has
multiple paths, they must all have the same delay. An IP internet is not isochronous. Variance in
delay, jitter, is especially pervasive. Protocol support is required to solve this problem. Each
transmission must contain a sequence number.
Jitter and Playback Delay
The receiver must implement a playback buffer.
A playback buffer cannot compensate for datagram loss. In that case output pauses for a time
period corresponding to the missing data. If K is too small, a small amount of jitter exhausts the
buffer. If K is too large, the extra delay may be noticeable to users.
Real-Time Transport Protocol (RTP)
RTP does not ensure timely delivery. Such guarantees must be made by the underlying system.
RTP provides two key facilities: a sequence number in each packet and a timestamp that allows a
receiver to control playback. Since it carries a wide variety of data, RTP does not enforce a
uniform interpretation of semantics. Fields in the header specify how to interpret remaining
header fields and how to interpret the payload.
The interpretation of several header fields depends on the seven-bit PTYPE field. For example,
time granularity depends on the payload type.
Streams, Mixing, and Multicasting
A key part of RTP is its support for translation (changing the encoding at an intermediate station)
or mixing (receiving streams of data from multiple sources and combining them into a single
stream). Mixing is useful for conference calls. Fields in the header identify the sender and

Adri Jovin J.J.
Page | 142
indicate whether mixing has occurred. RTP is designed to work with IP multicasting, an
environment in which mixing is especially attractive.
RTP Encapsulation
As its name implies, RTP is a transport-level protocol, but it runs over UDP. RTP does not use a
reserved UDP port number. Instead, a port is allocated for use with each session, and the remote
application must be informed about the port number. By convention, RTP uses an even
numbered UDP port. A companion protocol, RTCP, uses the next port number.
RTP Control Protocol (RTCP)
RTP allows a receiver to reproduce content. Another aspect of real-time transmission is equally
important: monitoring of the underlying network and providing out of band communication
between the endpoints. An application might choose a lower-bandwidth encoding when the
underlying network becomes congested. A receiver might vary the size of its playback buffer
when delay or jitter changes. Information could be sent in parallel with the real-time data. RTCP
allows senders and receivers to send reports to each other about the data and network
performance.
The above figure lists the five basic message types. A sender transmits a bye message when
shutting down a stream and an application specific message to define a new message type, eg, to
send closed caption information along with a video stream. Receivers periodically transmit
receiver report messages that inform the source about conditions of reception. These allow
receivers to learn about reception conditions of other receivers and allow receivers to adapt their
rate of reporting to avoid using excessive bandwidth. Control traffic will remain less than 5% of
the real-time data traffic, and receiver reports will generate less than 75% of control traffic. Each
receiver report identifies one or more synchronization sources with a separate section for each. A
section specifies the highest sequence number packet received from the source, the cumulative
and percentage loss experienced, time since the last report, and the inter- arrival jitter. Senders
periodically transmit a sender report message that provides an absolute timestamp which is the
only mechanism a receiver has to synchronize multiple streams, ie, video and audio. Senders also
transmit source description messages describing the user who controls the source, one for each
outgoing RTP stream. The contents are human readable. They must contain user@host and may
contain an email address, telephone number, geographic location, and an application program or
tool.

Adri Jovin J.J.
Page | 143
IP Telephony and Signaling
aka voice over IP. What additional technologies are needed before IP can be used in place of the
existing isochronous telephone system? There is no simple answer, but researchers are
investigating three components. Something like RTP, a mechanism to establish and terminate
telephone calls, ways an IP internet can function like an isochronous network. Signaling refers to
the process of establishing a telephone call. Specifically, Signaling System 7 (SS7) which
performs call-routing prior to sending any audio data. Given a phone number, it forms a circuit
through the network, rings the designated telephone, and connects the circuit when the phone is
answered. It also handles details such as call forwarding and error conditions (busy signal).
Before IP can be used to make phone calls, signaling functionality must be available. IP
telephony must be compatible with current standards so as to interoperate. The general approach
to interoperability uses a gateway. Two groups have proposed standards.
H.323 Standards
It was created by the ITU to allow transmission of voice over local area network technologies. It
specifies how multiple protocols can be combined to form a functional IP telephony system.
H.323 defines devices known as gatekeepers that each provides a contact point for telephones
using IP. Each IP telephone must register with a gatekeeper. H.323 allows participants to transfer
data. A pair of users engaged in an audio-video conference can also share an on-screen
whiteboard.
H.323 relies on four major protocols
with the following relationship

Adri Jovin J.J.
Page | 144
Session Initiation Protocol (SIP)
Proposed by the IETF, SIP only covers signaling. SIP uses client/server interaction, with servers
divided into two types. A user agent server runs in a SIP telephone. The second type is
intermediate and functions as a proxy server that can forward an incoming call request along the
path to the called phone or a redirect server that tells a caller how to reach a destination. SIP
relies on a companion protocol, the Session Description Protocol (SDP) which is especially
important in a conference call because participants join and leave dynamically. SDP specifies
details such as the media encoding, protocol port numbers, and multicast address.
Quality of Service Controversy
Quality of Service (QoS) refers to statistical performance guarantees regarding loss, delay,
throughput, and jitter. An isochronous network engineered to meet strict performance bounds is
said to provide QoS guarantees while a packet switched network using best effort delivery
provides no QoS guarantee. Is guaranteed QoS needed for real-time transfer of voice and video
over IP? If so, how should it be implemented?
There is a difference of opinion on these questions. IP designers insist that the Internet works
reasonably well without QoS guarantees and that adding per-flow QoS is infeasible because
routers will make the system expensive and slow. The controversy has produced many proposals
and experiments. The internet is already used to send audio. Technologies like ATM provide
QoS guarantees for each individual connection or flow. The IETF adopted a conservative
Differentiated Services (DiServ) approach that divides traffic into separate classes. It sacrifices
fine grain control for less complex forwarding and is sometimes called a Class of Service (CoS)
approach.
QoS, Utilization, and Capacity
The central issue in the debate over QoS is utilization: when a network has sufficient resources
for all traffic, QoS constraints are unnecessary; when traffic exceeds network capacity, no QoS
system can satisfy all users' demands. Proponents for QoS mechanisms assert that they achieve
two goals. They make the system more “fair". By shaping traffic from each user, they allow the
network to run at higher utilization without danger of collapse. One of the major arguments
against complicated QoS mechanisms arises from improvements in the performance of
underlying networks. Capacity has increased dramatically. As long as rapid increases in capacity
continue, QoS mechanisms merely represent unnecessary overhead. However, if demand rises
more rapidly than capacity, QoS may become an economic issue and ISPs may use cost to ration
capacity.
IntServ Resource Reservation (RSVP)
To provide QoS the IETF developed two protocols: the Resource ReSerVation Protocol (RSVP)
and the Common Open Policy Services (COPS) protocol. They both require changes to basic
infrastructure - all routers must agree to reserve resources for each flow between a pair of
endpoints. Before data is sent the endpoints must send a request that specifies the resources
needed, and all routers along the path must agree to supply those resources. Routers use

Adri Jovin J.J.
Page | 145
monitoring (traffic policing) and control traffic forwarding to ensure that traffic sent on a flow
does not exceed the specified bounds. Control of queueing and forwarding is needed for two
reasons. Routers must implement a queueing policy that meets the guaranteed bounds on delay
and must smooth packet bursts. The latter is called packet shaping. For example, a flow that
specifies an average throughput of 1Mbps may have 2 Mbps of traffic for a millisecond followed
by no traffic for a millisecond. A router can reshape the burst by temporarily queueing incoming
datagrams and sending them at a steady rate. RSVP handles reservation requests and replies
before any data is sent. To initiate an end-to-end flow and endpoint first sends an RSVP path
message to determine the path to the destination; the datagram carrying the message uses the
router alert option to guarantee that routers examine the message. After it receives a reply, the
endpoint sends a request message to reserve resources for the flow. Any router along the path
may deny the request, but if routers agree, RSVP returns a positive reply. Each RSVP flow is
simplex (unidirectional). If an endpoint requires QoS guarantee in two directions, each endpoint
must use RSVP to request a flow. Because RSVP uses existing forwarding, there is no guarantee
that the two flows will pass through the same routers.
IntServ Enforcement (COPS)
When a router receives an RSVP request there are two considerations: feasibility (are the
resources available) and policy (does the request lie within the policy constraints). Feasibility is a
local matter, but policy enforcement requires global cooperation - all routers must agree to the
same set of policies. To implement global policies, the IETF architecture uses a two-level model,
with client-server interaction between the levels. When a router receives a request, it becomes a
client that consults a server known as a Policy Decision Point (PDP) to determine whether the
request meets policy constraints. If the request is approved, the router acts as a Policy
Enforcement Point (PEP). The COPS protocol defines the interaction between a router and a
PDP.
DiffServ and Per-Hop Behavior
DiffServ differs from IntServ in two significant ways. It allocates service to a class, not a flow,
and allows each node along the path to define the service that a given class will receive. A router
can choose to divide bandwidth so that a given class known as Expedited Forwarding (EF)
receives 50% of the bandwidth. The next router might decide to give class EF 90. We use the
term per-hop behavior to describe this approach. There are no end-to-end guarantees.
Traffic Scheduling
There are four items to consider when constructing a scheduler:
• Fairness: The resources consumed by a flow fall within the assigned bounds.
• Delay: Packets on a given flow should not be delayed excessively.
• Adaptability: If a given flow does not have packets to send, the scheduler should divide
the extra bandwidth among other flows in proportion to their assigned resources.
• Computational Overhead: A scheduler must not incur much computational overhead. In
particular, theoretical algorithms such as Generalized Processor Scheduling (GPS) cannot
be used.

Adri Jovin J.J.
Page | 146
The most straightforward practical traffic scheduling scheme is name Weighted Round Robin
(WRR) because it assigns each flow a weight and attempts to send data from the flow according
to the flow's weight. Imagine three flows, A, B, and C, with weights 2, 2, and 4 respectively. It
may seem that a WRR scheduler could achieve the desired weights by selecting from the flows
in the following order: C C A B C C A B C C A B ... This does not take into account the size of
the datagrams.
Deficit Round Robin (DRR) computes weights in terms of total bytes sent rather than number of
packets. Initially, allocate a number of bytes to each flow. When a flow is selected, DRR
transmits as many packets as possible without exceeding the allotted number of bytes. The
deficit is added for the next round. Eventually, the deficit will add up to an extra packet. Round-
robin algorithms are efficient, but they have drawbacks. Delay depends on the number of other
flows, and use of these algorithms can introduce jitter.
Traffic Policing
A Traffic policer is needed to see that no flow receives more than its fair share. The leaky bucket
approach uses a counter to control the packet rate. Periodically the counter is incremented, and it
is decremented when a packet arrives. If the counter becomes negative, the incoming flow has
exceeded its allocated packet rate. A token bucket mechanism extends this approach by making
the counter correspond to bits rather than packets. These policing mechanisms consult the clock
when a packet arrives and uses the elapsed time since the last packet to compute an increment for
the counter.
MPEG-2 over IP/ATM
ATM has well-known capabilities for the delivery of real-time data, but it has not been widely
deployed in enterprise networks, except as a backbone networking technology. In carrier
networks, ATM is the technology of choice for the delivery of multiple services due to its ability
to deliver Quality of Service and high-bandwidth profile.
Many years ago the networking industry debated the advantages of ATM vs. IP. Those debates
were settled with the explosion of the Internet, and IP became the de facto network protocol.
Today, the most common use of ATM is to transport IP traffic∗. In fact, it is estimated that some
80% of all Internet traffic is actually carried over ATM, and most DSL service is delivering IP
over ATM (although the ATM layer is as invisible to the user as SONET, SDH, and other higher
layer transport mechanisms).
DVD-quality MPEG-2 video can be easily and seamlessly delivered over ATM networks using
IP, and this paper explains the basics.
MPEG over ATM
There are standards that describe MPEG-2 over ATM. In fact, the very architecture of MPEG-2
was guided by ATM’s 53-byte (48-byte payload) cell structure. MPEG-2 Transport Streams are
delivered in 188-byte sections which fit precisely into ATM’s AAL-5 layer. These standards are
promoted by the ATM Forum, DAVIC, and were adopted by early MPEG over ATM systems.
The main advantage of direct MPEG-to- ATM mapping is that it takes advantage of ATM’s
ability to deliver Quality of Service in order to deliver very robust performance. The

Adri Jovin J.J.
Page | 147
disadvantage is that there is no IP layer, and therefore deployment requires a complete end-to-
end ATM solution.
MPEG over ATM structure
Such a solution is fine for homogeneous ATM networks, but it eliminates the possibility of
delivering that video to Ethernet-connected devices such as PC’s or other IP-connected devices.
MPEG over IP/ATM
MPEG-2 video that is formatted for delivery over UDP/IP can take advantage of the existing
standard, well-supported mechanisms for the delivery of IP data over ATM. For example, a
VBrick connected to an OC3c ATM switch can interoperate with a VBrick or PC that is
connected to the network via Ethernet. Furthermore, ATM-connected devices can be managed
with SNMP, a web browser, or Telnet only when the device encapsulates IP into ATM, because
the management occurs at the IP layer. There are several methods for sending IP traffic over
ATM. Some of these methods encapsulate layer 2 data (RFC 1483 bridging) and some
encapsulate layer 3 data (IPoA). These encapsulation processes are considered layer 2 or layer
2.5 protocols, as layer 3 and other layer 2 protocols rely upon them for transport over ATM.
IPoA, Classical IP (RFC 1577) -- User data in the form of IP packets (Layer 3) is encapsulated
into AAL-5 PDUs for transport over ATM per the RFC-1483 Routed method. The fact that the
user data is routed at the IP layer instead of bridged at the MAC layer allows the source and
destination to be on different subnets. This is the preferred standard for the delivery of IP MPEG
data over ATM PVC’s and is supported by VBrick MPEG-2 products.
(RFC 1483 Bridged) provides a simple method of connecting end stations over an ATM
network. User data in the form of Ethernet packets (Layer 2) is encapsulated into AAL-5 PDUs
for transport over ATM. This method is simple, but since it operates at the MAC layer, routing is
not provided and there is no exposure to the IP contents carried.
LAN Emulation (LANE), as the name applies, is a method that causes connection-oriented
ATM circuits to appear as a shared-media LAN. In the simplest case, computers connected to a
ATM network run a special software client (LAN Emulation Client, or LEC). To the computer,
the LEC “looks” like an Ethernet segment. To the ATM network, the LEC “looks” like an ATM
client. The LEC communicates with a LAN Emulation Server (LES) and LAN Emulation
Configuration Server (LECS) that must be present in the ATM network. Another component is
the Broadcast Unknown Server (BUS), required for the ATM network to send data to more than
one point at the same time. In operation, the components automatically setup and tear down

Adri Jovin J.J.
Page | 148
ATM circuits as needed, and effectively providing a connectionless service (IP) with a
connection- oriented technology (ATM). Today, much of the complexity in getting these
components to work together has been eliminated, and LANE works quite well. One
disadvantage to LANE is that the commonly deployed version does not support wide scale IP
multicasting, although some implementations do. Typically, IP multicast traffic will be sent via
the BUS (performance issue) to all users of the network (traffic issue). Because LANE
establishes point-to-point (a so called “data direct”) connections after establishing a call, LANE
can work very well for the unicast transport of video over ATM.
VBrick IPoA
VBrick’s MPEG-2 Video Network Appliances send MPEG-2 over ATM using IP Over ATM
(IPoA). As a result, a VBrick connected to an ATM network can interoperate with a VBrick
connected to an Ethernet Network, enabling universal video networking. For integrated ATM/IP
Video Networking, the conversion of IP Over ATM to Ethernet, if necessary, occurs within the
network:
• An ATM port on a router that supports RFC-1577 will accept the IP data and deliver it as
native IP packets to other interfaces on the router.
• Edge Switches that have an ATM uplink and Ethernet user ports provide internetworking
services. An ATM-connected VBrick is fully interoperable with an Ethernet-connected
VBrick.
• ATM-to-ATM VBricks are supported via ATM PVC. The IP layer is invisible to the
ATM network.
Service providers, cable operators, and content aggregators benefit from IP over ATM because,
at some point along the delivery chain, the network infrastructure will presuppose IP. Some
Digital Subscriber Line Access Multiplexes (DSLAM) accept ATM, making it possible to
provide higher density Head End solutions with fewer inside ports while maintaining the QoS
necessary for premium content delivery.
Architecture
The VBrick IPoA facility takes advantage of VBrick’s integral IP routing function. This function
exchanges RIP tables with other routers, making it virtually transparent to the user and enabling
seamless IP connectivity where each ‘port’ of the router may be a unique IP network.
The block diagram illustrates how the IPoA works.

Adri Jovin J.J.
Page | 149
Each MPEG encoder or decoder may be configured with a different IP address, and a routing
facility can direct that traffic to an appropriate ATM VPI/VCI where the IPoA and AAL-5
encapsulation occurs. Similarly, the local Ethernet Data Port supports native IP data, allowing
high-speed access to the Internet and other common network resources. The ATM Virtual
Circuits are delivered over the ATM OC3c interface.
VBrick Video Network Appliances can be configured with multiple address assignments, to
concentrate IP traffic across the ATM network. The VBrick internal routing function
automatically knows which ATM interface is most appropriate to use based upon routing
knowledge leaned through the RIP processing. VBrick can seamlessly convert between Ethernet
and ATM IP layers, and automatically extract video and management IP packets. The local
Ethernet interface port may be used as a data access port, such that IP data presented to it is
automatically routed to a particular ATM PVC.
Importantly, VBrick allows the MPEG-2 video to be delivered on any valid ATM Virtual
Circuit, and allows the same or different ATM Virtual Circuits to be used for various
combinations of video or data sources. For example, you might use:
• one VC for receiving and a different VC for sending
• one VC for encoder #1 and a different VC for encoder #2
• one VC for all video
• one VC for Ethernet User Data and a different one VC for video
ATM Advantages
ATM, and IPoA has several important advantages. First, ATM networks are widely supported
and available from public carriers and within large corporations.
• An ATM PVC may be configured with certain Quality of Service parameters, and the
ATM network enforces these parameters on an end-to-end basis. To the extent that
MPEG video is transported over a particular PVC via IPoA, the IP layer inherits the QoS
for that PVC. This means that even though IP is used, ATM’s robust support for low
delay, high bandwidth service is preserved though a public or private portion of any
ATM network.
• ATM establishes virtual circuits within a physical media (e.g. a fiber). Because each
virtual circuit may have its own unique bandwidth characteristics, and because VBrick
enables video or data traffic to be sent over IP on different ATM VC’s, Quality of
Service is maintained even though it is sent via IP. Mission-critical data applications,
voice, web traffic, email, etc. cannot interfere with the video traffic and vice-versa.
• IP over ATM also enables end-to-end QoS in a Multi-Protocol Label Switching (MPLS)
network, since VBrick supports DiffServ. VBrick’s IP traffic can be mapped into MPLS
flows without additional processing because the video packets are tagged with DiffServ
priority at the source.
• ATM operates over fiber, providing extended reach. Inexpensive multimode fiber is
typically used for distances up to 2 km, and single mode fiber for distances to 40 km.

Adri Jovin J.J.
Page | 150
VBrick Advantages
VBrick’s MPEG-2 products provide significant advantages for the delivery of MPEG-2 over
ATM:
• Seamless ATM to Ethernet, end-to-end, one-way or two-way DVD quality television
• Quality of Service preservation
• Elimination of network edge devices for ATM-to-IP conversion
• Reduced complexity
• Professional Composite, S-Video and SDI video
• High Density (Dual Encoder)
• Integral Web Server
• Standard MPEG-2 Transport Streams, interoperable with VBrick decoders, desktop
software decoder (StreamPlayerII), and 3rd party Set Top Boxes
• Highest Value Proposition in the industry
Recording and Remote Control
Experiences of interactive Internet multimedia applications, especially conferencing, indicate
that there is a need for a mechanism to record and playback the media of these conferences.
When recording, both audio and video streams need to be archived, and possibly the shared
workspace, with the possibility of having stream synchronisation. As some conferences only
require a subset of the conferees to be recorded, the record mechanism should allow a selection
of the streams to be specified. The retrieval of recorded material needs to be easy for users to
access, either for direct playback to the user or for inclusion in another multicast multimedia
conference. Users also need to see which conferences have been recorded and are online for
access in order to select streams to play. As real-time multimedia applications generate vast
amounts of data, any recording system must therefore be fully equipped with large storage
capabilities.
To meet the above requirements, the objectives of a recording and playback system need to be:
• the ability to record multicast session data whose source may be any of the conferees;
• the ability to playback recorded material either directly to one user or into another
multicast session;
• to provide synchronisation between streams from each source;
• to allow users to create their own segments of recorded material, and create their own
material for playback;
• to provide a single point of contact for recording and playback;
• to supply a large repository of archive space which is accessible to authorised network
users who wish to record or playback multimedia data;
In the next section we discuss previous attempts at recording and playback of multimedia
conferences, and see how they meet the above objectives. Then the design of a system to achieve
the above objectives is presented, showing what is actually required for recording, playback, and
editing of multimedia streams. As the media streams are large, it is pertinent to provide indexes

Adri Jovin J.J.
Page | 151
into these streams. Multimedia indexes are a novel approach to accessing such data, and these
techniques will be discussed later.
After this, we describe the Real Time Streaming Protocol, which is becoming the standard for
remote control facilities for multimedia servers.
Recording
First we consider multimedia recording and playback systems which are not specifically
designed with multimedia conferencing in mind. Most of these aim to be video-on-demand
systems. Some of them aim to produce high quality, synchronised video with attention paid to
playback strategies in order to generate 30 frames per second of media. The designers of such
systems are interested in high speed discs and storage structures using, for example, RAID disc
technology in order to get high volumes of video data from storage to the user. Other work
focuses on designing models for presentation of media, and has been be extended into the arena
of distributed presentation of media, admission control for multiple users, and the mix of both
analogue and digital components within the infrastructure.
Using IP Multicast
Within a multimedia conference the media tools utilize the IP multicast mechanism by having
every instance of a tool join the same multicast address. For each media there is a unique address
and port pair, on which the tool is started. For example, the addresses used by MICE conferences
are shown in the table 9.1 below
Media Address
audio 224.2.17.11/3456
video 224.2.17.11/2232
whiteboard 224.2.17.11/32415
Table 9.1: Multicast address use in session
Note that here the address is the same but the port changes. In other conferences both the address
and port change. Furthermore, each tool may use more than one port, which is not always
advertised. The tools send data to one port and non-core data to another port. The non-core data
usually consists of session information such as the user name of the sender or some statistics
regarding the quality of the received data. These session messages are sent at low bandwidth as
they are secondary to the real data. It is most common for the session message port to be one
higher than the advertised port number. So, for the previous examples we would see session
messages for audio on port for video on port and for whiteboard on port
To record data from a multicast conference, the recorder need not be an active part of the
conference by executing any of the conference tools, but it must attach to that conference to
enable the multicast data to reach the recorder and hence be collected. As data for each media
within a conference will be seen on a different address and port pair, the recorder must

Adri Jovin J.J.
Page | 152
accommodate this. The collected data for each media can then be saved in an archive for later
use. To playback the recorded data to a multicast conference, a player must join the multicast
address for each media in the conference to enable the data from the archive to be sent into the
conference. Again, each media's data will go to a different address and port pair. In figure 9.1 we
see that before a recording or playback, the recorder or player respectively, is independent of the
conference, and that the data packets and session messages are on different addresses.
Figure 9.1: The recorder is independent of the conference
In figure 9.2 we see that the recorder or player has attached to the conference in order to collect
the data from each media.
Figure 9.2: The recorder attaches to the conference to record the data
Current Media Tools and Protocols
Multicast conferencing tools have been written and improved continuously by their developers
over the past three or four years, and new tools are still appearing.

Adri Jovin J.J.
Page | 153
As described in chapter five, the media tools send media data inside RTP packets on one
multicast address, and session message information inside RTCP packets on another multicast
address. For example, the audio tool used by each conference participant sends audio data in
small chunks of about 20 ms duration. Each chunk of audio data is preceded by an RTP header;
the RTP header and data are in turn contained in a UDP packet. The RTP header indicates what
type of audio encoding, such as PCM, ADPCM or GSM, is contained in each packet so that
senders can change the encoding during a conference, for example, to accommodate a new
participant or from indications of network congestion. Similarly, the session message data which
uses the RTCP format is also contained in a UDP packet.
As all media data and session messages are sent within UDP packets there is no guarantee of
packet delivery to the receivers. Due to the nature of busy packet-switched networks, some
packets get seriously delayed and others are lost in transit. To accommodate this most media
tools have a playout mechanism that uses and adaptive playout buffer in an attempt to hide these
effects from the end user. In some circumstances the media tool cannot cope and the media is
presented to the user with loss.
Reliable Multicast
Many IP multicast tools use a reliable multicast mechanism to maintain consistency between
each instance of the tool. Reliable multicast is a mechanism by which data which is multicast
from a sender is guaranteed to reach the receivers at some time. The reliability mechanism
requires some form of acknowledgement scheme to say which packets did arrive and did not
arrive at a receiver, a kind of retransmission scheme to send out lost or late packets, and support
within the tools to participate in these schemes.
For media such as video and audio, data arrives so fast it is better to present it quickly and
tolerate some reduction in quality if packets do not arrive on time, or are lost in transmission.
This can be done because humans are quite tolerant of loss, particularly for video, and the
information content is not drastically reduced if loss does occur. Furthermore, with a reliable
transmission scheme late or lost data could be resent to the receiver at a significant amount of
time after the original send time, which would cause the immediacy of any interaction between
users to be eliminated. Clearly, this is not suitable for live, interactive conferences. However, this
level of reliability is very important for shared workspace tools where whole objects need appear
with all their data eventually, but where there is no immediacy lost if a packet is retransmitted
and arrives at some later time.
Both of the well known shared workspace tools, wb and nt, use a reliable multicast mechanism.
The scheme used by Wb is described in and that for Nte is described in Both schemes resend
parts of objects that get lost on the network. The RTP protocol is not used because the data is not
real-time data, but partial-object data. In nt, and possibly wb, the RTP timestamp does not have
enough resolution to hold all the timing information required. Therefore both nt and wb have
their own packet format to support their reliability and retransmission mechanisms. It is
unfortunate that the schemes are not described in enough detail to build a new tool that could
participate in these schemes.

Adri Jovin J.J.
Page | 154
A Multimedia Recording Server
To allow easy deployment, the interface to playback systems should be made compatible with
WWW browsers, such as Mosaic or Netscape. This enables the media to be selected and
replayed using a GUI, and this approach avoids writing a special playback client but requires the
server playback interface to look like a WWW server. However, using a naive approach, the user
is able to playback the whole conference, but is not able to select their own start point or end
point.
Many of the problems associated with the delivery of multimedia to end-users are issues of
synchronisation. It has become apparent that synchronisation is more complicated than one may
think. We have determined that there are at least three kinds of synchronisation that are
important for media synchronisation. They are:
lip synchronisation
where the movement of the lips as seen on the video and the delivery of the audio persuade the
user that the person is really speaking at that time.
time synchronisation
where a specification of time, either relative or absolute, goes to the same point in time in each
stream of each media, regardless of the volume of data in the streams.
inter-packet synchronisation
where packets are presented to the network, by a player, in such a way that they arrive at the
media tools in the same order and inter-packet gap as when they arrived at a recorder. Where
more than one stream is being played some kind of inter stream multiplexing is required to
maintain this synchronisation.
It is desirable to have a unified approach which makes the tasks of recording and playback
simple to achieve, and overcomes the problems just discussed. By considering the problems of
the existing tools, it is apparent that a system that does recording and playback must have:
1. a tool for recording multiple media within the same conference
2. a tool for playing multiple media into a new conference
3. user specifiable streams on recording or playback
4. synchronisation between the media streams
5. a single point of contact for accessing the recordings
6. a large repository of disc space for the recordings

Adri Jovin J.J.
Page | 155
Server Interfaces
Due to the multiple but specialized functionality required, a server has different interfaces. The
main include interfaces for recording, playback, and editing are discussed here, but the design
allows the server to have as many specialized interfaces as required.
Recording
The record interface of the server allows a client start a recording of any of the media used in
conference, to specify the duration of the recording, to specify one, many, or all of the source
streams, and to add a title and text description of the recording. If more than one media is
selected for a source, then the user may choose to have time-based synchronisation between
those streams at the server. This will enable synchronisation within the playback mechanisms.
The server system collects all or some of the streams of media for a conference, and a client
allows the user to select which media to record and then instructs the server to record them. The
client does not need to be part of the conference. With the server listening to the media tool
session messages or observing the source addresses of incoming data each media of a
conference, the server can decide who is in each conference. Another feature of the server is the
ability to allow synchronisation between the incoming streams. In this way playback can be
presented to the user with the media streams for a source being synchronised.
A client connected to the record interface allows the user to select a media to record, and then
instructs the server to record that stream. The server can inform its clients who it is currently
recording a conference in order to stop more than one client attempting to record the same media
within that conference. It is expected that a new client will be started for each different multicast
conference. Figure 9.3 shows that the media tools send and receive data via IP multicast, and that
the server under instruction from the client receives that multicast data and records it.
Figure 9.3: Client informs server which collects multicast data
Using this client/server configuration, many sites may have servers which are able to record and
playback conferences. These sites may choose to record data for any authorised user on the

Adri Jovin J.J.
Page | 156
network, not just for local users. It is not necessary any special designated site to be the
conference recording hub for the whole of the MBone. With full deployment, there could be a
large collection of servers on the Internet. A client can connect to a server anywhere on the
network to request a recording.
Playback
The playback interface of the server allows a client to peruse all the online archives, to allow the
selection of one or more media and one or more streams for playback, to specify the destination
conference for the playback, and to control the media during playback.
For playback a different client interface is used to access the server. The server presents this
client with information showing which conferences have been recorded, which streams are
available for playback, and which conferences are currently being played back and on which
address.
The playback of the recorded data can be presented to the user in various ways:
1. the user can choose to play a stream or many streams from an old conference into a new
conference. In this scenario, all members of the conference can see the played-back
streams.
2. the user can choose to see a stream or streams presented to his workstation only. In this
scenario only the user sees the streams.
3. the user can choose to play back a recording as a presentation to a small group of people.
It would be possible to make the playback interface compatible with WWW browsers such as
netscape or Mosaic. This would enable the media to be selected and replayed using a GUI, and
avoids writing a special playback client but requires the server playback interface to look like a
WWW server by using the http protocol [#!http!#] As we have already seen a mechanism similar
to this has been successfully deployed by Anders Klemets of KTH in Sweden. The approach
suggested here does require the playback interface of the server to look like a WWW server.
It may be possible to have a client that combined playback with recording in order to have a GUI
that allows both recording and search and access to pre-recorded data. This may be desirable if
one needed to provide an interface that looked like a traditional tape recorder. In this instance the
client would connect to more than one server interface.
In some instances it may be desirable to have a full conference replay. As the server is able to
provide synchronisation data for the media streams, then during playback it is possible to have

Adri Jovin J.J.
Page | 157
all the media streams synchronised. The synchronisation means that the data is presented in near
real-time rather than at an arbitrary rate which is dependent on the speed of the machine.
Editing
The editing interface will allow the user to make copies of a whole media stream or to select a
segment of a media stream in order to create their own personal archives. When editing, the user
will be able to create a text description of arbitrary parts of the media to aid playback selection.
Further editing tools can be used to combine pre-selected segments of media, from possibly
different conferences, into new presentation material. The new presentations could then be used
as lecture material, for summaries of work, or for presentations of project work. As these newly
created presentations will be independent of one another it allows multiple uses of one recording
and thus allow a more flexible use of the whole archive.
A full playback of a video conference can be time consuming and often not what is required. It is
envisaged that a set of media editors be designed to enable each media to be made presentable
for different uses. This will allow for:
• text annotations to be added at multiple positions in a recording
• the creation of conference summaries
• the elimination of quiet sections
• the creation of lecture material
• the creation of special presentations
The editing functions required are the addition of annotations, the selection of segments, and the
ability to do post-processing of the data.
Addition of Annotations
The addition of annotations allows users to add meta-data to a recording. These annotations can
take on many forms, such as text, to interface to browsing and selection tools; HTML data or
URLs, to interface with the World Wide Web; content classification data, to interface to search
engines; start of video frame or start of speech burst markers, to interface to special player; and
so on. The addition of text descriptions allows users to specify what is contained in different
sections of a recording. These descriptions can be presented via the playback tool, and allow the
user to play from a specified point. Figure 9.4 show a stream with three added text annotations.

Adri Jovin J.J.
Page | 158
Figure 9.4: Having multiple text descriptions for a media
Segment Selection
The editing process allows segments of a recording to be selected and saved for later use. These
segments are independent of the original recording and can be manipulated freely for other uses.
Some related techniques to the one suggested here are described by Hampapur in Figure 9.5
shows that some segments that have been selected from the original media stream. One selection
has three small segments, whilst the other has one larger segment. This results in three instances
of media for the one recording; the original media, the 3 small segments, and the larger segment.
Each can be used independently of the others.
Figure 9.5: Selecting segments from a recording
For each type of media, and for each different format for that media, a different segment
selection editor may be required. For example:
• For H.261 video, when using or an editor could select on Macro Block (MB) or Group Of
Block (GOB) boundaries closest to the selection made by the user.
• For audio, the selection boundary could be on either each audio sample or on each
talkspurt
• For shared workspace applications, such as the whiteboard, an editor may not be useful if
selections were done on a time-slice basis. Both video and audio are continuous media
whereas the whiteboard is a non-continuous media, such that selecting data from a time
period would give a random set of objects. Selection would only be useful on an object-
by- object basis and therefore some kind of post-processing is required.
When using stream synchronisation, the segments of the media should be synchronised such that
when one media segment is selected an equivalent part of some associate media is also selected.

Adri Jovin J.J.
Page | 159
Figure 9.6 shows that the user is able to select segments (shown in grey) of video and audio
which are equivalent with respect to the synchronisation information stored by the server.
Figure 9.6: The synchronisation of 3 segments of audio and video
Analysis and Post-Processing Tools
Although the server maintains all the recording information, and has interfaces to allow
recording, playback, and editing, this functionality is often not enough. For extended use of the
data it is beneficial to provide tools for analysis of the media streams held in the server, and tools
for post-processing the media streams in order to modify them
These tools are different for each kind of processing and each kind of media.
With audio, for example, the tools envisaged are (i) for an analysis to graphically present the
temporal aspects of the audio for each user in a conference, and (ii) for an analysis to determine
the percentage of audio sent by each user in a conference. The analysis data can be used to delete
audio from some users in a recording.
With video, for example, it is envisaged that the media streams are post-processed to generate
more intra frames than exist in the original stream. Having the extra intra frames enables
playback from arbitrary places in the stream. With analogue video a new whole frame is sent to
the display every 25th of a second. Due to the way H.261 digital video is compressed using intra
frames and inter frames, whole new frames are rarely (if ever) sent to the display. These intra
and inter frames are the differences from the previous image. A whole frame is reconstructed
from the differences of the image contained in the intra and inter frames.
For shared workspace the post-processing can reconstruct workspace objects from the stream of
sub-objects contained in packets.
This list is not exhaustive, but is meant to give a flavour of the kind of processing required.
Clients
The clients of the multimedia server initiate the recorded, playback, or editing for the user. Each
interface uses a TCP stream in order to allow the building of both simple text based user
interfaces, based on the telnet model, or graphical user interfaces based on windows. Each client
can be designed to have a specialized task such as recording or playback only by connecting to
the relevant server interface. It is possible to build more complex user interfaces by having the
client connect to more than one server interface and provide compound functionality for the user.

Adri Jovin J.J.
Page | 160
Multiple clients can connect to the server, and clients can connect to as many interfaces as they
require. There is no specific limit within the server pertaining to the number of client
connections.
Server Storage
The hardware available for the media include CD and DVD jukeboxes plus some large magnetic
discs. The hard discs are used as fast, temporary storage within the server, whereas the CDs are
used for mass storage. When recording, media streams are sent from the network to the magnetic
discs, and then onto CDs for permanent storage. When replaying data, the media which is stored
on CDs is taken from the CD and sent to the magnetic drives for subsequent playback to the user.
To manage this hardware and to present media streams back to the user, some software is needed
to select CDs from the CD store and place them into the CD players of the jukebox.
A jukebox is a large collection of CD discs in a box with a few CD drives. These jukeboxes have
a huge capacity, going up to Terabytes. We have seen that a video stream with a high
compression ratio may generate over 1 Gb of data an hour, therefore these jukebox devices seem
to be ideal for video and audio because of their large capacity, however their throughput could be
a limitation. Currently a high density CD has a capacity of 650 Mb per side. Figure 9.7 shows
how the discs, the network, and the server software are connected.
Figure 9.7: Storage Devices for Multimedia on Demand Servers
Given that an hour of compressed video can be expected to require about 240 Mb of storage per
source, it is possible to evaluate the amount of storage space needed by an average conference
recording. From this figure we can extrapolate to determine how much storage space such a
system will require to be effective.

Adri Jovin J.J.
Page | 161
In terms of cost, as of December 1995, a 9 Gbyte disc drive costs about 1,500, and a fully
populated jukebox with 144 platters costs about 85,000. With a 9 Gbyte disc drive one gets 6
Gbytes for 1,000, with a jukebox of 144 platters, each with 650 Mbytes per side, giving a total of
172 Gbytes, one gets 2 Gbytes for 1,000. Clearly there is a factor of 3 in the cost of owning a
jukebox, and to replicate this amount of storage space on magnetic media with just 20 magentic
drives would only cost 30,000. The main benefit of the optical device is that there is no need to
buy extra backup devices and media.
Playout volumes
A real service needs to accommodate a large configuration, with a large optical jukebox, a fast
workstation front-end, plus a fast disc cache or just a workstation with a few Gbytes of disc
space.
For a larger configuration with a jukebox we can expect to store more low bandwidth media than
high bandwidth media. A jukebox with 144 platters, each with a total capacity of 1.2 Gbytes has
a total storage area of 172 Gbytes. The amount of media which can be stored is:
• for a video stream which is presented at 2 Mbps then 500 seconds of video would use 1
Gb. Therefore, in 172 Gb we could store 86,000 seconds worth, which is about 1,440
minutes or 24 hours.
• for a video stream which is presented at 128 Kbps then 8,000 seconds of video would use
1 Gb. Therefore, in 172 Gb we could store 1,376,000 seconds worth, which is about
23,000 minutes or 380 hours or 16 days.
• for a video stream which is presented at 32 Kbps then 32,000 seconds of video would use
1 Gb. Therefore, in 172 Gb we could store 5,504,000 seconds worth, which is about
92,000 minutes or 1530 hours or 64 days.
The most common configuration is expected to be a system in which a workstation with a few
Gb of disc space can be used as a server. Such a system could still expect to store many hours of
audio and video.
The factors to consider for the storage requirements of a large server are related to (a) the total
amount of physical data there is, and (b) the volume of data that needs to be recorded or played
in either Kbps or Mbps. For the amount of physical data we need to consider file size, partition
size, disc size as well as file structure; for the volume of data we need to consider: disc speed,

Adri Jovin J.J.
Page | 162
disc bandwidth, network speed network bandwidth as well as read/write delay seek times, platter
load times.
A full investigation of storage for high bandwidth video-on-demand servers can be found in
[#!cherve!#]. In this section there is just an overview of the issues that need to be considered for
the storage requirements of the server.
Size and Structure
For storing large media streams, the use of a database which is amenable to continuous media, or
a special video filing system [#!anders!#] seems pertinent. Using a traditional relational database
is not suitable as these RDBMS are optimised for indexed tables of small data items.
Use of raw partitions
Here we consider why a server may choose to use raw partitions on a disc rather than going
through the filing system mechanism. The potential reasons for using a raw partition rather than
a UNIX filing system are:
• UNIX cannot create files greater than 2GB
• UNIX generally cannot address disc drives greater than 2GB, but this limit has been
increased under Solaris
• when creating a large file the indirection, double indirection, and triple indirection to data
blocks is an unnecessary overhead.
• because video archives are so large we would not get many archives on a maximally
large filing system.
• the overhead of i-nodes, superblocks, etc. are unnecessary
• the server may need to map an archive address which may be greater than 2GB into a
disc specific address; having access to a raw partition will aid this.
It is pertinent to note that many relational database management systems, such as Oracle, use raw
partitions to increase performance, because the filing system is too much of an overhead.
To fully evaluate the use of raw partitions on any particular architecture would require some tests
to be undertaken to determine the speed difference between a the data transfer of a huge file
representative of a video archive in a filing system and the data transfer of an equivalent amount
of data from a raw partition. These tests would confirm if double and triple indirection really is
very expensive, or if the caching and read ahead techniques used by the file system make the
speed difference between a raw partition and a file system negligible.

Adri Jovin J.J.
Page | 163
Use of a Jukebox
There are many approaches to managing the jukebox. Something like a Hierarchical Storage
Manager can make many CDs look like one big partition. However, as each high-density CD
holds about 650 Mb per side, and UNIX generally allows a partition to have a maximum size of
2 Gb, it only takes 3 CD sides to make a maximum sized partition. To store video and audio
requires partitions of hundreds of gigabytes.
Current jukebox hardware and software supports a range of the features required. However,
because some of the other features are not suitable, it may be desirable to write a new jukebox
manager. In particular, most jukebox software makes the jukebox look like a collection of NFS
discs, with each disc having a UNIX file system. For video archiving a different model is
required. There will be far fewer files, but their size will be huge. are:
Speed and Delay
To fully evaluate a hardware configuration the speedy and delays of the relevant devices needs to
be considered, particularly, their actual performance rather than their nominal manufacturer
specifications. Currently, magnetic devices are significantly faster and have significantly lower
latency than optical devices.
Magnetic drives
In the Video Conference Recorder magnetic drives are used as a front end to the CD jukebox.
These magnetic drives have fast access times and high bandwidth in comparison to CD
jukeboxes which have slow access times and low bandwidth but very high storage capacity.
Currently high-end disc drives such as 9 Gbyte drives are cheap.
Jukeboxes
Currently a high density CD has a capacity of 650 Mb per side. As a video stream may be more
than 650 Mb, the server needs a mechanism to load a stream from many CDs onto the magnetic
media. When changing from one CD to another there is a delay. The server mechanism should
aim to hide the CD changeover time from the user. If the changeover time is not hidden, the
media streams would stop until the new CD was loaded. The magnetic discs provide a buffering
mechanism whereby media data can be loaded from CDs onto the magnetic discs and played out
to the user with an initial delay. This delay can be large enough to hide the changeover time.
When changing CDs, the number of CD drives within the jukebox affects the performance of the
server. Consider the process of accessing some data on a CD if there are:

Adri Jovin J.J.
Page | 164
2 drives
When using two drives it is possible to read from a CD in one drive and to load another CD
into the other drive in advance of its use. When one CD is completely read, then access to
another CD is just a case of flipping to the other drive. The CD in the original drive can now be
replaced with a new CD. The delay is the time it takes to swap drives.
1 drive
When using one drive, the data is read from the CD until completion. Then there will be a
delay whilst the CD in the drive is removed and put back into the jukebox. A new CD has to be
chosen from the jukebox and placed into the drive. The delay is relatively large.
As stated, the use of front end magnetic discs can hide some of the delay by doing large amounts
of buffering.
Indexing Techniques
There are various facilities for utilizing the data which enhance the basic recorded material;
namely the selection of segments, the addition of annotations, and the post-processing and
analysis of data. Furthermore, as the data sets are real-time multimedia, they are rather large. To
enable the flexible use of the recorded data and to enable access to these large data sets, the
concept of the real-time multimedia index has been devised.
These indexes allow access to the source streams in a multitude of ways. The primary index used
is created when a stream is originally recorded. For each source of a media, the incoming data
will be saved together with an entry in this index. Each index entry contains a reference to the
data, the time it arrived at the recorder, and a reference to some meta-data, which will initially be
empty. At the end of a recording, each source will have a stream of data and a stream of index
entries. It is the editing client which allows the user to manipulate these indexes in order to gain
the flexibility required. For example, the user can add a text annotation to any part of the
recording, and the server will attach this to the source stream in the relevant place in the index by
updating the meta-data field of an index entry.
These indexes are different but complementary to Rowe's indexes [#!rowind!#] which are used
in his Continuous Media Player. His indexes are for content categorisation and are used to aid
users in searching particular material in the database. For example, in Rowe's system the user
might ask: List all videos about dinosaurs, and the system would search the category index for
dinosaur. Similar work in the categorisation area has been done by Niblack et al, who build
indexes based on the colour, the texture, and the shape of the content [#!nibla!#] Also
complementary is the work by Hampapur on Video Data Management Systems [#!hampa!#]

Adri Jovin J.J.
Page | 165
Other kinds of index devised have been used for finding the site at which material is held in a
distributed multimedia system.
The index is one of the main design concepts and they allow access to the multimedia data in a
multitude of ways. When a stream is originally recorded is index is created . This is called the
primary index and used for access to the stream of data. For each source of a media, every
component of the incoming data will be saved together with an entry in the primary index.
An important factor of indexes is that the kind of data being indexed is not important nor is
neither its size nor its frequency. Implementors can write specialised recorders and players for
each specific data kind, all of which utilize, but are independent of the index itself.
Index Approach and Implementation
An index has a name together with four structural components:
1. the data
2. the index elements
3. some annotations associated with the data
4. some header information
Each index has a header file that contains information such as where to find the data, where to
find the index elements, where to find the annotations, the type of the data being indexed.
Each index entry contains a reference to the data, the time it arrived at the recorder, and a
reference to some meta-data annotations, which will initially be empty. At the end of a recording,
each source will have a stream of data and a stream of index entries (called the index track). This
is shown in figure 9.8.
Figure 9.8: The structure of an index element

Adri Jovin J.J.
Page | 166
Each annotation in the annotation list is an attribute-value pair, where the attribute is a name and
the value is any data, either text or binary, that is associated with the particular index element.
Each element of the attribute list contains the following data:
• the total number of bytes in the attribute.
• some status information, such as ACTIVE or INACTIVE for the attribute.
• the number of bytes in the name
• the name of this attribute
• the number of bytes in the value
• the value of this attribute
This structure is show in figure 9.9.
Figure 9.9: The structure of an attribute element
Using Indexes for Annotations
After a recording has been completed it can then be manipulated using the editing tools. In figure
9.10 we show an example where there are 3 text annotations for a stream of a recording. The text
annotations are linked into the recording by the meta-data annotations of an index entry.
Figure 9.10: The data for a source and its index
The primary index provides a simple and fast access mechanism to any part of a recording.
Any kind of annotation can be added to any point of a recording, and any index element can have
multiple annotations for each index element. The annotations are independent of one another,
and can be added at arbitrary times. Because no meaning is imposed on the annotations by the
indexing mechanism itself, their use is application specific. The annotations could have values
which are text, HTML, URL's, PostScript, references to other indexing systems, and many more.

Adri Jovin J.J.
Page | 167
Using Indexes for Analysis and Post-Processing
It is possible to create tools for both the analysis of the media streams and the post-processing
the media streams in order to create new modified streams. For video it is envisaged that the
media streams are post-processed to generate periodic full intra frames in order to provide
multiple entry points into the inter coded original stream. This will enable playback from
arbitrary places in the stream.
With analogue video a new whole frame is sent to the display every 25th of a second. Due to the
way digital video is compressed using intra frames and inter frames, whole new frames are rarely
(if ever) sent to the display. These intra and inter frames are the differences from the previous
image. A whole frame is reconstructed from the differences of the image contained in the intra
and inter frames. Our technique will have the benefit of having a full image at all times during
playback. Without having these full inter frames it will be possible to replay the video from an
arbitrary place but only see a few flickering blocks of changing inter coded video. Furthermore
as this technique will generate whole images at a regular frequency it will be possible to view a
video sequence in a fast- forward mode.
When using video data compression, such as the H.261 encoding, the encoded data stream
contains changes from the previous image. The changes take the form of Inter frames and Intra
frames. Inter frames are used for small local changes, and Intra frames are used for large
changes.
By doing an analysis on the Intra frames it is possible to determine when large changes have
occurred within a video sequence because a large change in the motion or a scene change will
cause the encoder to generate Intra frames. The analysis could determine where Intra frames
occur within a compressed video stream and automatically create a new index into that video.
For fast perusal video, the compressed domain again provides some benefits. As it is known
where scene changes or major motion events occur, then an indexing mechanism could be
devised based on this information which will create a new index with references only to the
specified points. When such an index is created, the playback mechanism could go to each
indexed point in the video and play back a few seconds to the user, and then skip to the next
index point. In this way the user will see all of the main events of the original video without the
need to do a random search. The user could then choose to go back to any point and start playing
the video in full.

Adri Jovin J.J.
Page | 168
To aid searching a continuous media stream such as video we plan to do automatic scene-change
analysis to determine when a scene has changed to a new one and do motion analysis to
determine if there has been a lot of motion within one scene. It is at these points that the user will
be interested.
Index References as Data
It is possible that the data referenced by an index is not a packet collected from the network, but
is a reference to an element of another index. That is, a level of indirection is allowed. Once
index references are available it is possible to build arbitrary structures of indexes, including
hierarchies and networks of inter-connected indexed material.
There are two components that constitute an index reference. First, the name of index being
referenced, and second, an offset into that index. The offset is the number of index elements
from the start of the referenced index. Using this name-offset pair as the data new indexes can be
built that reference other indexes.
For example, one way want an index that only has entries when there are intra-frames in a piece
of video so that playback of video only happens when there is a whole screenful of data. As
intra-frames only occur occasionally, a smaller index can be built that references the original
index elements which point to the intra-frame packets. In figure 9.12 the two indexes are shown.
The index index0 is the original primary index for the video media, and index1 is the index with
the index references. index1 only has 3 elements as there are only 3 packets with intra-frames in
the original data.
Figure 9.12: An index of index references
In the new index the value for the milliseconds since start field is application specific and may be
a copy of the time from the index element being referenced, or it may be set to some other value.
However, the annotations field will not be copied as they are specific to the original index. The

Adri Jovin J.J.
Page | 169
new index can have its own, independent annotations. It is the responsibility of a player to
determine the cause and effect of these values.
Lists of Index References
By allowing the data field to an index reference rather than a packet different structures of
indexes can be built. However, there is often a requirement to have one index element refer to
multiple indexes. That is, the data field is a list of index references rather than a single index
reference. This allows one index to reference a whole conference for example, and have
playback via that index, and have all the conference specific annotations attached to that index.
Using Indexes for Time Synchronisation
Once lists of index references are available then time-based indexes can be build. These have an
entry every n seconds, with the references to all of the media and the sources for those elements.
If there were, say, 12 sources then each new index element would have a list of 12 index
references. Each of the 12 references would be for a time closest to the second boundary in the
original indexes.
In figure 9.13 there is an example showing a time-based index referencing three other indexes.
There are 3 index references for each element of the time-based index. The time-based index has
an element for each second of elapsed time, and the references associated with each element
point to the index element whose time is closest to the number of elapsed seconds.
Figure 9.13: A time-based index referencing three other indexes

Adri Jovin J.J.
Page | 170
Using Indexes for Sender Timestamps
Instead of relying on receiver based timestamps, which are in the main index for a media, it is
possible to create an index where the timing is based on the sender timestamps. A mechanism is
required which analyses each source packet for the sender timestamp and creates an index entry
based on that.
Using Indexes for Joining Segments
We have to consider how separate segments from different media can be combined using the
index. This process needs extra information in the index header. Also, because the segments may
come from arbitrary conferences the timestamps in the indexes may not allow for
synchronisation. Therefore, some normalising process is required for all the timestamps within
the joined segments.
Enhanced Use of Indexes
Researchers are investigating enhanced uses of index tracks, to aid both replay and search of
media streams. When playback access to digitized audio streams is done from an arbitrary point
the resulting sound is a comprehensible signal. However, this approach is not suitable for
compressed digitized video.
For video, it is difficult to go to exact places as one can in a book for instance. Video is a
continuous media without obvious demarcation points for the information, at present it is not
possible to peruse a video in the same way as a book. There is no alternative but to watch large
parts of the video by using the fast-forward and rewind buttons to skip to a certain place and then
watch a section to determine if it is the correct one.
Researchers are designing some techniques that may be developed to improve the ability to find
information in continuous media such as video. Existing work in this area uses just digitized
video to extract information. Our approach is novel in that it uses compressed digitized video
which has been stored in a multimedia storage system, such as the Video Conference Recorder,
to extract the indexing information. We believe that it is possible to devise techniques for video
that are equivalent to flicking through a book. This technique constructs a thumbnail video with
just key frames presented. For example, one could show 1 seconds worth of frames for every 20
seconds or minute of real video. This will give a fast forward view of the video.
Techniques for encoding indexes into video have been devised at many places including MIT
and Berkeley. These access digital video to create index information. Because multimedia

Adri Jovin J.J.
Page | 171
conferencing and distance learning as a whole use data compression, it is pertinent to devise
these new techniques which use the compressed video directly.
The techniques which will be developed here will help users search and peruse the online
archives. If we compare finding information in a book with finding information in a video we see
there are significant differences. A book is structured by separating information into chapters,
sections, and paragraphs; a structuring which aids human comprehension and allows non-
sequential access to the data. The book can be accessed one page at a time, or by going directly
to a page which is the start of a chapter or a section. This is because a book is a collection of
discrete sections. Further information about the book can be created by building indexes into the
book which can be used to go to parts of the book which are about a subject in the index.
Once one can build indexes for the video, then the perusal of the stored video material can be
readily used in various applications. One of the main applications for this kind of storage and
indexing is in distance education in which compressed video and audio are heavily used. In
distance education most of the remote classrooms and students do not have access to the raw
video of the main classroom, they receive a compressed video stream which is stored for later
use. It is expected that the students use the indexing information to scan existing video
recordings in order to playback relevant parts of lectures and seminars. The indexes allow the
student to access and peruse many hours of video rapidly in a similar way to perusing a book.
Strategies for Recording and Playing
This section describes the strategies that are needed in both the recorders and players to record
and playback different kinds of media each possibly using a different protocol.
The Recorder
The recorder must firstly deal with multiple media within each conference, and secondly with
multiple sources for each of the media.
Each media will be received on a different multicast address and port pair. It is the responsibility
of the recorder to collect data from the multicast address for each media and determine the
source of the sender. The utilization of the multicast address is in effect a multiplexing process
by the senders. Each source sends packets to the same multicast address, and the underlying
virtual network delivers those packets to all the members of that multicast address. There is
neither a guarantee of delivery nor a guarantee of ordering for packets so each receiver may see
the data arriving differently. The recorder, which has to join the multicast address to receive the

Adri Jovin J.J.
Page | 172
data and is therefore a member, has to demultiplex information from the multicast address in
order to determine the original source. This process is shown in figure 9.14.
Figure 9.14: The recorder de-multiplexes data from a single multicast address
When the recorder receives packets for a media they are de-multiplexed from the media's
multicast address, the source is determined, and the packet data is then stored as-is using an
index for each of the sources. This is the simplest approach to recording, and can be used for data
on any multicast address. The data does not need to be a traditional media, but can be any
multicast data.
Many media applications use two (and occasionally more) multicast addresses, one for data and
another for session messages. If two multicast addresses are used the recorder needs to
accommodate this. Given the different kinds of media and their associated session messages it is
pertinent to have specialised per-media recorders that can deal with this.
Once the issue of multiple multicast addresses is dealt with, some recorders may need to look at
the contents of the packets before doing the indexing process. Of particular interest are sender
timestamps, which are contained in the packet header for a media. Much of the RTP
specification deals with sender timestamps and how to process them. It is expected that a
specialised RTP recorder can utilize this functionality.
A further enhancement of the recorders is to deal with cross-media synchronisation signals.
These will probably contain sender timestamps for multiple media, in order to allow a receiver to
calibrate themselves and facilitate lip synchronisation. Without media tool support, lip
synchronisation is not possible. Furthermore, any recorder of such media must also cope with
these calibration signals.

Adri Jovin J.J.
Page | 173
The Player
The player needs a mechanism for media selection and source selection. It must also deal with
the problem of address allocation, particularly with respect to uniqueness. The player must deal
with merging multiple media and multiple sources of each media, and try and address the
problem of multiple sources coming from a single host.
The player has to play each of the sources in a synchronised way. It has to cope with the fact that
the recorder may have started before any data arrived for a particular source. We observe that
there is a different delay between when the recording started and when data arrived for each
source, and for each media. This delay is shown in figure 9.15. The delay between the recording
start time and the arrival of the first packet may be very significant; running into hours. To avoid
players replicating this huge delay it is possible for the player to determine the minimum time
between the recording start time and the arrival of the first packet for all the sources, and then to
skip this amount of time during playback. Using this mechanism at least one source would start
to play immediately.
Figure 9.15: The delay between record start time and source data arrival
For a simple playback a player can join a multicast address, read data from an index and send
packets onto the network with the same inter-packet gap as when they were recorded. If there is
more than one index then the data from each of the indexes has to be multiplexed onto the same
multicast address. It is the responsibility of the receiving tool to determine the source stream, but
this can be problematic as all data seems to come from one IP source.

Adri Jovin J.J.
Page | 174
When a recording has session messages a specialised player is needed to accommodate these.
Whilst this special player is sending data packets to one port, the session messages, or a modified
form of them, should be played back onto the session message port. With the addition of session
messages in the player, the receiving tool has a better chance to determine the source as extra
information is available.
As some media use RTP for transmission special recorders can utilize the sender timestamps in
the RTP packets. Similarly, special players can be made to do the same. In this way, a special
RTP recorder-player pair can be devised which can send data back to the network independently
of the way it arrived at the recorder, but with respect to the timestamps at the sender. Also, RTP
has source identification that is independent of the IP source so there is no reliance on where data
came from. This is beneficial for a system as each RTP source will have a different RTP source
identifier even though all the packets come from the one player. However, not all tools currently
utilize the RTP source identification correctly which undermines its usefulness.
If recorders can deal with RTP together with cross-media calibration, then the players also need
to deal with it. This too will involve even more specialisation in the player. It is of particular note
that cross-media calibration requires the media tools to participate in the scheme as well as
recorders and players.
Because the user can create new indexes after the recording has finished it is necessary to have
special players that accommodate this feature too. This type of player has to know how to player
indexes that contains index references rather than just data packets. Once these players have been
devised then it will be possible to have playback via time synchronised indexes. Such indexes
have elements for regular time intervals rather than having elements for when packets arrive.
These players will be able to skip backwards and forwards in time across every media and every
source because each index element will address a constant time interval as opposed to the raw
indexes which are recorded at irregular intervals.
Command Propagation
Each object in the hierarchy communicates by sending messages from one object to another. In
fig 9.16 we see how the play command to the conference object, is duplicated and propagated to
both the media objects and source objects in turn. Each source object is responsible for playing
each source, and does this by doing operations on the index objects, and by using timer events in
the event handler.

Adri Jovin J.J.
Page | 175
Figure 9.16: The propagation of commands through the object hierarchy
The similar approach is used when determining the minimum time between the conference
recording start time and the arrival of the first packet. The conference object needs to know the
minimum for all sources. The method uses is as follows:
• the conference object requests the minimum delay time from all of the media objects.
• each media object requests the minimum delay time from all of its associated source
objects.
• each source object will report to its associated media object the time between the record
start time and the arrival of the first packet for that source.
• each media object evaluates the minimum delay from the set of delays reported by its
sources. This per-media minimum is then reported back to the conference object.
• the conference object evaluates the minimum delay from the set of delays reported by
each media object. This is the minimum for all of the sources. This minimum value is
sent to all the media objects.

Adri Jovin J.J.
Page | 176
The minimum value is then reported, by the media objects, to the source objects. Each source
subtracts this minimum delay from the time to play its first packet.
After this process is completed, at least one source will have no delay on start up, and the media
for that source will be sent to the user immediately.
Reliable Multicast
The recording and playback of most media involves the recorder collecting packets from the
network, and the player sending the packets back to the network. However, for media that use a
reliable multicast mechanism this approach is not possible. It is necessary for the recorder and
player to have an implementation of the reliable multicast code within themselves.
Consider the case where a recorder notices that some of the shared workspace data has not
arrived. It is the recorder's responsibility to request that the lost data be retransmitted, but without
full knowledge of the details of the retransmission scheme this cannot be done. Therefore, the
data stays lost within the recording. At playback time, when then player sends out the recorded
data this loss will be observed by the current receivers. They will send a retransmission request.
However, a player without a retransmission scheme implementation will be in a poor situation.
Firstly, it will not have the data to retransmit because the recorder never recorded it, and
secondly it cannot reply to the requests because it does not have the relevant implementation.
This area of recording and playback is particularly tricky, and further work needs to be applied to
this. It also requires the tool writers to have stand-alone implementations of their reliability and
retransmission code, which can be slipped into servers as required.
Address Allocation
One of the problems when playing back media to a single user or small group of users is
choosing a multicast address for the playback. This problem does not arise when doing playback
to a conference because conferences are usually on well known or well defined addresses. In
choosing an address for a user there must be no clash with existing, in-use multicast addresses.
As well as uniqueness of the address it has to be decided who makes that decision. There seems
to be four choices for who does this:
1. the user
2. a client
3. the server
4. an external service
For the first three choices it is hard to determine how they can guarantee some uniqeness given
they have no mechanism for deciding which addresses are already in use, and therefore which
are free to use. The most appealing choice is for some external service to make the decision. This
service can be used by all systems that require a new multicast address for their purposes.

Adri Jovin J.J.
Page | 177
Although such a service does not currently exist, it is in the design phase. The session directory
tool [#!sdp!#] already has an intelligent mechanism for choosing new addresses for conferences
started under its auspices, but is not usable by other systems as it is standalone. Work described
in chapter seven will culminate in the definition of a session description protocol, and should
lead to the implementation of a server that does allocate new multicast addresses on demand.
Once this server has been implemented any program can get unique multicast addresses.
Source Identification On Replay
When data is replayed by the server the receivers of this data observe that all packets come from
one host. Many media tools have been designed to expect that each source comes from a
different host. If multiple sources for each media are being replayed then all the sources seem to
be one host, which can confuse some media tools. As there are some mechanisms available to
overcome this problem other media tools are more tolerant.
There are currently two known approaches to this problem:
1. in the vat protocol it is possible to add an IP address to the packet header to signify that
the original source of the packet is not the sender but is that put in the packet.
2. in the RTP protocol there is a field for a host independent channel identifier. This allows
streams of media to be recognised by their channel id rather than their source address. As
this data is in the packet it will be recorded and when replayed it will be used by the new
receivers.
For problem free source identification on replay media tools need to be designed with media
recording and playback in mind. Using RTP is a good approach for this as it allows host
independent channel identification, however not all media tools are compliant with this
mechanism.
Remote Control of Playback
We are now used to being in control of the boxes which generate our media. Hifi equipment have
controls which allow us to start, stop and pause pre-recorded audio playing from CDs, tapes or
records. We can even use the controls to select which parts of the recording we wish to listen to.
Alternatively, we can tune into live sounds from radio stations, and switch channels. If we have a
spare tape, we can record the live broadcast for posterity, or perhaps to listen to it at a more
convenient time as we drive home. Similar controls are available on our video devices, allowing
movies to be recorded from TV broadcasts, and giving us the opportunity to see our favourite
sporting events time after time, in excruciating slow motion, pausing to find the exact moment
when the pass was released for the winning goal.
If our audio and video streams come to us over a network, and are available in the computer's
memory, then at the very least we should expect to be able to control the streams in the same
fashion as we control traditional media. We should be able to start the program playing, pause

Adri Jovin J.J.
Page | 178
the stream if we wish, skip further along the stream to locate the interesting bits, and slow or
speed up the programme as required. Since the medium for the stream is a network connected to
a computer, then building the controls is simply a matter of designing a program and the
associated techniques for talking to other programs.
Earlier in the book, we talked about how the streams of data can be compressed audio, video, or
even data intended for updating some application, such as stock market figures, or a sequence of
polygons to be rendered in a Virtual Reality environment. The stream is generated by a node on
the network which we will term the media server, and is sent as a stream of packets over the
network to the receiver(s), which appropriately process and pass the data up to the application.
We have two possible ways of controlling the media streams - we could send messages back
over the network to the source of the stream to ask the source to play, stop, pause etc., or we
could allow the stream to come to us, and simply manipulate the media in memory, keeping
control within the receiving computer. Both techniques are possible, but are suitable in different
environments.
When the stream of data coming over the network is shared by many different users, such as in a
multicast event, it becomes more appropriate to keep the controls local to the receiver. Since the
source has many different receivers, there is a potential implosion problem if the receivers all
sent control messages to the source. The source may then be overwhelmed by messages.
Alternatively if there are many receivers and no floor control, the source may have difficulty in
resolving the conflicting demands from the receivers - what to do if one asks to stop, whilst the
other asks to play? If authority over the view is delegated equally amongst all the receivers, then
each receiver should manipulate the stream locally. If they decide to start the stream, then they
should join the group and let the routing take care of bringing the data to the computer. If they
decide to pause, they should simply stop displaying any more stream and buffer incoming data,
and if they decide to stop the stream, then they should leave the multicast group, allowing the
routing to prune their branch from the multicast tree.
When the stream of data coming over the network is owned by just one user, then the network
and server resources are more efficiently used if the media stream is manipulated at source. Note
that even though the media streams are multicast, floor control within the session may have
designated an owner who has control over the streams.
Remote invocation of stream controls
Stream control requires the receiver to send a message to the media server, asking it to perform
some action. There may or may not be some response. This paradigm of request/response over a
network has been well-known for many years, and has been formulated as a remote procedure
call or rpc.
In a remote procedure call, the remote server offers a selection of procedures that the local
machine can call. These calls are labelled in a unique fashion across the server. When the local
machine wishes to make a call, the calling program constructs a packet which references the

Adri Jovin J.J.
Page | 179
label on the remote server, adds any arguments to the call encoded in a machine independent
format, and sends the packet to the server. The server receives the packet, routes the request to
the appropriate bit of code and unpacks the arguments so that they can be understood and
executes the request. If there are any return values, or any error responses, these are packed into
a packet and sent back to the sender. The strength of rpc mechanisms lay in the fact that the code
to construct the packets at the local machine, and demultiplex and deconstruct the arguments at
the server can be automatically generated from descriptions of the procedures that are similar to
programming languages with which progrmamers are already familiar, such as C. The canonical
reference for rpc is the work by Birrell and Nelson from Xerox PARC in
[#!birrell84:_implem_remot_proced_calls!#]. The most widely used rpc system has probably
been the sun rpc system, whilst every vendor with an interest in providing network solutions has
some variation upon rpc, such as DCOM from Microsoft, the Java RMI system and the DCE rpc
system.
Most projects which have designed networked multimedia systems have used an rpc system to
build the stream controls. An influential project was Pandora out of Cambridge. This work
pioneered the use of ATM type systems to deliver real-time media such as audio and video, but
also used RPC mechanisms to implement the network controls.
Rpc systems make life very simple for the application programmer, and for the conceptually
simple controls for a media stream, they would seem to provide an ideal solution. However, as
ever, there are a few problems.
Rpc mechanisms are intended to support the networking of heterogeneous machines, but there is
no widespread rpc system available across all platforms. CORBA provides a generic object-
oriented technology for integrating operating systems, and is seen by many to be the key
integrating technology. Currently, CORBA technology is costly, but increasing use should see its
use by more than major corporations.
Using rpc mechanisms makes defining the control interfaces easy. However there remain some
subtleties in interpreting the precedence of control requests. What happens if a number of play
requests are sent - should they be queued? How then should the streams be interrupted if the
requests are queued? How should the server track the current state of the stream? There is the
need for a small stateful protocol at the server to deal with the ordering of play and pause
requests Inherent in controlling streams of data remotely is the necessity to keep state at the
server about the requests that have been made. Requests must have an associated sequence
number, so that the server can service the incoming requests in the correct order, despite any re-
ordering that may occur.
If a client requests the transmission of a stream, the server has to return some identifier to the
client so that this stream can be distinguished from others. The mapping between this identifier
and the stream is held both on the server and on the client and can be the source of major
difficulties in designing a complete solution. The problem comes in deciding what happens when
the state is lost at either the client or the server. If the state is lost at the server, then the server
can no longer associate control requests with the outgoing stream. If the client loses state, then it

Adri Jovin J.J.
Page | 180
will not be able to generate control requests for the stream. If the stream consumes many
resources, then this may be disastrous, since a long lived stream which no one is using, and
which is uncontrolled is just a resource hog, and may block other legitimate uses.
Solutions to the problem rely on either being able to recreate the state, or in ensuring that the
protocol is fail-safe, whereby unless control is continuously asserted, the stream stops.
An Aside - The Hypertext Transfer Protocol as a Universal RPC Mechanism
Underlying the World Wide Web is the Hypertext Transfer Protocol or HTTP. This was
orginally designed purely to serve HTML documents, so provided a simple GET of a document
URL, which received a response from the receiver consisting of the HTML in plain ASCII. But it
is in the nature of all good ideas to become more complicated. Why should documents simply be
ASCII? Why not allow other character sets? Documents don't consist entirely of words - why not
allow pictures and other forms of media? And so it came to pass that HTTP used the MIME
standard (REF?) to encode the documents transferred from server to browser, allowing a range of
media types and encodings. Since HTTP was now passing generic objects around, identified
using the URI and typed by MIME, the current set of designers thought to themselves, ``Hey, we
have the basis of a generic request response protocol which we can use for rpc, but encoded in
ASCII. Let's turn the next generation of HTTP into an rpc protocol''
So, the Hypertext Transfer Protocol in its current formulation of version 1.1
[#!fielding97:_hyper_trans_protoc!#] has extended its capabilities beyond a simple protocol to
transfer html documents to becoming a generic request-response protocol for the Internet. Its
authors describe it as an ``application-level protocol'' which is a ``generic, stateless object-
oriented protocol which can be used for many tasks, such as name servers and distributed object
management systems through extension of its request methods''.

Adri Jovin J.J.
Page | 181
Figure 9.17: Basic HTTP Exchange
Like many other protocols emerging from the IETF, the protocol is based around ASCII header
fields9.1. The model is that the client sends a request to a server, possibly via one or more
proxies. If the request isn't serviced by any of the intermediate proxies, then it eventually reaches
the server. The server responds to the request with a response which returns via the proxies. The
fields of the messages are split into four separate headers which relate to different parts of the
protocol. We give a brief overview below, which is sufficient to understand the structure of the
RTSP protocol; for full details the reader should go to the HTTP standard itself, in
[#!fielding97:_hyper_trans_protoc!#].
The HTTP messages have a first line which contains the type of request or the response,
followed by a general header, a request or response header, and an entity header. If appropriate,
there is also a message body.
The first line of any request indicates the type of method;``GET'', ``HEAD'', ``OPTIONS,
``POST'', ``PUT'', ``DELETE'', ``TRACE'' are currently defined. After the type of request comes
the Uniform Resource Identifier (generally a URL) upon which the request is to act, followed by
the version of the protocol. The first line of any response gives the status of the request, which
has the version of the protocol, a numeric code and a textual description of the response, eg
numeric code 200 is ``OK'', whilst 404 is ``Not Found''.
The general header applies to both request and responses, but not to the body. They include the
fields Cache-ontrol, Connection, Date, Pragma, Transfer-Encoding, Upgrade and Via.

Adri Jovin J.J.
Page | 182
The request header includes additional information for the request, such as the range of
encodings that the client can accept, the most recent entity the client has, and the client software.
Response headers give information about the age of the response, when to retry on failure and
the type of server.
The entity header gives optional information about the body of the message, such as the length
(Content-Length), the type of the entity (Content-Type), when it expires (Expires) and when it
was last modified (Last-Modified). There may be an entity header for each of the entity bodies
present.
Finally, the bodies contain the actual data associated with the request and the resource, such as
the HTML page, the image, or the audio file.
The Real Time Stream Protocol – RTSP
In the words of the authors,
The Real-Time Streaming Protocol (RTSP) establishes and controls either a single or several
time-synchronized streams of continuous media such as audio or video...In other words, RTSP
acts a ``network remote control'' for multimedia servers.
As we have already discussed above, the interactions of a client with a server using a network
remote control is best modelled by the use of an rpc mechanism. However, rather than using an
rpc mechanism directly, the designers of RTSP decided to use a variation on HTTP, since in its
current incarnation of version 1.1, it approximates an application level rpc mechanism. The
designers felt that they could leverage the work already done in producing HTTP to produce
production code for RTSP clients and servers9.2, and in particular, use the technology developed
for proxies and security.
Multimedia presentations are identified by URLs, using a protocol scheme of ``rtsp''. The
hostname is the server containing the presentation, whilst the port indicates which port the RTSP
control requests should be sent to. Presentations may consist of one or more separate streams.
The presentation URL provides a means of identifying and controlling the whole presentation
rather than coordinating the control of each individual steam. So,
rtsp://media.example.com:554/twister/audiotrack
identifies the audio stream within the presentation twister, which can be controlled on its own. If
the user would rather stop and start the whole presentation, including the video, then they would
use the URL:
rtsp://media.example.com:554/twister/
RTSP add a number of new requests to the existing HTTP requests. These are
DESCRIBE

Adri Jovin J.J.
Page | 183
Causes a server to return a description of the protocol using the Session Description Protocol.
ANNOUNCE
Allows a client or server to register a description of a presentation.
OPTIONS
Causes a server to return the list of supported methods.
SETUP
Causes a server to allocate resources for a stream and starts an RTSP session. This
manipulates state.
PLAY
Starts data transmission on a stream allocated by SETUP. This manipulates state.
RECORD
Starts a server recording an allocated stream. This manipulates state.
PAUSE
Temporarily halts transmission of a stream without freeing server resources. This manipulates
state.
TEARDOWN
Frees resources associate with a stream, so that the RTSP session ceases to exist. This
manipulates state.
GET_PARAMETER and SET_PARAMETER
Placeholder methods to allow parameters of presentations and sessions to be manipulated.
REDIRECT
Causes a client to go to another server for parts of a presentation.
Putting this altogether, to send a control request, the client constructs a line consisting of the
method, the request URL, and the procol version number. Then it includes a general header, a
request header and possibly an entity header, as for the http protocol. This is sent to the server,
who executes the request if possible. It then returns a response containing a status-line and
general, response and entity headers. The status line contains the protocol version, the numeric
status code and a textual description.
The media streams are left unspecified by RTSP. These could be RTP streams, or any other form
of media transmission. RTSP only specifies the control and its up to the client and server
software to maintain the mapping between the control channel and the media streams.
A key concept in RTSP is the notion of a session. RTSP works by first requesting a presentation
to be started by a server, receiving in return a session identifier which it then uses in all
subsequent controls. Eventually, the client can request the teardown of session, which releases
the associated resources. The session identifier represents the shared state between the client and
server. If the state is lost, for example through one of the machines being rebooted, then the
protocol relies on the transport of the media stopping automatically, eg. through not receiving
RTCP messages if using RTP, or the implementation using the GET_PARAMETER method
below as a keep-alive.
The control requests and responses may be sent over either TCP or UDP. Since the order of the
requests matters, the requests are sequenced, so if any requests are lost, they must be

Adri Jovin J.J.
Page | 184
retransmitted. Using UDP thus requires the construction of retransmission mechanisms, so there
are very few occasions when the application can get away with using UDP.
The most obvious additions to the request header fields are a Cseq field to contain the sequence
numbers of requests generated by the client, and a Session field to both request and response
headers to identify the session. Session identifiers are generated in response to a SETUP request,
and must be used in all stateful methods. The Transport field allows the client and server to
negotiate and set parameters for the sending of the media stream. In particular, it allows the
client and server to set ports and multicast addresses for the RTP streams. There are a number of
other header fields, such as the time range of the presentation upon which the method executes
(Range), and various fields which interact with caches and other proxies.
Descriptions of session use the Session Description Protocol, which provides a generic technique
for describing the details of the presentation, such as transport and media types of the stream, and
the presentation content. Importantly, it also provides the start and end times of the presentation,
so that the client can PLAY form and to any point in the presentation they wish.
Media streams are referenced through specification of their times, either relative to the start time
of the presentation, or in real time. Rtsp allows the use of the standard time codes used in
industry such as SMTPE or Normal Play Time, or by specifying an absolute time for
presentations in real-time.
To display a presentation, the client software first requires the Rtsp URL of the presentation. If it
has this URL, it can then display the presentation by following these steps.
1. The client first requests the description of the presentation using the DESCRIBE method.
This supplies details of the media streams, so that the client can start the appropriate
media applications.
2. The client then requests that the session is SETUP, receiving a session identifier in return.
The server would allocate state, such as the sockets through which the media will be sent,
plus any reservations for bandwidth.
3. The client requests that the media streams of the session are PLAYed, by specifying the
URL and the session identifier, and a time range to start and finish playing at.
4. At any point they may PAUSE the presentation, and continue form any other point in the
presentation by using specifying a new range in the PLAY request.
5. When the client has completed, the client issues a TEARDOWN request to destroy the
session, and deallocate any resources.
Rtsp is intended to be a generic protocol for manipulation of continuous media over the Internet.
For a given presentation or server, there may be limitations upon the controls the client can use.
For instance, if a company has created a media presentation, they may well desire that a session
is not recorded. Since the controls may sometimes be present and at other times not, this presents
a problem in the design of a user interface. If a control is represented in the user interface, but
isn't available for a particular presentation, then the user may attempt to use the control and be
confused by the subsequent interactions.

Adri Jovin J.J.
Page | 185
Fortunately, the OPTIONS command allows the client to interrogate the server to determine the
available methods, and the component streams. Once these have been determined, the client
software can then build an appropriate interface, representing only the available control, and
presenting an indication of the components of the stream.
Movies on Demand
To show how RTSP is used, we shall demonstrate the implementation of a Media on Demand
Service, borrowed from the standard itself. The client wants to receive audio and video streams
from two different media servers A and V respectively. The actual description of the movie is
held on a web server W, which contains sufficient information to allow the client to set up the
appropriate receiving applications. We assume that the reader is familiar with the Session
Description Protocol.
C->W: DESCRIBE rtsp://foo/twister RTSP/1.0
CSeq: 1
W->C: RTSP/1.0 200 OK
CSeq: 1
Content-Type: application/sdp
Content-Length: 164
v=0
o=- 2890844526 2890842807 IN IP4 192.16.24.202
s=RTSP Session
m=audio 0 RTP/AVP 0
a=control:rtsp://audio.example.com/twister/audio.en
m=video 0 RTP/AVP 31
a=control:rtsp://video.example.com/twister/video
The client first requests the description of the protocol. Notice that the request has a sequence
number CSeq associated. The response returns with the numeric status code of 200, indicating
``OK''. It returns the sequence number, along with entity headers describing the body of the
message. The description of the presentation is held within the message body using the session
description protocol. The session consists of two streams, an audio and a video stream.
C->A: SETUP rtsp://audio.example.com/twister/audio.en RTSP/1.0
CSeq: 1
Transport: RTP/AVP/UDP;unicast;client_port=3056-3057
A->C: RTSP/1.0 200 OK
CSeq: 1
Session: 12345678
Transport: RTP/AVP/UDP;unicast;client_port=3056-3057;

Adri Jovin J.J.
Page | 186
server_port=5000-5001
C->V: SETUP rtsp://video.example.com/twister/video RTSP/1.0
CSeq: 1
Transport: RTP/AVP/UDP;unicast;client_port=3058-3059
V->C: RTSP/1.0 200 OK
CSeq: 1
Session: 23456789
Transport: RTP/AVP/UDP;unicast;client_port=3058-3059;
server_port=5002-5003
The client then requests the SETUP of the audio and the video streams, using the Transport
header field to indicate the preferred parameters for the RTP streams. The responses confirm
these are ok, adding the necessary ports to receive from the server. Importantly, the SETUP
responses return session identifiers to be used in associated commands with the appropriate
streams in all subsequent messages.
C->V: PLAY rtsp://video.example.com/twister/video RTSP/1.0
CSeq: 2
Session: 23456789
Range: smpte=0:10:00-
V->C: RTSP/1.0 200 OK
CSeq: 2
Session: 23456789
Range: smpte=0:10:00-0:20:00
RTP-Info: url=rtsp://video.example.com/twister/video;
seq=12312232;rtptime=78712811
C->A: PLAY rtsp://audio.example.com/twister/audio.en RTSP/1.0
CSeq: 2
Session: 12345678
Range: smpte=0:10:00-
A->C: RTSP/1.0 200 OK
CSeq: 2
Session: 12345678
Range: smpte=0:10:00-0:20:00
RTP-Info: url=rtsp://audio.example.com/twister/audio.en;
seq=876655;rtptime=1032181
The client then requests that each of the streams are played, specifying the range using SMTPE
timestamps. The start times of each of the streams is actually specified as 10 minutes into the
stream. The servers respond OK, returning the end of the time range that will be played.

Adri Jovin J.J.
Page | 187
C->A: TEARDOWN rtsp://audio.example.com/twister/audio.en RTSP/1.0
CSeq: 3
Session: 12345678
A->C: RTSP/1.0 200 OK
CSeq: 3
C->V: TEARDOWN rtsp://video.example.com/twister/video RTSP/1.0
CSeq: 3
Session: 23456789
V->C: RTSP/1.0 200 OK
CSeq: 3
Finally the client requests that the session be ended; tearing down the state associated with the
media streams.
----X-----------X----

UNIT V INTERNETWORKING MULTIMEDIA
Adri Jovin J.J.
Page | 188
UNIT – V
APPLICATIONS
MIME
To work with multimedia on the Internet, you must work within the requirements of the
appropriate protocol, using recognizable documents and formats. A voice attachment to an e-
mail message, for example, must be identified by the Post Office daemon for what it is, and then
be transmitted with the correct coding to the receiving computer. The receiver must have the
proper software (and hardware) for decoding the information and playing it back. To identify the
nature of the data transmitted and, by inference, the purpose of that data, the Internet uses a
standard list of filename extensions called Multipurpose Internet Mail Extensions (MIME-types).
Most browsers allow you to define MIME-types and map “helper apps” to the type for decoding
and playing. For example, with Netscape Navigator you can define Adobe’s Acrobat files (PDF
files) as a MIME-type and select the Acrobat Reader as the player application. These are not just
used by the e-mail daemon but, by convention, by other Internet daemons, including the Web’s
HTTP daemon. Perhaps the most widely installed HTTP software for managing web pages is
the open-source application called Apache (www.apache.org). Table 12-3 shows a list of
common MIME-types and their uses. (Note that many come from the Unix world, where the
Internet was born.) You can also visit www.file-ext.com for more information.
Table 12-3 Some Common MIME-Types Illustrate the Variety of Data Types and Formats Used
on the Internet

Adri Jovin J.J.
Page | 189
on the Internet (Continued)

Adri Jovin J.J.
Page | 190
on the Internet (Continued)
Multimedia elements are typically saved and transmitted on the Internet in the appropriate
MIME-type format and are named with the proper extension for that type. For example,
Shockwave Flash animation files end in .swf; image files end in .jpg, .jpeg, .gif, or .png; sound
files end in .au, .wav, .aif, .mp3, or another conforming format; QuickTime and other video clips
end in .qt, .mov, mp4, or avi.

Adri Jovin J.J.
Page | 191
Plug-ins and Delivery Vehicles
Plug-ins add the power of multimedia to web browsers by allowing users to view and interact
with new types of documents and images. Helper applications, or players, also provide
multimedia power by displaying or running files downloaded from the Internet by your browser,
but helpers are not seamlessly integrated into the operation of the browser itself. When an
unrecognized embedded MIME-type that can’t be displayed within your browser is called from
an HTML document (sounds, movies, unusual text or image files), most browsers will
automatically launch a helper application (if it is specified in the browser’s preferences) to view
or run it. However, this helper starts up and runs separately from the browser. Many plug-ins are
designed to perform special tasks not available without the plug-in installed. If you land on a
web page containing embedded, compressed images, for example, and the proper plug-in to
decompress those images is not installed, you will not be able to view the images.
Designers work around this problem by including hyperlinks in their pages, which direct the user
to the site where the missing plug-in may be found. Users must then download and install the
required plug-in, and then restart their browser. This is all a bit cumbersome. Until the
marketplace determines which plug-ins will become de facto standards for the Web, however,
developers have no alternative. Because downloading and installing plug-ins is perceived as a
hassle for the end user, many tool developers use the Java and JavaScript capabilities already
built into today’s web browsers. To offer a plug-in’s functionality to visitors at your own web
site, you may need the addition of MIME-type information to a special setup file on your server
that many plug-ins require. If you do not control or operate your own server, you should let your
service provider know the MIME-types that you need to have supported. Setting up servers for
some of the multimedia plug-ins is not a trivial task, and many Internet service providers will not
support high-bandwidth data streams for fear of overwhelming their Internet connection by
serving your streaming voice or video to the world. Indeed, while a plug-in or a player may be
free and readily available to anyone who wishes it, the software to actually build, compress,
manipulate, and serve the special data (such as for compressed images, compressed audio,
streaming video, animations, and VRML worlds) may be difficult and expensive, since the
company makes money from the development tool, not the client software.
Text
Text and document plug-ins such as the popular Adobe Acrobat Reader get you past the display
limitations of HTML and web browsers, where fonts are dependent on end users’ preferences
and page layout is primitive. In file formats provided by Adobe Acrobat, for example, special
fonts and graphic images are embedded as data into the file and travel with it, so what you see
when you view that file is precisely what the document’s maker intended.

Adri Jovin J.J.
Page | 192
Images
Browsers enabled for HTML5 will read and display bitmapped JPEG, GIF, and PNG image files
as well as Scalable Vector Graphics (SVG) files. Vector files are a mathematical description of
the lines, curves, fills, and patterns needed to draw a picture, and while they typically do not
provide the rich detail found in bitmaps, they are smaller and can be scaled without image
degradation. Plug-ins to enable viewing of vector formats (such as Flash) are useful, particularly
when some provide highoctane compression schemes to dramatically shrink file size and shorten
the time spent downloading and displaying them. File size and compression sound a recurring
theme on the Internet, where data-rich images, movies, and sounds may take many seconds,
minutes, or even longer to reach the end user. Vector graphics are also device-independent, in
that the image is always displayed at the correct size and with the maximum number of colors
supported by the computer. Unlike bitmapped files, a single vector file can be downloaded,
cached, and then displayed multiple times at different scaled sizes on the same or a different web
page.
Sound
Sound over the Web is managed in a few different ways. Digitized sound files in various
common formats such as MP3, WAV, AIF, or AU may be sent to your computer and then
played, either as they are being received (streaming playback) or once they are fully downloaded
(using a player). MIDI files may also be received and played; these files are more compact, but
they depend upon your computer’s MIDI setup for quality. Speech files can be specially encoded
into a token language (a “shorthand” description of the speech components) and sent at great
speed to another computer to be un-tokenized and played back in a variety of voices. Sounds
may be embedded into QuickTime, Windows Media, and MPEG movie files. Some sounds can
be multicast (using the multicast IP protocols for the Internet specified in RFC 1112), so multiple
users can simultaneously listen to the same data streams without duplication of data across the
Internet. Web-based (VoIP, or Voice over Internet Protocol) telephones also transmit data
packets containing sound information.
Animation, Video, and Presentation
The most data-intense multimedia elements to travel the Internet are video streams containing
both images and synchronized sound, and commonly packaged as Apple’s QuickTime,
Microsoft’s Video for Windows (AVI), and MPEG files. Also data rich are the files for
proprietary formats such as Keynote, Microsoft PowerPoint, and other presentation applications.
In all cases, the trade-offs between bandwidth and qualities are constantly in your face when
designing, developing, and delivering animations or motion video for the Web.

Adri Jovin J.J.
Page | 193
Peer-to-Peer Computing
When computers moved into mainstream use, PCs (personal computers) were connected together
through LANs (local area networks) to central servers. These central servers were much more
powerful than the PCs so any large data processing took place on these servers.
Since then, PCs have become much more powerful, and they are able to handle the data
processing locally rather than on central servers. Because of this, PC to PC, or peer-to-peer
computing can now occur when individual computers bypass central servers to connect and
collaborate directly with each other.
Three Distinct Peer-to-Peer Computing Models
There are three distinct peer-to-peer computing models:
1. Multiple Peer Relationship
PCs are connected / networked to each other through servers, and files can be shared and
collected from anyone else on that same network. One key problem is this can lead to major
breeches in security and intellectual property issues.
Examples:
Napster - http://www.napster.com (music sharing)
Note: Due to legal issues, Napster has temporarily stopped file sharing services on their site.
Kazaa - http://www.kazaa.com (multimedia sharing)

Adri Jovin J.J.
Page | 194
2. Distributed Peer Relationship
A group of computers connected together to combine their computing and processing abilities to
search the Internet or solve very complex problems requiring massive process crunching.
Examples:
Infrasearch- http://www.infrasearch.com (search engine)
Entropia - http://www.entropia.com (climate simulations and astrological calculations)
3. Collaborative Peer Relationship
A small group of people agree to collaborate through a common interface, such as on-line
gaming, chat rooms, instant messaging, or e-learning environment.
Examples:
Jeopardy - http://www.station.sony.com/multiplayerjeopardy/ (on-line gaming)
Chat Here - http://www.chat-here.com/ (chat room)
HorizonLive - http://www.horizonlive.com (e-learning seminars)

Adri Jovin J.J.
Page | 195
Significance and Impact of Peer-to-Peer Computing
So what is the significance of peer-to-peer computing? It changes the way people interact in the
areas of information sharing, collaboration, and learning.
First, peer-to-peer computing changes the way people share information. Prior to the Internet and
even the pervasive use of PCs, information was mostly disseminated through television, radio, or
via written (not digital) correspondence. But now information is digitized via e-mail, electronic
documents/files, or web pages and can be distributed to many different people in a short period
of time. The format of information has also greatly changed and ranges from short, sloppy,
opinionated text to lengthy, well organized, and thoroughly researched documents. People now
have to “filter” through a lot of junk data to get to the information they may be seeking. This
becomes even a greater challenge when people now have access to so many other computer
systems and consequently, the masses of information on those systems.
Second, peer-to-peer computing changes the way people collaborate. People no longer have to be
in the same room to plan a project, conduct a workshop, or learn a computer software program.
Ideas and discussions can be captured electronically and distributed to groups of people in
separate locations for review and input. But in order for this to be effective, people have to learn
the technical tools in order to communicate and collaborate (such as using an electronic
whiteboard or a video conferencing system) and must learn to express themselves in different
ways.
Lastly, peer-to-peer computing changes the way people learn. Traditional classroom
environments are no longer the only way to facilitate effective learning. Well designed media-
rich web content can be a very effective learning format and can cover the primary ways people
learn by incorporating sound, text/visual queues, combined with hands-on exercises and
examples. What is different, however, is that the learner has to become more responsible for their
learning. Traditional training “pushes” information to the learner and puts the onus of learning
more on the instructor. Whereas virtual training requires the learner to “pull” out the information
he/she is seeking, thus shifting the onus of learning onto the learner.
Future Trends and Challenges
Peer-to-peer computing “represents a swing of the pendulum back toward user control”.
Powerful PCs and the Internet enable people to come together through a common environment.
More and more people are taking information sharing into their own hands, and companies are
finding themselves in a situation of trying to control people and influence the use of the Internet.

Adri Jovin J.J.
Page | 196
Shared Applications
Shared applications that currently exist in the Mbone are what is termed Collaboration Aware,
which is to say that they have been written specifically to be used by multiple simultaneous
users, rather than simply being adapted through some kind of wrapper such as a shared
windowing system with floor controller, to such use.
Wb and Nt represent a fairly radical departure from previous shared applications, in that they are
engineered in the context of very large numbers of users (1000s) and over very long haul
networks, where there are very poor guarantees of full connectivity for the duration of a
conference that is using the application, and where the capacity and delay from any given source
to the group of users of the shared application may be varying by orders of magnitude.
To deal with this, the authors of both of these tools recognize up front that consistency of views
for all users is a holy grail, and simply unachievable within finite resources. However,
correctness of view should be possible up to some point.
The starting point for solving this is that an application specific reliability is required, and that it
is different for Wb and Nt.
Wb defines a set of operations that each user can carry out on the whiteboard, each of which is
idempotent, or can be repeated without danger. The reliability in Wb is achieved through
retransmission of missing operations. Again, to gain scaling to large numbers of participants,
there is a very ingenious scheme to decide who requests a retransmission: Conventional schemes
really on recipients notifying senders of missing packets through negative acknowledgement
packets or NACKs. However, a NACK/Retransmit style protocols would not scale, as typically,
a loss that is incurred somewhere in the MBone might be perceived by entire cohorts of receivers
when a subsequent operation arrives; an implosion of NACKs from all of them would then
result. Wb uses multicast to its own benefit. All messages for all functions are always multicast.
They also all contain timestamps, and all participants monitor their closeness to each other.
When a recipient wishes to request a repair retransmission, they first throw a dice with a number
of sides depending on how near they are to the source of the missing message, and wait for the
time indicated by the dice. If they subsequently see a repair request from one of their cohorts,
they suppress timer that would otherwise have gone off and caused them to make the request.
The Wb scheme is self tuning, and works very well
Nt is somewhat different, in that it is not a whiteboard, but a shared document editor (Network
Text). Thus the operations by different users cannot be conveniently made idempotent. Nt

Adri Jovin J.J.
Page | 197
attempts a stronger ordering than Wb, but uses similar schemes to achieve the repair. In the event
of a partition of the network, followed by a healed network, Nt will offer the users a choice of
which branch of the now disjoint edits to follow (or to revert to the pre-partition version of the
document).
Background to NTE
Since 1992, the multicast backbone (Mbone) on the internet has been used for a large amount of
multiparty multimedia conferencing experimentation. Although several audio and video tools
have been developed and used on the platform, only one shared workspace tool has seen
significant usage, namely the wb shared whiteboard tool from Van Jacobson of Lawrence
Berkeley Labs. Although wb is a good shared whiteboard tool, we have often seen it used in
circumstances where a shared text editor would have been a better choice of tool.
Wb is not a shared editor - for good reasons it keeps separate users' drawings and text separate,
as this greatly simplifies both its usage and its data consistency problems. We believed that for a
relatively simple environment - that of shared text editing - these consistency problems could be
solved within a loose consistency framework that had excellent scaling properties similar to
those of wb.
Starting from some general guiding principles - those of IP multicast, Application Level Framing
and Light-weight Sessions, we set out to develop a scalable shared text editor.
Scalability
Typical usage of a multiway shared editor is envisaged as being in a ``meeting'' with a number of
people at geographically distributed sites. Although the number of people actually making
changes to a document at any time is likely to be relatively small, we wish to avoid placing
constraints on the number of people able to participate in such a meeting. In particular, we do not
wish the load on any participating component to scale with the number of participants.
Resilience
We wish a shared editor to be resilient to network conditions including packet loss, link failure,
and failure of any participating end system. In combination with the scalability requirement, the
necessity to tolerate link or end-system failures implies that a distributed, replicated data model
is appropriate. Resilience to packet loss can be achieved through redundancy, through
retransmission, or through network reservation. Network reservation as is not widely available
and we will aim to show that it is unnecessary for this sort of application.

Adri Jovin J.J.
Page | 198
Loose Consistency
Observations from the MICE project of people using a shared whiteboard (wb) in conjunction
with an audio tool lead us to conclude that the users of such tools will happily tolerate temporary
inconsistencies between the view of a document at one site and the view of it at another, so long
as eventual consistency is achieved and so long as the time they have to wait is predictable. This
is fortunate as the requirements of resilience and scalability dictate that total consistency of view
is not possible unless mechanisms requiring unacceptable delays are employed.
Interactivity
As a general rule, constraints on interactivity between users via a shared application should be
kept to a minimum, and only imposed as a policy where the users actually require such
constraints rather than imposed by the mechanism of the shared application. Thus with a shared
editor, the mechanism implementing the editor should allow multiple users to edit the document
simultaneously. Thus if the sharing mechanism requires a lock to be imposed on a part of the
data set, then this lock should be imposed on as small a part of the data set as possible to
maximise the potential interactivity that can be achieved between users through the editor.
However, the imposing of locks of any sort is at odds with the resilience/scalability requirements
above, and because we do not require global consistency at any time, the use of locks should be
avoided if suitable alternative solutions are available.
Usage modes
Generally speaking there are two main forms of usage for a shared editor:
• Collaborative editing of a passage of text
• Annotation of a passage of text
In practice, when working in groups, we observe that shared editors also get used to a degree as
additional communication channel, but this can largely be though of as annotation, even though
these meta-annotations are not directly associated with existing text. Thus it is important for a
shared editor to be able to easily support both these communication modes as transparently as
possible.
If we satisfy the interactivity requirement above, it is vitally important, whether annotating or
modifying text, that the person making a particular change is easily identifiable, even though
several changes may be being made simultaneously.

Adri Jovin J.J.
Page | 199
Design
The basic building block for nt is IP multicast [#!deer:88!#][#!mac:94!#]. This provides us with
many-to-many communication at constant cost, irrespective of the number of receivers. To
achieve resilience, we adopt a distributed, replicated data model, with every participant
attempting to hold a copy of the entire document being shared. This means that end-systems or
links can fail, but that the remaining communicating sites still have sufficient data to continue if
they desire to do so. Updates to the distributed data model are made using IP multicast, although
this means that due to packet loss, not all sites will receive a particular update - how this is
resolved is discussed later.
In designing nt, we attempted to apply the guiding principle of application level framing (ALF,
[#!ddc:91!#]) - thus the application and its communication model employ the same data units.
Application Data Units
Our choice of application data units is driven in part by the model for data distribution and our
choice of data distribution model is driven in part by the choice of application data units.
Although we present the two as separate sections for the sake of readability, it should be clear
that the two really are closely interrelated.
The guiding factors in determining nt's data model come from the interactivity requirement listed
above - many users must be able to work on the same document simultaneously - and the
observation of usage modes, particularly the need to be able to keep annotations separate from
the primary text being worked on.
These guiding factors led us to choose a hierarchical data model based around blocks of text,
each consisting of a number of lines of text. Each block of text is independent of other blocks of
text - it can overlap them if required although this does not aid readability. An example of blocks
of text used for annotation, with each block in a different font, is given in figure
As it is not expected that most annotations will be modified by multiple users simultaneously,
this by itself allows a number of users to be working simultaneously on the document in separate
blocks.

Adri Jovin J.J.
Page | 200
However only allowing multiple people simultaneously to annotate a document imposes too
great a constraint on the potential usage modes of the editor. Thus we also make each line of a
block of a document a separate entity. This allows users to be working on separate lines in the
same block without there being a potential conflict.
We could potentially have taken this model further, and treated each character of a line as an
independent entity. There are, however, a number of reasons why this is undesirable. Firstly, the
amount of state that needs to be kept for each separate entity to ensure eventual consistency is
significant. In addition, if we choose a line rather than a character as an ADU, we do not need to
receive all the individual changes to the line as a user types - the most recent version of the line
is sufficient, which gives us a large degree of redundancy in the face of packet loss. Lastly, there
are potential transmission failure modes that with either line ADUs or character ADUs render us
with no globally consistent ordering for the ADUs - however, due to the nature of the changes to
text, these are significantly less likely to occur with line ADUs than with character ADUs. We
shall discuss these failure modes and also the implications of two users attempting to modify the
same line simultaneously later, in the light of the loose consistency model described below.
It is perhaps not immediately obvious that either a line or a block is truly an independent entity.
A block of lines is an independent entity because it has no interaction with, or dependency on,
other blocks. However, within a block, lines are dependent on the block that contains them as
their location is dependent on its position. In addition, a line has a position within a block. This
can be represented in a number of ways - the simplest being an absolute line number, or relative
links to next and previous lines. However, given that we wish to avoid any form of locking, it is
possible that a message conveying the creation of a new line by one user crosses with a message
conveying deletion of an earlier line by another user - in this and other similar cases it is clear
that an absolute line number is insufficient to uniquely place the line within a block. Positioning
a line relative to neighbouring lines is more robust, as even if a line creation crosses with the
deletion of the line immediately previous to the new line, the location of the previous line can be
retained after its deletion and the location is still unambiguous. Perhaps as importantly, so long
as lines are only created or deleted (never moved) relative location of lines means that only
changes in the immediate location of the line in question can cause possible confusion, which
makes it much easier for the user interface to make the possible confusion obvious to the users
involved.
However, although relative location information for lines is more flexible, it is insufficient to be
able to display a block of text as it arrives complete with lost packets - such as might happen
when a file is loaded into the editor. In such a case, adding additional line numbering allows the
parts of the block that have been received to be displayed immediately irrespective of whether
there are any missing lines at this stage. Whether this is an important requirement depends on the

Adri Jovin J.J.
Page | 201
importance placed on timeliness of display - we took the viewpoint that it is important to be able
to display any change as soon as it is received, and so added line numbers to the line ADUs
which are ignored after all previous lines in the block have been received.
Given the choice of ADUs as lines and blocks, there is a certain amount of meta-data that must
be associated with each. Thus a block contains, amongst other things, the following:
• position of the block on the page
• the line-id of the first line in the block
and a line contains, amongst other things:
• the text of the line
• the block-id of the block this line forms a part of
• the line-id of the previous line
• the line-id of the next line
• the line number of the block (ignored if all previous lines in the block are present)
Thus, although lines and blocks are not completely independent, blocks can be moved without
modifying the lines contained in the block, and lines can be created, deleted and edited
independently of other lines or blocks. There are however a number of desirable operations on
lines that cannot be carried out independently - we shall discuss these and their consequences
later.
Distributing the data model
The choice of a line as the ADU was made in part due to the simple observation that most
consecutive changes are made to a single line - generally because a user continues to type. Thus
if the whole line is sent for every character typed the additional overhead (over just sending the
changes) is not great, the data transfer is idempotent (assuming a version of the block has already
been received), and a great deal of ``natural'' redundancy is available - so long as a user
continues typing on the same line, lost packets are unimportant. However this is only the case if
we take a loose consistency approach - changes are displayed as soon as the arrive, irrespective
of whether other sites have received them.
In order to be able to use this ``natural redundancy'' property, it must be possible to identify
whether a version of a line that just arrived is more recent than the copy of it we have already.
This is necessary for two reasons - firstly to cope with misordered packets from a single source,
and secondly to cope with retransmitted information from hosts with out of date versions of the
data.

Adri Jovin J.J.
Page | 202
Misordered packets could be easily spotted using a sequence number; however out of date
retransmitted data is not possible to identify in this way. If we assume synchronised clocks at all
sites, recognising out of date information is very easy to achieve by simply timestamping every
line with the tome of its last modification, and sending these timestamps along with the data.
Packets containing a line or block with an out of date timestamp can be ignored at a receiver with
a later version of the same data. If we wish to take advantage of redundancy by not requiring
retransmission of many lost packets, and we have modification timestamps in the packets, then
we have no use for a sequence number in the packets. In general a receiver does not care if it
receives all the changes to a line as they happen; rather it only cares that it receives the final
version of a line, and that it receives sufficient changes that users understand what it happening
as it happens.
In practice, we can't assume synchronised clocks, but we can implement our own clock
synchronisation protocol in the application. Also if a receiver sees a retransmission of out of date
data, it should attempt to update the sender of that data - we shall discuss how this can be
achieved scalably under retransmission schemes below.
Clock Synchronisation
Given that all changes to a document are multicast, and that all changes are timestamped, we
have a simple mechanism for clock synchronisation amongst the members of a group:
• if a site has not sent any data and receives data from another site, it sets its clock
(application clock) to the timestamp in the received message.
• if a site has not received any data, and needs to send data, it sets its application clock to
its own local clock time.
• if a site then receives a message with a timestamp greater than its current application
clock time, it increases its application clock time to match that of the received message.
These simple rules ensure that all sites application clocks are synchronised sufficiently
accurately for our purposes.

Adri Jovin J.J.
Page | 203
Figure 8.2: An example of application based clock synchronization
Figure 8.2 illustrates this process. In this figure, source S2 sends the first message, and S1 and S3
synchronise to the timestamp in the message. Note S1 can only move its clock backwards
because it has no data. Two new sources then join, S4 and S5, and before any of the original
sources send a message, S5 does so. As S4 has sent no message (therefore has no data), it now
synchronises to S5. The three original sources have data therefore do not synchronise to S5. One
of the three original sources then sends, and both S4 and S5 synchronise to the timestamp in the
message. A new source S6 then joins, and sends before anyone else does. As its local clock is
ahead, all the existing sources now synchronise to it.
To illustrate in more detail, this consider three sites A, B, & C, with three application clocks tA,
tB, and tC, and positive transmission delays dAB, dAC, etc.
if A sends the first message, we have
tB=tA-dAB
tC=tA-dAC
if dAB ever decreases when A sends, then B will increase its clock to match the new delay, and
tA and tB become closer.
if dAB increases, B continues to use tB

Adri Jovin J.J.
Page | 204
Now consider a message sent by C:
This message arrives at B with timestamp tA-dAC and it arrives at time tA-dAB+dCB. A
comparison is made and only if dAC < dAB-dCB is the clock at B increased to be tA-dAC. Thus
the clock at B can only get closer to the clock at A when a message is received from C.
The process continues so long as messages are sent.
As all messages are timestamped anyway, global clock synchronisation to less than the minimum
delay between any two active sites is provided for free, and no explicit clock synchronisation
protocol is required. Naturally this assumes that all local clocks run at the same rate. This is a
reasonable assumption for almost all of today's workstations. Even if there is a clock in a
conference that does not run at the same rate as the rest, this does not cause the application any
real problems. The worst that can happen is that all clocks increase whenever a message from the
fastest clock is received, and that timestamps bear no relation to real time, and do not get
successively more accurate than the first pass approximation.
There are many algorithms that synchronise the clocks more accurately than the algorithm given
above, but for the purposes of consistency control, a necessary feature is that clocks are never
decreased, and the algorithm given is simple and sufficient.
Implementation of this algorithm reveals that there is a case where clocks do not stabilise to the
fastest clock. This occurs when two sites with a clock tick of length t are connected by a network
with a transit delay of less than t, as illustrated in figure 8.3. This can happen with some Unix
workstations with a 20ms clock resolution connected by a local Ethernet. Under these
circumstances, the receiver will synchronise to the sender to a resolution of less than t. If the two
clocks are not in phase, then the receiver can be ahead of the sender for part of each clock cycle.
If their roles as sender and receiver are reversed and the new sender now sends a packet at a
point in the clock cycle where its clock is ahead, the old sender then increments its clock to
match the new sender. If both sites send alternately, this can result in both clocks being
incremented indefinitely. This can simply be avoided if the clock tick interval is known, by
simply ignoring and clock differences of less than the clock tick interval. This is not noticeable
from the user's point of view as clock tick intervals are generally less than 20ms.

Adri Jovin J.J.
Page | 205
Figure 8.3: Clock synchronisation failure due to clock granularity being less than transmission
delay
Towards Reliability
Due to the redundancy inherent in the data distribution model - namely that a user that continues
typing will refresh and repair earlier packet losses on the same line of text - nte will often
perform acceptably with no mechanism for ensuring reliability. However, there are also many
situations where this is not the case, and so we need a mechanism to detect and repair the
resulting inconsistencies.

Adri Jovin J.J.
Page | 206
Inconsistencies may result from:
• Simple packet loss not corrected by subsequent changes (particularly where the last
change to a line has been lost, or where a block is the result of loading from a file)
• Temporary (possibly bi-directional) loss of large numbers of modifications due to
network partition.
• Late joining of a conference.
• Effectively simultaneous changes to the same line.
Inconsistency Discovery
Discovery of inconsistencies is not a difficult problem - however unlike the mechanisms used in
RMP [#!rmp!#] and SRM [#!van:95!#] inconsistencies due to simple packet loss cannot be
discovered simply from the absence of a packet as we wish most such changes to be repaired by
redundancy, and therefore do not need to see every packet at a receiver.
Instead we use a mechanism that ensures inconsistencies are resolved, irrespective of the number
of packets lost. There are three parts to this inconsistency discovery scheme.
Two schemes are based on the session messages each instance of the application sends
periodically to indicate conference membership. These session messages are sent by each site
with a rate that is dependent on the total number of sites in the conference. Thus the total session
message rate is constant (albeit a low one). To detect inconsistencies, each session message
carries two extra pieces of information - the timestamp of the most recent modification seen, and
a checksum of all the data. If the timestamp given by another site is later than the latest change a
receiver has seen, the receiver can request all changes from the missing interval without knowing
what that data actually was. This is most useful when changes have happened during a network
partition but all sites were passive when the partition was resolved. However, if a network
partition was a resolved as a site was transmitting, the receiver may have the must recent
changes, but not some older ones - hence the checksum is a last resort to discover that a problem
has occurred, if not which data is missing.
The third scheme is a more active scheme designed to prevent the above schemes from
needing to be used where possible. We have a concept of the current site - this an instance of the
application at the site which has most recently been active. If more than one site is active, any of
those sites can be chosen as current site. The current site periodically multicasts out a summary
packet giving the timestamps and IDs of all the most recently changed objects (lines and blocks).

Adri Jovin J.J.
Page | 207
If a receiver has a different version (either older or newer) of one of these objects then it is
entitled to either request the newer version from the current site, or to send its later version. The
current site may change each time a new user modifies the document; however the rate that these
summary packets are sent is a constant whilst any users are modifying the document - a new
current site simply takes over from the previous one. In fact sometimes more than one site may
think it is the current site at any one time.
An alternative to sending explicit summary packets might be for session and data packets to have
an additional object ID and its modification timestamp added to them, and for all sites to take
turns to report the state of the most recently modified objects. Indeed, depending on the choice of
retransmission scheme, this may be preferable, but we have chosen not to use this scheme. We
shall discuss why not after we have discussed the choice of retransmission scheme.
Scalable Retransmissions
When a receiver discovers there is an inconsistency between its data and that of another site, it
cannot just send a message to resolve the inconsistency immediately - if it did so there is a high
probability that its message would be synchronised with identical messages from other receivers
causing a NACK implosion. Instead we require a mechanism to ensure that approximately one
site sends the message. If such a message is an update, it should be multicast as this may update
other sites with out of date information. In fact, if such a message is a retransmission request, it
should also be multicast, as then the reception of the request can be used at other sites to
suppress their retransmission requests.
Van Jacobson uses such a scheme in wb[#!van:95!#] where a retransmission request is delayed
by a period of time dependent on the round-trip time between the receiver and the original source
plus a random amount. This scheme requires all participants to calculate a delay (round trip time)
matrix to all other sites, and this is done using timestamps in the session messages.
wb is more dependent on its retransmission mechanism than nte is, as wb has no natural
redundancy, and thus it requires its retransmission scheme to be extremely timely. nte does not
wish its retransmission scheme to be as timely as wb's scheme, as it expects most of its loss to be
repaired by the next character to be typed. In fact, in many cases this results in very significantly
fewer packet exchanges because in a large conference on the current Mbone, the probability of at
least one receiver losing a particular packet can be very high. Thus what we require is a
retransmission scheme that ensures that genuine inconsistencies are resolved in a bounded length
of time, but that temporary inconsistencies due to loss which will be repaired anyway do not
often trigger the retransmission scheme.

Adri Jovin J.J.
Page | 208
Wb's scheme can be simply tailored to do this effectively by adding a ``dead time'' to the
retransmission timer to allow a window during which the next modification to the line could
arrive. If we used wb's randomised time based scheme, then we would probably opt for not
sending summary messages but instead adding ID/timestamp pairs to the session messages as
described above. These would then tend to spread the retransmission requests more evenly.
In fact, wb's mechanism has not been described in detail at the time nte was designed and
implemented, and we used a different sender driven retransmission request scheme.
Sliding Key Triggered Retransmissions
When a instance of nte sends a summary packet, it starts upon the process of sending a sequence
of keys. When a receiver matches a key sent by the sender, it can immediately send its
retransmission request (which can be many objects if necessary) along with the key that was
matched. On receiving this request, the sender then starts the retransmission of the missing data.
Figure 8.4: Sliding Key Triggered Retransmission Requests

Adri Jovin J.J.
Page | 209
What causes this to be scalable is the nature of the keys involved. The sender generates a random
integer when it creates its summary message. Upon receipt of the summary message, the receiver
also generates a random key. Then the sender sends its key along with a key mask which
indicates the bits in the sender's key that must be matched in order for the receiver to send a
retransmission request. This key/mask pair is sent several times (typically 2 or 3 times) and then
if no retransmission request is forthcoming, the number of bits indicated by the mask is reduced
by one, and the key/new mask pair is sent again. If no retransmission request is forthcoming by
the time the mask indicates no bits need to be matched, then the process is started again with a
new random key, a new summary report, and possibly a new current site. If no change has
occurred since the previous summary report, the rate of sending sliding keys is reduced to half
the rate for the previous round or until it reaches a preset lower rate limit. This process is
illustrated in figure 8.4.
This is based on a scheme[#!ian:94!#] devised by Ian Wakeman for the solicitation of congestion
report messages in multicast based adaptive video coders.
As the session messages give us a reasonable approximation of the size of the conference at the
point when we generate the summary message, the sliding key can be started close to the point
where it would be expected to elicit the first response if all receivers need a retransmission. For a
typical thousand way conference, where only one receiver requires a retransmission, with each
key/mask pair sent twice per round and an estimated worst case RTT of 250ms, this results in 4
(small) packets per second and a maximum delay of 5 seconds before requesting retransmission.
Note that this is a much shorter time than can be achieved by having each receiver randomly
choose a delay from a sufficiently large uniform interval to ensure approximately one
retransmission request is made. If the conference was smaller, or more sites had suffered loss,
this time would be reduced.
Limitations of the Data Model
We have described a data model and a distribution model which are oriented towards building a
scalable distributed shared text editor. However, the data model and more importantly its
distribution model impose a set of limitations on the functionality of the shared editor, or on the
way this functionality is implemented.
Whilst a data model based on blocks and lines allows different blocks or lines to be modified
simultaneously without any problem, the lock-free distribution model and the choice of a line as
ADU mean that it is possible for more two or more users to attempt to modify the same line
effectively simultaneously.

Adri Jovin J.J.
Page | 210
In addition, network partitions can result in more complex inconsistencies arising. We shall show
below that a few simple choices can result in all such more complex inconsistencies becoming
equivalent to a detectable case of effectively simultaneous insertion of lines into the same place,
and that this can be resolved as all the alternative information is available at each site.
Effectively Simultaneous Changes to the Same Line
By ``effectively simultaneous'' we mean changes to the same line where due to network delay, or
partitioning, two users attempt to change the same line without seeing the effects of the others
changes first. Thus the timestamps of the changes may not be identical.
Effectively Simultaneous Changes due to Transmission Delay
NTE is designed for use in multimedia conferences where there are other channels of
communication between users than just NTE. Thus two users whose sites are not partitioned
from each other attempting to modify the same line effectively simultaneously will see the
change made by the later site being substituted for the change made by the earlier site. If such
changes do not involve adding a line break to the middle of a line, then no lasting confusion
should remain, and at least one of the users will be notified of what is happening. Hopefully, at
this point the users will talk to each other and decide who should actually make the change. Even
if the users do not have an out of band communications channel, they can always create a new
block to make meta-comments to resolve the confusion.
It should be noted that although there are circumstances when one site does not even see the
change made by the other site (for example, Site 1 does not seer Change 2 in figure 8.5), it is
possible for the user interface to flag the fact that another user is attempting to modify the line,
so that both users do realise exactly what is happening.
Figure 8.5: Two users attempting to simultaneously modify the same line

Adri Jovin J.J.
Page | 211
If one of the users adds a line break to the middle of a line, this is mapped into a truncation of the
existing line, and the creation of a new line after the existing line with the remainder of the text.
The overlapping request, if it happens after the line break insertion, but before the receipt of the
message (see figure 8.6) will then re-assert the previous line. This will be confusing for users, as
suddenly text has been duplicated.
Figure 8.6: Undesirable behaviour due to simultaneously splitting a line and modifying it.
One way to deal with this is to maintain a very short history, and if a remodification of a line we
just modified arrives within the expected round trip time, we reassert (and retransmit with a new
timestamp) the old unmodified line, and delete the extra created line. This strategy can of course
be defeated by packet loss, but may reduce confusion in those cases where the problem can be
detected.
It should be noted that packet loss can be treated as an abnormally long transmission delay for
these purposes.
An alternative depends on the model we use for deletion - if deletion is an irreversible operation
(i.e., deletion overrides modification), then simply deleting the line to be split and inserting two
new lines in its place prevents this undesirable behaviour.
Effectively Simultaneous Changes due to Network Partition
Whereas in the case of simultaneous modification due to transmission delay, we could
effectively backup with a minimum of confusion to the user (so long as we inform them why
we've done so), we cannot do so in the case of simultaneous modification due to network
partitioning.

Adri Jovin J.J.
Page | 212
If only single lines have been modified, there is no real problem, as the later change will be
asserted, and although this may not be what the user whose change just got replaced actually
wants, at least the document ends up in a consistent state.
However, if the user making the earlier modification adds any new lines (particularly if they do
so by inserting a line-break into the middle of a line), then there is a potential for the document to
end up in an inconsistent state. We shall show later that we can detect simultaneous insertion,
thus to maintain internal consistency, we should treat the splitting of a line (by adding a
linebreak to the middle of an existing line) as deletion of the old line, plus insertion of two new
lines.
It is also possible to keep a local copy of the state of any line we have modified at the time we
last modified it in order to be able to re-assert the state if the conflict resolution was not what we
actually wanted. However, it is always possible to envisage ways to defeat such a scheme if it
were to be performed automatically, so such a re-assertion should only be performed with
explicit consent from the user for each step.
We hope that these mechanisms will not be often called upon, but choosing a line as our
minimum unit means that there will always be occasions when such mechanisms are called upon.
How effective they are depends largely upon how well it is communicated to the user what has
happened, and what options they have to deal with it.
Deletion of Lines of Text
Deleted lines need to be stored as if they were modified lines, with a marker that they are no
longer visible. If they were not stored as actual objects, then they may be re-asserted by sites
which have missed seeing the deletion event. Unfortunately it is necessary for deleted line ID's
and deletion times to actually be transmitted and stored at all sites to prevent unintentional re-
assertion of deleted lines after the deleting site has left the conference.
If an entire block of text has been deleted, sites only need hold the ID and deletion time of the
block - the holding of information of lines within the block is unnecessary.
To prevent the re-assertion of lines or blocks after deletion when a network partition is resolved,
it is even necessary to send deleted block and line ID's and timestamps to new sites joining the
conference, so that they can continue to suppress re-assertion of the deleted line or block after all
the other members of this partition have left the conference.
To prevent unnecessary usage of bandwidth and memory, line and block IDs could be re-used for
new lines and blocks. However, this is dangerous as a deletion event and the subsequent re-use
of the line ID in one partition, followed by the modification of the original line in another
partition can lead to undesired consequences. It would be possible to consider schemes where a
line ID can only be re-used in a different block from the one it was previously used in, and thus
the line's block id field can be used for collision detection, but if the site originating the ID is no

Adri Jovin J.J.
Page | 213
longer reachable, it is far from obvious who can resolve the conflict without introducing further
complications. Thus in NTE, we have not attempted to implement any ID re-use system.
The most significant design decision here concerns the effects of network partitioning - if a user
at a site in one partition deletes a line, and subsequently another user in the other partition
modifies the line, we have two choices when the partition is resolved:
• Rely only on modification time (i.e. treat deletion as just another modification that can be
rectified later if required)
• Impose an ordering on operations such that deletion is of ``higher order'' than normal
modification, such that deletion always prevails over modification.
If we treat deletion as modification, then when several neighbouring lines are deleted in one
partition, and one of the middle lines is subsequently modified in the other partition, a single line
would be re-asserted with no surrounding context. There are mechanisms we could employ to
deal with this, but they tend to add unnecessary complexity, and generally are less natural to
users.
If we treat deletion as a higher order operation, then a different scenario causes potential
problems - a line can be split into two lines in one partition (one modified line and one new one),
and subsequently deleted in another partition, leading to only half a line remaining after the split
when the partitioning is resolved.
We took the decision that deletion is a higher order operation, as this provides us with a
mechanism that is irrespective of causal ordering in a partitioned network, and this causal
ordering independence provides a mechanism which we can utilise to achieve internally
consistent global consistency.
Thus the problem above where a line is split in two (a ``newline'' is added to the middle of the
line) can thus be resolved by deleting the line we wish to split and inserting two new lines in its
place. This achieves internal consistency because even if the line split in one partition is
modified or deleted in the other partition, the deletion operator overrides the modification
operator, and the two new lines then take its place. Of course if a line is split in both partitions,
then we now get both versions of the line after partition resolution, with no globally consistent
view of the line ordering in the block. Thus we've moved the problem to that of effectively
simultaneous insertion of lines. However, effectively simultaneous insertion is a detectable
situation, as we shall show below.
In fact to achieve inconsistency detection, there is one further requirement with respect to deleted
lines; they must not only be kept at all sites to prevent re-assertion of the line from a site that
missed the deletion, but they must also be kept in their original place in the list of lines at each
site. Thus, a line preceding a deleted line must still be transmitted with its "next line" field

Adri Jovin J.J.
Page | 214
indicating the deleted line. This is necessary so that simultaneous insertion is a detectable
situation.
Moving of sections of text
There are two ways to view the moving of a chunk of text:
• Deletion and then insertion of the contents of the deleted text
• Changing the bordering lines neighbours (sending modifications of the top and bottom
lines of text specifying new neighbours)
Deletion and then re-insertion is wasteful of bandwidth and causes extra state to have to be held
(and sent) for the deleted lines. We are sure however that no-one can have already changed the
new lines, assuming deletion is not overriden by modification, that the original lines cannot be
re-asserted.
Modifying the context information of the bordering lines and just re-sending this information has
the possible advantage that someone modifying the moved text in its old position due to
temporary partitioning sees her changes reflected in the new positioning after the partitioning has
been resolved. However, it is possible that one or more of the bordering lines are modified in a
partitioned site after the move has occurred, and that the resolution of the partition then undoes
or partially undoes the move, resulting in a block where no-one knows the correct line ordering.
Allowing the possible situation of no-one knowing the correct line ordering is extremely
undesirable, and so we treat moving as deletion and subsequent re-insertion, despite the
additional overheads this entails.
Garbage Collection and Check-pointing
Treating deletion as a higher order operation than modification gives us a mechanism for both
check-pointing of blocks and of garbage collection of deleted lines. Both of these can be done by
deleting a block, and then re-asserting the same data as a new block and lines. Although this
might seem wasteful, it means that for the additional cost of keeping deletion information for a
single block, we no longer need to store the deleted lines that used to be in the block. In addition,
it means that if there is a network partition that is subsequently resolved, any changes made in
the other partition will not be asserted. Of course this has to be implemented carefully, as the
users in the other partition may not wish to see their changes simply discarded, but there are
times when it is desirable to assert the current state. If both partitions checkpoint the same block,
then the block will be duplicated at the time of partition resolution, which may or may not be
what is desired.

Adri Jovin J.J.
Page | 215
Effectively Simultaneous Insertion of Lines
Effectively simultaneous modification of a single line always results in a block that is internally
consistent and the document in a globally consistent state. We've shown that if deletion is treated
as a higher order operation than deletion, then combinations of moving, deletion line splitting
and so forth can be dealt with, but with one potential side-effect - all these operations can result
in effectively simultaneous insertion of lines.
Effectively simultaneous insertion of lines is a slightly harder problem than the problems
discussed above, because it not only results in a document that is not internally consistent (we
have two copies of essentially the same line or lines) but also the document is not globally
consistent either (different sites can have different views of the document). This occurs because
two (or more) lines each have the same neighbouring lines. The problem can be detected at all
sites, because a set of lines is received that should fit in the same place in a block as a set of lines
we already have.
At this point the choices we made above, about deletion being a higher order operation than
modification but that deleted lines are retained, and moving of lines being treated as deletion and
re-insertion, ensure that if a newly received line has a particular previous line, then there are only
three choices for where the next line is:
Figure 8.7: Simultaneous Insertion is a Detectable Situation

Adri Jovin J.J.
Page | 216
Figure 8.8: Simultaneous Deletion and Insertion - no inconsistency
Figure 8.9: Simultaneous Deletion and Insertion resulting in inconsistency
1. There is no next line. This line is the end of the block.
2. We have seen the next line, and it is later in the block (possibly deleted).
3. We have not seen the next line.

Adri Jovin J.J.
Page | 217
If we have not seen the next line, then we know we have a potential inconsistency, and thus we
should not display the received line until we have the next line too. We wait until the line's ``next
line'' arrives, and again go through this process again. If the next line is a line we've seen, or the
end of the block, then we have enough information to know whether we have a problem or not.
We can therefore treat a connected set one or more new lines as if they were a single line - if the
line in our existing block following the ``previous line'' of the new data is not the same as the
``next line'' of the new data, then we have an inconsistency which must be resolved (see figure
8.7); otherwise we can unambiguously display the lines.
Now, so long as we keep deleted lines in their original place in the block and simply don't
display them (see figures 8.8 and 8.9 for examples), and so long as we treat moving as deletion
and subsequent insertion, then there is only one way such an inconsistency could occur - that
alternative new lines were inserted effectively simultaneously. Thus the question comes down to
which of the two sets of lines to keep and which to delete. So long as this matter is resolved
globally consistently, then global consistency can be restored to the block. In fact it is not quite
so simple as this, because if we delete one version of the lines, we must ensure that the block is
globally consistent including the position of the deleted lines or we may not be able to detect
further inconsistencies at all sites. However, we do not care about the order of a set on
consecutively deleted lines, so instead of deleting one of the two alternative sets of lines, we
must actually delete both alternative sets of lines and re-insert one of them again.
The choice of which alternative set of lines to retain can be made automatically (by the ``current
site'') or it can be made clear to the users that there is a problem that requires resolution. Either
way, if more than one site attempts to resolve the conflict, the cycle is simply repeated again.
Summary of Inconsistency Avoidance Mechanisms
To summarise the limitations necessary to ensure eventual consistency after the resolution
network partitioning:
• Simultaneous modification of a single line does not result in eventual inconsistency.
• Deletion must always override modification, irrespective of the timing of the two
operations.
• Deleted items must be transmitted to and stored at all sites to prevent re-assertion.
• Deleted lines must be kept in their original place in a block, and must remain referenced
by their neighbouring lines - this is a precondition for simultaneous insertion detection.
• Moving of text must be performed by deleting all the original lines containing the text to
be moved and inserting the new text in new lines, thus preserving the original line
ordering. This includes splitting a single line by adding a line break to the middle of it.

Adri Jovin J.J.
Page | 218
• If all the above are performed, line ordering within a block is only changed by adding
new lines. This makes simultaneous insertion of lines during a network partition a
detectable and resolvable situation.
These restrictions will ensure that eventual consistency is achievable, even in the face of
continuing modification during a network partitioning. However, they are not always sufficient
to ensure that the contents of the document eventually converge on what all of the users actually
wish it to be. Although this could be achieved by locking of blocks to ensure that only one
person can modify the a block at a time, we believe this restriction is unnecessary and would
restrict usage of the editor. Indeed, under many circumstances, the usage patterns of the editor
are likely to be such that large scale simultaneous editing of a block during a network
partitioning will not happen because the vocal discussion needed to do so will be possible. If
users are concerned about simultaneous editing of a block during a network partition, they
should checkpoint the block to ensure no unseen changes can be made to it. For paranoid users,
this checkpointing procedure could be automated, although we do not believe this is desirable or
necessary.
Usability Issues
Much work in the field of Computer Supported Collaborative Working (CSCW) has started from
a detailed set of usability requirements, and attempted to engineer a system to support these
requirements. In single user systems, this works well, but for near-synchronous shared tools,
there is a real danger that this result in systems that do not perform well in real-world networks.
We have started with a high-level set of usage and performance requirements, and worked
through a solution that satisfies these high-level requirements. Such a solution imposes a tight set
of constraints on the operations that can be performed on the data set. In designing wb, it appears
that Van Jacobson also pursued this design path, although as the high-level requirements were
slightly different, their resulting constraint set is different. There is, of course, a danger that in
pursuing this type of path, the resultant system performs well but does not properly support any
required task. It is dangerous to attempt to draw general conclusions from a small number of
examples, but we believe that both nte and wb have resulted in systems that support the users'
tasks well. Furthermore, we believe that engineering a CSCW tool around a set of constraints
that ensure good performance is likely to result in a tool that is more likely to be used.
Having said this, it is vital that detailed usability requirements are given a great deal of attention
when working within our constraint set. The detailed requirements we used to design nte include:
• Nt should be as simple as possible to use. It is not a word processor, as is not intended for
single user usage.
• It should always be possible to tell who is in the conference.

Adri Jovin J.J.
Page | 219
• It should always be obvious who is performing any change at any time.
• WYSIWIS (What You See Is What I See) is not a requirement. Different users should be
able to edit different parts of the document simultaneously, even if these parts are
separated by several screens of text.
• The primary user operations are:
o Block creation
o Text addition
o Text deletion
o Block deletion
o Block moving
o Cut, Copy and Paste of text
o Search and replace within a block
o Setting the font and colour of a block for identification purposes.
o Using a Shared Pointer
o Loading text from a file
o Saving text to a file
• We expect that no other user operations on the text need to be supported as they are
unnecessary and increase the complexity of the user interface. Other operations should be
added only where they aid the above operations.
• It should be possible to prevent other users from editing a particular block of text - a kind
of ``annotation only'' mode.
• It should be possible to save the resulting text as either ASCII text or as structured text
maintaining the block structure.
• It should be possible to save selected blocks only.
There is little point in discussing all of these detailed design decisions here, as most of them are
uninteresting. However, a few are worthy of mention, as they were not obvious (at least to the
author).

Adri Jovin J.J.
Page | 220
Asynchronous Events - informing the user
One requirement is that it should always be possible to see who is changing what at any time. In
nte we achieve this by placing a labelled graphical icon at the place a change is happening. This
is necessary not only because it identifies who is making a change, but also because it is not
always very obvious that a change has been made. For example, single character changes are not
easy for a user to spot if his attention was focused somewhere else at the time. Thus the
appearance of the icon serves to mark for a few seconds the location of a change, the nature of
the change, and (because the icons is labelled) who made the change.
In nt, all changes are marked in this way. However, when we started to use nte in group
conferences, it became clear that the screen could get very congested, and also that when a block
of text was deleted, users often could see that a block of text had been deleted and who deleted it
from the eraser icon and who deleted it, but could often not remember which text it was that was
deleted. The reason for this was that their attention was elsewhere, and all they could tell was
that something had gone. The solution was simple - not only display the eraser icon and its label,
but also fade the block away so they can see what is happening as it happens. In practice, this
fade need take no longer than a second to remove the sense of surprise that the users were feeling
when blocks were deleted.
Block Locking
As mentioned above, one detailed requirement is to lock a block so that only its creator can
modify it. There are a number of cases where this is important, but the most likely one is a
scenario where a piece of pre-prepared text is included in the conference for discussion. In this
case, comments may be added as annotations, but the original text most not be modified.
So long as a block has associated with it the meta-data of who its creator is, then locking a block
(at least amongst co-operating applications) is simply a case of marking that block as not being
editable.
However, given the unreliable nature of the underlying communications protocol, if a block is
created as unlocked, and then subsequently, locked there is no guarantee that all sites will receive
the ``lock'' message. Given the scalability and resilience requirements, we do not even know for
certain exactly who is currently in the session. Thus any site that (temporarily) missed a lock
message is still free to modify the locked block until such time as the retransmission scheme
succeeds in propagating the lock message. Given a network partitioning, this could be a
significant length of time. Whilst this circumstance will probably be rare in occurring, we would
like the block locking to be a definite thing - if a user thinks a block is locked, there should be no
way for that block to be modified. A similar, though slightly less strong, argument also holds for
unlocking blocks.
There are a number of possible ways to ensure that block locking is unambiguous, but many of
them infringe our previously stated requirements. The following is in line with our requirements:

Adri Jovin J.J.
Page | 221
1. Allow new blocks to be created as either ``locked'' or ``unlocked''. Do not allow a block to
ever be changed between the two states.
2. Only allow a block to change state from locked to unlocked or vice-versa by deleting the
block and retransmitting its data as a new block.
In the current version of nt, we have only implemented 1. above. We do not believe that block
locking is frequently required, and when it is, it is most often required on a block whose data has
been loaded from a file. Making the decision about whether or not a block should be locked at
load time is quite natural. The default for blocks created on-the-fly is for them to be unlocked.
Generalising the Models
Nt, its data model, and its underlying protocol were all designed to solve one specific task - that
of shared text editing. We used general design principles - those of IP multicast, light-weight
sessions, and application level framing as starting points. However, the application data model is
intended only for text. The data distribution model uses the redundancy achieved through
treating a line as an ADU combined with the fact that most successive modifications are to the
same line to avoid the need for most retransmissions.
However, the restrictions the data distribution model impose on a data structure consisting of a
doubly linked list of application data units can perhaps be generalised somewhat. The imposition
of a strict ordering of ADUs, combined with marking deleted ADUs whilst leaving them in
position in the ordering, allows the detection of inconsistencies caused by network partitioning in
a loose consistency application. It is perhaps not easy to see how this could be applied to other
near-synchronous shared applications, so an example is in order.
In nt, the stacking order of blocks of text is a local issue so that overlapping blocks can be
simultaneously edited. However, in a shared drawing tool, stacking order must be a global issue,
as most drawing tools use overlaying of one object over another to produce a more complicated
object. In a shared drawing tool, each drawing object (circle, polygon, rectangle, line, etc) is the
drawing equivalent of a line of text. The concept of a block does not exist as such, but instead
there is a strict ordering (analogous to line ordering in a block) which is imposed by the stacking
order. Thus, the same set of constraints that apply to lines of blocks in nte should also be applied
to the stacking order of drawing objects in this hypothetical drawing tool.
The retransmission mechanism used in nte is novel, in that its requirements are perhaps atypical
of shared applications because of the wish to exploit redundancy. For many applications, SRM is
a more appropriate choice of retransmission mechanism, as, given a stream of packets with
sequence numbers, it is likely to be more timely. However, for applications where retransmission
is a relatively rare phenomena due to redundancy or other relaxed consistency requirements, and
where we desire a sender controlled system, NTE's retransmission scheme has some possible
benefits. Its sliding key messages can carry the latest ID and timestamp, which can be used to

Adri Jovin J.J.
Page | 222
ensure that partitions are resolved at the earliest opportunity. Although we do not use the
property in nt, sliding key schemes can be used to ration retransmission requests - this might be
useful where the reverse path from receivers to senders is bandwidth limited.
Video Conferencing
Videoconferencing may be described as a method of conferencing between two or more
locations where both sound and vision are transmitted and received so as to enable simultaneous
interactive communication.
Due to its cost it was originally only used by multi-national companies to link worldwide sites.
However, as the technology has improved and costs have fallen dramatically, it is now used
extensively in education and commerce. Videoconferencing can save significant amounts of
money in terms of both travel costs and time it can also open up new methods of communication
e.g. linking several schools together to enhance the learning experience. Videoconferencing is
certainly growing very rapidly, and can save a great deal of money, but is it a new technique?
No, videoconferencing has been with us for decades. Multinational corporations, for example,
have been routinely using it since the 1980’s.
Most television viewers would have experienced videoconferencing as a passive observer
without realising it. The television broadcasters have routinely used two way interviews (for
sports coverage etc) from as early as the 1960’s, where a link presenter in the studio interacts
with a football manager; for example, at a remote football ground. These two are actively
videoconferencing, but are not normally considered in this light. This type of conferencing is
achieved at a high technical standard with broadcast quality sound and pictures.
So why has videoconferencing become topical and much more popular since the 1990’s? The
main reasons are the advances in technology and equipment, making these now available at a
reasonable cost. Both of the examples outlined above historically used dedicated methods of
connection between the sites. Dedicated connections, whether by landline or satellite link,
provide high quality but are very expensive so the need had to be overwhelming to justify the
cost.
Introduction to Video Conferencing
The introduction of dial up digital connections by the telephone companies and access to the
Internet coupled with parallel advances in technology have now made reasonable quality
conferences to a remote site affordable to a much wider group of users. At an even lower quality
and perhaps not really applicable to education, two-way videoconferencing is fairly common
between mobile phone users. The Microsoft Network (MSN) is another platform which enables
this low end videoconferencing.

Adri Jovin J.J.
Page | 223
Basics
Videoconferencing can conveniently be broken down into 3 components:
1. The conference environment, i.e. the classroom or conference room
2. The conference equipment, that converts the images and speech of the participants into a
format that enables transmission to a remote site over the network
3. The conference network that links the sites together
All videoconferences must have these three basic elements, however simple or complex; the
conference can always be synthesised into environment, equipment and network. Conferencing
is not limited only to talking heads, i.e. the participants; slides from a PowerPoint presentation
running on a PC may be introduced to illustrate a particular point, or perhaps moving sequences
from a PC application or DVD player. It is even possible for a PC application running at one site
to be shared with another site so that either site can annotate and take control of the software.
When PC documents are shared in this way it is termed “Data Sharing”.
Application of Video Conferencing
Meetings: this is probably the most popular application. The cost savings can be appreciable
especially for international conferences where not only the travel costs are saved but also the
significant time spent travelling to destinations. A company with branches in Europe, the United
States and the Far East can use videoconferencing to effectively communicate and run its
business. Geographical distance is no longer a barrier. This “travel replacement”, though, can
sometimes act as a disincentive for some people to videoconference, as despite the real costs and
inconvenience of travel many actually enjoy the experience especially when they are able to
cover the cost with expenses.
Teaching: videoconferencing can open up new dimensions to teaching. With videoconferencing,
the teacher no longer has to be in the same room as the students. A national or even an
international subject specialist can be linked to any number of sites in the United Kingdom or
even the world. The lack of a local expert need not prevent students following a particular
course. Several sites may also collaborate to present a subject more comprehensively. For
example, Monkseaton High School in North Tyneside regularly shares language teaching with
other schools; they also conference with a local school as a member of the “Partner Scheme” to
assist students who have physical and or communication difficulties.
The ability to introduce illustrative material either as still images or moving sequences to both
the local and remote audiences enhances the presentation. Videoconference sessions can take
many forms including:

Adri Jovin J.J.
Page | 224
1. There is a now a range of software which facilitates one-to-one conferencing from PC
desktops. The quality and available features vary widely.
2. A teacher by him or herself teaching remotely to a group of students.
3. A teacher in a classroom teaching his own and students in another school.
4. A teacher with both local and remote students but with the added dimension of full interaction
from both student groups.
In the second scenario, the teacher can conference from any convenient room suitably equipped,
even an office (see Figure 1). They can select appropriate sources for their lesson i.e. camera,
Whiteboard, PC, DVD player etc. as appropriate. The remote site will be able to interact and
interject in a similar same way that would occur during a normal lesson even though the teacher
may be hundreds of miles away. The teacher’s main concern would be to transmit decent images
and accompanying sound. A link person would coordinate the students and equipment at the
remote site.
The third situation requires a different approach. If a local teaching session to a group of students
is also simultaneously relayed by videoconferencing to a remote site then the room needs to be

Adri Jovin J.J.
Page | 225
specially arranged. The teacher will now need to face the local students but also be seen clearly
at the remote site. The room would require to be laid out in a similar way to Figure 2 below. The
local students can view PC illustrations etc. on the local picture monitor.
Figure 2: Teaching Local Students and Relayed to a Remote Classroom

Adri Jovin J.J.
Page | 226
The teacher will also need to view their own image together with any illustrations that they are
using while facing the local audience. These illustrations may originate from a PC, a DVD player
or from a Whiteboard now in common usage throughout schools. A small television monitor
termed a preview monitor is shown on the desk in Figure 2 and is used to view any of these local
images prior and during transmission to the remote site. As the lesson is simply being relayed to
the remote site, interaction is not expected so remote site pictures and sounds are not relevant.
Finally, where teaching is being conducted both locally and remotely and full interaction is
needed, we have a more complex situation. Pictures and sound are now required from the remote
site. The local students and the teacher will need to see and hear the remote students. A second
television display facing the students will normally be provided but the teacher also requires a
second television preview monitor on their desk facing them in order to view the remote students
(see Figure 3).
Television picture monitors have been illustrated above, but in many cases data or large screen
projectors will be better for the student displays as the images can be much larger than those

Adri Jovin J.J.
Page | 227
produced by television monitors and still be acceptably bright under ambient lighting levels.
However, care has to be taken with some data projectors as the noise from their cooling fans can
be quite intrusive.
In higher education, large scale teaching over videoconferencing is carried out to overcome
geographical problems. The Welsh Video Network links most Universities and Colleges in
Wales. In Scotland, the remote Highland and Island areas have been linked together to create
The University of the Highlands and Islands so that students may link with partner institutions
for collaborative teaching.
Management: colleges with split campuses need to communicate regularly and efficiently, and
videoconferencing is seen as a cost-effective solution. A good example is the Open University
who conference between sites and also link remote sites to their regional centres.
Remote Diagnosis: where specialist medical resources are scarce e.g. in a rural area, cottage
hospitals and GP surgeries can be linked to the regional teaching hospital. Patients can then
receive rapid specialist diagnosis.
Interviews: this use is growing in popularity. It is found by many users to be just as effective as
face-to-face interviews, particularly when data sharing is used to compare CVs etc. For
international interviews, videoconferencing could include candidates that previously would have
been excluded purely on grounds of cost. This is particularly relevant for local authorities. Some
interviewers, however, regard the conference experience as quite inferior when compared to a
face-to-face meeting and complain that the medium lacks spontaneity, and that nuances of body
language are lost. Other interviewers use videoconferencing as a first stage filter in the selection
process. They are thus able to view many more potential candidates due to the cost savings,
especially where these applicants are based overseas.
Legal Work: the nature of many legal cases can involve communication between parties in
different locations, even different continents. Videoconferencing helps to reduce costs. It is also
being used for live in-court sessions. In particular, sensitive cases e.g. those involving child
abuse, intimidation and rape, videoconferencing can make the experience far less traumatic for
the victims.
Advantages
• Convenience
• Cost savings for travel, accommodation and staff time
• Ability to link several sites simultaneously
• Access to remotely located experts

Adri Jovin J.J.
Page | 228
• In child abuse or other court proceedings, victims’ evidence provided via
videoconferencing can reduce the potential of intimidation
• Having a set time for the meeting encourages more control and less time wasted on non-
agenda items
Disadvantages
• The quality of the received images can be compromised by the technology
• On lower quality links, movement can be jerky
• Body language can be lost if movement is jerky and/or picture quality is reduced
• There may be a delay on the sound that participants need to get accustomed to
• Some believe that the atmosphere of a normal face-to-face meeting is lost
Certainly some of these disadvantages may affect the experience, especially for the lower quality
links. For meetings, it is said that if you already know the participants it is a distinct advantage.
This is probably true for most meetings but may be especially important for videoconferences
where the technical quality is compromised.
The Conference Environment
School rooms that are effective for conventional teaching and face-to-face meetings may not be
suitable for videoconferencing, but music rooms may provide a good starting point for a
conferencing room as they could already possess some of the desirable characteristics. The
provision of an effective environment for videoconferencing requires particular attention to the
room’s characteristics; this is true both for sophisticated teaching sessions and simple desktop
conferencing from a PC with plug in cards. The room needs to be quiet, as microphones can
accentuate unwanted sounds. The human ear may be quite tolerant of ambient noise such as
traffic, air-conditioners, students in corridors etc. but a microphone is not so discriminatory and
the intrusive noise can effectively render a conference unworkable. Echoes within a room (or
reverberation) must be controlled otherwise the conference sound will be degraded. This is
particularly relevant in schools where some classrooms can suffer severely from echoes.
Television cameras are sensitive to changes in light levels so windows pose a special problem, as
light levels can fluctuate wildly. The overall lighting level also needs to be high to produce the
best possible images from the camera. Decorations within the room need to be chosen carefully.
Busy patterned backgrounds are not ideal; plain matt surfaces are to be preferred.
All of these factors and several more are covered in greater detail in the VTAS document
Videoconferencing Rooms. An acceptable environment is the starting point; without this, sound
and vision quality will be impaired. Unfortunately many potential users of the medium have been

Adri Jovin J.J.
Page | 229
put off by a bad experience in an unsuitable room. If the characteristics of the room are
unsuitable, no amount of expensive equipment will correct the resulting poor quality of the
pictures and sound.
Videoconferencing Equipment
In its most basic form, a videoconference involves transmission of the images and the voices of
the participants to a remote site. Optional sources include still images and/or moving sequences
from a video recorder, DVD or a PC. The basic conference requires:
• A television camera to capture images of the participants
• A microphone to pick up their speech
• A means of transferring this sound and vision information to the remote location
The Television Camera and Lens
The camera converts light images into an electrical signal so that it may be displayed, recorded
or transmitted. The camera lens focuses the images onto a light sensitive layer sometimes
referred to as a Charge Coupled Device or CCD. The type of lens determines the angle or field of
view. A wide-angle lens can capture a large group, a narrow angle lens a close up. The
ubiquitous “Web-Cam” is a very low cost (and low quality) camera with a fixed focus lens
designed only to provide an image of the PC operator. Lenses suitable for group conference
applications will have a variable field of view these are known as Zoom lenses. Zoom lenses can
“zoom out” to a wide-angle shot to include a group or “zoom in” to a close up shot of a single
participant. The sensitivity of the lens is determined by the amount of light entering the lens;
most cameras adjust sensitivity automatically but sometimes there is manual adjustment. Both
methods move an “iris-diaphragm” in front of the lens to effectively increase or reduce the
diameter (or aperture) of the entry pupil of the lens. Most cameras control the iris automatically
to optimise image quality for the available light. It is important to realise, however, that in low
light conditions, as the iris is opened up to increase the sensitivity there will be less depth of
field. The resultant picture overall may not appear sharply focussed. Television cameras are
capable of operating in very low light conditions but they produce much higher quality pictures
at higher light levels. At high light levels the lens aperture will also be reduced to give the added
advantage of a greater depth of field. In broadcast television this increase in depth of field may
not necessarily be seen as an advantage. For artistic effect, directors may require the foreground
to be in sharp focus with the background out of focus. This gives separation to the images.
However, in videoconferencing the general rule is the more depth of field the better.
Microphones
A microphone converts sound energy into an electrical signal to enable it to be recorded or
transmitted to a remote location. It has already been mentioned that microphones also pick up

Adri Jovin J.J.
Page | 230
unwanted sounds together with the participants’ voices. If these unwanted sounds are significant
they will interfere or even completely obscure the desired parts of the sound. The correct choice
of microphone and its position in the room are vital to achieve high quality sound. The
conference environment will also influence sound quality, as has been mentioned above both low
ambient noise levels and low levels of echo are essential requirements.
The CODEC
The CODEC has two components: the COder and the DECoder. The COder takes the local
sound and vision signals and converts them into a form that can be transmitted over a digital
network. The DECoder performs the reverse function i.e. it takes the remote site’s digital signals
from the network and converts or decodes them into a form that enables the picture monitor to
display images and the loudspeaker to radiate sound from the remote site. A CODEC is thus
required at each end of the link for a videoconference. CODECs can be packaged as plug in PC
boards for personal desktop use, within purpose designed complete systems for group
conferencing known as “Rollabouts”, or as compact “Portable” units that sit on top of picture
monitors.
From 8.1 above, the electrical signals generated by the camera may after suitable processing be
recorded. They may also be displayed on a picture monitor or data display. Similarly from 8.2,
microphone signals can be recorded or heard via a loudspeaker. It is also possible to transmit
these electrical signals directly to another site. These electrical facsimiles of the sound and
picture are termed analogue signals. Analogue television signals require a network with a huge
capacity for transmitting the information, i.e. a very wide bandwidth. Analogue networks are
capable of very high quality but can be very expensive to install. They also generally need a
dedicated circuit i.e. only available between specific sites for a particular application. For a short
link e.g. between rooms, dedicated circuits are cost effective as they can be achieved simply by a
length of cable, but for anything further the costs can be prohibitive. Videoconferencing using
analogue methods was established by multinational corporations in the 1980’s, it was also
employed by the broadcasters for sports interviews etc. The potential cost savings
(multinationals) and the importance of the message (broadcasting) outweighed the high network
costs involved. With advances in technology, analogue transmission has virtually ceased; digital
transmission is now the norm for videoconferencing and both radio and television broadcasting.
For videoconferencing to appeal to a wider audience it had to prove cost effective. This was
solved by the arrival of digital networks that were easily accessible.
Analogue and digital signals are very different. Human senses respond directly to analogue
stimuli. A typical example of an analogue signal is shown in Figure 4, the sound wave generated
by a vibrating tuning fork. A microphone close by will convert the sound variations into an
electrical signal that mimics the original. The microphone signals vary both in level (amplitude)
and frequency (pitch) dependant on the sound being picked up. The electrical signal is a

Adri Jovin J.J.
Page | 231
facsimile of the sound wave. Analogue networks therefore need to respond to both amplitude and
frequency variations to transmit the information accurately.
Figure 4: Analogue Signals
Digital signals are transmitted as a series of electrical pulses that vary in sympathy to the original
signal in various ways, (dependant on the digitisation system in use). The level or amplitude of
the pulses is of no significance. Only the presence or absence of a pulse needs to be detected.
Digital transmission is thus much more tolerant of poor networks.
The Network
Digital networks that are now almost universal have encouraged the current expansion in
videoconferencing. They have enabled the medium to be affordable to a much wider market.
The Internet
All schools now have access to the World Wide Web via the Internet. This technology uses a
common method of transmitting and receiving data called IP or Internet Protocol standard to
ensure information passes flawlessly between networks. The success of the WWW could not
have been achieved without this single ubiquitous standard.
A disadvantage of IP transmission is that because all data traffic e.g. e-mails, WWW downloads
or videoconferencing has to share the same pipes, they all have to compete for the available
space or bandwidth. With e-mails and WWW downloads, this may cause delays between sending
and receiving data dependant on the traffic, but all of the transmitted data should arrive
eventually at its destination although some parts of the data may have taken different routes
before finishing its journey. Videoconferencing is not so tolerant. Videoconferencing transmits
digitised sound and vision signals and both are very intolerant of missing or delayed parts of the
data.
When a network is congested and some parts of the data are routed via an alternative but longer
path, the recovered sound or vision can be impaired. The degradation in quality may vary from a
missing speech syllable or image fuzziness to completely incoherent speech and an
unrecognisable image. IP transmissions do not normally guarantee quality of service at present.

Adri Jovin J.J.
Page | 232
However, there are techniques being used more frequently by the telecommunications providers
that prioritise vision and sound signals so that they can pass relatively unscathed.
Integrated Services Digital Network (ISDN)
Before the Internet achieved its present ubiquitous state and reached every school the telephone
network offered its own digital communication via telephone lines. Termed ISDN (Integrated
Services Digital Network), this uses the existing telephone infrastructure to carry digital signals.
Offered by telecommunications providers it is available in many parts of the world. The service
is accessed through dial up in the same way as a telephone, so the network does not have to be
dedicated between the videoconference centres. The costs include installation (for the special
interfaces), rental and call charges. The quality of ISDN videoconferencing is limited by the
quality (or bandwidth) of the connection varying from 64kbit/s (ISDN-1) to 2Mbit/s (ISDN-30).
The normal bandwidth used for meetings is either 128kbit/s (ISDN-2) or 384kbit/s (ISDN-6).
One important feature of ISDN is its Guaranteed Quality of Service (GQOS) which means that
if, for example, a 384kbit/s link is dialled then unless there is a fault condition the full 384kbits/s
will be provided for the duration of the conference. The Internet, because of the way the data
traffic has to compete for space, is currently unable to offer GQOS. ISDN maintains identical
down and upload speeds.
IP and ISDN CODECs
As has been previously mentioned, a CODEC (see 8.3 above) is necessary to enable the sound
and vision signals to be converted and then transmitted from the local site. Another CODEC is
also required at the remote site to convert the digital signals back into sound and vision signals
for the audience. The method of transmission for IP and ISDN are different and so require
different CODECs. ISDN, once the mainstream medium for videoconferencing, has been
overtaken by IP so that IP CODECs are now usually offered as standard with an ISDN option if
required.
Broadband Networks
Computers connected to the Internet via a modem and telephone line are limited to a data rate of
around 56Kbit/s. Broadband connections using ADSL (Asymmetric Digital Subscriber Line)
technology achieve rates from 512Kbit/s up to around 16 Mbit/s. The speed of both these
connections will however depend on the amount of data traffic currently on the network and as
the name “Asymmetric” implies, the download and upload speeds differ. Generally data can be
downloaded approximately twice as fast as it can be uploaded, which for domestic users is
convenient. If a connection with guaranteed speed is required then the telephone provider may be
able to install ISDN digital lines (see 9.2) with speeds from 64Kbit/s to 2Mbit/s. ISDN lines also
have identical up and download speeds. When numerous data streams from subscribers converge
to travel to other towns or countries, very high capacity networks are needed purely to cope with
the density of traffic. Other methods of transmission such as M-JPEG have been used, especially

Adri Jovin J.J.
Page | 233
by the broadcasters who have access to higher bandwidths, but many of these have been replaced
by IP as the technology and capacity of the transmission lines have improved.
Implementing Videoconferencing - the Options
Point to Point and Multipoint
Conferencing between two sites is termed “Point to Point” and is the most frequently used
method. It only requires equipment at each site and the network of choice. If more than two sites
require a simultaneous conference, this is termed “Multipoint” conferencing, and an additional
specialised piece of equipment is required, called a “Multipoint Control Unit” or MCU. Some
videoconferencing CODECs can support limited multipoint working (e.g. up to 4 sites).
Generally, MCU’s are very expensive and are rented from service providers just for the duration
of a conference. Educational and research organisations which are entitled to register with the
JANET Video Conferencing Service (JVCS) have use of JANET(UK)’s MCU’s provided for the
academic community. The factsheet JANET Videoconferencing Booking Service gives details of
registration and the service.
Voice Switched Conferences
Multipoint conferences introduce additional constraints. With Point to Point conferences both
sites can see and hear each other throughout the conference. With Multipoint conferences it
would be unworkable for all sites to be seeing all other sites simultaneously. Normally, therefore,
the conference is “voice switched”. In other words, the image of the site speaking takes
precedence and is seen by all the other sites.
Chairman Control
Another option with Multipoint working is “Chairman Control” where all sites receive pictures
from the Chair site alone. The Chair site will see images from the remote site currently speaking.
All sites receive all sound.
Continual Presence
A third option is “Continual Presence” where the picture is segmented to show images of each
site continuously, with the sound being voice switched. With continual presence it is important to
appreciate that the greater the number of sites and the smaller their individual images the less
easy it is to recognise anything meaningful on the viewing screen. It does depend on viewing
distance etc. but generally, continual presence is unworkable for more than four sites. The JVCS
MCUs currently have an upper limit of 9 sites in continual presence mode, although this is likely
to increase in the near future.

Adri Jovin J.J.
Page | 234
Centralized Conference Control
It has been argued that the problem with the Internet model of multimedia conferencing is that it
doesn't support simple phone calls, or secure closed ``tightly managed'' conferences.
However, it is easy to add this functionality after one has built a scalable system such as the
Mbone provides, rather than limiting the system in the first place. For example, the management
of keys can provide a closed group very simply. If one is concerned about traffic analysis, then
the use of secure management of IP group address usage would achieve the effect of limiting
where multicast traffic propagated. Finally, a telephone style signaling protocol can be provided
easily to "launch" the applications using the appropriate keys and addresses, simply by giving the
users a nice GUI to a distributed calling protocol system.
Currently, the IETF working group on Conference Control is liasing with the T.120 standards
work in the ITU and have made some statements about partial progress.
Distributed Conference Control
In this section, we will discuss some of the lessons that we have garnered from previous work
involving computer based multimedia conferencing, and use these as a basis for developing an
architecture for the next generation of conference control applications, suitable for conferencing
over wide area networks. We show that a simple protocol acting over a conference specific
communications channel, named the Conference Control Channel or CCC, will perform all tasks
within the scope of conference control.
The previous generation of conferencing tools, such as CAR, mmconf, etherphone and the
Touring Machine ([#!mice!#], [#!crowley!#], [#!mmcc!#], [#!touring!#]), were based on
centralised architectures, where a central application on a central machine acted as the repository
for all information relating to the conference. Although simple to understand and simple to
implement, this model proved to have a number of disadvantages, the most important of which
was the disregard for the failure modes arising from conferencing over the wide area.
An alternative approach to the centralised model is the loosely coupled model promoted by Van
Jacobson and exemplified by the vat[#!ietfmbone!#][#!vat!#] and wb[#!vic!#] applications. In
the ``loose session model'', the network is regarded as inherently unreliable. Our observations of
the Mbone show that humans can cope with a degree of inconsistency that arises from
partitioned networks and lost messages, as long as the distributed state will tend to converge in
time. This model makes less demands on the network, and recognises the possibility of failure
modes up front.
We have taken these and the other lessons we have derived from experience with conferencing
tools, and derived two important aims that any conference control architecture should meet:-

Adri Jovin J.J.
Page | 235
1. The conference architecture should be flexible enough so that any mode of operation of the
conference can be used and any application can be brought into use. The architecture should
impose the minimum constraints on how an application is designed and implemented.
2. The architecture should be scalable, so that ``reasonable'' performance is achieved across
conferences involving people in the same room, through to conferences spanning continents with
different degrees of connectivity, and large numbers of participants. To support this aim, it is
necessary explicitly to recognise the failure modes that can occur, and examine how they will
affect the conference, and then attempt to design the architecture to minimise their impact.
We model a conference as composed of an (possibly unknown) number of people at
geographically distributed sites, using a variety of applications. These applications can be at a
single site, and have no communication with other applications or instantiations of the same
application across multiple sites. If an application shares information across remote sites, we
distinguish between the cases when the participating processes are tightly coupled 7.1 - the
application cannot run unless all processes are available and contactable - and when the
participating processes are loosely coupled, in that the processes can run when some of the sites
become unavailable. A tightly coupled application is considered to be a single instantiation
spread over a number of sites, whilst loosely coupled and independent applications have a
number of unique instantiations, although they possibly use the same application specific
information (such as which multicast address to use...).
The tasks of conference control break down in the following way:
• Application control - Applications as defined above need to be started with the correct
initial state, and the knowledge of their existence must be propagated across all
participating sites. Control over the starting and stopping can either be local or remote.
• Membership control - Who is currently in the conference and has access to what
applications.
• Floor management - Who or what has control over the input to particular applications.
• Network management - Requests to set up and tear down media connections between
end-points (no matter whether they be analogue through a video switch, a request to set
up an atm virtual circuit, or using RSVP[#!rsvp!#] over the internet), and requests from
the network to change bandwidth usage because of congestion.
• Meta-conference management - How to initiate and finish conferences, how to advertise
their availability, and how to invite people to join.
We maintain that the problem of meta-conference management is outside the bounds of the
conference control architecture, and should be addressed using tools such as sd, traditional
directory services or through external mechanisms such as email. The conference control system

Adri Jovin J.J.
Page | 236
is intended to maintain consistency of state amongst the participants as far as is practical and not
to address the social issues of how to bring people together, and co-ordinate initial information
such as encryption keys.
We then take these tasks as the basis for defining a set of simple protocols that work over a
communication channel. We define a simple class hierarchy, with an application type as the
parent class and subclasses of network manager, member and floor manager, and define generic
protocols that are used to talk between these classes and the application class, and an inter-
application announcement protocol. We derive the necessary characteristics of the protocol
messages as reliable/unreliable and confirmed/unconfirmed (where `unconfirmed' indicates
whether responses saying ``I heard you'' come back, rather than indications of reliability).
It is easily seen that both tightly and loosely coupled models of conferencing can be
encompassed if the communication channel is secure.
We have abstracted a messaging channel, using a simple distributed inter-process
communication system, providing confirmed/unconfirmed and reliable/unreliable semantics. The
naming of sources and destinations is based upon application level naming, allowing wildcarding
of fields such as instantiations (thus allowing messages to be sent to all instantiations of a
particular type of application).
Finally, we briefly describes the design of the high level components of the messaging channel
(named variously the CCC or the triple-C). Mapping of the application level names to network
level entities is performed using a distributed naming service, based upon multicast once again,
and drawing upon the extensive experience already gained in the distributed operating systems
field in designing highly available name services.
Requirements
Multicast Internet Conferencing
Since early 1992, a multicast virtual network has been constructed over the Internet. This
multicast backbone or Mbone[#!mbone!#] has been used for a number of applications including
multimedia (audio, video and shared workspace) conferencing. These applications involved
include vat (LBL's Visual Audio Tool), ivs (INRIA Videoconferencing System[#!ian:94!#]), nv
(Xerox's Network Video tool, [#!frederick!#]) and wb (LBL's shared whiteboard) amongst
others. These applications have a number of things in common:
• The are all based on IP Multicast.
• They all report who is present in a conference by occasional multicasting of session
information.
• The different media are represented by separate applications

Adri Jovin J.J.
Page | 237
• There is no conference control, other than each site deciding when and at what rate they
send.
These applications are designed so that conferencing will scale effectively to large numbers of
conferees. At the time of writing, they have been used to provide audio, video and shared
whiteboard to conference with about 500 participants. Without multicast 7.3, this is clearly not
possible. It is also clear that, with unreliable networks, these applications cannot achieve
complete consistency between all participants, and so they do not attempt to do so - the
conference control they support usually consists of:
• Periodic (unreliable) multicast reports of receivers.
• The ability to locally mute a sender if you do not wish to hear or see them. However, in
some cases stopping the transmission at the sender is actually what is required.
Thus any form of conference control that is to work with these applications should at least
provide these basic facilities, and should also have scaling properties that are no worse that the
media applications themselves.
The domains these applications have been applied to vary immensely. The same tools are used
for small (say 20 participants), highly interactive conferences as for large (500 participants)
dissemination of seminars, and the application developers are working towards being able to use
these applications for ``broadcasts" that scale towards millions of receivers.
It should be clear that any proposed conference control scheme should not restrict the
applicability of the applications it controls, and therefore should not impose any single
conference control policy. For example we would like to be able to use the same audio encoding
engine (such as vat), irrespective of the size of the conference or the conference control scheme
imposed. This leads us to the conclusion that the media applications (audio, video, whiteboard,
etc) should not provide any conference control facilities themselves, but should provide the
handles for external conference control and whatever policy is suitable for the conference in
question.
Computer Based Multimedia Conferencing Requirements
Computer based multimedia conferencing has the somewhat different requirements from POTS
or ISDN based video conferencing which has made use of custom terminals. It is also a
requirement that such systems interwork with ISDN and POTS based conferencing systems. All
told:
• Multicast based applications running on workstations/PCs where possible.
• Hardware codecs and the need to multiplex their output.

Adri Jovin J.J.
Page | 238
• Sites connecting into conferences from ISDN.
• Interconnecting all the above.
These requirements have dictated that we build a number of Conference Management and
Multiplexing Centres to provide the necessary format conversion and multiplexing to interwork
between the multicast workstation based domain and unicast (whether IP or ISDN) hardware
based domain.
Traditionally such a multiplexing centre would employ a centralised conference control system.
However, this is not scalable and we wish the users of such systems to participate in large
multicast based conferences. We also do not wish to change the multicast media applications
when they switch from a entirely multicast based conference to one using a central system for
some participants.
It is inevitable that translators, multiplexors, format converters and so forth will form some part
of future conferences, and that large conferences will be primarily multicast based.
Specific requirements
Modularity
Conference Control mechanisms and Conference Control applications should be separated. The
mechanism to control applications (mute, unmute, change video quality, start sending, stop
sending, etc) should not be tied to any one conference control application in order to allow
different conference control policies to be chosen depending on the conference domain. This
suggests that a modular approach be taken, with for example, a specific floor control modules
being added when required (or possibly choosing a conference manager tool from a selection of
them according to the conference).
A unified user interface
A general requirement of conferencing systems, at least for relatively small conferences, is that
the participants need to know who is in the conference and who is active. Vat is a significant
improvement over telephone audio conferences, in part because participants can see who is
(potentially) listening and who is speaking. Similarly if the whiteboard program wb is being used
effectively, the participants can see who is drawing at any time from the activity window.
However, a participant in a conference using, say, vat (audio), ivs (video) and wb (whiteboard)
has three separate sets of session information, and three places to look to see who is active.
Clearly any conference interface should provide a single set of session and activity information.
A useful feature of these applications is the ability to ``mute" (or hide or whatever) the local
playout of a remote participant. Again, this should be possible from a single interface. Thus the

Adri Jovin J.J.
Page | 239
conference control scheme should provide local inter-application communication, allowing the
display of session information, and the selective muting of participants.
Taking this to its logical conclusion, the applications should only provide media specific features
(such as volume or brightness controls), and all the rest of the conference control features should
be provided through a conference control application.
Flexible floor control policies
Conferences come in all shapes and sizes. For some, no floor control, with everyone sending
audio when they wish, and sending video continuously is fine. For others, this is not satisfactory
due to insufficient available bandwidth or a number of other reasons. It should be possible to
provide floor control functionality, but the providers of audio, video and workspace applications
should not specify which policy is to be used. Many different floor control policies can be
envisaged. A few example scenarios are:
• Explicit chaired conference, with a chairperson deciding when someone can send audio
and video. Some mechanism equivalent to hand raising to request to speak. Granting the
floor starts video transmission, and enables the audio device. Essentially this is a
schoolroom type scenario, requiring no expertise from end users.
• Audio triggered conferencing. No chairperson, no explicit floor control. When someone
wants to speak, they do so using ``push to talk". Their video application automatically
increases its data rate from, for example, 10Kb/s to 256Kb/s as they start to talk. 20
seconds after they stop speaking it returns to 10Kb/s.
• Audio triggered conferencing with a CMMC[#!mice!#] Conference Management and
Multiplexing Centre - essentially one or more points where multiple streams are
multiplexed together for the benefit of people on unicast links, ISDN and hardware
codecs. The CMMC can mix four streams for decoding by participants with hardware
codecs. The four streams are those of the last four people to speak, with only the current
speaker transmitting at a high data rate. Everyone else stops sending video automatically.
• A background Mbone engineering conference that's been idle for 3 hours. All the
applications are iconised, as the participant is doing something else. Someone starts
drawing on the whiteboard, and the audio application plays an audio icon to notify the
participant.
Scaling from tightly coupled to loosely coupled conferences
CCCP originates in part as a result of experience gained from centralised Multimedia Conference
Control system. centralised system intended for use over ISDN. As described earlier in this
chapter, the functionality such systems provided can be summarised up by listing its basic
primitives:

Adri Jovin J.J.
Page | 240
• Create conference
• Join/Leave Conference
• List members of conference
• Include/exclude application in conference
• Take floor
In addition, there were a number of asynchronous notification events:
• Floor change
• Participant joining/leaving
• Application included/excluded
The application model was modeled around applications that could replicate either themselves or
their display onto remote machines if they were given a list of addresses or displays, hence the
include/exclude functionality. However, these are the basic primitives required to support a
tightly coupled conference, although for some uses others may be added.
Any conference control system that claims to be fairly generic must be able to support these
primitives with reasonable reliability. (Absolute consistency is not really a feasible option in a
multiway conference)
Loosely coupled conferences put less constrains on the protocols used, but must scale to much
larger numbers, and must be very tolerant of loss and network segmentation.
Taking the modular approach described above, we would expect to change conference
controllers when moving from one regime to another, but we do not wish to change the media
applications too.
The Conference Control Channel (CCC)
To bind the conference constituents together, a common communication channel is required,
which offers facilities and services for the applications to send to each other. This is akin to the
inter process communication facilities offered by the operating system. The conference
communication channel should offer the necessary primitives upon which heterogeneous
applications can talk to each other.
The first cut would appear to be a messaging service, which can support 1-to-many
communication, and with various levels of confirmation and reliability. We can then build the
appropriate application protocols on top of this abstraction to allow the common functionality of
conferences.

Adri Jovin J.J.
Page | 241
We need an abstraction to manage a loosely coupled distributed system, which can scale to as
many parties as we want. In order to scale we need the underlying communication to use
multicast. Many people have suggested that one way of thinking about multicast is as a
multifrequency radio, in which one tunes into particular channels in which we are interested in.
We extend this model to build an Inter Process Communications model, on which we can build
specific conference management protocols. Thus we define an application control channel.
What do we actually want from the system?
• We want to ask for services
• We want to send requests to specific entities, or groups of entities and receive responses
from some subset of them, with notifications sent out to others.
Figure 7.4: CCCP Conceptual Design
CCCP originates in the observation that in a reliable network, conference control would behave
like an Ethernet or bus - addressed messages would be put on the bus, and the relevant
applications will receive the message, and if necessary respond. In the internet, this model maps
directly onto IP multicast. This is illustrated in figure 7.4. In fact the IP multicast groups concept
is extremely close to what is required. In CCCP, applications have a tuple as their address:

Adri Jovin J.J.
Page | 242
(instantiation, application type, address). We shall discuss exactly what goes into these fields in
more detail later. In actual fact, an application can have a number of tuples as its address,
depending on its multiple functions. Examples of the use of this would be:
Destination Tuple Message
------------------------ ------------
(1, audio, localhost) <start sending>
(*, activity_management, localhost) <receiving audio from host:> ADDRESS
(*, session_management, *) <I am:> NAME
(*, session_management, *) <I have media:> {application list}
(*, session_management, *) <Participant list:> {participant list}
(*, floor_control, *) <REQUEST FLOOR>
(*, floor_control, *) <I HAVE FLOOR>
and so on. The actual messages carried depend on the application type, and thus the protocol is
easily extended by adding new application types. In keeping with the underlying multicast
message reception paradigm, a CCC application needs to register its interest in messages before
it will receive them. Where possible, a CCC system uses the underlying multicast delivery to
optimise where messages are filtered.
CCC Names
Our model of the CCC is a broadcast bus, where the receivers filter the messages according to
what information and which protocols they need to receive and participate in. Using this model,
we based our naming scheme upon the attributes of an application that could be used in deciding
whether to receive a message. We thus build a name tuple from three parts:
(instantiation, type, address)
An application registers itself with its CCC library, specifying one or more tuples that it
considers describe itself. Note that there is no conference identifier currently specified as part of
the tuple, but this is liable to change, since a conference identifier may be useful in unifying
conference management and conference meta-management, and considerably simplifies the
design of applications which may be part of multiple conferences simultaneously. In the current
prototype design, a control group address or control host address or address list are specified at
startup, and that Meta-conferencing (i.e., allocation and discovery of conference addresses) is
outside the scope the CCC itself. The parts of the tuple are:
address
In our model of a conference, applications are associated with a machine and possibly a user at a
particular machine. Thus we use a representation of the user or the machine as a field in the

Adri Jovin J.J.
Page | 243
tuple, to allow us to specify applications by location.. The address field will normally be
registered as one of the following:
• hostname
• username@hostname
When the application is associated with a user, such as a shared whiteboard, the
username@hostname form is used, whereas applications which are not associated with a
particular user, such as a video switch controller register simply as hostname. For simplicity, we
use the domain naming scheme in our current implementation, although this does not preclude
other identifiable naming schemes. Note that the hostname is actually shorthand for no-
user@hostname, so that when When other applications wish to send a message to a destination
group (a single application is a group of size 1), they can specify the address field as one of the
following:
• username@hostname
• hostname
• *@hostname - Note that the hostname is actually shorthand for no-user@hostname, so
that this matches all applications on the given host.
• - this is used to address applications regardless of location.
The CCC library is responsible for ensuring a suitable multicast group (or other means) is chosen
to ensure that all possible matching applications are potentially reachable (though depending on
the reliability mode, it does not necessarily ensure the message got to them all).
It should be noted that in any tuple containing a wildcard (*) in the address, specifying the
instantiation (as described below) does not guarantee a unique receiver, and so normally the
instantiation should be wildcarded too.
type
The next attribute we use in naming applications is based on hierarchical typing of the
application and of the management protocols. The type field is descriptive both of the protocol
that is being used and of the state of the application within the protocol at a particular time. For
example, a particular application such as vat may use a private protocol to communicate between
instantiations of the application, so a vat type is defined, and only applications which believe
they can understand the vat protocol and are interested in it would register themselves as being of
type vat. An alternative way of using the type field is to embed the finite state machine
corresponding to the protocol within the type field - thus a floor management protocol could use
types floor.management.holder and floor.management.requester in a simple floor control
protocol, that can cope with multiple requests at once. A final way of using the type field is to
allow extensions to existing protocols in a transparent fashion, by simply extending the type field
by using a version number. Some examples of these techniques can be found in the examples
given.

Adri Jovin J.J.
Page | 244
Some base types are needed to ensure that common applications can communication with each
other. As a first pass, the following types have been suggested:
• audio.send - the application is interested in messages about sending audio
• audio.recv - the application is interested in messages about receiving audio
• video.send - the application is interested in messages about sending video
• video.recv - the application is interested in messages about receiving video
• workspace - the application is a shared workspace application, such as a whiteboard
• session.remote - the application is interested in knowing the existence of remote
applications (exactly which ones depends on the conference, and the session manager)
• session.local - the application is interested in knowing of the existence of local
applications
• media-ctrl - the application is interested in being informed of any change in conference
media state (such as unmuting of a microphone).
• floor.manager - the application is a floor manager
• floor.slave - the application is interested in being notified of any change in floor, but not
(necessarily) in the negotiation process.
It should be noted that types can be hierarchical, so (for example) any message addressed to
audio would address both audio.send and audio.recv applications. It should also be noted that an
application expressing an interest in a type does not necessarily mean that the application has to
be able to respond to all the functions that can be addresses to that type. Also, (if required) the
CCC library will acknowledge receipt on behalf of the application.
Examples of the types existing applications would register under are:
• vat - vat, audio.send, audio.recv
• ivs - ivs, video.send, video.recv
• nv - nv, video.send, video.recv
• wb - wb, workspace
• a conference manager - confman, session.local, session.remote, media-ctrl, floor.slave
• a floor ctrl agent - flooragent, floor.manager, floor.slave
In the current implementation, the type field is text based, so that debugging is simpler, and we
can extend the type hierarchy without difficulty.
instantiation
The instantiation field is purely to enable a message to be addressed to a unique application.
When an application registers, it does not specify the instantiation - rather this is returned by the

Adri Jovin J.J.
Page | 245
CCC library such that it is unique for the specified type at the specified address. It is not
guaranteed to be globally unique - global uniqueness is only guaranteed by the triple of
(instantiation, type, address) with no wildcards in any field. When an application sends a
message, it uses one of its unique triples as the source address. Which one it chooses should
depend on to whom the message was addressed.
Reliability
CCCP would be of very little use if it were merely the simple protocol described above due to
the inherent unreliable nature of the Internet. Techniques for increasing the end-to-end reliability
are well known and varied, and so will not be discussed here. However, it should be stressed that
most (but not all) of the CCCP messages will be addressed to groups. Thus a number of
enhanced reliability modes may be desired:
• None. Send and forget. (an example is session management messages in a loosely
coupled system)
• At least one. The sending application wants to be sure that at least one member of the
destination group received the message. (an example is a request floor message which
would not be ACKed by anyone except the current floor folder).
• n out of m. The sending application wants to be sure that at least n members of the
destination group received the message. For this to be useful, the application must have a
fairly good idea of the destination group size. (an example may be joining of a semi-
tightly coupled conference)
• all. The sending application wants to be sure that all members of the destination group
received the message. (an example may be ``join conference" in a very tightly coupled
conference)
It makes little sense for applications requiring conference control to reimplement the schemes
they require. As there are a limited number of these messages, it makes sense to implement
CCCP in a library, so an application can send a CCCP message with a requested reliability,
without the application writer having to concern themselves with how CCCP sends the
message(s). The underlying mechanism can then be optimised later for conditions that were not
initially foreseen, without requiring a re-write of the application software.
There are a number of ``reliable" multicast schemes available, such as [#!rmp!#] and [#!isis!#],
which can be used to build consensus and agreement protocols in asynchronous distributed
systems. However, the use of full agreement protocols is seen to be currently limited to tightly
coupled conferences, in which the membership is known, and the first design of the CCC library
will not include reliable multicast support, although it may be added later as additional
functionality. Unlike distribute databases, or other automated systems that might exploit causal

Adri Jovin J.J.
Page | 246
ordered multicast discussed in chapters 2 and 3, communications systems for humans can exploit
the users tolerance for inconsistency.
We believe that sending a message with reliability ``all'' to an unknown group is undesirable.
Even if CCCP can track or obtain the group membership transparently to the application through
the existence of a distributed nameserver, we believe that the application should explicitly know
who it was addressing the message to. It does not appear to be meaningful to need a message to
get to all the members of a group if we can't find out who all those members are, as if the
message fails to get to some members, the application can't sensibly cope with the failure. Thus
we intend to only support the all reliability mode to an explicit list of fully qualified (i.e. no
wildcards) destinations. Applications such as joining a secure (and therefore externally
anonymous) conference which requires voting can always send a message to the group with "at
least one" reliability, and then an existing group member initiates a reliable vote, and returns the
result to the new member.
Ordering
Of course loss is not the only reliability issue. Messages from a single source may be reordered
or duplicated and due to differing delays, messages from different sources may arrive in
``incorrect'' order.
Single Source Reordering
Addressing reordering of messages from a single source first; there are many possible schemes,
almost all of which require a sequence number or a timestamp. A few examples are:
1. Ignore the problem. A suitable example is for session messages reporting presence in a
conference.
2. Deal with messages immediately. Discard any packets that are older than the latest seen.
Quite a number of applications may be able to operate effectively in the manner. However, some
networks can cause very severe reordering, and it is questionable as to whether this is desirable.
3. Using the timestamp in a message and the local clock, estimate the perceived delay from the
packet being sourced that allows (say) 90% of packets to arrive. When a packet arrives out of
order, buffer it for this delay minus the perceived trip time to give the missing packet(s) time to
arrive. If a packet arrives after this timeout, discard it. A similar adaptive playout buffer is used
in vat for removal of audio jitter. This is useful where ordering of requests is necessary and
where packet loss can be tolerated, but where delay should be bounded.
4. Similar to above, specify a fixed maximum delay above minimum perceived trip time,
before deciding that a packet really has been lost. If a packet arrives after this time, discard it.

Adri Jovin J.J.
Page | 247
5. A combination of [3] and [4]. Some delay patterns may be so odd that they upset the running
estimate in [3]. Many conference control functions fall into this category, i.e. time bounded, but
tolerant of loss.
6. Use a sliding window protocol with retransmissions as used in TCP. Only useful where loss
cannot be tolerated, and where delay can be unbounded. Very tightly coupled conferences may
fall into this category, but will be very intolerant to failure. Should probably only be used along
with application level timeouts in the transmitting application.
It should be noted that all except [1] require state to be held in a receiver for every source. As not
every message from a particular source will be received at a particular receiver due to CCCP's
multiple destination group model, receiver based mechanisms requiring knowing whether a
packet has been lost will not work unless the source and receiver use a different sequence space
for every (source, destination group) pair. If we wish to avoid this (and I think we usually do!),
we must use mechanisms that do not require knowing whether a packet has been lost.
Thus the above list of mechanisms becomes:
1. Have CCCP ignore the problem. Let the application sort it out.
2. Have CCCP deal pass messages to the application immediately. Discard any packets that are
older than the latest seen.
3. As above, estimate the perceived delay within which (say) 90% of packets a particular
source arrive, but delay all packets from this source by the perceived delay minus the perceived
trip time.
4. As above, calculate the minimum perceived trip time. Add a fixed delay to this, and buffer
all packets for this time minus their perceived trip time.
5. A combination of [3] and [4], buffering all packets by the smaller of the two amounts.
6. Explicitly ack every packet. Do not use a sliding window.
Note that just because CCCP cannot provide more elaborate mechanisms, this does not mean an
application itself (with some semantic knowledge) cannot build a more elaborate mechanism on
top of any of [1]..[5]. However it does mean that message timestamps and sequence numbers
must be available at the application level.
Multiple Source Ordering
In general we do not believe that CCCP can or should attempt to provide ordering of messages to
the application that originate at different sites. CCCP cannot predict that a message will be sent
by, and therefore arrive from, a particular source, so it cannot know that it should delay another
message that was sent at a later time. The only full synchronisation mechanism that can work is
an adaptation of [3]..[5] above, which delays all packets by a fixed amount depending on the trip
time, and discards them if they arrive after this time if another packet has been passed to the user
in the meantime. However, unlike the single source reordering case, this requires that clocks are
synchronised as each site.

Adri Jovin J.J.
Page | 248
CCCP does not intend to provide clock synchronisation and global ordering facilities. If
applications require this, they must do so themselves. However, for most applications, a better
bet is to design the application protocol to tolerate temporary inconsistencies, and to ensure that
these inconsistencies are resolved in a finite number of exchanges. An example is the algorithm
for managing shared teleconferencing state proposed by Scott Shenker, Abel Weinrib and Eve
Schooler [she].
For algorithms that do require global ordering and clock synchronisation, CCCP will pass the
sequence numbers and timestamps of messages through to the application. It is then up to the
application to implement the desired global ordering algorithm and/or clock synchronisation
scheme using one of the available protocols and algorithms such as NTP [lam],[fel],[bir].
CCCP Messages
Message format:
(SRC triple){list of (DST triple)s} FUNCTION parameter list
The Naming Service
CCC can be run on the bus model, passing all messages around a single multicast group per
conference. This will scale reasonably, since it scales with the number of participants in the
conference. Name resolution occurs at each host, matching the destination naming tuple in the
message against the list of tuples that are registered at this particular host. However, it it does not
scale indefinitely, because the load on each host and the network increases with the complexity
of the conference and the number of messages. To improve scaling, the communications should
be optimised so that messages are only propagated to the machines that are interested. Thus we
need a service that maps the naming tuples to locations, so that intelligent mapping of message
paths to locations can be performed (aka intelligent routing and placement of multicast groups).
This name location service (or naming service as it is more generally known) has a number of
properties that differentiate it from other naming services such as X.500 and the DNS:
• Dynamic and frequent updates.
• Fast propagation of changes.
• Ability to fall back to broadcast to interested parties when uncertain about the
consistency of a refined addressing group, since names are unique per conference and are
included in each message.

Adri Jovin J.J.
Page | 249
The last property is important, since it allows a relaxed approach to maintenance of consistency
amongst the naming servers, saving greatly in the messages and complexity of the internals of
the service.
We intend to implement a nameserver suitable for loosely coupled conferences as the default in
the CCC library. However, CCCP will also allow the use of an external nameserver to
supplement or replace the internal nameserver behaviour, which will allow much greater use of
the nameserver to made in more tightly coupled conferences, for instance by using the
nameserver to keep an accurate list of members.
Security
CCCP will implement two levels of security - a very simple host access model similar to the
xhost mechanism in the X window system[xse], and encryption of all CCCP packets with a key
provided.
Host access security
The application is started with an access control string, which is a list of hosts it is prepared to
accept commands from. It passes this to the CCC library, and the CCC library then filters all
messages whose source is not in the access control list.
This very simple level of security is intended primarily to prevent external attacks such as
switching media tools transmission on or off, and thus compromising the privacy of users.
Note that the X magic-cookie mechanism is not too useful here, as the cookie would have to be
carried in over CCC packet, which lays it open to attack from anyone who can capture multicast
packets.
Encryption
We recognise that the only way for CCCP to be really secure is to use encryption of all CCC
packets, and CCCP will support an encryption scheme. The key distribution problem is
considered to be outside the scope of CCCP itself, and CCCP will require the application to pass
the key to it. After this, all CCCP messages from this library will be encrypted, and non-
encrypted messages will be ignored.
CCCP will allow an encryption key per conference id, and a key for messages not associated
with any conference. Which encryption key to use for outgoing messages is chosen by the CCC
library according to the conference id. Once the application has passed the set of keys to the
CCC library, it no longer has to concern itself with encryption.
Encryption and Host Access can be used simultaneously.

Adri Jovin J.J.
Page | 250
Conference Membership Discovery
CCCP will support conference membership discovery by providing the necessary functions and
types. However, the choice of discovery algorithm, loose or tight control of the conference
membership and so forth, are not within the scope of CCCP itself. Instead these algorithms
should be implemented in a Session Manager on top of the CCC.
Distributed Virtual Reality
Distributed VR is a relatively new area of research. Virtual reality systems are now largely
software components, rather than requiring the dedicated head-up-display input controllers and
renderer hardware of the past. Current high-end workstations can now render scenes described in
VRML and other languages in near real time.
The introduction of audio and video input and output on desktop machines led to the deployment
of software based multimedia conferencing, we expect to see the deployment of multi-user
virtual environments over the Internet shortly.
In this paper, we present an architecture for distributed virtual reality. We outline the necessary
network support and a transport protocol, and the way that distributed virtual reality applications
would interact (the API if you like) with each other using these mechanisms.
It is a goal of the architecture to provide policy free mechanisms for distributed VR application
builders. It is not a goal to make it easy to program such applications, since it is all too easy in
providing easy-to-program distributed system tools that let the application builder overload the
network, and at the same time provide suboptimal performance for the user.
The architecture provides necessary and sufficient hooks only for distributed VR.
Multicast routing is a mature area of research in the Internet. The system that we now call the
``Mbone''[#!mac:94!#] has its roots in the research by Cheriton and Deering where they
developed the Internet multicast model, including the idea of host groups[#!igmp!#] and the
basic service model for IP multicast[#!deer:88!#]. It is now widely deployed in the network, and
available in most common host operating systems and routers.
Some functions in a distributed system can best be performed in intermediate nodes or routers,
whilst others can best be performed in end systems or hosts. The end to end
principle[#!end2end!#] is used to select where to place a function. The principle is that a function
that requires end system knowledge to carry out correctly should only exist in the end systems;
where it appears in the network, it should only be as a performance enhancement.
Two principles behind the design of high performance, low cost protocols that have proved
themselves in this environment are Application Layer Framing (ALF) and Integrated Layer
Processing (ILP)[#!ddc:91!#]. These state simply that: the unit of synchronisation and recovery

Adri Jovin J.J.
Page | 251
should as far as possible be the same as the unit of communication (packet); and that where
possible, modular functionality that is specified in layered form should not be implemented as
such, and that new designs communications systems should factor this in so that processing
associated with layered modules can be integrated in one pass.
Combining these principles with the use of multicast for many-to-many communication, a
number of further techniques have arisen for protocol design:
• Multicasting everything is a good idea. In applications with relatively high packet rates,
the use of multicast for control information as well as for user data is not a high load, and
can greatly simplify convergence of protocols for correctness, as well as performance and
synchronisation.
• As error rates in modern networks have decreased, end to end recovery from packet loss
or re-ordering has been seen to be a more optimal design than hop-by-hop recovery. As
we move from one-to-one, through one-to-many and on to many-to-many applications,
we can see that the same principle has to be changed. Neither the original sender nor the
network can deal with the task of delivering packets in order, and senders cannot know
when (or which) receivers are missing packets. It is not a good idea to use a positive-
acknowledgement plus timeout-retransmission scheme for multicast applications because
of the well known ``implosion'' problem (congestive collapse caused by multiple
acknowledgements returning to the sender or senders)[#!uclmtp!#].
• Scalable reliable multicast[#!van:95!#] is a technique that has seen wide deployment in
LBL's whiteboard application (the so-called ``wb''). Wb uses the principles above to
provide a reliable delivery. The protocol and repair algorithm are briefly as follows
(paraphrased from[#!van:95!#]):
• Messages are sent with a sequence number plus a timestamp. There are three basic types
of messages: data messages, negative acknowledgements and heartbeat messages.
All participants keep state for all other participants, which includes the following:
1. Source address, plus last seen in order sequence number
2. The estimated distance (delay) from this participant to each of the others.
In addition, participants keep a number of the most recently received messages available.8.3
On detecting a missing packet from a gap in the sequence number space (between last
received in order and newly received packet), a receiver prepares to send (multicast) a negative
acknowledgement, which acts as a request for a repair from any other participant. However, the
receiver defers from sending the negative acknowledgement for a time. This time is set so as to
cause the set of potential participants sending a negative to (implicitly) conspire so that usually

Adri Jovin J.J.
Page | 252
only one (or a small number) of them make the request. To do this, the timer is drawn from a
uniform distribution over the range [c1*dsa, (c1+c2)*dsa], where c1 and c2 are constants and da
is the requesting participant's estimate of the delay to the source. This time is subject to a binary
exponential backoff in the usual manner if there is no response.
Participants that receive the request for repair, and wish to try and honour it also dally before
sending the repair. Their hiatus is drawn from the distribution [d1*dab, (d1+d2)*dab], to ensure
that it is likely that only one (or at least only a few) send the repair.
Repair request messages suppress repair requests from other sites missing the same packet.
Repair responses suppress other responses (as well as hopefully satisfying the request!).
Finally, the delay estimation is based on the same algorithm used in NTP[#!ntp!#]. Heartbeat
messages are sent by each participant carrying a list of other participants, together with the
timestamp from the last message seen from each participant t1, and the difference, d, t3-t2,
between its arrival time, t2 and the heartbeat send time t3. On receiving a heartbeat message at
t4, the delay can be estimated as (t4 - t1 - d)/2. This can be calculated using a rolling average,
and the mean difference kept for safety if required. (So long as paths are reasonably symmetric,
and clock rates not too different, this gives suffices for the repair algorithm above).
• Some applications require packets to have a particular inter-arrival rate. So long as the
network can support the average rate (i.e. there is no long term congestion), ``Receiver
makes good'' is generally a low cost solution to dealing with jitter or partial packet re-
ordering. It is hard to provide a globally transmission clock in large heterogeneous
networks. Essentially, if timestamped, and a receiver clock does not drift or wander w.r.t
a sender clock too quickly, a receiver can run an adaptive playout buffer to restore the
playout times of packets. The size of the playout buffer is essentially twice the inter-
arrival variation, to ensure a significant percentage of packets arrive within the worst case
time. A rolling average of the inter-arrival times (kept in any case if the algorithm
described above is in use). If the mean delay varies (due to increased or decreased load
on the network, or due to route changes) then quiet times can be used to make
adjustments.
• Cheriton[#!cher:95!#] describes a a scheme called log based receiver repair which was
devised for distributed simulations in the DIS (the DSINet is an ARPA funded program
of work whose target is the development of a suite of systems to support the Synthetic
Theatre of War demonstration in 1997. from which much of this research stems). This is
similar in spirit to the SRM approach, but has separate log servers rather than expecting
all applications to participate in the repair algorithm. The upside of this is that a larger
history may be kept (and for applications where the entire history is necessary to
reconstruct current state, it may be too costly to distribute to all sites). The downside is

Adri Jovin J.J.
Page | 253
that a distinguished server type needs a distinguished protocol to maintain replicant
servers and so forth.
• Synchronisation of messages from different sources is generally a bad thing.[#!lws!#]
Open loop protocols such as the types we have described above (and many other
heartbeat style protocols such as routing update and reachability protocols) are prone to
synchronise send times. This can be avoided by careful selection of randomising timers
based on unique participant data (own address is a good example).
• Multicast applications cannot use positive feedback for reliability. Nor can they use
positive explicit feedback for congestion or flow control. Instead, implicit, and
aggregated information may be more effective. One scheme for congestion Control and
multicast is described by Wakeman et al[#!ian:94!#]. It is important to be flexible.
Depending on whether group communication can proceed at the speed of the slowest
participant (or link to them) or the average, or be completely heterogeneous, we need
different schemes for flow and congestion control. We can separate these also in to
sender and receiver based adaptation, and the next item refers to work in this area. In
Wakeman's scheme, the sender elicits responses from a receiver set at a given distance by
multicasting out a packet with a sliding key, which essentially acts as a selector key to
choose some small percentage of recipients to act as samples to report n traffic conditions
seen at their pint in the network w.r.t the sender.
• In the LBL work, this idea is generalized: Multicast receivers keep state about all senders.
As with SRM, periodically, they send heartbeat messages, which contain this state
(perhaps spreading the state over a set of heartbeat messages to keep the size of the
updates small). This state can be used by senders to estimate the conditions at all points
in the network. To keep heartbeat/session traffic to a reasonable level, the rate of the beat
is reduced in proportion to the number of participants. Although this means that the rate
of samples from any given system is decreased for a larger group, the number of samples
stays the same. In fact, as a group gets larger, it is statistically reasonable to assume that it
is more evenly and widely distributed, and may give better and better samples concerning
traffic conditions throughout the multicast distribution fabric.
• McCanne, on his work on multicast video, has looked at layered media encodings, and
how they may be mapped onto multiple groups.[#!mccanne:96!#] In this work, different
levels of quality (or urgency, or interest) are sent to different group addresses. The
number of levels, and amount that is sent in each can be adjusted using schemes such as
the one described just before. However, receivers can independently adjust the rate of
traffic that arrives at them simply by joining (to increase) or leaving (to decrease) one of
more groups, corresponding to the appropriate coding levels.

Adri Jovin J.J.
Page | 254
• Flow and Congestion Signal actions are generally the same for multicast applications as
for unicast. The stable, and safe algorithm used in TCP since 1988[#!van:88!#], with a
slow start cycle, a congestion control cycle with exponential backoff and linear increase,
and a fast retransmit cycle to avoid short lived congestion can be employed.
The rest of this paper is structured as follows: In the next section, we outline the structure of a
distributed virtual reality system. After that we look at some of the real requirements from
distributed VR, both from systems and the human perspective. Following that we present the
transport protocol. Finally we look at further work that needs to be done.
General Idea and Problems
Virtual Reality systems are really glorified renderers combined with simulations, and some fancy
display and pointer technology. The software components typically include the roughly what you
would expect, as illustrated in figure 8.10.
Figure 8.10: Virtual Reality Software Structure
The performance requirements for distributing VR are surprising. Once a system is up and
running, it transpires that objects are introduced/created and destroyed, with relatively low
frequency. If a distributed VR system runs the collision detection and rendering at all receivers,
it is only POV and object locations and attitudes that need updating. Here, though, there may be
an extremely stringent latency requirement because of multiple interactions with other objects.

Adri Jovin J.J.
Page | 255
We need three main components to a protocol to distribute this functionality:
1. A Virtual World bulk load protocol
2. An object naming system
3. An update protocol
The virtual world bulk load protocol could be based in a number of existing protocols for large
scale information dissemination. We assume that virtual worlds are potentially very large
databases (although many VRML examples are not that large, we would anticipate this changing
as time progresses and people develop a richer taste in virtual environments). However, it may
be possible to use the update protocol that we are devising here too, if it is engineered for high
throughput as well as low latency.
The object naming system requires hierarchical naming and wildcarding, as well as a clean
mapping from object to multicast group. Such a system has been devised in earlier work, and is
partially implemented in the CCCP system[#!cccp!#].
The update protocol is what we will concentrate on here. Essentially, we propose an extension of
SRM, together with the congestion avoidance mechanisms taken from Wakeman and McCanne's
work. We propose basing the protocol on RTP, as is described in Parnes' work[#!rtpsrm!#], since
it is likely that eventually VR worlds may include media such as audio and video, and we see no
reason to use a different packet header to support point of view and object motion, and new
object upload.
We propose that the constants of repair can be adjusted in favour of fast recovery rather than low
numbers of repeated requests and repairs, since the load caused by this (given the size of repairs)
is relatively small.
The sender application needs to specify the semantics of the operation, and this is conveyed in
single field in the protocol so that receivers can work out whether a repair request is relevant or
whether a new update will overwrite the effect anyhow.
Virtual Reality Operations, User Views and Network Considerations
For a distributed multi-party application, there are a number of constraints which we might try to
satisfy.
• Convergence, or does the interface show a system that reaches eventual consistency.
• Determinism, or does the system behave according to users' expectations of the
underlying model.

Adri Jovin J.J.
Page | 256
• Multiple Modifiers - does the system allow more than a single user to modify an object
(write/update state associated with an object).
• Interactivity - does the system let users lock an object so that they then have exclusive
write/update access.
A key point in the design of the system is that when operating over a network with loss,
reordering, and different levels of throughput/latency between different senders and recipients, it
is clear that it is impossible to satisfy all four of these at any one time.
It should be sufficient to relax one of the constraints at one time, but it will dependent on a
particular application and user requirements which, constraint is the appropriate one to relax.
Network conditions that pertain to each of these constraints are:
• Capacity - bits per second - may differ between a sender and a set of recipients - a
distribution tree rooted at each sender will have paths to recipients, and different parts of
the sub tree may have different capacity.
• Errors/Loss - packet loss may be due to noise on the line, transient outage, or temporary
congestion. Again, in a multi-party application, this may be different for different
receivers.
• Availability - networks can partition. Some senders or recipients may crash, or reboot at
different times during a multi-party session.
• Delay - delays over long-haul networks8.4, even without queues, can be significant.
Again, they will usually differ from one recipient to the next.
Application Model
Initially, we are making the big assumption here, that the model run at each site/host is the same,
and that the only differences are POVs and the user at each host. However, it is not necessary for
all users at all sites to ``see'' all objects. In fact, real estate probably precludes this. This means
that certain (large) optimisations are possible in terms of delivering operation/messages to each
host.The set of typical operations between a VR application and the underlying system range
from moving point of view to moving objects themselves and altering the attitudes.
A more advanced approach, one could consider sending trajectories around, and having receivers
apply the model world to the object. (eg. gravity, wind, friction, etc etc).
Messages may be idempotent (describe an absolute position for an object, or position and
velocity, and so on), or not (maybe they just say how far the object has moved).

Adri Jovin J.J.
Page | 257
Interactions between users of a distributed VR application are complex. An object must in some
senses have an "owner". This may be the user at a site (e.g. if the object is the POV itself, or the
glove/pointer, etc), or an object currently under the control of a user.
Thus we should associate a set of objects (and users) with a portion of the multicast address
space - the idea is that a user is in some locale and that only operations relative to that locale
need be delivered to their system.
Furthermore, there most be some notion of ``real'' virtual world distance. The set of objects in
VR in a locale that are ``near'' a user are subject to interactions which can reasonably be modeled
without having to worry about propagation delay across the net. Beyond some distance, and the
time to get a message from the originator of the action to the hosts that a set of POVs that include
this locale may simply be too high to consider. For example, imagine playing catch in a virtual
world simulated between two machines on the net. The time for the ball to reach the ground is
less than the time for the message to cross a terrestrial line more than a few thousand miles.
However, it may be possible to introduce some element of representation of distance into the
user interface - two approaches suggest themselves based on physics models of the real world:
1. Uncertainty - as an object gets further away, we can introduce less certainty about its
position (note that the SRM protocol above gives us enough information to do this accurately!).
2. Relativity - we could warp spacetime in the virtual environment to produce a similar effect
to gravity - action at a distance could become much slower.
Given this, different VR applications may need to relax a different one of the four constraints,
possibly at different times. In fact, we can envisage single applications that need to relax
different constraints for different objects at different times.
The Distributed Virtual Reality Multicast Protocol, DVRMP
DVRMP runs on top of RTP[#!rtp!#] and UDP. All DVRMP messages are multicast. As in
Parnes' proposal, we use RTP data packets as is, for data. If reliability is needed, the RTP PT
field is used to indicate SRM, otherwise it simply conveys normal data.
Table 8.1: RTP Header

Adri Jovin J.J.
Page | 258
The SSRC is the source VR application. the CSRC can be used for input from specific devices.
We use RTCP as is for session quality reports, and respect the PARNES protocol for reliability:
More than one multicast group is used. Session messages are used within one multicast group to
advertise the existence of participants and objects. Objects have owners. At any one time, the
owner is the source of the session messages about an object.
Messages can be marked as reliable or not reliable, however, this is not binding on recipients.
Recipients can operate the repair algorithm on reliable object messages in the normal way. Other
messages are deemed ephemeral, and can be lost (or discarded) at will.

Adri Jovin J.J.
Page | 259
Light Weight Session Philosophy
Multimedia Sessions come in many shapes and sizes, but there are only really two models for
session control: light-weight sessions and tightly coupled conferencing.
Light-weight sessions are multicast based multimedia conferences that lack explicit session
membership and explicit conference control mechanisms. Typically a lightweight session
consists of a number of many-to-many media streams supported using RTP and RTCP using IP
multicast.
The rendezvous mechanism for light-weight sessions is a multicast based session directory. This
distributes session descriptions[#!sdp!#] to all the potential session participants. These session
descriptions provide an advertisement that the session will exist, and also provide sufficient
information including multicast addresses, ports, media formats and session times so that a
receiver of the session description can join the session. As dynamic multicast address allocation
can be optimised by knowing which addresses is in use at which times, the session directory is an
appropriate agent to perform multicast address allocation.
In the Internet, the User Datagram Protocol, UDP is used to carry Interactive or realtime
multimedia data around. UDP is chosen because it is lightweight, but also because it interposes
no unpredictable or unnecessary functions between the application and the network.
Multimedia transport protocols do not have to be complex or ``heavyweight'', in the sense that an
all-purpose transport protocol such as the Transmission Control Protocol, TCP is ``heavyweight''.
In fact, TCP is not really heavyweight, but it does have, albeit parsimoniously, mechanisms for
flow control, reliability, and ordering, that are counter-productive for multimedia traffic. It is
also completely impossible to use in a scalable way for many-to-many
communication[#!uclmtp!#].
The goal of transport protocols is to provide end to end services that are specific to some modest
range of applications, and provide specific services that are not common to all applications, and
therefore not required in the network service.
However, UDP, being originally designed for lightweight management tasks such as local time
and name lookup services, does not always provide enough functionality for multimedia data
transport. Multimedia transport protocols need to provide hooks for a common set of multimedia
services, that can be distinguished quite clearly from conventional data services, for example:
A basic framing service is needed, defining the unit of transfer, typically common with the
unit of synchronisation (in keeping with the Application Layer Framing principle[#!ddc:91!#]).
Multiplexing may be needed, since we may wish to identify separate media in separate
streams, but we may also wish to stack media together in the same IP packets.

Adri Jovin J.J.
Page | 260
Timely delivery is needed where possible, and can be traded off against reliability. This means
that it is unlikely in most scenarios that a error detection and timeout/retransmission based error
recovery scheme is appropriate. Instead, mostly we would employ media specific forward error
recovery.
Synchronisation is a common service to almost all networked multimedia applications, so
providing a common timestamping framework seems like a good component of a transport
protocol, especially since it is between the sender (who knows media sample times) and the
receiver (who has to play out media in the right order, and without gaps, gasps or unnecessary
jitters that the synhronisation is needed. Synchronisation is also needed between different media,
and this is discussed further later in this chapter.
The basic unit of transfer in the Internet is the IP packet, which carries application data, and a
transport protocol header preceded by the IP header. Packets are demultiplexed at the host based
on the transport protocol port numbers, and routed in the network based IP addresses.
RTP, the preferred carrier/framing protocol for multimedia traffic, is carried on top of UDP. It
has the potential for further multiplexing information, since it carries source identifiers. The
main other item in the RTP header is the media timestamp (for the sample carried) which is
formatted specifically for each media type. This is used for the playout and synchronisation
algorithms. RTCP, the Real Time Control Protocol, is used to convey additional information
such as participant details and statistics of packet arrivals and loss. It is typically sent on one
UDP port number higher than the UDP port used for the associated RTP packets.
One might envisage using the Internet to provide a rich medium for a Distributed Orchestra or
even a cattle Auction. However, there are limits imposed by the speed of light, to the end to end
and round trip delays, however fast the lines and switching, and however ingenious the
implementation of RTP. Musicians require extremely short delays between hearing each other,
and reacting, and reacting to the reaction - for example, consider a rallentando when an entire
orchestra slows down the beat evenly - this may require delays of a few tens of milliseconds at
most. This precludes such activities over distances of more than a few hundred miles at absolute
best!
----X-----------X----

CU9255 internetworking Multimedia

More Related Content

What's hot

Viewers also liked

Similar to CU9255 internetworking Multimedia

More from Adri Jovin

Recently uploaded

CU9255 internetworking Multimedia