multimedia

Prof. Dr. Eng. Ali M. Abo-Zaid 1

Introduction to multimedia

1


Multimedia? Five senses are mechanisms to communicate

with others and environment

Most important?

• Text • Images • Sound • Animation

• Video

http://images.google.ca/imgres?imgurl=http://nsstc.uah.edu/~sundar/Teaching/SRS2/globe.jpg&imgrefurl=http://nsstc.uah.edu/~sundar/Teaching/SRS2/SRS2Index.htm&h=542&w=539&sz=46&tbnid=TByteEAaIdCDrM:&tbnh=130&tbnw=129&hl=en&start=34&prev=/images%3Fq%3Dglobe%26start%3D20%26svnum%3D10%26hl%3Den%26lr%3D%26sa%3DN



Multimedia Features

Multimedia components • Text

• Images

• Sound

• Animation

• Video

used in combination with the following features:

1.Interactivity

2.Hyperlinking

Computer-based multimedia applications integrate the

various media components and allow interactivity

and hyperlinking




What is multimedia task?

Combining materials in many media—text, graphic art, sound, video and animation— and

delivering it via digital computer or other electronic means.

Interactive multimedia: Giving an end user some control over what elements to present and when in a hypermedia.

Hypermedia: providing a linked structure through which a user can navigate through elements.


Multimedia every where:

At home: games, entertainment,

education, etc.

In public places: in hotels, malls, ...

In business: marketing and training

In schools: pre-school through post-grad

E-learning is now big business drive

multimedia


Why E-Learning?

Why use computers to teach?

Cost of instruction by other methods is high

Extensive learner practice is needed

Logistical difficulties, getting to classroom

Convenience of just-in-time learning.

Intended learners may have special needs or

different learning styles


Education: • E-learning will become a ‗real area‘ not just a substitute for lack of teachers • All learning will be technology mediated in some way

Shopping and Retail: • Traditional shopping replaced by on-line shopping entirely • Shopping experience will totally change (CRP,…..)

Entertainment: • Electronic games – more virtual reality and on-line

Communications: • Video phone • Internet telephone and video phone systems

Multimedia applications now


Implementation

Multimedia development is different from systems programming !

That is why is prototyping a good idea.

Programming uses authoring tools as well as multimedia.

Media development involves special tools for graphics, sound, video, etc.

User testing and user observations are important.


Required Equipment What pieces of hardware do you think a multimedia developer would need to have either contained in his/her computer or attached to it?

• Sound Card

• Video Card

• Microphone

• Headphones/Speakers

• Camera, Camcorder (video camera recorder) • Fast Processor

• Lots of storage capacity

• DVD burner/player

Software to develop the applications:

•Image creation •Website creation •Video •Animation


Delivery, maintenance and term

CD-ROM/DVD/memory stick versus or

Web delivery.

Corrective: fixing errors after delivery

Should be adaptive to a new environments

Should be perfective: improving behavior or

performance

Should be preventive: improving its

maintainability


Apply concept of multi-finality


Multimedia Definition


Multimedia Data Representation

1- Text


Text Representation:

Use of text in multimedia applications varies on:

Educational, Entertainment, Business.

More, Old and type of people ie.

Children, teens, adults, Elderly, ….


Change the look of your webpage

by changing:

Text Attributes

Design/Layout/Placement of the text

Here is some text

with different

attributes

Design/Layout/Placement

Here is some text with different:

• Placement

• Layout

• Design

Text Attributes

Text Representation Continued


Text Attributes: Text doesn't have to be boring!

• Emphasis can be added by varying the text

attributes

• Font type - Arial, Times New Roman, Comic Sans

• Style - Regular, bold, italics

• Kerning – space between cha r a cters

• Leading - vertical space between lines of text.

• Size - pts vs. pixels ( 8 pt, 8 px, 10 pt, 10 px, 36pt.)

• Color - (red, blue, black… )

• Special effects - underline, shadow,

superscript,subscript,


Serif Sans-Serif

Font Types

See Western Site: www.uwo.ca Use Verdana, Arial

http://www.uwo.ca/


Font Styles:

Variations in the appearance that lets you emphasize parts of the text.

- Bold, Italics, Superscript, Subscript etc. Some examples

Bold Times New Roman

Times New Roman Underline Times New Roman Bold Comic Italics Comic Underline Comic

IMPORTANT: Never underline in a website unless it is a link


Text Leading

the amount of __________________of text

Measured in either positive or negative points or

zero

Greatness is not found in possession, power, position or prestige. It is discovered in goodness, humility, service and character.

Greatness is not found in

possession, power, position or

prestige. It is discovered in

goodness, humility, service

and character.


Text Size PIXEL (px) • the smallest discrete

component of an image or picture on a monitor

• Pixels are a measurement for text and images for the web

• A monitor display setting of 800px x 600px = 480,000 pixels

1280px x 1024px = 1.3 million pixels

POINT (pt) • A linear unit measuring

size of text

• Points are a print unit of measure (Word processing)

• 72 pt = 1 inch • 1 pt = 1/72 inch


NOTE: A point size of 72, will always give you a font that is one inch high

when printed.

If you sent this MS Word Document to a printer and measured between the lines, it would be one inch. For print it doesn‘t matter if you print from a Windows machine or a Mac, a 72 point font size on paper is always one inch high.

Text Size: Points – when printing


Comparison table of points and pixels

Text Size: Pixels – when viewing on a

monitor


Text Color – representation for

the web

COLOUR

Hexadecimal code

represented as number of six

digits

made up of decimals (0–9)

and letters (A-F)

Red Color is #FF0000

Represents RRGGBB What is Purple Western

Colour


Text Design Considerations Readability Is the text easy to read? Avoid dark text against dark

background

Visual Appeal Does the text complement the graphics? Choose text that co-ordinates with any graphics Position text carefully to achieve a good balance with the other multimedia elements Text Layout – simple, clear, white space


Text Design

Considerations

Design in General

Use a max of 2 to 3 different types of fonts

Use a max of 2 to 3 colours in a document/website

Never use underlining in a webpage

Mood Creation

Set the mood using appropriate font

attributes and layout

Headings usually looks better Sans-

Serif, body usually looks better Serif

Sample websites that create a mood:

http://www.courtjesters.co.nz/

http://www.circleworks.co.nz/

http://www.courtjesters.co.nz/

http://www.circleworks.co.nz/


Examples from: “The Non-Designer’s Design Book by Robin Williams

Both boxes above say the same thing.

Which design do you like better?

Can you put your finger on what makes one “Better” than the other.

Before we begin building websites, let‘s

get some design basics:

Good Design Is As Easy as 1-2-3

1. Learn the principles.

They‘re simpler than you might think 2. Recognize when you’re not using them.

Put it into words – name the problem. 3. Apply the principles.

You‘ll be amazed.

Good Design is as easy as . . .

Learn the principles

They‘re simpler than you might think Recognize when you’re not using them.

Put it into words – name the problem. Apply the principles.

You‘ll be amazed.


When designing a webpage, think about:

Contrast

Avoid making 2 elements just similar

Either make them the same (same font, colour, etc…) or make them VERY different.

Items are aligned - creates stronger cohesive unit

Group related items together

Repeat some aspect of the design throughout the entire design i.e. Bold font, thick rule, bullet, colors, font types

Alignment

Repetition

Proximity


History Grotte de Lascaux, France

15,000 - 13,000 BC

1914

1928

1928 – 1931

•Prehistoric humans paint images on the walls of their caves Silent movies incorporated multiple media by using film and text captions together

Walt Disney debuts the first cartoon to use synchronized sound. Movies with sound replace silent movies

http://images.google.ca/imgres?imgurl=http://upload.wikimedia.org/wikipedia/en/4/4e/Steamboat-willie.jpg&imgrefurl=http://www.tutorgig.com/ed/Mickey_Mouse&h=300&w=400&sz=58&hl=en&start=1&um=1&tbnid=1ZOlp96SyrBnFM:&tbnh=93&tbnw=124&prev=/images%3Fq%3Dsteamboat%2Bwillie%2Btrailer%26svnum%3D10%26um%3D1%26hl%3Den%26sa%3DN


1930s

1937

1969

Technicolor is introduced in film and most movies are filmed in color after 1940. Bell Laboratories had a breakthrough in creating dual sound tracks on film. Fantasia was the first commercial movie with a complete surround soundtrack in 1940 NETWORK TECHNOLOGY INTRODUCED

….Arpanet is created ….Advance Research Projects Agency Network …. (discussion 1962) …..First packet switching network and the predecessor to the Internet.

History


Creation of Arpanet 1969

4 locations (1969)

•UCLA

•Stanford

•UC Santa Barbara

•University of Utah

Began talks in 1962

Objective: Build a network technology to allow researchers at various locations across USA to share information

1969

1970 – East coast


1977

1981

1991

1992

1992

1993

Apple starts to dominate PC market. IBM PC announced and captures market share in 18 months. World Wide Web debuts thanks to Tim Berners-Lee. MS Windows 3.1 is released HTML debuts The first graphical browser is born (allows us to see web pages containing IMAGES), it is called Mosiac, by Marc Andreessen, Erin Brina &Tim Clark

History

1969: Network technology introduced: ARPANET


1994

1995

1996

1998 1999

The Rolling Stones become the first major band to broadcast a live performance over the Internet. Disney releases Toy Story, the first feature length computer generated movie (77 minutes long, 4 years to make, 800,000 machine hours to render). Affordable digital cameras widely available. Google Search Engine operates by Larry Page & Sergey Bri Napster debuts, allowing users to download and share MP3s.

History


History (continues)

2000s Integration of computer, memory storage, digital

data camcorders, MP3 players, IPods, speakers,

telephones, HD TV and other technologies

explodes!

2001 Apple introduces iTunes and the iPod

2005 youTube.com launches

2007 Search engine giant Google surpasses

Microsoft as "the most valuable global brand," and

also is the most visited Web site. End


The Multimedia Principle

The power of multimedia is that it

include both words and graphics

Why?

Graphics facilitate active learning,

mentally making connection between

pictorial and verbal representations

Words alone may cause shallow

learning


Two kinds of pictures

Decorative vs. explanative illustrations

What‘s the difference?

Decorative pictures are eye candy

Explanative illustrations help learner

understand the material

Instructional designer‘s job is to enable

learner to make sense of information


How to match graphics to content

It is necessary to:

Illustrate procedures with screen captures

Show a process flow with arrows or

animated graphics

Organize topics by using rollover buttons to

show different graphics


Psychology of multimedia

Information delivery theory: learning

consists of acquiring information

Information format shouldn‘t matter

Cognitive theory: learning is actively

making sense of information

Active learning involves constructing and

connecting visual and verbal representations of

material


Evidence for multimedia effect

Ten lessons teaching scientific or

mechanical processes, such as how pumps

work

Students who receive multimedia lesson

perform better on post-test than students

who receive same information in words

Improvement of 55-121% more correct

solutions to transfer problems

Similar results in experiments was done to

confirm the above results.



2- Sound


(1) Digital Audio

Sound is a continuous wave that travels through the air

The wave is made up of pressure differences. Sound is detected by measuring the pressure level at a location.

Sound waves have normal wave properties (reflection, refraction, diffraction, etc.).

A variety of sounds source generate sounds:

Air Pressure changes Acoustic -- Direct Pressure Variations

Electrical -- Microphone produces electric signal Ears -- Responds to pressure hear sound

Electrical -- Loud Speaker


what a sound actually is

If sound is required input into a

computer: it needs to sampled or

digitized:

Microphones, video cameras produce

analog signals (continuous-valued

voltages)


How analog-to-Digital

to digitize sound (convert it into a stream of numbers) we need to convert Analog signal to Digital signal by Specialized Hardware ―A/D converter‖ by sampling the

sound signal. Two parameters had to be settled:

Sampling - divide the horizontal axis (the time dimension) into discrete pieces. Uniform sampling is broad.

Quantization - divide the vertical axis (signal strength) into pieces. Sometimes, a non-linear function is applied.


Digitization process

1-Recording


Digitization process 1-lisnting


Analog-to-Digital Conversion

parameters 1. How often do you need to sample the

signal?

2. How good is the signal?

3. How is audio data formatted?

A typical examples include: • Volume

• Cross-Fading

• Looping

• Echo/Reverb/Delay

• Filtering

• Signal Analysis


Digital audio

Windows uses WAV format, Apple uses AIFF

Advantages: Reliable playback (―what you hear is what you get‖)

Required for speech playback

Sound effects in \windows\media or the web How so? Where are you getting your sound

effects?


Quantization

More sampling and more data improves quality and resolution

Value of each sample is rounded off to nearest integer (hence, quantization)

8-bit sampling size provides 256 bits to describe dynamic range of amplitude

16-bit sampling size provides over 65 thousand bits for dynamic range, but significantly increases space requirements


Digitized sound is sampled

Sampling frequency/rate: how often the sound sample is taken Sampling size: how much information is stored per sample


Clipping

Occurs if amplitude exceeds intervals

Clipping produces hiss or other distortions


Sample Rates and Bit Size

Sample Rates and Bit Size: How do we store each sample value (Quantization)?

8 Bit Value (0-255), 16 Bit Value (Integer) (0-65535)

How many Samples to take? • 11.025 KHz for Speech (Telephone 8KHz)

• 22.05KHz Low Grade Audio (WWW Audio, AM Radio)

• 44.1 KHz -- CD Quality

Now if we sample at twice the sample frequency, i.e. Nyquist Rate, we start to make some progress. This is the minimum sampling rate.

Nyquist rate -- For lossless digitization, the sampling rate should be at least twice the maximum frequency responses. Indeed many time more the better.


Measuring sound

Sound is measured in decibels (db), the

pressure or volume:

typical voice conversation is ~70db,

a soft whisper is 30db,

a jackhammer is 120db

Decibels measure the energy required to make

sounds logarithmically

A jackhammer’s noise is about 1 watt, but voice

conversation is 1/100,000 watt


Implications of Sample Rate and Bit

Size

SNR is a measure of the quality of the signal.

Affects Quality of Audio: – 16-Bit has a signal-to-noise ratio of 98 dB -- virtually

inaudible

– 8-bit has a signal-to-noise ratio of 50 dB

– Therefore, 8-bit is roughly 8 times as noisy (Prove?)

– 6 dB increment is twice as loud Signal to Noise Ratio

(SNR)

noise

signal

2

noise

2

signal

V

V log 20

V

V log 10 SNR


What to do?

Each bit adds about 6 dB of resolution (little bit more), so 16 bits => 98 dB.

There is therefore a trade off between Audio Quality vs. Data Rate.


Typical applications

Quality Sample Rate Bits per Mono/ Data Rate Frequency

(KHz) Sample Stereo (Uncompressed) Band

--------- ----------- -------- -------- ----------------- ------------

Telephone 8 8 Mono 8 KBytes/sec 200-3,400 Hz

AM Radio 11.025 8 Mono 11.0 KBytes/sec

FM Radio 22.050 16 Stereo 88.2 KBytes/sec

CD 44.1 16 Stereo 176.4 KBytes/sec 20-20,000 Hz

DAT 48 16 Stereo 192.0 KBytes/sec 20-20,000 Hz


Practical Implications of Nyquist Sampling Theory

We must use a Low Pass Filter before Sampling, Otherwise strange artifacts from high frequency signals appear and are audible. We'll finish off with a question: Why are CD Sample Rates 44.1 KHz? The answer should be obvious if you have paid attention to the above notes


Typical Audio Formats

Popular audio file formats include .au (Unix workstations), .aiff (MAC, SGI), .wav (PC, DEC workstations)

A simple and widely used audio compression method is Adaptive Delta Pulse Code Modulation (ADPCM). Based on past samples, it predicts the next sample and encodes the difference between the actual value and the predicted value.

Delivering Audio over a Network

Trade off between desired fidelity and file size

Bandwidth Considerations for Web and other media.


A synthesizer is an electronic musical instrument

designed to produce artificially generated sound,

using techniques such as additive, subtractive, FM

and physical modelling synthesis to create sounds.

There are two major kinds of synthesizers, analog

and digital.

There are also many different kinds of synthesis

methods, each applicable to both analog and

digital synthesizers.

Synthetic Sound

http://www.fact-index.com/a/ad/additive_synthesis.html

http://www.fact-index.com/f/fr/frequency_modulation_synthesis.html


•In the 1970s the development of miniaturized solid-state components allowed synthesizers to become self-contained, portable instruments. By the early 1980s companies were selling compact, modestly-priced synthesizers and the development of Musical Instrument Digital Interface (MIDI) made it easier to integrate and synchronize synthesizers and other electronic instruments. In the 1990s synthesizers began to appear as computer software. •FM (Frequency Modulation) Synthesis - used in low-end Sound Blaster cards, OPL-4 chip, Yamaha DX Synthesizer range popular in Early 1980's.

•Wavetable synthesis: wavetable generated from sound waves of real instruments •Modern Synthesizer use a mixture of sample and synthesis.

Synthetic Sound (continued)

http://en.wikipedia.org/wiki/Musical_Instrument_Digital_Interface


MIDI

A notation similar to a musical score

A communications standard describing how electronic instruments and synthesizers play musical sound

Advantages: Often smaller than digital wave form files

Can be stretched or edited more easily

Exploits quality of good playback instruments

Disadvantages: Playback not reliable, depends on instruments

Can‘t play speech


Typical connection


Example:Q104 MIDI Interface The Q104 MIDI Interface will accept a MIDI

signal from a keyboard or other MIDI device

and creates a control voltage for pitch, a

control voltage for velocity, a gate signal, and

a trigger signal. These signals can be used to – sequencers.

– control oscillators, filters, and to trigger envelope generators

and


Computing the size of a digital recording

Sampling rate * duration (seconds) * (bit resolution / 8) * 1 (mono) or 2 (stereo)

E.g., for a 10 second recording at 20.05 kHz, 8-bit resolution, monophonic (good for speech):

22050 kilohertz * 10 seconds * 8/8 * 1 = 220,500 bytes

Or, for 10 seconds of good music quality at 44.1 kHz, 16-bit resolution, stereo:

44100 kHz * 10 * 16/8 * 2 = 1,764,000 bytes

―Red Book‖ standard (ISO 10149) for CD-quality audio

Expensive for multimedia

Playing 16-bit resolution requires 16-bit sound cards


Compression of digital audio

Compression algorithms can reduce file size by about an order of magnitude

Downside? More clipping, sound degradation

Authorware: Shockwave Audio (SWA) Xtras->Other->Convert WAV to SWA

Also Voxware (better for speech)

Flash: on import, converts wav to 128kb mp3 Stored automatically as symbols in library

Select imported sound from frame properties

Why might it be better to convert some wav files to mp3 outside of Flash?


Compression of digital audio

Compression algorithms can reduce file size by about an order of magnitude

Downside? More clipping, sound degradation

Authorware: Shockwave Audio (SWA) Xtras->Other->Convert WAV to SWA

Also Voxware (better for speech)

Flash: on import, converts wav to 128kb mp3 Why might it be better to convert some wav files to mp3

outside of Flash?


Tips for recording sound

Use a decent quality microphone Should eliminate noise when silent

Choose narrators who speak clearly Coach your narrator for pronunciation, emphasis

A sound editor lets you manipulate sound Trimming: removing white or empty space

Splicing: combining parts of several sounds

Amplifying: adjusting volume (for consistency)

Re-sampling (e.g., from 16-bit down to 8-bit)

Conversion and compression (e.g., wav to mp3)



2- Image &

graphics


Example:

multimedia

Documents

interactive multimedia

multimedia developer

multimedia task

big business drive multimedia

time learning

grad elearning

video phone internet

online shopping