the final stages

1

Spring 2002© University of Stirling

IT82: Multimedia 1

Testing and Delivery


IT82: Multimedia 2

The Final StagesWe have scoped the project with our clients… signedthe proposal...

We have brainstormed… storyboarded andfinalised our design...

We have selected our authoring environment andproduced our prototype… to make sure our ideasare feasible and practical and of course thecustomer has approved the approach...

Now we flesh out our prototype and we havecompleted the project… so we just deliver it…Yes?… Errrrrrrrrrrrrrrrrrrrrrrrr?

2


IT82: Multimedia 3

Perhaps we should TEST our presentation first...


IT82: Multimedia 4

Testing a safety-critical piece of software, such as thecontrol software for an aeroplane may seem obvious…

• Does the plane fly without crashing?

• What happens if… .

• An engine fails

• Severe turbulence

• etc

Testing Software

3


IT82: Multimedia 5

But any computer program needs testing…

• Does it produce the correct answer when…

• Does it crash if…

• Can the user do…

Testing Software (2)

Is it functional?

Is it useable?


IT82: Multimedia 6

But do we need to test our multimedia presentation?Surely if we have built it using an authoring systemlike Director it will work just fine!

Hmmm… … ..

Hopefully a presentation built using a tried andtested authoring system will be more stable thansomething you have programmed from scratch…But...

Testing Multimedia

4


IT82: Multimedia 7

Functionality:

• Does clicking this button take you to the rightscreen?

• Is the audio track synchronized with the video?

Testing Multimedia (2)

Usability:

• Can a user navigate as you expect?

• Does the message get across?


IT82: Multimedia 8

Testing Functionality• Functionality includes the entire operation of your

multimedia system– layout of media– navigation

• Specified by storyboards, nav maps and taskanalysis

• May be simpler than your average computerprogram, but still needs testing!– And there could be complex code (even in Lingo) underlying

the dynamics, interactivity and navigational strategies

5


IT82: Multimedia 9

There are a number of techniques we can use to testthe functionality of our application.

They are the same as in software engineering...

These include:

• Black Box Testing• White Box Testing• Walk thru’s

Testing Functionality (2)


IT82: Multimedia 10

Black Box testing is the most commonly used methodfor functional testing...

INPUT OUTPUT

We consider what goes in and what comes out.

We are asking, “Does the presentation match thespecification?”

• verification

Black Box Testing

6


IT82: Multimedia 11

For a multimedia presentation, this means

• Are all the strands of the presentation there?

• Is all the necessary information included?

• Can all the goals specified in a task analysisactually be reached?

BEWARE ! - specifications are not always astatement of what the user wants!

Black Box Testing (2)


IT82: Multimedia 12

White box testing involves testing “all” the functionalityof the program.

This can be simple in a simple program…

But if the program is large it will require splitting to givea function by function test approach.

White Box Testing

Once each individual function within the applicationhas been tested and is working as expected…

We re-combine the modules and test the programas a whole - known as “Integration”

7


IT82: Multimedia 13

White Box Testing (2)

• This applies to a multimedia presentation as much asfor any software program

• Detailed check of functionality against storyboards,nav maps and task analysis

• Check of control code for interactivity and navigation• Very easy to have a button with a link to the wrong

place, or accidentally include an image on the wrongpage or in the wrong place!

• White box testing precedes black box testing– will be going on throughout the development of the prototype


IT82: Multimedia 14

This process doesn’t involve the computer at all.

It involves someone sitting down with a print out of theprogram and “walking through” it.

It is best that the person carrying out the process isnot the person who wrote the program.

It is sometimes called “Inspection” testing.

The inspector looks at the code and compares it withthe specification.

Walk Thru’s

8


IT82: Multimedia 15

In computer programs, the “inspector” looks for:

Variables being initialisedLoops initialised and terminatedetc

Walk Thru’s (2)

For a multimedia presentation a “walk thru” can bedone to:

• check feasibility of task analysis againststoryboards and nav maps• check control code


IT82: Multimedia 16

Alpha and Beta TestingSoftware engineers will understand the differencebetween these two types of testing:

Alpha - Alpha releases are typically used by softwaredevelopers to describe levels of product developmentwhen testing is done and feedback is sought - they arepassed amongst a select group of mock users.

Beta - These are sent to a wider but still selectaudience with the caveat that the software maycontain bugs - here is where the battle for a goodreputation for the presentation can be won or lost!

9


IT82: Multimedia 17

Usability TestingWe want our presentation to be usable… All issues andtechniques of usability testing from HCI apply… But also...

We want our multimedia project to be suitable for the user.

We differ in terms of usability in that we are also looking tosee if our presentation “works” and to evaluate whetherwe achieve our goals by using multimedia or if we havechosen the correct delivery environment for thepresentation.

We obviously don’t have time to look at all of theseaspects . So let’s look at a few...


IT82: Multimedia 18

Gathering Information

How do we gather information about the usability ofour presentation?

There are many approaches we can adopt…

• Questionnaires

• Field trials

• Focus groups

10


IT82: Multimedia 19

QuestionnairesThe questionnaire approach can prove very useful.

Firstly we have to decide what it is we want to find out:

• Validation

• Verification

• Functionality

• Enhancement

Structuring questionnaires is a distinct skill where thedesigner can, if not careful, ask the wrong question to findout the information.


IT82: Multimedia 20

Questionnaires (2)Questionnaires are used

• on a cold call basis - e.g. here is a sample of ourproduct…

• in field trials

The major problem with a questionnaire... given thatyou have designed one exactly how it is required! …is that it requires processing... i.e. effort and time…that means money.

11


IT82: Multimedia 21

Field TrialsField trials are an essential component in the testing of amultimedia project.

This is where we throw the presentation at the real usersto see how they interact and how the interfacecomponents perform.

In field trials there are a number of approaches that canbe adopted dependant on how much guidance is givento the users.

Typically a number of users are given the program to runand are either asked to fill in a questionnaire or areasked pointed comments by the facilitator...


IT82: Multimedia 22

Field Trials (2)

Field trials rely on trained observers or facilitators to get atthe information required.

Indeed many firms outsource this whole aspect of testingto specific experts in the area.

These firms may well have dedicated usability labs.

The whole subject of usability testing including taskanalysis and user interaction is employed during the fieldtrials.

12


IT82: Multimedia 23

Focus GroupsFocus groups form an added input into both the designand re-design process.

Here we can conduct guided field trials or discuss thewhole aspect of the presentation i.e.

What would you want from a tour of Stirling?

What kinds of things would you like this program orpresentation to involve?

Here we can also attempt to gain a better insight into whatthe user expects etc

What do you understand by the term MultimediaPresentation?


IT82: Multimedia 24

Focus Groups (2)

The other format of a focus group is to run a lab session orintroduction to the software or presentation and then allowthe focus group to utilise the program without supervision.

This approach is especially useful for educational titleswhere the teacher can effectively run a completelesson and observe the students whilst they operatethe interface.

13


IT82: Multimedia 25

Delivery to the ClientSo we have tested, gained feedback, modified theproduct so that everything is now ready to deliver tothe customer…

But exactly what do we deliver?

This will depend on the goal of the presentation… isit aimed at a kiosk (stand-alone installation) or fordelivery in a magazine (CD-Rom) or are wedelivering a web-based presentation etc

Each has it’s own foibles and pitfalls… the biggest ismaintenance and upgrades.


IT82: Multimedia 26

Delivery and MaintenanceThe user will push you to maintain the presentation,especially if it is web-based or kiosk-based. Thesepresentations are strong on customer interfacing and assuch feedback on quality or wishlist items can be fast andfurious…

When you get the final delivery “signed off” it is essentialin that process that a structure is put in place to accountfor any upgrading or maintenance of the production.Especially in terms of costing or in the form of amaintenance agreement!!!

14


IT82: Multimedia 27

What to DeliverRemember to deliver only the protected presentation,unless you want everyone to copy your code or to“borrow” your artwork etc

Also be aware that in terms of Director and Authorware,for example, the Xtras need to be included.

Once the CD-Rom or other delivery medium is sorted,another phase of testing on a cleaned down platform orvirgin platform is essential!

You need to have a platform which contains ONLY theminimum spec that you have specified for the production.NOTHING ELSE!!!!


IT82: Multimedia 28

The End of Design As We Know It!

the final stages

Documents