friday, may 14, 2004 isys3015 analytical methods for is professionals school of it, the university...
Post on 19-Dec-2015
219 views
TRANSCRIPT
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 1
Factorial DesignsFactorial Designs
Week 9 Lecture 2
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 2
AgendaAgenda
• Basic factorial design concepts
• Main and interaction effect
• Factorial design in computer system performance analysis
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 3
What are factorial designsWhat are factorial designs
• Two or more independent variables are manipulated in a single experiment
• They are referred to as factors• The major purpose of the research is to
explore their effects jointly• Factorial design produce efficient
experiments, each observation supplies information about all of the factors
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 4
A simple exampleA simple example
• Investigate an education program with a variety of variations to find out the best combination– Amount of time receiving
instruction• 1 hour per week vs. 4 hour
per week– Settings
• In-class vs. pull out• 2 X 2 factorial design
– Number of numbers tells how many factors
– Number values tell how many levels
– The result of multiplying tells how many treatment groups that we have in a factorial design
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 5
Null outcomeNull outcome
• None of the treatment has any effect
• Main effect– is an outcome that is a
consistent difference between levels of a factor.
• Interaction effect– An interaction effect
exists when differences on one factor depend on the level you are on another factor.
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 6
Main effectsMain effects
• Main effect of time• Main effect of setting• Main effects on both
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 7
Interaction effectInteraction effect
• An interaction effect exists when differences on one factor depend on the level of another factor
• How do we know if there is an interaction in a factorial design?– Statistical analysis will report all main effects and
interactions.– If you can not talk about effect on one factor
without mentioning the other factor– Spot an interaction in the graphs – whenever there
are lines that are not parallel there is an interaction present!
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 8
Interaction effectInteraction effect
• Interaction as a difference in magnitude of response
• Interaction as a difference in direction of response
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 9
Factorial design variationsFactorial design variations
• A 2 X 3 example• study the effect of
different treatment combinations for cocaine abuse. – Factor 1: treatment
• psychotherapy • behavior modification
– Factor 2:• inpatient • day treatment• outpatient
– Dependent variable• severity of illness rating
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 10
Factorial designs in computer Factorial designs in computer system performance analysis system performance analysis
• Personal workstation design– Processor: 68000, Z80, 8086– Memory size: 512K 2M or 8M bytes– Number of disks: one, two or three– Workload: Secretarial, managerial or
scientific– User education: high school, college, post-
graduate level
• Dependent variable– Throughput, response time
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 11
2222 factorial design factorial design
• Two factors, each at two levels
• Example: workstation design– Factor 1: memory
size– Factor 2: cache size– DV: performance in
MIPS0
20
40
60
80
4M 8M
Memory size
Perf
orm
ance
in M
IPS
1K
2K
Cache size
Memory size
4M byte 8M byte
1K 15 45
2K 25 75
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 12
Quantify effectsQuantify effects
• We want to learn which factor contribute more to the performance.– Define two variable
– The regression model
Missizememoryif
Missizememoryifxa 161
41
Kissizecacheif
Kissizecacheifxb 21
11
baabbbaa xxqxqxqqy 0
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 13
Quantifying results (cont)Quantifying results (cont)
• Resolving those coefficients
• We get
• How do you read this?
abba
abba
abba
abba
qqqq
qqqq
qqqq
qqqq
0
0
0
0
75
25
45
15
baba xxxxy 5102040
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 14
Quantify effects by sign tableQuantify effects by sign table
• Sign table method
I A B AB y
1 -1 -1 1 15
1 1 -1 -1 45
1 -1 1 -1 25
1 1 1 1 75
160 80 40 20 Total
40 20 10 5 Total/4
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 15
22KK factorial design factorial design
• K factors, each at two level
• 2K experiments• 23 design example
– In designing a personal workstation, the three factors needed to be studied are: cache size, memory size and number of processors
Factor Level -1 Level 1
Memory size 4Mbytes 16Mbytes
Catch size 1Kbytes 2Kbytes
Number of processors
1 2
Cache size (Kbytes)
4 Mbytes 16 Mbytes
1 2 1 2
1 14 46 22 58
2 10 50 34 86
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 16
22kk factorial design with factorial design with replicationreplication
• r replications of 2k experiments– 2Kr observations– Allows estimation of experimental errors– 223 design example
• The memory-cache experiments were repeated three times each. The result is shown below
Cache size Memory size
4M 8M
1 K 15, 18, 12 45, 48,51
2K 25, 28, 19 75,75,81
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 17
Full and fractional factorial Full and fractional factorial design design
• Full factorial design– Study all combinations– Can find effect of all factors– May try 2K factorial design first
• Fractional (incomplete) factorial design– Leave some treatment groups empty– Less information– May not get all interactions– No problem if interaction is negligible
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 18
22k-pk-pFractional factorial design Fractional factorial design
• Large number of factors– Large number of experiments– Full factorial design too expensive– Use a fractional factorial design
• 2k-p design allows analyzing k factors with only 2k-pexperiments.– 2k-1 design requires only half as many
experiments– 2k-2 design requires only one quarter of the
experiments
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 19
Example: 2Example: 27-47-4 DesignDesign
Exp No. A B C C E F G
1 -1 -1 -1 1 1 1 -1
2 1 -1 -1 -1 -1 1 1
3 -1 1 -1 -1 1 -1 1
4 1 1 -1 1 -1 -1 -1
5 -1 -1 1 1 -1 -1 1
6 1 -1 1 -1 1 -1 -1
7 -1 1 1 -1 -1 1 -1
8 1 1 1 1 1 1 1
• Study 7 factors with only 8 experiments• When quantify the effects, just calculate the main
effects• Will be able to eliminate some factors in further study.
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 20
Quantify modelQuantify model
ggffeeddccbbaa xqxqxqxqxqxqxqqy 0
I A B C C E F G
1 -1 -1 -1 1 1 1 -1 20
1 1 -1 -1 -1 -1 1 1 35
1 -1 1 -1 -1 1 -1 1 7
1 1 1 -1 1 -1 -1 -1 42
1 -1 -1 1 1 -1 -1 1 36
1 1 -1 1 -1 1 -1 -1 50
1 -1 1 1 -1 -1 1 -1 45
1 1 1 1 1 1 1 1 82
317 101 35 109 43 1 47 3 Total
39.62 12.62 4.37 13.62 5.37 0.125 5.87 0.37 Total/8
Friday, May 14, 2004 ISYS3015 Analytical Methods for IS Professionals
School of IT, The University of Sydney 21
Preparing the sign tablePreparing the sign table
• Choose k - p factors and prepare a complete sign table for a full factorial design with k-p factors
• Of the 2k-p –k +p -1 column on the right, choose p columns and mark them with the p factors that were not chosen in step 1.