techniques of data analysis assoc. prof. dr. abdul hamid b. hj. mar iman director centre for real...
TRANSCRIPT
![Page 1: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/1.jpg)
Techniques of Data Analysis
Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman
DirectorCentre for Real Estate Studies
Faculty of Engineering and Geoinformation ScienceUniversiti Tekbnologi Malaysia
Skudai, Johor
![Page 2: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/2.jpg)
Objectives
Overall: Reinforce your understanding from the main lecture
Specific: * Concepts of data analysis * Some data analysis techniques * Some tips for data analysis
What I will not do: * To teach every bit and pieces of statistical analysis techniques
![Page 3: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/3.jpg)
Data analysis – “The Concept”
Approach to de-synthesizing data, informational, and/or factual elements to answer research questions
Method of putting together facts and figures to solve research problem
Systematic process of utilizing data to address research questions
Breaking down research issues through utilizing controlled data and factual information
![Page 4: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/4.jpg)
Categories of data analysis
Narrative (e.g. laws, arts) Descriptive (e.g. social sciences) Statistical/mathematical (pure/applied sciences) Audio-Optical (e.g. telecommunication) Others
Most research analyses, arguably, adopt the firstthree.
The second and third are, arguably, most popular in pure, applied, and social sciences
![Page 5: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/5.jpg)
Statistical Methods
Something to do with “statistics” Statistics: “meaningful” quantities about a sample of
objects, things, persons, events, phenomena, etc. Widely used in social sciences. Simple to complex issues. E.g. * correlation * anova * manova * regression * econometric modelling Two main categories: * Descriptive statistics * Inferential statistics
![Page 6: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/6.jpg)
Descriptive statistics
Use sample information to explain/make abstraction of population “phenomena”.
Common “phenomena”:* Association (e.g. σ1,2.3 = 0.75)
* Tendency (left-skew, right-skew)* Causal relationship (e.g. if X, then, Y)* Trend, pattern, dispersion, rangeUsed in non-parametric analysis (e.g. chi-
square, t-test, 2-way anova)
![Page 7: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/7.jpg)
Examples of “abstraction” of phenomena
Trends in property loan, shop house demand & supply
0
50000
100000
150000
200000
Year (1990 - 1997)
Loan to property sector (RM
million)
32635.8 38100.6 42468.1 47684.7 48408.2 61433.6 77255.7 97810.1
Demand for shop shouses (units) 71719 73892 85843 95916 101107 117857 134864 86323
Supply of shop houses (units) 85534 85821 90366 101508 111952 125334 143530 154179
1 2 3 4 5 6 7 8
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
Batu P
ahat
Joho
r Bah
ru
Kluang
Kota T
ingg
i
Mer
sing
Mua
r
Pontia
n
Segam
at
District
No
. o
f h
ou
ses
1991
2000
0
2
4
6
8
10
12
14
0-4
10-1
4
20-2
4
30-3
4
40-4
4
50-5
4
60-6
4
70-7
4
Age Category (Years Old)
Pro
po
rtio
n (
%)
Demand (% sales success)
120100806040200
Pri
ce (
RM
/sq
. ft
of
bu
ilt a
rea
)
200
180
160
140
120
100
80
![Page 8: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/8.jpg)
Examples of “abstraction” of phenomena
Demand (% sales success)
12010080604020
Pri
ce
(R
M/s
q.f
t. b
uilt
are
a)
200
180
160
140
120
100
80
10.00 20.00 30.00 40.00 50.00 60.00
10.00
20.00
30.00
40.00
50.00
-100.00
-80.00
-60.00
-40.00
-20.00
0.00
20.00
40.00
60.00
80.00
100.00
Distance from Rakaia (km)
Distance from Ashurton (km)
% prediction
error
![Page 9: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/9.jpg)
Inferential statistics
Using sample statistics to infer some “phenomena” of population parameters
Common “phenomena”: cause-and-effect * One-way r/ship
* Multi-directional r/ship
* Recursive
Use parametric analysis
Y1 = f(Y2, X, e1)Y2 = f(Y1, Z, e2)
Y1 = f(X, e1)Y2 = f(Y1, Z, e2)
Y = f(X)
![Page 10: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/10.jpg)
Examples of relationship
Coefficientsa
1993.108 239.632 8.317 .000
-4.472 1.199 -.190 -3.728 .000
6.938 .619 .705 11.209 .000
4.393 1.807 .139 2.431 .017
-27.893 6.108 -.241 -4.567 .000
34.895 89.440 .020 .390 .697
(Constant)
Tanah
Bangunan
Ansilari
Umur
Flo_go
Model1
B Std. Error
UnstandardizedCoefficients
Beta
StandardizedCoefficients
t Sig.
Dependent Variable: Nilaisma.
Dep=9t – 215.8
Dep=7t – 192.6
![Page 11: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/11.jpg)
Which one to use?
Nature of research * Descriptive in nature? * Attempts to “infer”, “predict”, find “cause-and-effect”, “influence”, “relationship”? * Is it both? Research design (incl. variables involved). E.g. Outputs/results expected * research issue * research questions * research hypotheses
At post-graduate level research, failure to choose the correct data analysis technique is an almost sure ingredient for thesis failure.
![Page 12: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/12.jpg)
Common mistakes in data analysis Wrong techniques. E.g.
Infeasible techniques. E.g. How to design ex-ante effects of KLIA? Development
occurs “before” and “after”! What is the control treatment? Further explanation! Abuse of statistics. E.g. Simply exclude a technique
Note: No way can Likert scaling show “cause-and-effect” phenomena!
Issue Data analysis techniques
Wrong technique Correct technique
To study factors that “influence” visitors to come to a recreation site
“Effects” of KLIA on the development of Sepang
Likert scaling based on interviews
Likert scaling based on interviews
Data tabulation based on open-ended questionnaire survey
Descriptive analysis based on ex-ante post-ante experimental investigation
![Page 13: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/13.jpg)
Common mistakes (contd.) – “Abuse of statistics”
Issue Data analysis techniques
Example of abuse Correct technique
Measure the “influence” of a variable on another
Using partial correlation
(e.g. Spearman coeff.)
Using a regression parameter
Finding the “relationship” between one variable with another
Multi-dimensional scaling, Likert scaling
Simple regression coefficient
To evaluate whether a model fits data better than the other
Using R2 Many – a.o.t. Box-Cox 2 test for model equivalence
To evaluate accuracy of “prediction” Using R2 and/or F-value of a model
Hold-out sample’s MAPE
“Compare” whether a group is different from another
Multi-dimensional scaling, Likert scaling
Many – a.o.t. two-way anova, 2, Z test
To determine whether a group of factors “significantly influence” the observed phenomenon
Multi-dimensional scaling, Likert scaling
Many – a.o.t. manova, regression
![Page 14: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/14.jpg)
How to avoid mistakes - Useful tips
Crystalize the research problem → operability of it!
Read literature on data analysis techniques. Evaluate various techniques that can do similar
things w.r.t. to research problem Know what a technique does and what it doesn’t Consult people, esp. supervisor Pilot-run the data and evaluate results Don’t do research??
![Page 15: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/15.jpg)
Principles of analysis
Goal of an analysis:
* To explain cause-and-effect phenomena
* To relate research with real-world event
* To predict/forecast the real-world
phenomena based on research
* Finding answers to a particular problem
* Making conclusions about real-world event
based on the problem
* Learning a lesson from the problem
![Page 16: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/16.jpg)
Data can’t “talk” An analysis contains some aspects of scientific reasoning/argument: * Define * Interpret * Evaluate * Illustrate * Discuss * Explain * Clarify * Compare * Contrast
Principles of analysis (contd.)
![Page 17: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/17.jpg)
Principles of analysis (contd.)
An analysis must have four elements: * Data/information (what) * Scientific reasoning/argument (what? who? where? how? what happens?) * Finding (what results?) * Lesson/conclusion (so what? so how? therefore,…)Example
![Page 18: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/18.jpg)
Principles of data analysis
Basic guide to data analysis:
* “Analyse” NOT “narrate”
* Go back to research flowchart
* Break down into research objectives and
research questions
* Identify phenomena to be investigated
* Visualise the “expected” answers
* Validate the answers with data
* Don’t tell something not supported by
data
![Page 19: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/19.jpg)
Principles of data analysis (contd.)
Shoppers Number
Male
Old
Young
6
4
Female
Old
Young
10
15
More female shoppers than male shoppers
More young female shoppers than young male shoppers
Young male shoppers are not interested to shop at the shopping complex
![Page 20: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/20.jpg)
Data analysis (contd.)
When analysing:
* Be objective
* Accurate
* TrueSeparate facts and opinionAvoid “wrong” reasoning/argument. E.g.
mistakes in interpretation.
![Page 21: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/21.jpg)
Introductory Statistics for Social SciencesIntroductory Statistics for Social Sciences
Basic conceptsBasic conceptsCentral tendencyCentral tendency
VariabilityVariabilityProbabilityProbability
Statistical ModellingStatistical Modelling
![Page 22: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/22.jpg)
Basic Concepts
Population: the whole set of a “universe” Sample: a sub-set of a population Parameter: an unknown “fixed” value of population characteristic Statistic: a known/calculable value of sample characteristic
representing that of the population. E.g.
μ = mean of population, = mean of sample
Q: What is the mean price of houses in J.B.?
A: RM 210,000
J.B. houses
μ = ?
SST
DST
SD
1
= 300,000 = 120,0002
= 210,0003
![Page 23: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/23.jpg)
Basic Concepts (contd.)
Randomness: Many things occur by pure chances…rainfall, disease, birth, death,..
Variability: Stochastic processes bring in them various different dimensions, characteristics, properties, features, etc., in the population
Statistical analysis methods have been developed to deal with these very nature of real world.
![Page 24: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/24.jpg)
“Central Tendency”
Measure Advantages Disadvantages
Mean(Sum of all values ÷
no. of values)
Best known average
Exactly calculable
Make use of all data
Useful for statistical analysis
Affected by extreme values Can be absurd for discrete data
(e.g. Family size = 4.5 person)
Cannot be obtained graphically
Median(middle value)
Not influenced by extreme
values Obtainable even if data
distribution unknown (e.g.
group/aggregate data) Unaffected by irregular class
width
Unaffected by open-ended class
Needs interpolation for group/
aggregate data (cumulative
frequency curve) May not be characteristic of group
when: (1) items are only few; (2)
distribution irregular
Very limited statistical use
Mode(most frequent value)
Unaffected by extreme values
Easy to obtain from histogram
Determinable from only values
near the modal class
Cannot be determined exactly in
group data
Very limited statistical use
![Page 25: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/25.jpg)
Central Tendency – “Mean”,
For individual observations, . E.g.
X = {3,5,7,7,8,8,8,9,9,10,10,12}
= 96 ; n = 12 Thus, = 96/12 = 8 The above observations can be organised into a frequency
table and mean calculated on the basis of frequencies
= 96; = 12
Thus, = 96/12 = 8
x 3 5 7 8 9 10 12
f 1 1 2 3 2 2 1
f 3 5 14 24 18 20 12
![Page 26: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/26.jpg)
Central Tendency–“Mean of Grouped Data”
House rental or prices in the PMR are frequently tabulated as a range of values. E.g.
What is the mean rental across the areas?
= 23; = 3317.5
Thus, = 3317.5/23 = 144.24
Rental (RM/month) 135-140 140-145 145-150 150-155 155-160
Mid-point value (x) 137.5 142.5 147.5 152.5 157.5
Number of Taman (f) 5 9 6 2 1
fx 687.5 1282.5 885.0 305.0 157.5
![Page 27: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/27.jpg)
Central Tendency – “Median”
Let say house rentals in a particular town are tabulated as follows:
Calculation of “median” rental needs a graphical aids→
Rental (RM/month) 130-135 135-140 140-145 155-50 150-155
Number of Taman (f) 3 5 9 6 2
Rental (RM/month) >135 > 140 > 145 > 150 > 155
Cumulative frequency 3 8 17 23 25
1. Median = (n+1)/2 = (25+1)/2 =13th. Taman
2. (i.e. between 10 – 15 points on the vertical axis of ogive).
3. Corresponds to RM 140-145/month on the horizontal axis
4. There are (17-8) = 9 Taman in the range of RM 140-145/month
5. Taman 13th. is 5th. out of the 9
Taman
6. The interval width is 5
7. Therefore, the median rental can
be calculated as:
140 + (5/9 x 5) = RM 142.8
![Page 28: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/28.jpg)
Central Tendency – “Median” (contd.)
![Page 29: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/29.jpg)
Central Tendency – “Quartiles” (contd.)
Upper quartile = ¾(n+1) = 19.5th. Taman
UQ = 145 + (3/7 x 5) = RM 147.1/month
Lower quartile = (n+1)/4 = 26/4 = 6.5 th. Taman
LQ = 135 + (3.5/5 x 5) = RM138.5/month
Inter-quartile = UQ – LQ = 147.1 – 138.5 = 8.6th. Taman
IQ = 138.5 + (4/5 x 5) = RM 142.5/month
![Page 30: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/30.jpg)
“Variability”
Indicates dispersion, spread, variation, deviation For single population or sample data:
where σ2 and s2 = population and sample variance respectively, xi = individual observations, μ = population mean, = sample mean, and n = total number of individual observations.
The square roots are:
standard deviation standard deviation
![Page 31: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/31.jpg)
“Variability” (contd.)
Why “measure of dispersion” important? Consider returns from two categories of shares: * Shares A (%) = {1.8, 1.9, 2.0, 2.1, 3.6} * Shares B (%) = {1.0, 1.5, 2.0, 3.0, 3.9} Mean A = mean B = 2.28% But, different variability! Var(A) = 0.557, Var(B) = 1.367
* Would you invest in category A shares or category B shares?
![Page 32: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/32.jpg)
“Variability” (contd.)
Coefficient of variation – COV – std. deviation as % of the mean:
Could be a better measure compared to std. dev.
COV(A) = 32.73%, COV(B) = 51.28%
![Page 33: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/33.jpg)
“Variability” (contd.)
Std. dev. of a frequency distribution The following table shows the age distribution of second-time home buyers:
x^
![Page 34: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/34.jpg)
“Probability Distribution”
Defined as of probability density function (pdf). Many types: Z, t, F, gamma, etc. “God-given” nature of the real world event. General form:
E.g.
(continuous)
(discrete)
![Page 35: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/35.jpg)
“Probability Distribution” (contd.)
Dice1
Dice2 1 2 3 4 5 6
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12
![Page 36: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/36.jpg)
“Probability Distribution” (contd.)
Values of x are discrete (discontinuous)
Sum of lengths of vertical bars p(X=x) = 1 all x
Discrete values Discrete values
![Page 37: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/37.jpg)
“Probability Distribution” (contd.)
2.00 3.00 4.00 5.00 6.00 7.00
Rental (RM/ sq.ft.)
0
2
4
6
8
Freq
uenc
y
Mean = 4.0628Std. Dev. = 1.70319N = 32
▪ Many real world phenomena take a form of continuous random variable
▪ Can take any values between two limits (e.g. income, age, weight, price, rental, etc.)
![Page 38: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/38.jpg)
“Probability Distribution” (contd.)
P(Rental = RM 8) = 0 P(Rental < RM 3.00) = 0.206
P(Rental < RM7) = 0.972 P(Rental RM 4.00) = 0.544
P(Rental 7) = 0.028 P(Rental < RM 2.00) = 0.053
![Page 39: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/39.jpg)
“Probability Distribution” (contd.)
Ideal distribution of such phenomena:
* Bell-shaped, symmetrical
* Has a function of
μ = mean of variable x
σ = std. dev. Of x
π = ratio of circumference of a
circle to its diameter = 3.14
e = base of natural log = 2.71828
![Page 40: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/40.jpg)
“Probability distribution”
μ ± 1σ = ? = ____% from total observation
μ ± 2σ = ? = ____% from total observation
μ ± 3σ = ? = ____% from total observation
![Page 41: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/41.jpg)
“Probability distribution”
* Has the following distribution of observation
![Page 42: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/42.jpg)
“Probability distribution”
There are various other types and/or shapes of distribution. E.g.
Not “ideally” shaped like the previous one
Note: p(AGE=age) ≠ 1
How to turn this graph into a probability distribution function (p.d.f.)?
![Page 43: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/43.jpg)
“Z-Distribution” (X=x) is given by area under curve Has no standard algebraic method of integration → Z ~ N(0,1) It is called “normal distribution” (ND) Standard reference/approximation of other distributions. Since there
are various f(x) forming NDs, SND is needed To transform f(x) into f(z): x - µ Z = --------- ~ N(0, 1) σ 160 –155 E.g. Z = ------------- = 0.926 5.4
Probability is such a way that: * Approx. 68% -1< z <1 * Approx. 95% -1.96 < z < 1.96 * Approx. 99% -2.58 < z < 2.58
![Page 44: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/44.jpg)
“Z-distribution” (contd.)
When X= μ, Z = 0, i.e.
When X = μ + σ, Z = 1 When X = μ + 2σ, Z = 2 When X = μ + 3σ, Z = 3 and so on. It can be proven that P(X1 <X< Xk) = P(Z1 <Z< Zk)
SND shows the probability to the right of any particular value of Z.
Example
![Page 45: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/45.jpg)
Normal distribution…Questions
Your sample found that the mean price of “affordable” homes in Johor Bahru, Y, is RM 155,000 with a variance of RM 3.8x107. On the basis of a normality assumption, how sure are you that:
(a) The mean price is really ≤ RM 160,000(b) The mean price is between RM 145,000 and 160,000
Answer (a): P(Y ≤ 160,000) = P(Z ≤ ---------------------------) = P(Z ≤ 0.811) = 0.1867Using , the required probability is: 1-0.1867 = 0.8133
Always remember: to convert to SND, subtract the mean and divide by the std. dev.
160,000 -155,000
3.8x107
Z-table
![Page 46: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/46.jpg)
Normal distribution…Questions
Answer (b):
Z1 = ------ = ---------------- = -1.622
Z2 = ------ = ---------------- = 0.811
P(Z1<-1.622)=0.0455; P(Z2>0.811)=0.1867P(145,000<Z<160,000) = P(1-(0.0455+0.1867) = 0.7678
X1 - μ
σ
145,000 – 155,000
3.8x107
X2 - μ
σ
160,000 – 155,000
3.8x107
![Page 47: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/47.jpg)
Normal distribution…Questions
You are told by a property consultant that the average rental for a shop house in Johor Bahru is RM 3.20 per sq. After searching, you discovered the following rental data:
2.20, 3.00, 2.00, 2.50, 3.50,3.20, 2.60, 2.00, 3.10, 2.70 What is the probability that the rental is greater than RM 3.00?
![Page 48: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/48.jpg)
“Student’s t-Distribution”
Similar to Z-distribution:
* t(0,σ) but σn→∞→1
* -∞ < t < +∞
* Flatter with thicker tails
* As n→∞ t(0,σ) → N(0,1)
* Has a function of
where =gamma distribution; v=n-1=d.o.f; =3.147
* Probability calculation requires information on
d.o.f.
![Page 49: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/49.jpg)
“Student’s t-Distribution”
Given n independent measurements, xi, let
where μ is the population mean, is the sample mean, and s is the estimator for population standard deviation.
Distribution of the random variable t which is (very loosely) the "best" that we can do not knowing σ.
![Page 50: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/50.jpg)
“Student’s t-Distribution”
Student's t-distribution can be derived by:
* transforming Student's z-distribution using
* defining
The resulting probability and cumulative distribution functions are:
![Page 51: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/51.jpg)
“Student’s t-Distribution”
where r ≡ n-1 is the number of degrees of freedom, -∞<t<∞,(t) is the gamma function, B(a,b) is the beta function, and I(z;a,b) is the regularized beta function defined by
fr(t) =
=
Fr(t) =
=
=
![Page 52: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/52.jpg)
Forms of “statistical” relationship
Correlation Contingency Cause-and-effect * Causal * Feedback * Multi-directional * Recursive The last two categories are normally dealt with
through regression
![Page 53: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/53.jpg)
Correlation “Co-exist”.E.g.
* left shoe & right shoe, sleep & lying down, food & drink Indicate “some” co-existence relationship. E.g.
* Linearly associated (-ve or +ve)
* Co-dependent, independent But, nothing to do with C-A-E r/ship!
Example: After a field survey, you have the following data on the distance to work and distance to the city of residents in J.B. area. Interpret the results?
Formula:
![Page 54: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/54.jpg)
Contingency A form of “conditional” co-existence:
* If X, then, NOT Y; if Y, then, NOT X
* If X, then, ALSO Y
* E.g.
+ if they choose to live close to workplace,
then, they will stay away from city
+ if they choose to live close to city, then, they
will stay away from workplace
+ they will stay close to both workplace and city
![Page 55: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/55.jpg)
Correlation and regression – matrix approach
![Page 56: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/56.jpg)
Correlation and regression – matrix approach
![Page 57: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/57.jpg)
Correlation and regression – matrix approach
![Page 58: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/58.jpg)
Correlation and regression – matrix approach
![Page 59: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/59.jpg)
Correlation and regression – matrix approach
![Page 60: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/60.jpg)
Test yourselves!
Q1: Calculate the min and std. variance of the following data:
Q2: Calculate the mean price of the following low-cost houses, in various
localities across the country:
PRICE - RM ‘000 130 137 128 390 140 241 342 143
SQ. M OF FLOOR 135 140 100 360 175 270 200 170
PRICE - RM ‘000 (x) 36 37 38 39 40 41 42 43
NO. OF LOCALITIES (f) 3 14 10 36 73 27 20 17
![Page 61: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/61.jpg)
Test yourselves!
Q3: From a sample information, a population of housing estate is believed have a “normal” distribution of X ~ (155, 45). What is the general adjustment to obtain a Standard Normal Distribution of this population?
Q4: Consider the following ROI for two types of investment:
A: 3.6, 4.6, 4.6, 5.2, 4.2, 6.5B: 3.3, 3.4, 4.2, 5.5, 5.8, 6.8
Decide which investment you would choose.
![Page 62: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/62.jpg)
Test yourselves!
Q5: Find:
(AGE > “30-34”)
(AGE ≤ 20-24)
( “35-39”≤ AGE < “50-54”)
![Page 63: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/63.jpg)
Test yourselves!
Q6: You are asked by a property marketing manager to ascertain whether
or not distance to work and distance to the city are “equally” important factors influencing people’s choice of house location.
You are given the following data for the purpose of testing:
Explore the data as follows:• Create histograms for both distances. Comment on the shape of the
histograms. What is you conclusion?• Construct scatter diagram of both distances. Comment on the output.• Explore the data and give some analysis.• Set a hypothesis that means of both distances are the same. Make
your conclusion.
![Page 64: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/64.jpg)
Test yourselves! (contd.)
Q7: From your initial investigation, you belief that tenants of “low-quality” housing choose to rent particular flat units just to find shelters. In this context ,these groups of people do not pay much attention to pertinent aspects of “quality life” such as accessibility, good surrounding, security, and physical facilities in the living areas.
(a) Set your research design and data analysis procedure to address the research issue(b) Test your hypothesis that low-income tenants do not perceive
“quality life” to be important in paying their house rentals.
![Page 65: Techniques of Data Analysis Assoc. Prof. Dr. Abdul Hamid b. Hj. Mar Iman Director Centre for Real Estate Studies Faculty of Engineering and Geoinformation](https://reader035.vdocuments.net/reader035/viewer/2022062511/5517ca05550346892b8b4c94/html5/thumbnails/65.jpg)
Thank you