3/21/17

1

Chapter5AnalysisofData

PHYS4840Prof.HannahJang-Condell

Announcements

• ProblemSet#4isdueWednesday3/22• Readsection5.1byThursday3/23(JITTquestiondueat10am)

• Exam#1:Tuesday3/28• NoclassThursday3/30• Readsection5.2forTuesday4/4• ProblemSet#5isdue4/12

3/21/17

2

AnalysisofData• sorting• measuringthemean,standarddeviation,variance,skewness,...

• fittingmodels,goodnessoffit• computingcorrelationcoefficientsfortwosetsofdata• aretwodistributionsdifferent?• computingconfidencelimitsonestimatedmodelparameters

Reference:DataReductionandErrorAnalysisforthePhysicalSciencesbyBevington &Robinson.

ErrorsandUncertainty

• statistical • systematic

3/21/17

3

MeanandstandarddeviationP(x) = 1

σ 2πexp −(x −µ)2

2σ 2

"

#$

%

&'

Exercise

• Meanmean

• Standarddeviationstd

• Computethemeanandstandarddeviationofthefollowingsetofnumbers:

[17,19,21,25,19,22,18,21,23,24]• Submityouranswerstopolleverywhere

µ =xi∑N

σ 2 =(xi −µ)

2∑N −M

3/21/17

4

Randomnumbergenerators

• Runtherandn commandtogenerate10randomnumbers.Calculatethemeanandstandarddeviation.

• Whatisthedifferencebetweenrand andrandn?

• HowdoIgenerate10randomnumberswithameanof20andastandarddeviationof5?

Discussion

• Whatdoesitimplywhensomeonesaysasamplemeanisμ±σ?

• Whatdoes±2σimply?±3σ?A2σdeviationshouldoccurhowoften?

• Whatistheuncertaintyinthemeanofadistribution?Assumetheuncertaintiesineachdatapointareroughlythesame.

• Whichoftheaboveparameterschangewhenthenumberofmeasurementsistripled?Byhowmuch?

3/21/17

5

Significant

https://xkcd.com/882/

Correlations

• Youcanmeasurehowwelltwodistributionsaredistributedby

• WhatdoesitmeanifR=1?• WhatifR = −1 ?R = 0 ?• http://www.tylervigen.com/spurious-correlations

R =αβ∑

α 2∑ β 2∑,

α = xi − x , β = yi − y

3/21/17

6

GoodnessofFit

• Youhaveasetofyi vs.xi andamodelY(x)

• ForGaussianerrors,youexpect

• So,foragoodfit,youwant

χ 2 =yi −Y (xi )"# $%σ i2∑

2

yi −Y (xi ) ≈σ i

χν2 =

χ 2

N −M≈1

“reducedchi-square”

LinearRegression

𝑌 = 𝑎𝑥 + 𝑏

3/21/17

7

LinearRegression

𝑎 = 𝑆Σ𝑥𝑦 − Σ𝑥Σ𝑦𝑆Σ𝑥, − Σ𝑥 ,

𝜎. =𝑆

𝑆Σ𝑥, − Σ𝑥 ,�

𝑏 = Σ𝑥,Σ𝑦 − Σ𝑥Σ𝑥𝑦𝑆Σ𝑥, − Σ𝑥 ,

𝜎0 =Σ𝑥,

𝑆Σ𝑥, − Σ𝑥 ,

�

Discuss

• Whatifσi = σ (constant)?Howdoa, σa, b, σbdependonσ?

3/21/17

8

Exercises

1. Downloadandrunlsfdemo andlinreg.Reproducefigures5.3and5.4.

2. Downloadpollsf.Reproducefigure5.5.

3. Usetherandn functioninMatlab togeneratedatacorrespondingto

𝑦1 = 𝑎𝑥1 + 𝑏 + 𝜎1whereσ isaGaussiandistributedrandomerror.Generateasetof10points,assumingaconstanterror,for

𝑥 = 0,1,2, … , 9 .Plotthesepointsalongwiththeerrorbars.

3/21/17

9

4. Fityourtenpointstoastraightline.Whatistheχ2 valueofyourfit?Howwelldoyourecovertheslopeandinterceptofyourline?

5. Fityour10pointstoa9thorderpolynomial.Whatistheχ2 valueofthisfit?Whichfitlooksbetter:thelinearfitorthepolynomialone?

Section5.2SpectralAnalysis

3/21/17

10

GJ876• Star:0.33Msun

– (Mdwarf,orreddwarf)

• Planetsdiscoveredusingradialvelocity(Dopplershift)technique

• b:2.3Mjup,61dayperiod(1998)

• c:0.7MJup,30dayperiod(2000)

Imagecredit:oklo.org

GJ876

Marcy,Butleretal.2001 Rivera&Lissauer2001

𝜒:, = 8.99 𝜒:, = 1.48

Oneplanet Twointeractingplanets

3/21/17

11

Timeseriesanalysis

• Signalprocessing• Onewaytounderstandphenomenainthetimedomain istoconvertitintothefrequencydomain

• Fouriertransforms

FourierTransform

• AFouriertransformisalinearoperation.• Whatdoesthismeanaboutthetransformofthesumoftwofunctions?

• Thetransformofaconstanttimesafunction?

H ( f ) = h(t)e2πift dt−∞

∞

∫

h(t) = H ( f )e−2πift df−∞

∞

∫

Hk =1N

hne2πi k n/N

n=0

N−1

∑

hn = Hk e−2πi k n/N

n=0

N−1

∑

3/21/17

12

PropertiesofFourierTransformsIfh(t) isreal, thenH(-f ) = H(f )*Ifh(t) isimaginary, then H(-f ) = -H(f )*Ifh(t) iseven, then H(-f ) = H(f ) (even)Ifh(t) isodd, then H(-f ) = -H(f ) (odd)Ifh(t) isreal andeven, then H(-f ) isrealandevenIfh(t) isrealandodd, then H(-f ) isimaginaryandoddIf h(t) isimaginaryandeven, then H(-f ) isimaginaryandevenIf h(t) isimaginaryandodd, then H(-f ) isrealandodd

*indicatescomplexconjugate.Example:ifA = 2 + 2i,thenA* = 2 – 2i

PropertiesofFourierTransforms

h(at) ⇔ |a|-1H(f/a) time scaling|b|-1h(t/b) ⇔ H(bf) frequency scaling

h(t - t0) ⇔ H(f) exp(-2πift0) time shiftingh(t) exp(-2πif0t) ⇔ H(f - f0) frequencyshifting

3/21/17

13

• Transformsdonothavetobecarriedoutinthetime/frequencydomains.Ifatransformoccursspatiallyasafunctionofposition(withunitsofmeters),thentheinversetransformwouldbecarriedoutwithunitsof_________________.

PowerSpectrum

P( f ) = H ( f )2= H ( f )H ( f )*

= Re(H ( f ))!" #$2− Im(H ( f ))!" #$

2

3/21/17

14

GJ876

Riveraetal.2005

Periodogram(bandcplanetsremoved)

⟵frequency(day-1)10-410-310-210-11

Riveraetal.2005

3/21/17

15

Phase-foldedresiduals

Riveraetal.2005

Improvedfitfromaddinga3rdplanet

Riveraetal.2005

3/21/17

16

Sampling

• IfΔ isthetimeintervalbetweenconsecutivesamplings,hn=h(nΔ), n=-∞ to +∞, then Δ-1 isthesamplingrate

Belowisa1kHzsinewavesampledat12kHz.Whatisthelowestfrequencysamplingratethat

couldfullyreconstructsuchawave?

3/21/17

17

Nyquist frequency

• IfasinewaveoftheNyquist frequencyissampledatitspeak,thenthenextsamplingwilloccuratitstrough,followedbyanothersamplingattheensuingpeak,andsoon.

• Nyquist samplingofasinewaveoccurstwicepercycle.

fNy =12Δ

Aliasing

Frequencyofsignal:1/21

Samplingrate:1/20