1

FITTING PSYCHOMETRIC FUNCTIONSFlorian Raudies

11/17/2011

Boston University

2

Overview

Definitions

Parameters

Fitting

Example: Visual Motion

Goodness of Fit

Conclusion

3

Definitions

Labels for the axes of a psychometric function

Stimulus level

Pro

port

ion

corr

ect

ExamplesExperiment Design

Proportion Correct

2AFC 50…100%

3AFC 33…100%

2IAFC 50…100%

2AFC Two alternative forced choice

4

Definitions

Special points of an psychometric function

Stimulus level

Pro

port

ion

corr

ect

50%

PSE

75%

25%

PSEPoint of subjective equivalence

JNDJust noticeable difference

2JND

Psychometric function

5

Definitions

Weibull function

Cumulative normal distribution function

Logit function

𝑓 𝑊𝑒𝑖𝑏𝑢𝑙𝑙 (𝑥 |𝛼 , 𝛽 )=1− exp (−( 𝑥𝛼 )𝛽

)

𝑓 𝑐𝑛𝑑𝑓 (𝑥 | μ ,𝜎 )=erf (𝑥 ;𝜇 ,𝜎 )

𝑓 𝑙𝑜𝑔𝑖𝑡 (𝑥 | μ ,𝜃 )= 1

1+exp (− 𝑥−𝜇𝜃 )

erf (𝑥 ;𝜇 ,𝜎 )= 1√2𝜋 𝜎 ∫

−∞

𝑥

exp (−(𝑠−𝜇)2

2𝜎)𝑑𝑠

6

Definitions in Matlabfunction Y = weibullFunction(X, alpha, beta)% weibullFunction% X - Input values.% alpha - Parameter for scale.% beta - Parameter for shape.%% RETURN% Y - Return values.%% DESCRIPTION% See http://en.wikipedia.org/wiki/Weibull_distribution.

Y = 1 - exp(-(X/alpha).^beta);

7

Definitions in Matlabfunction Y = cndFunction(X, mu,sigma)% cndFunction - Cumulative normal distribution function% Shift by one up and rescale because the integral for erf % ranges from 0 to value whereas the distribution uses the % boundaries -inf to value.Y = (1+erf( (X-mu)/(sqrt(2)*sigma) ))/2;

function Y = logitFunction(X, mu,theta)% logitFunction…Y = 1./( 1 + exp( -(X-mu)/theta ) );

8

Parameters

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

stimulus level

prop

ortio

n co

rrec

tPsychometric Functions

Weibull, =5, =7cndf, =7, =1logit, =2, =0.5

9

Parameters

0 1 2 3 4 5 6 7 8 9 100

0.5

1

Stimulus level

Pro

port

ion

corr

ect Weibull

=5, =2=5, =7=1, =7

0 1 2 3 4 5 6 7 8 9 100

0.5

1

Stimulus level

Pro

port

ion

corr

ect Cndf

=5, =1=5, =0.25=1, =0.25

0 1 2 3 4 5 6 7 8 9 100

0.5

1

Stimulus level

Pro

port

ion

corr

ect Logit

=5, =1=5, =0.25=1, =0.25

10

Parameters

Additional parameters for a psychometric function

with the parameter vector .

- scale

- shape

- guessing rate. For nAFC .

- miss rate. For a perfect observer .

11

Definition in Matlabfunction Y = psycFunctionMissGuess(psycFunction,X,Theta,Const)% psycFunctionMissGuess% psycFunction - Function handle for the psychometric function.% X - Input values.% Theta - Parameter values.% Const - Constants, here guess rate and miss rate.%% RETURN% Y - Output values.

Y = Const(1) … + (1 - Const(1) - Const(2)) * psycFunction(X,Theta(1),Theta(2));

12

Fitting

Assume: independent measurements with

strength of the percept and

response of participant in a 2AFC task.

Problem: Maximum likelihood estimation for parameters of the psychometric function

with

13

Fitting

Small values and log-likelihood

The term can lead to small values below the range of single or double precision. Thus, for optimization take the negative and apply the monotonic log-function function:

This expression is maximized for the parameters . Often additional constraints for the parameters are available.

This is a constraint nonlinear optimization problem with also referred to as nonlinear programming.

14

Fitting in Matlabfunction Theta = fittingPsycFunction(X, Y, opt)% fittingPsycFunction…ThetaMin = opt.ThetaMin; % Lower boundary for parameters.ThetaMax = opt.ThetaMax; % Upper boundary.ThetaIni = opt.ThetaIni; % Initial value for parameters.Const = opt.Const; % Constants in the psychometric function.psycFunction = opt.psycFunction; % Function handle for the psychometric function.% Optimization with the fmincon from the Matlab optimization toolbox.Theta = fmincon(@(Theta)logLikelihoodPsycFunction(... psycFunction, Theta, X,Y, Const), ... ThetaIni, [],[],[],[], ThetaMin,ThetaMax, [], opt); function L = logLikelihoodPsycFunction(psycFunction, Theta, X,Y, Const)% logLikelihoodFunction…Xtrue = X(Y==1);Xfalse = X(Y==0);L = -sum(log( psycFunctionMissGuess(...

psycFunction, Xtrue, Theta, Const) + eps))... -sum(log(1 - psycFunctionMissGuess(...

psycFunction, Xfalse, Theta, Const) + eps));

15

Example: Visual Motion

Objective: Measure the coherence threshold for motion-direction discrimination.

Design: 2AFC task between leftward and rightward motion for varying motion coherence by a

limited dot lifetime in an random dot kinematogram (RDK).

Use the method of constant stimuli for 11 coherence values. This requires usually more samples than adaptive thresholding techniques.

Use 10 trials for each coherence value.

This is a very simplified example!

16

Example: Visual Motion

17

Example: Visual Motion

A single trial

Fixation &

(Response) for500ms

Fixation &

1st Motion for400ms

Fixation &

Blank for100ms

Time

Overall time 10 x 11 x 1,9sec = 209sec or 3.48min.

A response is not expected before the first trail.

Are the motions equal?

Fixation&

2nd Motion for400ms

18

Example: Visual MotionCorrect / response

Motion coherence (%)

0 10 20 30 40 50 60 70 80 90 100

Trial

1 1/1 1/1 1/1 1/1 1/1 1/1 1/1 1 0 0 0

2 0/1 0/1 0/0 1/1 0/0 1/1 1/1 1 1 1 0

3 1/0 0/1 1/1 0/1 1/1 1/0 0/1 0 0 0 1

4 1/1 1/0 0/1 0/0 0/1 0/1 0/0 0 0 1 1

5 0/0 1/1 1/0 1/0 1/0 0/0 1/1 1 1 1 0

6 0/1 0/0 1/0 1/1 1/1 0/0 1/1 0 1 1 0

7 1/0 1/1 0/0 1/0 0/1 1/1 1/1 1 0 0 1

8 0/0 0/1 0/1 0/1 0/0 0/0 0/0 0 0 0 0

9 1/1 0/0 1/0 0/1 1/1 1/0 0/0 1 1 1 1

10 0/1 1/0 0/0 0/0 0/1 0/0 0/0 0 1 0 0

Correct (%)

50 50 50 50 60 70 90 100 100 100 100

19

Example: Visual Motion

0 20 40 60 80 10040

50

60

70

80

90

100

110

motion coherence (%)

perc

enta

ge c

orre

ctFitted Weibull function

datafitted Weibull

20

Example in Matlab% Load data files.DataStimulus = dlmread('./DataMotionCoherenceStimulus00.txt');DataObserver = dlmread('./DataMotionCoherenceObserver00.txt');% "Response = 1" encodes correct and "Response = 0" incorrect.Response = double(DataStimulus(2:end,:)==DataObserver(2:end,:));trialNum = size(Response,2);StimulusLevel = DataStimulus(1,:);

% Fit data.opt.ThetaMin = [ 1.0 0.5]; % alpha, beta to optimize.opt.ThetaMax = [100.0 10.0];opt.ThetaIni = [ 5.0 1.0];opt.Const = [ 0.5 0.0]; % gamma, lambda are fixed.opt.psycFunction = @weibullFunction;StimulusLevelMatrix = repmat(StimulusLevel,[trialNum 1]);Theta = fittingPsycFunction(StimulusLevelMatrix, Response, opt);

21

Goodness of Fit

Over dispersion or lack of fit

Dependency between trials

Non-stationary psychometric function (e.g. learning)

Under dispersion or fit is too god

Experimenter’s bias in removing outliers

22

Conclusion

Use maximum likelihood to fit your data, while leaving the lapse rate as parameter being optimized.

This is not the case in the presented code but can be adapted.

Assess goodness of fit to:

Ensure Parameter estimates and their variability are from a plausible model to describe the data.

Identify uneven sampling of the stimulus level or outliers by applying an objective criteria.

23

References

For fitting data

Myung, Journal of Mathematical Psychology 47, 2003

Treutwein & Strasburger, Perception & Psychophysics 61(1), 1999

For goodness of fit

Wichmann & Hill, Perception & Psychophysics 63(8), 2001

For detection theory

Macmillan & Creelman. Detection theory - A user’s guide Psychology Press (2009)