1

FITTING PSYCHOMETRIC FUNCTIONSFlorian Raudies11/17/2011Boston University

2

OverviewDefinitionsParametersFittingExample: Visual MotionGoodness of FitConclusion

3

DefinitionsLabels for the axes of a psychometric function

Stimulus level

Pro

porti

on c

orre

ct

ExamplesExperiment Design

Proportion Correct

2AFC 50…100%

3AFC 33…100%

2IAFC 50…100%

2AFC Two alternative forced choice

4

DefinitionsSpecial points of an psychometric function

Stimulus level

Pro

porti

on c

orre

ct

50%

PSE

75%

25%

PSEPoint of subjective equivalence

JNDJust noticeable difference

2JND

Psychometric function

5

DefinitionsWeibull function

Cumulative normal distribution function

Logit function

𝑓 𝑊𝑒𝑖𝑏𝑢𝑙𝑙 (𝑥 |𝛼 , 𝛽 )=1− exp (−( 𝑥𝛼 )𝛽)

𝑓 𝑐𝑛𝑑𝑓 (𝑥 | μ ,𝜎 )=erf (𝑥 ;𝜇 ,𝜎 )

𝑓 𝑙𝑜𝑔𝑖𝑡 (𝑥 | μ ,𝜃 )= 1

1+exp (− 𝑥−𝜇𝜃 )

erf (𝑥 ;𝜇 ,𝜎 )= 1√2𝜋 𝜎 ∫

−∞

𝑥

exp (−(𝑠−𝜇)2

2𝜎 )𝑑𝑠

6

Definitions in Matlabfunction Y = weibullFunction(X, alpha, beta)% weibullFunction% X - Input values.% alpha - Parameter for scale.% beta - Parameter for shape.%% RETURN% Y - Return values.%% DESCRIPTION% See http://en.wikipedia.org/wiki/Weibull_distribution.

Y = 1 - exp(-(X/alpha).^beta);

7

Definitions in Matlabfunction Y = cndFunction(X, mu,sigma)% cndFunction - Cumulative normal distribution function% Shift by one up and rescale because the integral for erf % ranges from 0 to value whereas the distribution uses the % boundaries -inf to value.Y = (1+erf( (X-mu)/(sqrt(2)*sigma) ))/2;

function Y = logitFunction(X, mu,theta)% logitFunction…Y = 1./( 1 + exp( -(X-mu)/theta ) );

8

Parameters

0 2 4 6 8 100

0.2

0.4

0.6

0.8

1

stimulus level

prop

ortio

n co

rrect

Psychometric Functions

Weibull, =5, =7cndf, =7, =1logit, =2, =0.5

9

Parameters

0 1 2 3 4 5 6 7 8 9 100

0.5

1

Stimulus level

Pro

porti

on c

orre

ct Weibull

=5, =2=5, =7=1, =7

0 1 2 3 4 5 6 7 8 9 100

0.5

1

Stimulus level

Pro

porti

on c

orre

ct Cndf

=5, =1=5, =0.25=1, =0.25

0 1 2 3 4 5 6 7 8 9 100

0.5

1

Stimulus level

Pro

porti

on c

orre

ct Logit

=5, =1=5, =0.25=1, =0.25

10

ParametersAdditional parameters for a psychometric function

with the parameter vector .

- scale - shape - guessing rate. For nAFC . - miss rate. For a perfect observer .

11

Definition in Matlabfunction Y = psycFunctionMissGuess(psycFunction,X,Theta,Const)% psycFunctionMissGuess% psycFunction - Function handle for the psychometric function.% X - Input values.% Theta - Parameter values.% Const - Constants, here guess rate and miss rate.%% RETURN% Y - Output values.

Y = Const(1) … + (1 - Const(1) - Const(2)) * psycFunction(X,Theta(1),Theta(2));

12

FittingAssume: independent measurements with

strength of the percept and response of participant in a 2AFC task.

Problem: Maximum likelihood estimation for parameters of the psychometric function

with

13

FittingSmall values and log-likelihoodThe term can lead to small values below the range of single or double precision. Thus, for optimization take the negative and apply the monotonic log-function function:

This expression is maximized for the parameters . Often additional constraints for the parameters are available.This is a constraint nonlinear optimization problem with also referred to as nonlinear programming.

14

Fitting in Matlabfunction Theta = fittingPsycFunction(X, Y, opt)% fittingPsycFunction…ThetaMin = opt.ThetaMin; % Lower boundary for parameters.ThetaMax = opt.ThetaMax; % Upper boundary.ThetaIni = opt.ThetaIni; % Initial value for parameters.Const = opt.Const; % Constants in the psychometric function.psycFunction = opt.psycFunction; % Function handle for the psychometric function.% Optimization with the fmincon from the Matlab optimization toolbox.Theta = fmincon(@(Theta)logLikelihoodPsycFunction(... psycFunction, Theta, X,Y, Const), ... ThetaIni, [],[],[],[], ThetaMin,ThetaMax, [], opt); function L = logLikelihoodPsycFunction(psycFunction, Theta, X,Y, Const)% logLikelihoodFunction…Xtrue = X(Y==1);Xfalse = X(Y==0);L = -sum(log( psycFunctionMissGuess(...

psycFunction, Xtrue, Theta, Const) + eps))... -sum(log(1 - psycFunctionMissGuess(...

psycFunction, Xfalse, Theta, Const) + eps));

15

Example: Visual MotionObjective: Measure the coherence threshold for motion-

direction discrimination.Design: 2AFC task between leftward and rightward

motion for varying motion coherence by a limited dot lifetime in an random dot kinematogram (RDK).

Use the method of constant stimuli for 11 coherence values. This requires usually more samples than adaptive thresholding techniques.

Use 10 trials for each coherence value.This is a very simplified example!

16

Example: Visual Motion

17

Example: Visual MotionA single trial

Fixation &

(Response) for500ms

Fixation &

1st Motion for400ms

Fixation &

Blank for100ms

Time

Overall time 10 x 11 x 1,9sec = 209sec or 3.48min.

A response is not expected before the first trail.

Are the motions equal?

Fixation&

2nd Motion for400ms

18

Example: Visual MotionCorrect / response

Motion coherence (%)0 10 20 30 40 50 60 70 80 90 100

Trial

1 1/1 1/1 1/1 1/1 1/1 1/1 1/1 1 0 0 0

2 0/1 0/1 0/0 1/1 0/0 1/1 1/1 1 1 1 0

3 1/0 0/1 1/1 0/1 1/1 1/0 0/1 0 0 0 1

4 1/1 1/0 0/1 0/0 0/1 0/1 0/0 0 0 1 1

5 0/0 1/1 1/0 1/0 1/0 0/0 1/1 1 1 1 0

6 0/1 0/0 1/0 1/1 1/1 0/0 1/1 0 1 1 0

7 1/0 1/1 0/0 1/0 0/1 1/1 1/1 1 0 0 1

8 0/0 0/1 0/1 0/1 0/0 0/0 0/0 0 0 0 0

9 1/1 0/0 1/0 0/1 1/1 1/0 0/0 1 1 1 1

10 0/1 1/0 0/0 0/0 0/1 0/0 0/0 0 1 0 0

Correct (%)

50 50 50 50 60 70 90 100 100 100 100

19

Example: Visual Motion

0 20 40 60 80 10040

50

60

70

80

90

100

110

motion coherence (%)

perc

enta

ge c

orre

ctFitted Weibull function

datafitted Weibull

20

Example in Matlab% Load data files.DataStimulus = dlmread('./DataMotionCoherenceStimulus00.txt');DataObserver = dlmread('./DataMotionCoherenceObserver00.txt');% "Response = 1" encodes correct and "Response = 0" incorrect.Response = double(DataStimulus(2:end,:)==DataObserver(2:end,:));trialNum = size(Response,2);StimulusLevel = DataStimulus(1,:);

% Fit data.opt.ThetaMin = [ 1.0 0.5]; % alpha, beta to optimize.opt.ThetaMax = [100.0 10.0];opt.ThetaIni = [ 5.0 1.0];opt.Const = [ 0.5 0.0]; % gamma, lambda are fixed.opt.psycFunction = @weibullFunction;StimulusLevelMatrix = repmat(StimulusLevel,[trialNum 1]);Theta = fittingPsycFunction(StimulusLevelMatrix, Response, opt);

21

Goodness of FitOver dispersion or lack of fitDependency between trialsNon-stationary psychometric function (e.g. learning)

Under dispersion or fit is too godExperimenter’s bias in removing outliers

22

ConclusionUse maximum likelihood to fit your data, while leaving the lapse rate as parameter being optimized. This is not the case in the presented code but can be adapted.

Assess goodness of fit to:Ensure Parameter estimates and their variability are from a plausible model to describe the data.Identify uneven sampling of the stimulus level or outliers by applying an objective criteria.

23

ReferencesFor fitting dataMyung, Journal of Mathematical Psychology 47, 2003Treutwein & Strasburger, Perception & Psychophysics 61(1), 1999

For goodness of fitWichmann & Hill, Perception & Psychophysics 63(8), 2001

For detection theoryMacmillan & Creelman. Detection theory - A user’s guide Psychology Press (2009)