www.mubeena.net the situational interview
TRANSCRIPT
www.mubeena.net
The Situational InterviewThe Situational Interview
www.mubeena.net
www.mubeena.net
DefinitionDefinition
Job-related situational questions Situations are specific on-the-job activitiesApplicants put in hypothetical situationsQuestions can refer to past experience or future intentionsDiffers from structured format by benchmarking applicant responses
www.mubeena.net
EvolutionEvolutionTreeTree
Legal IssuesSelection Process DilemmaGriggs vs. Duke Power, 1971
Workforce PredictionQuality of workforce will deteriorate
Dyer, 1981
Redefinition of Selection Tool“A valid test cannot be developed until the organization agrees upon an acceptable definition (measure) of employee behavior.”
Latham & Wexley, 1982
www.mubeena.net
Research LiteratureResearch LiteratureLatham, Saari, Pursell, & Campion, 1980
Examined reliability and validity of situational interview by conducting three studiesStudy 1: entry-level position, Study 2: first-line supervisory position, Study 3: concurrent predictive validityResults for Study 1: Interview scores were significantly correlated with every performance criterion including the overall global ratingResults for Study 2: Interview scores correlated significantly with three of four criteria – safety, work habits, org commitmentResults for Study 3: Interview scores significantly correlated with composite job performance scores, even for women and black employees
www.mubeena.net
Research LiteratureResearch LiteratureConclusions of Latham et al. (1980)
Intentions correlate with behaviorComprehensive job analysis reflect content validityInterviewee motivation influenced by face validityInter-rater reliability high because interviewers themselves developed scoring keyEmphasis on critical behaviors rather than traitsPast behavior based items could lessen dishonest responses, it can be verified by previous employer. However, adverse impact must be kept in check
www.mubeena.net
Research LiteratureResearch LiteraturePulakos & Schmitt, 1995
Compared validities of experience-based (past performance) questions and situational (future intentions) questionsResults: Past performance is better predictor of job performance
Lack of Validity for Situational Items Attributed to: 1. Responses were evaluated at end of interview. It is
necessary to make the ratings immediately2. Prior studies indicate that situational interviews involve
lower-level jobs. In this study, the job was complex and demanding
• Major conclusion was that interviews result in lower levels of adverse impact because they measure cognitive as well as non-cognitive performance dimensions
www.mubeena.net
Research LiteratureResearch LiteratureMaurer & Fay, 1988
Two hypotheses: Rater training Inter-rater agreement
Situational Inter-rater agreementInterpretation of results: Situational interviews are more effective in producing higher inter-rater agreement, even with little or no training, because of specific rating scalesThere was so significant main effect of training Situational interview is more cost-effective strategy in comparison with conventional structured interview
www.mubeena.net
Critical Incident Technique
Was first developed and used by John Flanagan and his students at the University of Pittsburgh in the late 1940s and early 1950sUsed to identify job behaviors that differentiate successful performance from unsuccessful performanceCan be useful in job analysis as well as training and performance appraisalThe development of the graduate situational interview was based on this technique
www.mubeena.net
Critical Incident DevelopmentCritical incidents were identified by reviewing the job
analysis (CMQ)Behaviors that could exemplify either good or poor behaviors of graduate students were chosenFourteen job dimensions were identified:Prioritizing LeadershipStress management EthicsAbility to work with others Public speakingReceiving feedback AltruismSelf Assessment DiversityDecision making RetentionGroup involvementParticipation
www.mubeena.net
Development of Situational Interview Questions
The critical incidents were used to form the questions to be asked in the situational interviewThe beginning of each question stem entailed a description of a circumstance involving a critical incidentExample: Critical Incident – Decision MakingExample: You are placed at a company in which your supervisor is critical and provides no direction to complete a taskThe questions ended with a general proposition Example: “How would you handle this situation?”Seventeen questions were written using this formatEach critical incident that we identified had a corresponding questionTwo of the critical incidents had more that one corresponding question, namely stress management and ethics
www.mubeena.net
BARS DevelopmentTo facilitate the scoring process a behaviorally anchored rating scale was developed for each questionThe behavioral scales were developed by brainstorming and determining KSA relevance and importance as rated by SMEs (job incumbents)Our anticipation of the responses that we would receive was used to develop a five point scaleThe five point scales included examples of high, average and low responses The examples that we agreed represented high, average and low responses were used as behaviors on the scale
www.mubeena.net
Example 5: Applicant tries to talk to the boss
4: Applicant tries to set own goals and direction no resolve of conflict.
3: Applicant says nothing. 2: Applicant complains, does not
start the job until the issue is addressed.
1: Applicant leaves the job/talks to boss in a negative fashion.
www.mubeena.net
Focus Groups
Our questions and behavioral scales were presented to a focus group The focus group (subject matter experts) recommended the following:- add multitasks to the scales- reword ambiguous and vague questions - make sure that each question corresponds to the critical incident it is describing in a situation
www.mubeena.net
Scoring of the Situational Interview
Each response on the behavioral scale had a corresponding scoreScores were rated from one to five - a 1 or 2 represented a low response - a 3 represented an average response - a 4 or 5 represented a high response The interviewers choose the response that best represents the interviewees answerA total score for the interview can be obtained by summing the ratings for each question
www.mubeena.net
Development of Cutoff Score
The Ebel method was used to determine the cutoff score- rate each item (1-5)- determine the percentage of items a minimally qualified candidate would respond to correctly (a correct response to each item represented a score of 3 or more)- multiply this percentage by the number of items
(3 x 17 = 51)- the cutoff score becomes 51 for the situational interview
www.mubeena.net
Administration
Participants: 30 I/O graduate studentsTwo groups of raters: 3 and 2Selection Tool was administered twice over a 2 week period.The Tool was administered in a classroom setting with other selection tests being administered at the same time.
www.mubeena.net
Possible Threats To Validity
Random Error:Noise/Disruption EffectsExposure to test questions before
administrationFakeablityRater effects(Halo, Contrasts,
Leniency Effects)
www.mubeena.net
Descriptive Data
Mean=64.21Standard Deviation=5.78Range=27.67Skewness=-.845
www.mubeena.net
Score Distribution
interview avg
72.5
70.0
67.5
65.0
62.5
60.0
57.5
55.0
52.5
50.0
47.5
45.0
7
6
5
4
3
2
1
0
Std. Dev = 5.78
Mean = 64.2
N = 30.00
www.mubeena.net
Validity
Correlation with-Graduate GPA r=.284, p=.064-Psychology 249 r=.360, p=.025-Psychology 283A r=.381, p=.019-Undergraduate GPA r=.252, p=.089-GRE r= -.007, p=.485
www.mubeena.net
Near Future
Standardize scores
Reliability and more validity analysis
Analyze individual questions
www.mubeena.net
Distant Future
How can we modify the interview to make it a better instrument for selecting graduate students?separate information gathering and
evaluation, evaluate after gatheringmodify behavioral anchorsre-categorize questions so that
questions which co-vary are in the same category