testing in industrial & business settings. how are people hired for a particular job or...
TRANSCRIPT
Testing in Testing in Industrial & Industrial &
Business Business SettingsSettings
How are people hired for a How are people hired for a particular job or position?particular job or position?
Employment interviewEmployment interview Can be influenced by extraneous factor Can be influenced by extraneous factor
such as age, disability status, gender, such as age, disability status, gender, racerace
Use of testsUse of tests When used in addition to interview & When used in addition to interview &
other sources of information, can other sources of information, can improve chances of successful selectionimprove chances of successful selection
Base Rates & Hit RatesBase Rates & Hit Rates
tests often used to make dichotomous decisions: Pass/fail Accept/reject Hire/don’t hire
E.g., use of an 1100 score on the GRE to decide on whom to reject or whom to process further
This value is a cut-off score or a cutting score
The problem with cutting The problem with cutting scoresscores
Some people who have scores above the cutting score will do poorly
Some people who have scores below the cutting score would have done well
Hits & MissesHits & Misses
HitHit MissMiss
MissMiss HitHit
Decision on basis of cutting scoreAccep
tReject
Actual Outcome
SuccessFailure
Hit RateHit Rate
Percentage of cases in which a test Percentage of cases in which a test accurately predicts success or accurately predicts success or failurefailure
Hits & MissesHits & Misses
2020 4040
1010 3030
Decision on basis of cutting scoreAccep
tReject
Actual Outcome
SuccessFailure
What is the hit rate?
Base RateBase Rate
The selection or pass rate obtained The selection or pass rate obtained withoutwithout using the test using the test
E.g., using only undergraduate E.g., using only undergraduate grades, letters of reference, etc. (no grades, letters of reference, etc. (no GRE), 70% of individuals selected GRE), 70% of individuals selected are successful in completing their are successful in completing their degreedegree
If hit rate using the test is only 60%, If hit rate using the test is only 60%, test is of no valuetest is of no value
False negatives & false False negatives & false positivespositives
There are two types of “misses”:There are two types of “misses”: False negativeFalse negative
Individuals not selected (because of a score below the Individuals not selected (because of a score below the cutting score) who would have been successful in job cutting score) who would have been successful in job or program (e.g., GRE has high false negative rate)or program (e.g., GRE has high false negative rate)
Concluding on the basis of a test that a tumor is Concluding on the basis of a test that a tumor is benignbenign
Concluding on basis of a test that someone is not Concluding on basis of a test that someone is not suicidal, when he/she actually issuicidal, when he/she actually is
False positiveFalse positive Individuals selected for job or program who failIndividuals selected for job or program who fail Concluding on the basis of a test that someone has Concluding on the basis of a test that someone has
AIDS, when they do not have the diseaseAIDS, when they do not have the disease
HitHit False False negativenegative
FalseFalse
PositivePositiveHitHit
Decision on basis of cutting score
Hire as pilot
Reject
Actual Outcome
Success
Failure
What is more dangerous, false +ve or false –ve?
HitHit False False negativenegative
FalseFalse
PositivePositiveHitHit
Decision on basis of cutting scoreDiagnose
Aids
Diagnose
No Aids
Actual DiseaseStatus
Has AIDS
Doesn’t
Have AIDS
What is more dangerous, false +ve or false –ve?
Costs of False NegativesCosts of False Negatives& False Positives& False Positives
For certain decisions, false negatives For certain decisions, false negatives will have a high costwill have a high cost E.g., diagnosing a person as not having E.g., diagnosing a person as not having
AIDS when she really does have it; that AIDS when she really does have it; that a person is not suicidal when he really isa person is not suicidal when he really is
For other decisions, false positives For other decisions, false positives will have a high costwill have a high cost E.g. hiring someone as a commercial E.g. hiring someone as a commercial
pilot when he is likely to crash the planepilot when he is likely to crash the plane
Test Performance
Actual
Outcome
Cutting Score
Success
Failure
False -ve
False +ve
False negative, positive & criterion validity
Changing Rate of False Changing Rate of False Negatives & False Positives Negatives & False Positives
by adjusting the cutting by adjusting the cutting scorescore
To reduce the number of false To reduce the number of false negatives,negatives, lower lower the cutting scorethe cutting score
To reduce the number of false To reduce the number of false positives, positives, raise raise the cutting score the cutting score
AA
88BB
22
CC
1515DD
7575
Decision on basis of cutting scoreDiagnose
Aids
Diagnose
No Aids
Actual DiseaseStatus
Has AIDS
Doesn’t
Have AIDS
Detection rate (sensitivity) = A/(A+B) = 8/10 Accuracy Rate = (A+D)/(A+B+C+D) = 83/100 (83%)
Taylor Russell TablesTaylor Russell Tables
helps evaluate validity of test in relation to amount it contributes beyond base rates
Requires Definition of success -- patient lived, success on
job, school Determination of base rate Definition of selection ratio -- % applicants
selected Validity coefficient
table gives likelihood that person selected on basis of test score will be successful
Taylor-Russell Table for a Base Rate of .60Taylor-Russell Table for a Base Rate of .60
Class ExerciseClass Exercise Suppose that you are director of graduate studies in psychology
at Wilfrid Laurier University. You can choose 10 students for the graduate program. A total of 50 students apply to the program, and normally you would make your decision on the basis of undergraduate grades, letters of recommendation, work and volunteer experiences, and the like. In the past, about 60% of the students whom you admit complete their degrees. Up to this point, your program has not required applicants to submit GRE scores, but you are considering adding a GRE requirement for new applications, even though it correlates only about .25 with success in graduate school.
What is the base rate of success in this instance?
WWhat is your success rate adding the GRE?
How many people will you reject that would probably have been successful in the program? (i.e., how many false negatives will there be?)
How many people will you accept who will likely not complete the program? (how many false positives will there be?)
77 2323
33 1717
Decision on basis of cutting scoreAccep
tReject
Actual Outcome
SuccessFailure
Base rate - .60
Selection ratio - .20
Validity coefficient - .25
30
20
10 40 50
Tests for Use in IndustryTests for Use in Industry
Bennett Mechanical Comprehension Bennett Mechanical Comprehension TestTest Mechanical abilityMechanical ability
Minnesota Clerical TestMinnesota Clerical Test Clerical skillsClerical skills
Computer Programmer Aptitude Computer Programmer Aptitude BatteryBattery Computer programming skillsComputer programming skills
Integrity TestsIntegrity Tests
Also known as “honesty tests”Also known as “honesty tests” Paper-and-pencil inventories used for Paper-and-pencil inventories used for
personnel selection to identify potentially personnel selection to identify potentially dishonest or “counterproductive” dishonest or “counterproductive” employeesemployees
American businesses lose from $15- to American businesses lose from $15- to $25-billion per year due to employee theft$25-billion per year due to employee theft
30% of all business failures attributed to 30% of all business failures attributed to employee theftemployee theft
Personnel Selection Personnel Selection Inventory (PSI)Inventory (PSI)
Includes scales that measure:Includes scales that measure: HonestyHonesty Drug avoidanceDrug avoidance Tenure or job commitmentTenure or job commitment SafetySafety Work valuesWork values Supervision attitudesSupervision attitudes Validity Validity
CandidnessCandidness Accuracy Accuracy
Sample ItemsSample Items
Responses are recorded on 7-point scale ranging from “far more than average” (1) to
“far less than average” (7)
Validity of PSIValidity of PSI In 1982 & 1983 a chain of home improvement centers screened In 1982 & 1983 a chain of home improvement centers screened
more than 4,000 applicants with PSI to help reduce on-the-job more than 4,000 applicants with PSI to help reduce on-the-job theft and drug abuse.theft and drug abuse.
PSI not used from 1979 to 1981PSI not used from 1979 to 1981 ““shrinkage” losses reduced from $7.5 million to $5.25 millionshrinkage” losses reduced from $7.5 million to $5.25 million Number of employees terminated for using drugs at work reduced Number of employees terminated for using drugs at work reduced
from 16 in 1981 to 1 in 1983 from 16 in 1981 to 1 in 1983
Measuring Characteristics Measuring Characteristics of the Work Settingof the Work Setting
Based on the premise that work Based on the premise that work environments will influence the environments will influence the productivity and satisfaction of individuals productivity and satisfaction of individuals who work in those environmentswho work in those environments
Rudolph Moos developed scales to measure Rudolph Moos developed scales to measure several different kinds of environments:several different kinds of environments: Work environmentsWork environments ClassroomsClassrooms Correctional institutionsCorrectional institutions
Work Environment ScaleWork Environment Scale
10 subscales, organized into 3 10 subscales, organized into 3 dimensions:dimensions: Relationship Relationship Personal growth Personal growth System maintenance & system changeSystem maintenance & system change
Relationship DimensionRelationship Dimension InvolvementInvolvement (extent to which employees are (extent to which employees are
concerned about & committed to their jobs)concerned about & committed to their jobs) ““people seem to take pride in the organization”people seem to take pride in the organization”
Peer cohesionPeer cohesion (extent to which employees (extent to which employees are friendly & supportive of one another)are friendly & supportive of one another) ““There’s not much group spirit”There’s not much group spirit”
Supervisor supportSupervisor support (extent to which (extent to which management is supportive of employees)management is supportive of employees) ““supervisors tend to talk down to employees”supervisors tend to talk down to employees”
Personal Growth Personal Growth DimensionDimension
AutonomyAutonomy (extent to which employees are (extent to which employees are encouraged to be self-sufficient)encouraged to be self-sufficient) ““people can use their own initiative to do things”people can use their own initiative to do things”
Task orientationTask orientation (degree of emphasis on (degree of emphasis on good planning, efficiency & getting the job good planning, efficiency & getting the job done)done) ““this is a highly efficient, work-oriented place”this is a highly efficient, work-oriented place”
Work pressureWork pressure (degree to which the press of (degree to which the press of work & time urgency dominate the job work & time urgency dominate the job milieu)milieu) ““there is constant pressure to keep working”there is constant pressure to keep working”
System Maintenance & System Maintenance & System Change DimensionSystem Change Dimension
ClarityClarity (extent to which employees know what to (extent to which employees know what to expect)expect) ““things are sometimes pretty disorganized”things are sometimes pretty disorganized”
ControlControl (extent to which management uses rules & (extent to which management uses rules & pressures to keep employees under control)pressures to keep employees under control) ““Supervisors keep a rather close watch on employees”Supervisors keep a rather close watch on employees”
Innovation Innovation (degree of emphasis on variety, change, (degree of emphasis on variety, change, new approaches)new approaches) ““New & different ideas are always being tried out”New & different ideas are always being tried out”
Physical comfortPhysical comfort (extent to which physical (extent to which physical surroundings contribute to pleasant work surroundings contribute to pleasant work environment)environment) ““work space is awfully crowded”work space is awfully crowded”
Sample WES ProfilesSample WES Profiles