quantitative assessment methods - student...

Quantitative Assessment Methods

Victor W. Barr, Ph.D.

Dan Reilly

Quantitative Assessment Methods

This program will focus on guiding you through the process of planning for gathering and analyzing quantitative data. Quantitative data is data that can be analyzed as numbers as opposed to qualitative data. In addition, this session will briefly cover issues of how to make decisions about how such data is gathered,

analyzed and used to make a decisions and arguments. Specific attention will be focused on how to build the structures that make gathering such data easier.

Learning Outcomes Examples

Participants will learn a process of making a decision about what data to collect and how simplify the collection of information.

Participants will be able to apply the methodologies of quantitative data to their own area of student affairs.

Participants will be better prepared to make decisions about how they will plan for using and presenting data.

“If we are to convince our colleges and universities of the worth of our centers we must begin to evaluate

the outcomes of our services.

This is especially important as we move into times of budgetary restraint.” (Victor Barr, 1997)

Quantitative vs. Qualitative.

• I think of the two (quantitative versus qualitative) as complementary ways of gathering and presenting evidence to support your case.

•

• Qualitative is both a first step (What's going on here? or Exploratory?)

• and frequently gives us the data to tell the story. It though can mostly tell us perceptions

•

• Quantitative helps us to look below the surface and see what is going on in a more definable way. It also provides data that for some is more convincing.

Need to think through ahead of time –What story will I need to tell and what data is needed to tell the story and be convincing?

First Decision Points

•What point do you need to be able to make?•Who is you audience(s)?•How do you want to make your point?

None stand alone

What point do you need to be able to make?

• What is the question at hand and how can you get to the best data to be able to answer it?

Question: Counseling Center

Are you really that busy?

Who is you audience(s)?

• Is it one targeted audience?

• Are there more than one?

• For each audience what type(s) of data are convincing?

Counseling Center

Question: Are you really that busy?

Audience: VC Rogers, Departmental Review, Chancellor

Who is you audience(s)?For each audience what type(s) of

data are convincing?

Counseling Center Audience:

VC Rogers

Departmental Review

Chancellor

What type of data is the best way to make your point?

• Survey

• Usage data

• Process data

• Comparison data

• Outcome data

At its heart evaluation is a three step sequence

Objectives ------->Means---------->Measures

Question:

Data Issues

• Validity of measures

• Ease of use of measures

• Data standardization – at UT and External to UT

Counseling Center Data Sources

Question: Are you really that busy? Audience: VC Rogers, Departmental Review, ChancellorMethodology: Usage data, Comparison Data.

Evaluations vs. Research

• Research is a proving causality and involved tight control, specificity and rigor. It can be artificial –random assignment.

• Evaluation, on the other hand, has come the way of technology rather than science. Its accent is not on theory building but on product delivery or mission accomplishment. Its essence is to provide feedback leading to a successful outcome defined in practical, concrete terms.

• “The purpose of evaluation is to improve, not to prove.” (Stufflebeam et al 1978)

Data to collect and how?Creating the Hook

What data will you collect?

How will you set it up easily collect the data?

How do you plan to use the data?

Will data storage be an issue?

Desk drawer is not adequate.

If on-going, keep and eye on the data as you collect it.

http://www.chatsworth.com/Product_Docs/13082_NPS.PDF

http://christiankocinski.files.wordpress.com/2009/12/hook_ver1.jpg

Weekly Data Gathered by Counseling Center


Caveats

• Check your data regularly.

Verify as you go.

Will it make your point at the end?

Does your data make sense?

• Back up you data regularly.


Presenting your data

Things to Remember

• Remember Point to be made and Audience

• Analyze data

• How will you present your results? What is the venue – report, PowerPoint, etc. Remember Point to be made and Audience.

Counseling Center

Use of Comparison Group


0

500

1000

1500

2000

2500

3000

3500

4000

#

School

Student ratio/Senior Staff

Use of Comparison Group

0

2000

4000

6000

8000

10000

12000

14000

8/11 9/11.45 8/10 9/13 12/12 12/18 8.5/11.5 10/17 13/13 7.15/7.15 8.4/10 14.47/ 22 9/13 11/16 18/20 13.3/13.2

Senior/Senior+Interns

What is the total number of sessions provided by your center?

Public Universities of 25001-30,000


0

10

20

30

40

50

60

70

#C

lie

nts

Week

Waitlist FY10

Waitlist 1

Waitlist 2

Waitlist 3

Waitlist 4

FY08

FY09

Counseling Center

• Issues to Consider

– We started putting more emphasis on group counseling 3 years ago.

– This last fall we made the walk-in briefer and mores structured.

– We added to out array of services Stress and Wellness Clinic .


Center Utilization Data

0

10

20

30

40

50

60

70

#

Week

Walk-Ins

FY08FY09FY10FY08FY09FY10



0

50

100

150

200

250

300

350

8/3

8/1

7

8/3

1

9/1

4

9/2

8

10/1

2

10/2

6

11/9

11/2

3

12/7

12/2

1

1/4

1/1

8

2/1

2/1

5

3/1

3/1

5

3/2

9

4/1

2

4/2

6

5/1

0

5/2

4

6/7

6/2

1

7/5

7/1

9

8/2

SESSIONS Offered FY10

2006-07

2007-08

2008-09

"20009-10

2006-07

2007-08

2008-09

"20009-10




0

50

100

150

200

250

300

350

7/3

0

8/6

8/1

3

8/2

0

8/2

7

9/3

9/1

0

9/1

7

9/2

4

10/1

10/8

10/1

5

10/2

2

10/2

9

11/5

11/1

2

11/1

9

11/2

6

12/3

12/1

0

12/1

7

12/2

4

12/3

1

1/7

1/1

4

1/2

1

1/2

8

2/4

2/1

1

2/1

8

2/2

5

3/3

3/1

0

3/1

7

3/2

4

3/3

1

4/7

4/1

4

4/2

1

4/2

8

5/5

5/1

2

5/1

9

5/2

6

6/2

6/9

6/1

6

6/2

3

6/3

0

7/7

7/1

4

7/2

1

#

Week

Delivered: Triage + Sessions + Group + SWC

"07/08"08-09"09-10"07/08"08-09"09-10


3000

4000

5000

6000

7000

8000

9000

10000

FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09

Sessio

ns

Year

Total Sessions Intake + Shown + Group + SWC

Public Universities of 25001-30,000


Questions

quantitative assessment methods - student...

Documents