FEVATOOLS: A TOOLKIT TO JUMP-START FORMATIVE

ASSESSMENT OF STUDENT LEARNING AND COURSE

DESIGN

Jim JuliusSDSU Course Design Institute

May 26, 2010

Guiding Questions

What does it mean to bring an inquiry mindset to course design?

How should one decide what kind of feedback to seek?

What tools are available to collect feedback?

What do I do with the data?

Summative

Formative

What (and why) are you measuring?

Outcomes: tell you what you got, not how or why

Inputs Processes

Seeking continuous improvement

Also formative

What (and why) are you measuring?

Outcomes

Inputs, Processes

Refining type and quality of learning materials &

activities

Cycle 1 Cycle 2 Cycle 3 Cycle 4

Optimized Hybrid or

Fully OnlineClasses

ConventionalFace-to-Face

Classes

Narrowing Your Inquiry + Selecting Useful Tools

http://fevatools.wikispaces.com

Data from M. Laumakis

pICT fellow in 2005 Began teaching parallel 500-student

sections of PSYCH 101 in 2006, one traditional and one hybrid

First fully online PSYCH 101, Summer 2008

Grade data, attendance data, IDEA survey, clicker survey, observational protocols all part of data gathering

SALG

How much did the following aspects of the class help your learning?

Rated from 1 (no help) to 5 (great help)

Summer 2008 Fully Online: SALG Data

Question Summer 2008 Online

Taking the test online 4.27

Discussion Forums 3.00

Introduction e-mail that explained the basics of the course

4.50

SALG Data over time & format

Question Fall 2007

Blended

Fall 2007F2F

Spring 2008

Blended

Spring 2008F2F

Summer 2008

Online

Summer 2009

Online

Questions, answers, and discussions in class

3.96 4.04 4.10 4.01 4.36 4.30

Live online class sessions

3.39 4.20 4.15 4.40

Archives of live online class sessions

4.15 4.50 4.44 4.70

Quality of contact with the teacher

3.41 3.48 3.94 3.90 4.26 4.40

Working with peers outside of class/online

3.12 3.22 3.31 3.39 3.82 3.40

Community of Inquiry Survey

Statements rated from 1 (strongly disagree) to 5 (strongly agree)

Based on the Community of Inquiry framework’s three elements:

1. Social Presence

2. Cognitive Presence

3. Teaching Presence

Community of Inquiry Survey

CoI Dimension Summer 2008 Ratings

Summer 2009 Ratings

Social Presence 3.94 4.16Affective Expression 3.56 3.95Open Communication 4.29 4.42Group Cohesion 3.97 4.12

Cognitive Presence 3.96 4.22Triggering Event 3.91 4.15Exploration 3.73 4.08Integration 4.09 4.33Resolution 4.10 4.33

Teaching Presence 4.38 4.52Design and Organization 4.50 4.67Facilitation 4.38 4.52Direct Instruction 4.23 4.33

What would you like to further explore?

Image sources http://www.flickr.com/photos/brockzilla/271302584/ http://www.flickr.com/photos/littledebbie11/3195586863/ http://www.flickr.com/photos/aaronjacobs/2346932630/ http://www.flickr.com/photos/shareski/3044437847/