are you using technology to assess learning or assessing learning to improve technology? marilee j....
Post on 20-Dec-2015
226 views
TRANSCRIPT
Are you Using Technology to Assess Learning or Assessing Learning to Improve Technology?
Marilee J. Bresciani, Ph.D.Associate Professor, Postsecondary Education andCo-Director of the Center for Educational Leadership, Innovation, and PolicySan Diego State University3590 Camino Del Rio NorthSan Diego, California, [email protected]
Overview
Overview of Outcomes-Based Assessment
Examples of Using Technology to Assess Learning or Assessing Learning to Improve Technology
Questions to Consider .
Ask Yourself These Questions
How would you describe what outcomes-based assessment is to your faculty, staff, and students?
How would you explain its importance?
Ask Yourself These Questions
How would your description change if you were describing assessment for
how well technology enhances student learning?
how well technology can be used to assess and report out on student learning ?
The Assessment Cycle (Bresciani, 2006)
The key questions…• What are we trying to do and why? or• What is my program supposed to accomplish?
or• What do I want students to be able to do and/or
know as a result of my course/workshop/orientation/program?
• How well are we doing it?• How do we know?• How do we use the information to improve or
celebrate successes?• Do the improvements we make contribute
to our intended end results?
The IterativeSystematicAssessment CycleBresciani, 2006
Mission/Purposes
Goals
Outcomes
Implement Methods to
Deliver Outcomes and
Methods to Gather Data
Gather Data
Interpret Evidence
Make decisions to improve programs; enhance student learning and development;
inform institutional decision-making, planning,
budgeting, policy, public accountability
Drivers
Regional Accreditation Requirement since 1985
National Commission on the Future of Higher Education (2006)
Demand for Public Information about Performance
• Transparency of outcomes and results• Comparable measures of quality• Demonstration of value-added of the
entire educational experience
Drivers, Cont.
Accountability requirements handed back to statesPerformance indicatorsStandardized tests
Discipline Standards could be designed by disciplines
Increased focus on private education Still no focus on evaluating
transferability of learning
Now turn to each other
Would you change any of your responses to the aforementioned questions?How would you describe what
outcomes-based assessment is?
How would you explain its importance?
This Assessment Process Can be Used To…
1. Assess learning, where technology is adata collection tool or teaching tool (to promote deep
learning)
2. Improve technology as a teaching tool or a data collection tool in order to make assessment of learning more effective
This Assessment Process Can be Used To…
3. Improve professional development for use of technology
4. Improve technological skills of students, faculty, and staff
5. Improve process of planning for improved student learning by using technology as an instructional organizational tool
So, One Process,
which is outcomes-based assessment,
for many uses
The Key is to Determine
What you are assessing?Therefore, what are your
outcomes?Informs DesignInforms MethodologyInforms Decisions
Student Learning Example
Students will be able to identify potential ethical research dilemmas and conflicts of interest and propose solutions to address themCase Study exercise in classAssignment - Student’s actual
research project using Blackboard discussion
Student Learning Example, Cont.
The majority of students were able to demonstrate this learning on blackboard and challenged their peers’ comments as well.
However, three students did not perform as well as they had on the in-class case studies.
What was the problem?
Lack of student learning
Or concern with technology as a data collection tool?
Two things Going on at Once
1. Assess learning, where technology is adata collection tool or teaching tool
2. Improve technology as a data collection tool in order to make assessment of learning more effective
Another Example
Assessing effectiveness of professional development for faculty using TracDat to manage their process for evaluating student learningEvidence of assessment plans and
reports entered
Another Example
All 10 who attended the workshop entered assessment plans But 6 of the 10 had poorly written
outcomes, and disconnected evaluation methods even though all items were in their “correct” box
Only 2 of the 10 faculty had entered assessment reports
What was the problem?
Poor professional development workshop
Or concern with faculty and their technology skills?
Three Things Going on at Once
2. Improve technology as a data collection tool in order to make assessment of learning more effective
3. Improve professional development for use of technology
4. Improve technological skills of students, faculty, and staff
One Final Example
Assessed instructional technology for its accessibility from the Pacific Islands Delays in assignments postedTroubles posting to discussion
board
Example, Cont.
Poor accessibility by students but could not discern if it was the software or the students
Interestingly, students did not complain about access and the designer said it was because the course expectations were so clear
What was the problem?
Poor technology
Or concern with students and their technological skills?
Three Things Going On
2. Improve technology as a teaching tool in order to make assessment of learning more effective
4. Improve technological skills of students, faculty, and staff
5. Improve process of planning for improved student learning by using technology as an instructional organizational tool
Re-Visiting: The Key is to Determine
What you are assessing?Therefore, what are your
outcomes?Informs DesignInforms MethodologyInforms Decisions
Take a moment and share
with each other a scenario where you determined that you needed to clarify whether you were assessing technology or learning
Questions to Consider (Bresciani, 2007)
When considering the use of Technology to promote outcomes-based assessment…Does the use of this technology reinforce the
reasons we are engaged in outcomes-based assessment?
How does the use of this technology promote formative assessment, summative assessment, or the assessment of transferable skills/knowledge?
Questions to Consider, Cont. (Bresciani, 2007)
Is our professional development to train users of our technology coupled with the professional development for outcomes-based assessment or instructional design?
How does the use of this technology enable us to better use the results to improve student learning and development?
Questions to Consider, Cont. (Bresciani, 2007)
How does the use of this technology improve efficiencies? And who benefits from those improved efficiencies?
How does this technology constrain or promote creativity/flexibility within each discipline?
How does this technology aid the coordination of the process? Or does it contribute to isolated thinking and doing?
Questions to Consider, Cont. (Bresciani, 2007)
How can data that this technological tool gathers help us evaluate the extent that we are transforming our organization through the use of evidence-based decision-making?“Keep your eye on the prize”
Challenge Questions to Consider (Bresciani, 2007)
How does this technology help illustrate the transferability of learning/skills?
How does this technology help us illustrate, evaluate, and improve the learning/skills as they relate to the requisite skills required by employers?
Challenge Questions to Consider , Cont. (Milne, 2007)
How does this technology help us determine whether the technology is promoting or prohibiting social interaction?
How do we ensure that the technology is assuring the deepest engagement with the subject matter?
Challenge Questions to Consider , Cont. (Bresciani, 2007)
How will you determine that technology is promoting deep learning?
What do you need to refine in order to make this happen?
Which questions do you need to focus on as you move forward with your learning at this conference?
References
Bresciani, M.J. (2006). Outcomes-Based Academic and Co-Curricular Program Review: A Compilation of Institutional Good Practices. Sterling, VA: Stylus Publishing
Bresciani, M.J. (2007). Implementing Lessons Learned: Establishing A Culture of Evidence-Based Assessment. Closing Keynote at Course Technology Conference. Las Vegas, NV.