1 blackboard outcomes system: enabling continuous improvement
TRANSCRIPT
1
Blackboard Outcomes System: Enabling Continuous Improvement
2
I. Critical Issues in Higher Education:Accountability, Accreditation, and Need for Insight
Key Issues
3
National Association of State Universities and Land Grant Colleges recommends Voluntary System of Accountability (VSA) using measures that:
• Improve educational outcomes of students• Focus on value added• Are transparent or publicly disclosed• Are compared against an institution’s past performance
Increasing demands for accountability
Commission on the Future of Higher Education“Colleges and universities must become more transparent about cost, price and student success outcomes and must willingly share this information with students and families.”• Creation of “consumer-friendly information database on higher education” to include student learning outcomes.• Tie federal funding to institutions sharing performance data.
4
State Challenges
The Pappas ReportJuly 2006
“A recent (June 20, 2006) report by Education Research Center utilizes a Cumulative Population Index to estimate the probability that a student in the 9th grade will complete high school on time with a regular diploma. North Carolina was at 66%, which is below the national average.”
“The increasing diversity will further challenge preparation as there are substantial educational achievement gaps among different socio-economic groups. NCCS reports that…12.76% of white enrollment were graduates in 2004-5, compared to 9.68% Black and 9.03% of Hispanic enrollment.”
5
PLAN ME
AS
UR
E
IMPROVE
CYCLE
SACS Standard 3.3.1. The institution -
Accreditation focus on learning outcomes
Identifies expected outcomes for its educational programs and its administrative and educational support services.
Assesses whether it achieves these outcomes
Provides evidence of improvement based on analysis based on analysis of those results
Establishes and evaluates program and learning outcomes
Ongoing (not just once each accreditation period) Institution-wide (academic units, administrative units) SACS Standard 2.5
6
II. Drive for Continuous Improvement
7
1. What specific knowledge, abilities, and experiences should the graduates of each of our programs possess?
2. How do we see (measure) these outcomes in our students – developing from entry to exit and then demonstrated post-graduation?
3. How do institutional services and policies affect these outcomes?
4. Do specific student attributes (e.g. race, gender, nationality, financial aid status) affect achieving these outcomes?
5. How does course sequencing, pedagogy, co/extra-curricular activities, and technology affect these outcomes?
6. What is the location of our longitudinal data supporting the answers to these questions?
7. Do stakeholders at our institution agree on the answers?
Higher Education Wants to Understand its Effectiveness
8
Cycle of assessment in higher education
• Plan program learning outcomes
• Plan course outcomes
• Define measurement methods
• Establish rubrics• Discussions
• Survey employers• Interview graduates• Collect examples of
student work• Run course
evaluations• Construct portfolios
• Review survey results• Look for demographic trends
on data• Discussions • Develop program change
recommendations
CYCLE
PLAN ME
AS
UR
E
IMPROVE
Drive for continuousimprovement in institutional effectiveness:• Academic outcomes - teaching and learning• Administrative outcomes
9
Multiple levels of assessment
Course
Program
Institution
– Strategic planning, institutional research, accreditation self studies, general education
– Program assessment, curriculum planning, discipline-specific accreditation, course design
– Course development, testing, assignments, portfolios, course re-design
Inte
nded
out
com
es
Measured outcom
es
10
III. Obstacles to effective assessment and continuous improvement
11
Obstacles to effective assessment and continuous improvement Lack of visibility into assessment data and knowledge
– Missed opportunity to learn and improve– No repository for data over time
Fragmented technology applications– Disconnected individual systems for planning, measuring, and
improving Lack of connection between different assessment
processes at multiple levels of institution– Duplication and overlap of activities and effort
Current processes are costly and time consuming– Paper and pencil
12
Fragmented and Disconnected Technology Applications
Assessment ManagementAssessment Management
InstitutionalResearch
InstitutionalResearch
Student OutcomePlanning/Tracking
Student OutcomePlanning/Tracking
EnterpriseTest & Survey
EnterpriseTest & Survey
•Institutional and Department Goal / Objective Management
•Executive Dashboard
•Curriculum Planning
•Project Management Tools
•3rd Party Standards
•Document Management
•Communication and Collaboration Tools
•Enterprise Test
•Enterprise Survey
•Enterprise Course Evaluation
•3rd Party Instruments
•Course Test Aggregate Reporting
•Bubble Sheets (OMR)
•Secure Testing
•Evaluation Portfolios
•Learning Content & Activity Alignment
•Executive Dashboard and Scorecards
•Student Performance Monitoring
•Early Warning Systems
•Lesson Planning
•Export
•Evaluation DataWarehouse
•Bb LS Extraction, Transformation, Loading (ETL)
•SIS / SRS ETL
•Correlations
•Analytics Engine
•Ad-hoc Reporting Manager
Institution Level
Program Level
Classroom Level
13
IV. Blackboard Outcomes SystemTM
14
A New Member of the Blackboard Academic Suite™
15
Co-Designed with Leading Partner Institutions
16
Guided by National Experts
Mary Allen, Ph.D.– Professor Emeritus of Psychology at California State University–
Bakersfield– Former Director, California State University Institute for Teaching and
Learning. – Author of “Assessing Academic Programs in Higher Education” and
“Assessing General Education Programs”
Bob Mundhenk, Ph.D.– Former Director of Assessment and Senior Scholar, American Association
of Higher Education– Former Dean, Director of Assessment, VP of Student Affairs
Ephraim Schechter, Ph.D.– Former University Director of Assessment, NC State University– Directed assessment components of North Carolina State University's
2004 SACS reaffirmation. – Maintains the well-known meta-list
Internet Resources for Higher Education Outcomes Assessment
17
Expected Outcomes and Benefits
Expected Outcomes– Enable creation of “systemic and systematic” assessment processes– Connect institutional assessment to program assessment to classroom
assessment– More visibility of data and knowledge about outcomes– Better evidence-based decision making
• Confidence in budgetary justification Benefits
– Better learning– More efficient and effective assessment activities– Easier accreditation– Cost savings
18
Uses for the Blackboard Outcomes System
Institution– Capturing Data for Institutional Research– Measuring Success of Strategic Plan Implementation– Managing Regional Accreditation Self Study– Providing Government Reporting
Program– Capturing Data for Program Reviews and Assessments– Showcasing Curriculum Planning– Managing Discipline-specific Accreditation
Classroom– Organizing Course Development– Capturing Data for Course Redesign– Improving Face-to-Face, Blended, and e-Learning Instruction
19
V. Key Capabilities
20
Provide critical information to make critical decisions
Reports Make evidence-based
decisions to improve academic and administrative outcomes
Inform analysis with data: From across
departments Linking planned
outcomes and measured outcomes
2005 2006
21
Measure student learning outcomes
Rubrics Employ rubrics to
communicate shared expectations and measure student learning outcomes
22
Encourage active and reflective learning
Outcomes Portfolios Apply rubrics to student work in
Outcomes Portfolios Measures course and program
learning outcomes
23
Collect and evaluate student work
Outcomes Artifact Collect student work for
evaluation Evaluate student work Apply rubric to measure
student learning outcomes
24
Promote a holistic view of the curriculum
Curriculum maps Identifies gaps
and redundancies in curriculum
Inform other departments about relationships
Document relationships between program outcomes and course outcomes
25
Understand your stakeholders
Surveys Learn from on-
campus and off-campus stakeholders via this email-based solution
Align questions with standards, goals, and objectives
Built-in flexible analysis and reporting
26
Reduce time and cost of assessment and accreditation Consolidate investments in numerous
disconnected technology with institution-wide access to key capabilities: Enterprise Survey Course Evaluation Outcomes Portfolio Rubric Builder Curriculum Mapping
Content Management Streamline documentation of
assessment processes for regional and specialized accreditation
Improvement Initiatives Manage and track progress for
numerous assessment process
