Transcript
Page 1: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

Robert Reason Josh Mitchell Kathleen Gillon

Iowa State University

Upper Midwest Civic Engagement Summit

Dubuque, IA

May 29, 2013

MEASURING CIVIC LEARNING:MEANINGFUL STRATEGIES FOR EDUCATING 21ST

CENTURY CITIZENS

Page 2: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

GETTING STARTED:

• Introductions:• Who we are

• Our interest in civic learning research and assessment

• What we hope to gain from this workshop

• Workshop Goals and Interests:

• Why Measure Civic Learning?

• What is Civic Learning?

• How do we Measure Civic Learning?

Page 3: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

GETTING STARTED:

• General Workshop Structure:• Assessing Civic Learning: Challenges and Opportunities

• Defining Civic Learning: What Works?

• A Crucible Moment

• History and Call to Action

• Key Findings and Recommendations

• Key recommendations for Researchers

• Campus-Based Assessment:

• Assessing Engagement

• Indirect vs. Direct Methods of Evaluation

• Connecting Assessment Models with Measurable Learning Outcomes

• Tools and Techniques

Page 4: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

WHERE WE SIT:

• Group Discussion - Major Issues in Assessing Civic Engagement on Campus

• What are some of the biggest obstacles in assessing civic learning and engagement on your campus?

• What are some of the solutions found to address these challenges?

Page 5: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens
Page 6: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

6

THE ORIGIN AND AUTHORS OF A CRUCIBLE MOMENT

• Commissioned by The Department of Education, initiated by Martha Kanter, Under Secretary of Education

• Partnership between Global Perspective Inventory, Inc. (GPI) and the Association of American Colleges and Universities

• National Task Force on Civic Learning and Democratic Engagement• A series of five national roundtables representing leaders from a

wide range of constituent groups inside and outside of higher education

• Several open sessions at a number of national and regional conferences

• A commissioned paper on what we know from research on the impact of civic learning and democratic engagement on student learning and development

Page 7: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens
Page 8: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

KEY RECOMMENDATIONS FOR HIGHER EDUCATION

1. Foster a CIVIC ETHOS across all parts of the campus and educational culture.

2. Make CIVIC LITERACY a core expectation for all students.

3. Practice CIVIC INQUIRY across all fields of study.

4. Advance CIVIC ACTION through transformative partnerships, at home and abroad.

Page 9: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

WHY STUDY CIVIC LEARNING??

Page 10: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

EVIDENCE SUPPORTING NEED FOR COMPREHENSIVE CIVIC LEARNING

• U.S. ranked 139th in voter participation of 172 world democracies in 2007.

• Only 24% of graduating high school seniors scored at the proficient or advanced level in civics in 2010. (NCES, 2011)

• Less than ½ of 12th graders reported studying international topics as part of a civics education. (NCES, 2011)

Page 11: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

STUDENTS VIEWS ON CIVIC LEARNING AND HIGHER EDUCATION

Page 12: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

FINDINGS RELATED TO CIVIC LEARNING AND HIGHER EDUCATION

• A group of 14,000 college seniors surveyed in 2006-07 averaged just over 50% in a civic literacy exam. (Intercollegiate Studies Institute, 2007)

• Just over 1/3 of college faculty surveyed strongly agreed their campus actively promotes awareness of US or global social, political, and economic issues.

• Only slightly more than one-third of students surveyed felt strongly that faculty publicly advocated the need for students to become active and involved citizens. (Dey et al. 2009)

Page 13: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

A FOUNDATION PARTIALLY LAID

Curricular civic pathways

• Making civic literacy a core expectation for all students in general education programs

• Integrating civic inquiry into a central field of study

Powerful civic pedagogies

• Intergroup and deliberative dialogue

• Service Learning and Volunteer Service

• Collective civic problem solving

Page 14: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

THE CHALLENGE OF THE NEXT DECADES OF CIVIC LEARNING

• From elective to expected

• From one time to progressive learning over time

• From individually oriented civic action to collaboratively oriented action

• From some departments, divisions, and people to everyone all over campus

• From parallel practices and programs to integrated ones

Page 15: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

PRIORITIES FOR FUTURE RESEARCH

Disseminate existing assessment tools for measuring civic learning.

Amass and publicize evidence that shows how civic learning, civic agency, and democratic engagement result in increased retention and college success.

Use the Civic Investment Plan matrix to identify specific action research projects that could be initiated at one’s own institution.

Establish standards in civic learning to serve as guidelines for measuring and reporting progress.

Sponsor and support further research on the impact of programs and partnerships that foster civic learning and democratic engagement on learning outcomes and student development.

Include additional research questions in routinely administered higher education surveys to explore how learning environments can enhance key civic competencies

Develop national civic indicators and report on levels of civic and democratic knowledge, skills, values, and action achieved by high school and college graduates.

Page 16: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

FIVE AREAS IN WHICH MORE EVIDENCE ON CIVIC LEARNING IS NEEDED:• Existing data on civic outcomes is almost entirely based on student self-reports of

their behaviors, attitudes, and growth over time. Multiple measures are needed to capture students’ ability to demonstrate civic competencies.

• Data on civic outcomes connected with the experiences of underserved students (underrepresented minorities, first-generation, transfer, and low income).

• Greater evidence is needed on outcomes more closely associated with civic competencies or capacity building, particularly for success in an increasingly global and diversifying nation (e.g. collaborative problem-solving, deliberative dialogue, teamwork).

• Research on outcomes or evidence related to global knowledge and skills is particularly thin

• Though there is strong evidence linking civic learning experiences with improved rates of retention and completion, greater evidence is needed on the ways in which variations in civic learning (i.e. forms of practice and levels of intensity) impact measures of student success.

- Excerpted from “A Brief Review of the Evidence on Civic Learning in Higher Education, Finley, 2011

Page 17: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

WHAT IS CIVIC LEARNING?

Page 18: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

ASSESSING CIVIC LEARNING: TERMS

• Direct measures

• Indirect measures

• Constructs

• Self-reports

Page 19: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens
Page 20: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

ASSESSING CIVIC LEARNING: MEASURABLE OUTCOMES

• Knowledge, skills, and values (Saltmarsh, 2005)

• Knowledge, skills, and behaviors (Cogan, 1999)

• Knowledge, skills, dispositions, identity (Stokamer, 2011)

• Knowledge, skills, practice, and disposition (The Center for Engaged Democracy, 2012)

Page 21: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

KNOWLEDGE

• Understanding of topics related to democratic society; systems and processes of democratic decision-making and governance

- Civic knowledge is understanding of how democracy operates

Page 22: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

SKILLS

• Necessary abilities to understand and advocate for a complex position

- includes intercultural communication, decision-making, critical thinking

Page 23: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

DISPOSITIONS

• Values and attitudes related to actions—the inclination to engage in civic actions

Page 24: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

CIVIC IDENTITY

• “Commitment to civic responsibility through efficacy in action” (Stokamer, 2012, p. 51).

- Requires the incorporation of civic responsibility into one’s self-concept

Page 25: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

HOW DO WE ASSESS CIVIC LEARNING?

Page 26: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

INDIRECT MEASURES

• Assume:

• that engagement in educationally-purposeful activities result in learning outcomes

• that engagement with educationally-purposeful campus climate supports learning outcomes

Page 27: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

INDIRECT CLIMATE MEASURES• National Survey of Student Engagement (http://nsse.iub.edu/)

• Benchmarks

• Active and Collaborative Learning

• Enriching Educational Experiences

• Supportive Campus Environment

• Global Perspectives Inventory (https://gpi.central.edu/index.cfm)

• The GPI reflects a global and holistic view of student learning and development and the importance of the campus environment in fostering holistic student development. The GPI measures how

• a student thinks,

• views herself as a person with a cultural heritage, and

• relates to others from other cultures, backgrounds and values.

Page 28: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

GLOBAL PERSPECTIVE INVENTORY

Interpersonal -- Social Responsibility

• I think of my life in terms of giving back to society.

• I work for the rights of others.

• I put the needs of others above my own personal wants.

• I consciously behave in terms of making a difference.

• Volunteering is not an important priority in my life.

Page 29: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

PERSONAL & SOCIAL RESPONSIBILITY INVENTORY (PSRI)STRIVING FOR EXCELLENCE: developing a strong work ethic and consciously doing one’s very best in all aspects of college

CULTIVATING PERSONAL

AND ACADEMIC INTEGRITY: recognizing and acting on a sense of honor, ranging from

honesty in relationships to principled engagement with a formal academic honors code

CONTRIBUTING TO A LARGER COMMUNITY: recognizing and acting on one’s responsibility to the educational community and the wider society, locally, nationally, and globally

TAKING SERIOUSLY THE PERSPECTIVES OF OTHERS: recognizing and acting on the obligation to inform one’s own judgment; engaging diverse and competing perspectives as a resource for learning, citizenship, and work

DEVELOPING COMPETENCE IN

ETHICAL AND MORAL REASONING

AND ACTION: developing ethical and moral

reasoning in ways that incorporate the other four responsibilities; using such reasoning in learning and in life

http://www.psri.hs.iastate.edu/

Page 30: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

PERSONAL & SOCIAL RESPONSIBILITY INVENTORY (PSRI)

Striving for Excellence

• Overall Climate for Excellence**

• Motivation to Develop a Strong Work

Ethic

• Communicating Expectations about

Excellence**

• Developing a Strong Work Ethic

Cultivating Academic Integrity

• General Climate for Academic

Integrity**

• Faculty Roles in Academic Integrity**

• Developing Academic Integrity

Contributing to a Larger Community

• General Climate for Contributing to a Larger Community**

• Advocating for Contributing to a Larger Community**

• Developing a Commitment to Contributing to a Larger Community

Taking Seriously the Perspectives of Others

• General Climate for Perspective Taking**

• Advocating for Perspective Taking**

• Developing Perspective Taking

Refining Ethical and Moral Reasoning

• General Climate for Ethical and Moral Reasoning**

• Sources of Support for Ethical and Moral Reasoning**

**Indicates a factor score that is comparable to professionals’ factor score. The factor is comprised of exact (or analogous) items as the students’ factor.

Page 31: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

DIRECT MEASURES

Attempt to measure (directly) the outcomes believed to be associated/comprise the construct of “civic competency” or “civic learning.”

Page 32: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

CIVIC ATTITUDES & SKILLS QUESTIONNAIRE(MOELY ET AL., 2002)

Factor 1: Civic ActionI plan to do some volunteer work

I am committed to making a positive difference.

 

Factor 2: Interpersonal and problem-solving skills

I can listen to other people’s opinions.

I can successfully resolve conflicts with others.

I try to place myself in the place of others in trying to assess their current situation.

I tend to solve problems by talking them out.

 

Factor 3: Political AwarenessI am aware of current events.

I plan to be involved in the political process.

Factor 4: Leadership SkillsI am a good leader.

I have the ability to lead a group of people.

I feel I can make a difference in the world.

 

Factor 5: Social Justice AttitudesPeople are poor because they choose to be poor. (Reverse scored)

In order for problems to be solved, we need to change public policy.

We need to change people’s attitudes in order to solve social problems.

 

Factor 6: Diversity AttitudesI find it difficult to relate to people from a different race or culture. (Reverse scored)

I enjoy meeting people who come from backgrounds very different from my own.

Cultural diversity within a group makes the group more interesting and effective.

Page 33: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

ACTIVISM ORIENTATION SCALE(CORNING & MYERS, 2002)

Conventional Activism• Display a poster or bumper sticker with a

political message?• Invite a friend to attend a meeting of a political

organization or event? • Purchase a poster, t-shirt, etc. that endorses a

political point of view? • Serve as an officer in a political organization? • Attend an informational meeting of a political

group? • Organize a political event (e.g., talk, support

group, march)? • Give a lecture or talk about a social or political

issue? • Go out of your way to collect information on a

social or political issue? • Campaign door-to-door for a political

candidate?

High-Risk Activism• Engage in a political activity in which

you knew you would be arrested? • Engage in a physical confrontation at

a political rally • Engage in a political activity in which

you feared that some of your possessions would be damaged?

• Engage in an illegal act as part of a political protest?

• Engage in a political activity in which you suspect there would be a confrontation with the police or possible arrest?

• Block access to a building or public area with your body?

Page 34: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

EXPECTED POLITICAL VOICE MEASURE(SEIDER, GILLMOR, & RABINOWIC, 2012)

Contact or visit a public official to express your opinion

Contact a newspaper or magazine to express your opinion on an issue

Call in to a radio or television talk show to express your opinion

Write a comment on a political blog

Take part in a protest, march, or demonstration

Sign a written or email petition about a political or social issue

NOT buy something because you dislike the conditions under which it was made

Go door to door to campaign for a political cause

Work together with others to solve a problem in the community where you live

Run for a political position at the local, state, or national level

Page 35: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

COMMUNITY SERVICE SELF-EFFICACY SCALE(REEB ET AL., 2010)

If I choose to participate in community service in the future, I will be able to make a meaningful contribution.

In the future, I will be able to find community service opportunities which are relevant to my interests and abilities.

I am confident that, through community service, I can help in promoting social justice.

I am confident that, through community service, I can make a different in my community.

I am confident that I can help individuals in need by participating in community service activities.

I am confident that, in future community service activities, I will be able to interact with relevant professionals in ways that are meaningful and effective.

I am confident that, through community service, I can help in promoting equal opportunity for citizens.

Through community service, I can apply my knowledge in ways that solve “real-life” problems.

By participating in community service, I can help people to help themselves.

I am confident that I will participate in community service activities in the future.

Page 36: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

OPEN ENDED QUESTIONS/NARRATIVE DATA

• Open ended question:

What experiences at this campus have helped you to further develop your work ethic? Please describe 1-2 examples below.

• Narrative Data

• Just the sheer amount of work that I need to do to stay afloat in a class. And the result of high grades, especially at such a rigorous institution, is rewarding

• Work-load: sink or swim. Without a strong work ethic, you will truly drown.

• Well, the academic standards are such that to "get by" you have to have at least a DECENT work ethic or be very passionate about your subject matter. While I've struggled with procrastination, small class sizes and the opportunity to work closely with professors has enabled me to grow as a student and a person.

Page 37: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

PURPOSES OF NARRATIVE DATA

Narrative data can…

1. Provide a Comprehensive Understanding of Civic Engagement

• Insight into Quantitative Measures

• Overall climate

• Recent situations

2. Allow for Engagement in Civic Literacy

• Key Recommendation for Higher Ed: Make CIVIC LITERACY a core expectation for all students.

• Can students articulate their experiences related to civic learning?

3. Inform future assessment and practice

Page 38: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

I am heavily involved in non-academic activities (service organizations, academic organizations, greek life, etc) and have taken on a number of leadership roles during my time here. It is my responsibility to be on top of my work so that I can perform my leadership responsibilities for each organization and also complete my coursework in a timely fashion.

I am captain of the Mock Trial team. My experience with this organization has honed me into being an even more responsible, organized, and motivated individual.

I am standards chair for my sorority. As such, it is my job to promote good academic and social behavior among our members.

Learning about events like the community garden and seeing all the opportunities that our institution offers have helped me deepen my commitment to contributing to the community.

Being a member of a student organization dedicated to non-partisan discussion of politics has helped me understand the need for civic engagement.

Our school has a culture of taking on many projects, clubs, leadership positions, volunteer experiences, jobs, etc. but this is different than work ethic. Students here collect titles, but don't necessarily spend the time to be efficient or good at any of these.

Page 39: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

I feel they are trying to put a gag on faculty with unpopular opinions! When the president received a vote of no confidence from the faculty, instead of firing him, the board of trustees helped the president to commit a retribution firing of the only untenured faculty member from the department involved in uncovering the administration's violation of shared governance. WHAT A FIASCO! Complete lack of integrity…This is why the past three board of trustee meetings have been packed with student protestors, but the board refuses to respond to student demands for justice.

Unfortunately the president of the school is involved in many despicable legal battles against academic freedom of the faculty. This has deepened my commitment to justice, especially when big institutions like this school abuse their power.

I find it extremely disheartening and disturbing that the President has waged a campaign AGAINST faculty free speech and shared governance. The department and faculty do, however, uphold, support and educate us about academic integrity in spite of the administration's abuse.

Students are actively discouraged from speaking any view that is contrary to the senior administrations’. When faculty try to advocate for students they are retaliated against by the administration by being denied chairships, tenure, and other faculty esources. These denials have resulted in costly litigation by the university, which drains scarce financial resources.

Seeing the immoral actions of the President and the Board of Trustees with regard to tenure and the Board's inattention to the academic health and needs of the University has discouraged many of us.

Page 40: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

USING NARRATIVE DATA

• Read

• Don’t let the data sit on the shelf

• Analyze

• Thematic

• Cross-case

• Present

• How can we present narrative data in a way that is concise, meaningful. and easy to share?

Page 41: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

INFORMING FUTURE ASSESSMENT & PRACTICE

• Using narrative data to guide additional assessment & practice

• Focus Groups

• Interviews

• Observations

• Texts & Websites

Page 42: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

CIVIC LEARNING: TOOLS AND RESOURCESLiterature

• A Crucible Moment: College Learning and Democracy’s Future: http://www.aacu.org/civic_learning/crucible/index.cfm

• A Brief Review of the Evidence on Civic Learning in Higher Education: http://www.aacu.org/civic_learning/crucible/documents/CivicOutcomesBrief.pdf

• Promising Practices for Personal and Social Responsibility: https://secure.aacu.org/source/Orders/index.cfm

• Making Progress: What We Know About the Achievement of Liberal Education Outcomes : https://secure.aacu.org/source/Orders/index.cfm

• Diversity and Democracy, Fall 2012 Issue – Civic Engagement and Student Success: A Resonant Relationship: http://www.diversityweb.org/diversitydemocracy/

Assessment Tools and Inventories

• Personal and Social Responsibility Inventory: http://www.psri.hs.iastate.edu/

• Global Perspective Inventory: https://gpi.central.edu/index.cfm

• AAC&U VALUE rubrics: http://www.aacu.org/value/index.cfm

Page 43: Measuring Civic Learning: Meaningful Strategies for Educating 21st Century Citizens

FOR FURTHER INFORMATION ON THE PERSONAL AND SOCIAL RESPONSIBILITY INVENTORY (PSRI), CONTACT

Robert ReasonPSRI Director / Associate Professor of Education

Iowa State [email protected] http://www.psri.hs.iastate.edu/


Top Related